2024-09-15T03:33:37.363 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T03:33:37.381 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T03:33:37.468 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects} email: skanta@redhat.com first_in_suite: false job_id: '7905302' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 2126 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 thrash_many: true - rados: clients: - client.0 objects: 50 op_weights: copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_00:54:39 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T03:33:37.468 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-15T03:33:37.469 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-15T03:33:37.469 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T03:33:37.473 INFO:teuthology.task.internal:Checking packages... 2024-09-15T03:33:37.495 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-15T03:33:37.495 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T03:33:37.495 INFO:teuthology.packaging:ref: None 2024-09-15T03:33:37.495 INFO:teuthology.packaging:tag: None 2024-09-15T03:33:37.495 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T03:33:37.495 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:33:37.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-15T03:33:37.647 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:33:37.649 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T03:33:37.659 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T03:33:37.659 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T03:33:37.665 INFO:teuthology.task.internal:Saving configuration 2024-09-15T03:33:37.682 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T03:33:37.689 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T03:33:37.714 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 03:27:46.520048', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T03:33:37.739 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 03:27:46.518381', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-15T03:33:37.739 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T03:33:37.747 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-09-15T03:33:37.747 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-09-15T03:33:37.747 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T03:33:37.807 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fde183c3ac0>, signals=[15]) 2024-09-15T03:33:37.807 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T03:33:37.814 INFO:teuthology.task.internal:Opening connections... 2024-09-15T03:33:37.814 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T03:33:37.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:33:37.909 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T03:33:37.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:33:38.004 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T03:33:38.013 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-09-15T03:33:38.020 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-09-15T03:33:38.020 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-09-15T03:33:38.069 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="Ubuntu" 2024-09-15T03:33:38.069 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T03:33:38.069 INFO:teuthology.orchestra.run.smithi026.stdout:ID=ubuntu 2024-09-15T03:33:38.069 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE=debian 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="20.04" 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_CODENAME=focal 2024-09-15T03:33:38.070 INFO:teuthology.orchestra.run.smithi026.stdout:UBUNTU_CODENAME=focal 2024-09-15T03:33:38.071 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-09-15T03:33:38.093 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-15T03:33:38.099 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-15T03:33:38.100 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="20.04" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-15T03:33:38.147 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-15T03:33:38.148 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=focal 2024-09-15T03:33:38.148 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=focal 2024-09-15T03:33:38.148 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-15T03:33:38.167 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T03:33:38.176 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T03:33:38.181 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T03:33:38.181 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-09-15T03:33:38.183 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-15T03:33:38.189 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T03:33:38.194 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T03:33:38.194 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-09-15T03:33:38.196 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-15T03:33:38.245 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T03:33:38.299 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T03:33:38.313 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T03:33:38.313 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T03:33:38.314 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T03:33:38.314 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T03:33:38.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:33:38.320 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-09-15T03:33:38.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:33:38.321 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-15T03:33:38.327 INFO:teuthology.orchestra.run.smithi026.stdout:5.4.0-124-generic 2024-09-15T03:33:38.327 INFO:teuthology.task.kernel:Running kernel on smithi026: 5.4.0-124-generic 2024-09-15T03:33:38.327 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-get clean 2024-09-15T03:33:38.371 INFO:teuthology.orchestra.run.smithi122.stdout:5.4.0-124-generic 2024-09-15T03:33:38.372 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.4.0-124-generic 2024-09-15T03:33:38.372 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-15T03:33:38.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-15T03:33:38.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-get update 2024-09-15T03:33:38.605 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:33:38.616 INFO:teuthology.orchestra.run.smithi026.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T03:33:38.618 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T03:33:38.683 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T03:33:38.721 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-15T03:33:38.721 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:33:38.802 INFO:teuthology.orchestra.run.smithi026.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-15T03:33:38.910 INFO:teuthology.orchestra.run.smithi026.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T03:33:38.983 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T03:33:39.090 INFO:teuthology.orchestra.run.smithi026.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T03:33:39.137 INFO:teuthology.orchestra.run.smithi026.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T03:33:39.140 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T03:33:39.143 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T03:33:39.144 INFO:teuthology.orchestra.run.smithi026.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T03:33:39.155 INFO:teuthology.orchestra.run.smithi026.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-15T03:33:39.156 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T03:33:39.159 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T03:33:39.159 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T03:33:39.160 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T03:33:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T03:33:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T03:33:39.183 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T03:33:39.186 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:33:39.186 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T03:33:39.188 INFO:teuthology.orchestra.run.smithi026.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:33:39.188 INFO:teuthology.orchestra.run.smithi026.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T03:33:39.195 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T03:33:39.200 INFO:teuthology.orchestra.run.smithi026.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T03:33:39.200 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T03:33:39.202 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T03:33:39.203 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T03:33:39.203 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T03:33:39.203 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T03:33:39.208 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T03:33:39.208 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T03:33:39.208 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T03:33:39.209 INFO:teuthology.orchestra.run.smithi026.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T03:33:39.210 INFO:teuthology.orchestra.run.smithi026.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T03:33:39.210 INFO:teuthology.orchestra.run.smithi026.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T03:33:39.211 INFO:teuthology.orchestra.run.smithi026.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T03:33:39.225 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T03:33:39.225 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T03:33:39.225 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T03:33:39.226 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T03:33:39.258 INFO:teuthology.orchestra.run.smithi026.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T03:33:39.258 INFO:teuthology.orchestra.run.smithi026.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T03:33:39.296 INFO:teuthology.orchestra.run.smithi026.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-15T03:33:39.322 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-15T03:33:39.657 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-15T03:33:39.679 INFO:teuthology.orchestra.run.smithi026.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-15T03:33:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-15T03:33:39.720 INFO:teuthology.orchestra.run.smithi026.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-15T03:33:39.721 INFO:teuthology.orchestra.run.smithi026.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-15T03:33:39.816 INFO:teuthology.orchestra.run.smithi026.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-15T03:33:39.816 INFO:teuthology.orchestra.run.smithi026.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-15T03:33:39.835 INFO:teuthology.orchestra.run.smithi026.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:33:39.835 INFO:teuthology.orchestra.run.smithi026.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-15T03:33:39.849 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-15T03:33:39.856 INFO:teuthology.orchestra.run.smithi026.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-15T03:33:39.864 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-15T03:33:39.864 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-15T03:33:39.887 INFO:teuthology.orchestra.run.smithi026.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-15T03:33:39.892 INFO:teuthology.orchestra.run.smithi026.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-15T03:33:39.892 INFO:teuthology.orchestra.run.smithi026.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-15T03:33:39.893 INFO:teuthology.orchestra.run.smithi026.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-15T03:33:39.924 INFO:teuthology.orchestra.run.smithi026.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-15T03:33:39.925 INFO:teuthology.orchestra.run.smithi026.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-15T03:33:39.925 INFO:teuthology.orchestra.run.smithi026.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-15T03:33:39.925 INFO:teuthology.orchestra.run.smithi026.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-15T03:33:39.955 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-15T03:33:39.956 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-15T03:33:39.968 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-15T03:33:39.968 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-15T03:33:39.989 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-15T03:33:40.005 INFO:teuthology.orchestra.run.smithi026.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-15T03:33:40.006 INFO:teuthology.orchestra.run.smithi026.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-15T03:33:40.007 INFO:teuthology.orchestra.run.smithi026.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-15T03:33:40.007 INFO:teuthology.orchestra.run.smithi026.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-15T03:33:40.019 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-15T03:33:40.026 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-15T03:33:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-15T03:33:40.027 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-15T03:33:40.094 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-15T03:33:40.095 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-15T03:33:43.621 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 21.8 MB in 3s (7,158 kB/s) 2024-09-15T03:33:43.660 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 21.8 MB in 3s (6,955 kB/s) 2024-09-15T03:33:45.201 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2024-09-15T03:33:45.209 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-15T03:33:45.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:33:45.233 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:33:45.315 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2024-09-15T03:33:45.324 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-15T03:33:45.541 INFO:teuthology.orchestra.run.smithi026.stdout:Building dependency tree... 2024-09-15T03:33:45.542 INFO:teuthology.orchestra.run.smithi026.stdout:Reading state information... 2024-09-15T03:33:45.543 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-15T03:33:45.544 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-15T03:33:45.754 INFO:teuthology.orchestra.run.smithi026.stdout:The following additional packages will be installed: 2024-09-15T03:33:45.754 INFO:teuthology.orchestra.run.smithi026.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:33:45.754 INFO:teuthology.orchestra.run.smithi026.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T03:33:45.754 INFO:teuthology.orchestra.run.smithi026.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T03:33:45.755 INFO:teuthology.orchestra.run.smithi026.stdout:Suggested packages: 2024-09-15T03:33:45.755 INFO:teuthology.orchestra.run.smithi026.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T03:33:45.755 INFO:teuthology.orchestra.run.smithi026.stdout:Recommended packages: 2024-09-15T03:33:45.755 INFO:teuthology.orchestra.run.smithi026.stdout: thermald 2024-09-15T03:33:45.758 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-15T03:33:45.758 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:33:45.758 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-15T03:33:45.758 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-15T03:33:45.759 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-15T03:33:45.760 INFO:teuthology.orchestra.run.smithi122.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-15T03:33:45.760 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-15T03:33:45.760 INFO:teuthology.orchestra.run.smithi122.stdout: thermald 2024-09-15T03:33:45.813 INFO:teuthology.orchestra.run.smithi026.stdout:The following NEW packages will be installed: 2024-09-15T03:33:45.813 INFO:teuthology.orchestra.run.smithi026.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:33:45.813 INFO:teuthology.orchestra.run.smithi026.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T03:33:45.814 INFO:teuthology.orchestra.run.smithi026.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T03:33:45.815 INFO:teuthology.orchestra.run.smithi026.stdout:The following packages will be upgraded: 2024-09-15T03:33:45.815 INFO:teuthology.orchestra.run.smithi026.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T03:33:45.821 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-15T03:33:45.821 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-15T03:33:45.822 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-15T03:33:45.822 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-15T03:33:45.822 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-15T03:33:45.822 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-15T03:33:45.874 INFO:teuthology.orchestra.run.smithi122.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:33:45.874 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 77.2 MB of archives. 2024-09-15T03:33:45.875 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T03:33:45.875 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T03:33:45.998 INFO:teuthology.orchestra.run.smithi026.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:33:45.998 INFO:teuthology.orchestra.run.smithi026.stdout:Need to get 77.2 MB of archives. 2024-09-15T03:33:45.998 INFO:teuthology.orchestra.run.smithi026.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-15T03:33:45.998 INFO:teuthology.orchestra.run.smithi026.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-15T03:33:46.151 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T03:33:46.243 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T03:33:46.723 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T03:33:46.724 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T03:33:46.724 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T03:33:46.817 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T03:33:46.827 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T03:33:47.149 INFO:teuthology.orchestra.run.smithi026.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-15T03:33:47.422 INFO:teuthology.orchestra.run.smithi026.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-15T03:33:47.552 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 77.2 MB in 1s (77.2 MB/s) 2024-09-15T03:33:47.734 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T03:33:48.743 INFO:teuthology.orchestra.run.smithi026.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-15T03:33:48.743 INFO:teuthology.orchestra.run.smithi026.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-15T03:33:48.743 INFO:teuthology.orchestra.run.smithi026.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-15T03:33:49.019 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-15T03:33:49.081 INFO:teuthology.orchestra.run.smithi026.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-15T03:33:49.465 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T03:33:49.469 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:33:49.558 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:33:49.825 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 77.2 MB in 3s (23.7 MB/s) 2024-09-15T03:33:50.006 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-15T03:33:51.699 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T03:33:51.713 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:33:51.800 INFO:teuthology.orchestra.run.smithi026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-15T03:33:51.805 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:33:51.806 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:33:51.886 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:33:52.868 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T03:33:52.883 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:33:52.926 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:33:53.728 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-15T03:33:53.744 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:33:53.840 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:33:54.990 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-15T03:33:55.004 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:33:55.048 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:33:57.588 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:33:57.739 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:33:58.049 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:33:58.176 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:33:58.520 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T03:33:58.534 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T03:33:58.578 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:33:59.762 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:33:59.888 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:34:00.232 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:34:00.358 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:34:00.686 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-15T03:34:00.704 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-15T03:34:00.752 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:34:03.999 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T03:34:04.022 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:34:04.065 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:06.045 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-15T03:34:06.068 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-15T03:34:06.112 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:06.773 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:34:06.901 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:34:07.235 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:08.483 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:08.888 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-15T03:34:09.010 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-15T03:34:09.334 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:09.626 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:34:09.627 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:09.769 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:10.582 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:10.945 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:34:11.105 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T03:34:11.240 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:11.368 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T03:34:11.494 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T03:34:11.620 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:11.688 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T03:34:11.688 INFO:teuthology.orchestra.run.smithi122.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:11.786 INFO:teuthology.orchestra.run.smithi026.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:34:11.786 INFO:teuthology.orchestra.run.smithi026.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:11.926 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:13.161 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-15T03:34:13.287 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-15T03:34:13.413 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:13.539 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-15T03:34:13.665 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-15T03:34:13.791 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-15T03:34:13.866 INFO:teuthology.orchestra.run.smithi026.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-15T03:34:13.867 INFO:teuthology.orchestra.run.smithi026.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:34.750 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T03:34:34.750 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub' 2024-09-15T03:34:34.770 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:34:34.791 INFO:teuthology.orchestra.run.smithi122.stdout:Generating grub configuration file ... 2024-09-15T03:34:35.084 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:34:35.103 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:35.462 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:34:35.474 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:34:36.190 INFO:teuthology.orchestra.run.smithi026.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-15T03:34:36.191 INFO:teuthology.orchestra.run.smithi026.stdout:Sourcing file `/etc/default/grub' 2024-09-15T03:34:36.209 INFO:teuthology.orchestra.run.smithi026.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:34:36.266 INFO:teuthology.orchestra.run.smithi026.stdout:Generating grub configuration file ... 2024-09-15T03:34:36.644 INFO:teuthology.orchestra.run.smithi026.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:34:36.674 INFO:teuthology.orchestra.run.smithi026.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:37.108 INFO:teuthology.orchestra.run.smithi026.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:34:37.121 INFO:teuthology.orchestra.run.smithi026.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:34:37.486 INFO:teuthology.orchestra.run.smithi122.stdout:done 2024-09-15T03:34:37.892 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 20 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.4.0.195.193 2024-09-15T03:34:37.939 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:34:37.940 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:34:37.940 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-15T03:34:37.940 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-15T03:34:37.940 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:34:37.940 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-15T03:34:37.940 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T03:34:37.940 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T03:34:37.940 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T03:34:37.940 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-09-15T03:34:37.941 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T03:34:37.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-15T03:34:38.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-15T03:34:38.145 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:34:38.153 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:34:38.170 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:34:38.264 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:34:39.819 INFO:teuthology.orchestra.run.smithi026.stdout:done 2024-09-15T03:34:39.991 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-15T03:34:40.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:34:40.090 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-15T03:34:40.236 DEBUG:teuthology.orchestra.run.smithi026:> dpkg -s linux-image-generic 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Package: linux-image-generic 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Status: install ok installed 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Priority: optional 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Section: kernel 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Installed-Size: 20 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: amd64 2024-09-15T03:34:40.283 INFO:teuthology.orchestra.run.smithi026.stdout:Source: linux-meta 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout:Version: 5.4.0.195.193 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout:Recommends: thermald 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout:Description: Generic Linux kernel image 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:34:40.284 INFO:teuthology.orchestra.run.smithi026.stdout: available. 2024-09-15T03:34:40.284 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-15T03:34:40.284 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T03:34:40.284 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T03:34:40.285 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T03:34:40.285 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T03:34:40.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-get clean 2024-09-15T03:34:40.285 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-15T03:34:40.285 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-15T03:34:40.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-get update 2024-09-15T03:34:40.501 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:34:40.515 INFO:teuthology.orchestra.run.smithi122.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T03:34:40.515 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:34:40.517 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-15T03:34:40.535 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-15T03:34:40.535 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-15T03:34:40.535 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-15T03:34:40.535 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 20 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.4.0.195.193 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:34:40.536 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-15T03:34:40.537 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T03:34:40.543 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.08DR5tOOea 2024-09-15T03:34:40.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub/grub.cfg /tmp/tmp.08DR5tOOea 2024-09-15T03:34:40.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.08DR5tOOea 2024-09-15T03:34:40.619 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:34:40.638 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.08DR5tOOea is 9KB 2024-09-15T03:34:40.654 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.08DR5tOOea 2024-09-15T03:34:40.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T03:34:40.701 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:34:40.717 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:34:40.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T03:34:40.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T03:34:40.753 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T03:34:40.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo update-grub 2024-09-15T03:34:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:34:42.161 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-09-15T03:34:42.167 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:34:42.181 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-15T03:34:42.426 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2024-09-15T03:34:42.439 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:34:42.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-15T03:34:42.453 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:42.527 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2024-09-15T03:34:42.728 INFO:teuthology.orchestra.run.smithi026.stdout:Building dependency tree... 2024-09-15T03:34:42.729 INFO:teuthology.orchestra.run.smithi026.stdout:Reading state information... 2024-09-15T03:34:42.795 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:34:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:34:42.968 INFO:teuthology.orchestra.run.smithi026.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-15T03:34:42.969 INFO:teuthology.orchestra.run.smithi026.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-15T03:34:42.970 DEBUG:teuthology.orchestra.run.smithi026:> dpkg -s linux-image-generic 2024-09-15T03:34:42.989 INFO:teuthology.orchestra.run.smithi026.stdout:Package: linux-image-generic 2024-09-15T03:34:42.989 INFO:teuthology.orchestra.run.smithi026.stdout:Status: install ok installed 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Priority: optional 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Section: kernel 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Installed-Size: 20 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Maintainer: Ubuntu Kernel Team 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: amd64 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Source: linux-meta 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Version: 5.4.0.195.193 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Recommends: thermald 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout:Description: Generic Linux kernel image 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: This package will always depend on the latest generic kernel image 2024-09-15T03:34:42.990 INFO:teuthology.orchestra.run.smithi026.stdout: available. 2024-09-15T03:34:42.991 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-15T03:34:43.037 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.Gi7elkuW5L 2024-09-15T03:34:43.038 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Gi7elkuW5L 2024-09-15T03:34:43.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.Gi7elkuW5L 2024-09-15T03:34:43.108 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.Gi7elkuW5L is 9KB 2024-09-15T03:34:43.123 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.Gi7elkuW5L 2024-09-15T03:34:43.129 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-15T03:34:43.186 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:34:43.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-15T03:34:43.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-15T03:34:43.220 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-15T03:34:43.220 DEBUG:teuthology.orchestra.run.smithi026:> sudo update-grub 2024-09-15T03:34:43.759 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-15T03:34:43.760 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-15T03:34:44.466 INFO:teuthology.orchestra.run.smithi026.stderr:Sourcing file `/etc/default/grub' 2024-09-15T03:34:44.474 INFO:teuthology.orchestra.run.smithi026.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-15T03:34:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-09-15T03:34:44.826 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-15T03:34:44.838 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-15T03:34:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-15T03:34:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-15T03:34:46.935 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-09-15T03:34:46.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-09-15T03:35:13.793 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-09-15T03:35:13.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:35:13.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:35:16.942 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T03:35:16.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-15T03:35:16.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:35:32.161 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-15T03:35:35.489 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-09-15T03:35:41.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:35:41.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:35:44.221 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-15T03:35:44.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-15T03:35:44.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:35:56.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:35:56.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:36:14.653 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-15T03:36:15.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-09-15T03:36:27.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-15T03:36:27.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:36:28.191 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-15T03:36:28.806 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-15T03:36:28.806 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-15T03:36:28.806 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-09-15T03:36:28.814 INFO:teuthology.orchestra.run.smithi026.stdout:5.4.0-195-generic 2024-09-15T03:36:28.815 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T03:36:28.815 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T03:36:28.815 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T03:36:29.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:36:29.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:36:29.815 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T03:36:29.816 DEBUG:teuthology.orchestra.run.smithi026:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T03:36:29.900 INFO:teuthology.orchestra.run.smithi026.stdout:ttyS1 2024-09-15T03:36:29.921 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:29.924 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-15T03:36:30.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:36:30.499 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.4.0-195-generic"... 2024-09-15T03:36:30.499 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-15T03:36:30.548 INFO:teuthology.orchestra.run.smithi122.stdout:5.4.0-195-generic 2024-09-15T03:36:30.549 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-15T03:36:30.549 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T03:36:30.549 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-15T03:36:31.550 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-09-15T03:36:31.550 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T03:36:31.642 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-09-15T03:36:31.665 DEBUG:teuthology.parallel:result is None 2024-09-15T03:36:31.665 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T03:36:31.673 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T03:36:31.674 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T03:36:31.676 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T03:36:31.683 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T03:36:31.689 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T03:36:31.697 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T03:36:31.698 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T03:36:31.700 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T03:36:31.760 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T03:36:31.766 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T03:36:31.767 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T03:36:31.769 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T03:36:31.784 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:36:31.786 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:36:31.795 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:36:31.797 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T03:36:31.799 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T03:36:31.805 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T03:36:31.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T03:36:31.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T03:36:31.863 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T03:36:31.871 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T03:36:31.872 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T03:36:31.874 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T03:36:31.880 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T03:36:31.886 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T03:36:31.933 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:36:31.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T03:36:31.969 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T03:36:31.976 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T03:36:32.029 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:36:32.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T03:36:32.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-09-15T03:36:32.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-15T03:36:32.101 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T03:36:32.109 INFO:teuthology.task.internal:Starting timer... 2024-09-15T03:36:32.109 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T03:36:32.119 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T03:36:32.127 DEBUG:teuthology.task.selinux:Excluding smithi026: OS 'ubuntu' does not support SELinux 2024-09-15T03:36:32.127 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-09-15T03:36:32.127 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T03:36:32.127 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-15T03:36:32.127 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T03:36:32.127 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T03:36:32.135 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T03:36:32.149 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T03:36:32.149 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T03:42:14.724 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-09-15T03:42:14.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-15T03:42:14.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:42:14.817 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-09-15T03:42:14.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-09-15T03:42:14.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:42:14.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T03:42:15.030 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-15T03:42:15.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T03:42:15.154 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T03:42:15.163 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T03:42:15.164 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T03:42:15.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T03:42:15.166 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T03:42:15.167 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T03:42:15.198 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T03:42:15.198 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Command line: ntpd -gq 2024-09-15T03:42:15.199 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: proto: precision = 0.067 usec (-24) 2024-09-15T03:42:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:42:15.199 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:42:15.200 INFO:teuthology.orchestra.run.smithi026.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listen normally on 3 ens1f0 172.21.15.26:123 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listen normally on 4 lo [::1]:123 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:cac0%3]:123 2024-09-15T03:42:15.201 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:15 ntpd[13616]: Listening on routing socket on fd #22 for interface updates 2024-09-15T03:42:15.202 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-15T03:42:15.202 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Command line: ntpd -gq 2024-09-15T03:42:15.202 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: proto: precision = 0.145 usec (-23) 2024-09-15T03:42:15.202 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:42:15.203 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-15T03:42:15.203 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: restrict ::: KOD does nothing without LIMITED. 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listen and drop on 0 v6wildcard [::]:123 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listen normally on 2 lo 127.0.0.1:123 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listen normally on 4 lo [::1]:123 2024-09-15T03:42:15.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-09-15T03:42:15.205 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:15 ntpd[13706]: Listening on routing socket on fd #22 for interface updates 2024-09-15T03:42:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:16 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:16 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:16 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:16.200 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:16 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:16 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:16.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:16 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:17.198 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:17 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:17 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:18 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:18 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:18 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:18 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:18 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:18 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:18.201 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:18 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:20 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.211 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.212 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.212 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.212 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.212 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.212 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.212 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.213 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.213 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.213 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:20.214 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:20.214 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.214 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.214 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:20.214 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:20.214 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:20.215 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:20.215 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:20.215 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:20 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:22.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:22 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.203 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stdout:15 Sep 03:42:22 ntpd[13706]: ntpd: time slew +0.000996 s 2024-09-15T03:42:22.204 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew +0.000996s 2024-09-15T03:42:22.205 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T03:42:22.205 INFO:teuthology.orchestra.run.smithi122.stderr:15 Sep 03:42:22 ntpd[13706]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T03:42:22.282 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T03:42:22.282 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-15T03:42:22.282 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:22.282 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:22.282 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:22.282 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:24.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:24.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:24.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:24.199 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: can't open /var/log/ntpstats/rawstats.20240915: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: can't open /var/log/ntpstats/peerstats.20240915: Permission denied 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stdout:15 Sep 03:42:24 ntpd[13616]: ntpd: time slew +0.001647 s 2024-09-15T03:42:24.200 INFO:teuthology.orchestra.run.smithi026.stdout:ntpd: time slew +0.001647s 2024-09-15T03:42:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-15T03:42:24.201 INFO:teuthology.orchestra.run.smithi026.stderr:15 Sep 03:42:24 ntpd[13616]: can't open /var/log/ntpstats/loopstats.20240915: Permission denied 2024-09-15T03:42:24.268 INFO:teuthology.orchestra.run.smithi026.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T03:42:24.269 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================== 2024-09-15T03:42:24.269 INFO:teuthology.orchestra.run.smithi026.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:24.269 INFO:teuthology.orchestra.run.smithi026.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:24.269 INFO:teuthology.orchestra.run.smithi026.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:24.269 INFO:teuthology.orchestra.run.smithi026.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-15T03:42:24.269 INFO:teuthology.run_tasks:Running task install... 2024-09-15T03:42:24.278 DEBUG:teuthology.task.install:project ceph 2024-09-15T03:42:24.278 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T03:42:24.279 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-15T03:42:24.279 INFO:teuthology.task.install:Using flavor: default 2024-09-15T03:42:24.285 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T03:42:24.285 INFO:teuthology.task.install:extra packages: [] 2024-09-15T03:42:24.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-key list | grep Ceph 2024-09-15T03:42:24.286 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-09-15T03:42:24.389 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T03:42:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-15T03:42:24.450 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T03:42:24.450 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T03:42:24.451 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T03:42:24.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:42:24.456 INFO:teuthology.orchestra.run.smithi026.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-15T03:42:24.456 INFO:teuthology.orchestra.run.smithi026.stdout:uid [ unknown] Ceph.com (release key) 2024-09-15T03:42:24.457 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-15T03:42:24.457 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:42:24.601 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T03:42:24.601 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:42:24.611 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default/ 2024-09-15T03:42:24.612 INFO:teuthology.task.install.deb:Package version is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:42:24.715 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:42:24.716 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T03:42:24.724 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:42:24.724 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-15T03:42:24.733 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-15T03:42:24.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-get update 2024-09-15T03:42:24.920 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:42:24.974 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T03:42:24.986 INFO:teuthology.orchestra.run.smithi026.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal InRelease 2024-09-15T03:42:24.995 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T03:42:25.008 INFO:teuthology.orchestra.run.smithi026.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-15T03:42:25.019 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T03:42:25.031 INFO:teuthology.orchestra.run.smithi026.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal Release.gpg 2024-09-15T03:42:25.037 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:42:25.040 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T03:42:25.060 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-15T03:42:25.060 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-15T03:42:25.060 INFO:teuthology.orchestra.run.smithi026.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T03:42:25.077 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main i386 Packages [3,392 B] 2024-09-15T03:42:25.094 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-15T03:42:25.119 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:42:25.130 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-15T03:42:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:42:25.211 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-15T03:42:25.568 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 28.5 kB in 1s (45.2 kB/s) 2024-09-15T03:42:25.751 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 28.5 kB in 1s (36.0 kB/s) 2024-09-15T03:42:26.728 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-15T03:42:26.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T03:42:26.823 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-15T03:42:26.956 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2024-09-15T03:42:26.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1643-gd75f2382-1focal cephadm=17.2.7-1643-gd75f2382-1focal ceph-mds=17.2.7-1643-gd75f2382-1focal ceph-mgr=17.2.7-1643-gd75f2382-1focal ceph-common=17.2.7-1643-gd75f2382-1focal ceph-fuse=17.2.7-1643-gd75f2382-1focal ceph-test=17.2.7-1643-gd75f2382-1focal ceph-volume=17.2.7-1643-gd75f2382-1focal radosgw=17.2.7-1643-gd75f2382-1focal python3-rados=17.2.7-1643-gd75f2382-1focal python3-rgw=17.2.7-1643-gd75f2382-1focal python3-cephfs=17.2.7-1643-gd75f2382-1focal python3-rbd=17.2.7-1643-gd75f2382-1focal libcephfs2=17.2.7-1643-gd75f2382-1focal libcephfs-dev=17.2.7-1643-gd75f2382-1focal librados2=17.2.7-1643-gd75f2382-1focal librbd1=17.2.7-1643-gd75f2382-1focal rbd-fuse=17.2.7-1643-gd75f2382-1focal 2024-09-15T03:42:27.042 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-15T03:42:27.042 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-15T03:42:27.054 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2024-09-15T03:42:27.247 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T03:42:27.248 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T03:42:27.248 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T03:42:27.249 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-15T03:42:27.249 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T03:42:27.249 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T03:42:27.249 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T03:42:27.249 INFO:teuthology.orchestra.run.smithi122.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T03:42:27.249 INFO:teuthology.orchestra.run.smithi122.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T03:42:27.251 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: socat xmlstarlet 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T03:42:27.252 INFO:teuthology.orchestra.run.smithi122.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-15T03:42:27.253 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-09-15T03:42:27.278 INFO:teuthology.orchestra.run.smithi026.stdout:Building dependency tree... 2024-09-15T03:42:27.278 INFO:teuthology.orchestra.run.smithi026.stdout:Reading state information... 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T03:42:27.374 INFO:teuthology.orchestra.run.smithi122.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T03:42:27.377 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-15T03:42:27.378 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 librbd1 2024-09-15T03:42:27.490 INFO:teuthology.orchestra.run.smithi026.stdout:The following packages were automatically installed and are no longer required: 2024-09-15T03:42:27.491 INFO:teuthology.orchestra.run.smithi026.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-15T03:42:27.491 INFO:teuthology.orchestra.run.smithi026.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-15T03:42:27.491 INFO:teuthology.orchestra.run.smithi026.stdout:The following additional packages will be installed: 2024-09-15T03:42:27.491 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-15T03:42:27.491 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-15T03:42:27.492 INFO:teuthology.orchestra.run.smithi026.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-15T03:42:27.492 INFO:teuthology.orchestra.run.smithi026.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-15T03:42:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: socat xmlstarlet 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout:Suggested packages: 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-15T03:42:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout:Recommended packages: 2024-09-15T03:42:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: btrfs-tools 2024-09-15T03:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T03:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 201 MB of archives. 2024-09-15T03:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T03:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T03:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T03:42:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:The following NEW packages will be installed: 2024-09-15T03:42:27.618 INFO:teuthology.orchestra.run.smithi026.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-15T03:42:27.618 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-15T03:42:27.618 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-15T03:42:27.618 INFO:teuthology.orchestra.run.smithi026.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-15T03:42:27.618 INFO:teuthology.orchestra.run.smithi026.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-15T03:42:27.619 INFO:teuthology.orchestra.run.smithi026.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-15T03:42:27.619 INFO:teuthology.orchestra.run.smithi026.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-15T03:42:27.620 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-15T03:42:27.621 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-15T03:42:27.621 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-15T03:42:27.621 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-15T03:42:27.621 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-15T03:42:27.621 INFO:teuthology.orchestra.run.smithi026.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-15T03:42:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:The following packages will be upgraded: 2024-09-15T03:42:27.622 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 librbd1 2024-09-15T03:42:27.717 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T03:42:27.767 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T03:42:27.779 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T03:42:27.786 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T03:42:27.786 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T03:42:27.789 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T03:42:27.790 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T03:42:27.797 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T03:42:27.815 INFO:teuthology.orchestra.run.smithi026.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-15T03:42:27.815 INFO:teuthology.orchestra.run.smithi026.stdout:Need to get 201 MB of archives. 2024-09-15T03:42:27.815 INFO:teuthology.orchestra.run.smithi026.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-15T03:42:27.815 INFO:teuthology.orchestra.run.smithi026.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-15T03:42:27.844 INFO:teuthology.orchestra.run.smithi026.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1643-gd75f2382-1focal [3,508 kB] 2024-09-15T03:42:27.890 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T03:42:27.892 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T03:42:27.901 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T03:42:27.967 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T03:42:27.974 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T03:42:28.017 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T03:42:28.030 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T03:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T03:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T03:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T03:42:28.213 INFO:teuthology.orchestra.run.smithi026.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-15T03:42:28.221 INFO:teuthology.orchestra.run.smithi026.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-15T03:42:28.265 INFO:teuthology.orchestra.run.smithi026.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-15T03:42:28.278 INFO:teuthology.orchestra.run.smithi026.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-15T03:42:28.331 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T03:42:28.339 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T03:42:28.403 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-15T03:42:28.409 INFO:teuthology.orchestra.run.smithi026.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-15T03:42:28.431 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T03:42:28.438 INFO:teuthology.orchestra.run.smithi026.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-15T03:42:28.508 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T03:42:28.508 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T03:42:28.508 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T03:42:28.513 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T03:42:28.514 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T03:42:28.516 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T03:42:28.516 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T03:42:28.520 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T03:42:28.521 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T03:42:28.553 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T03:42:28.555 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T03:42:28.556 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T03:42:28.575 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T03:42:28.578 INFO:teuthology.orchestra.run.smithi026.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-15T03:42:28.580 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T03:42:28.586 INFO:teuthology.orchestra.run.smithi026.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-15T03:42:28.603 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T03:42:28.604 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T03:42:28.605 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T03:42:28.606 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T03:42:28.606 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T03:42:28.609 INFO:teuthology.orchestra.run.smithi026.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-15T03:42:28.609 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T03:42:28.610 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T03:42:28.616 INFO:teuthology.orchestra.run.smithi026.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-15T03:42:28.617 INFO:teuthology.orchestra.run.smithi026.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-15T03:42:28.624 INFO:teuthology.orchestra.run.smithi026.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-15T03:42:28.624 INFO:teuthology.orchestra.run.smithi026.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-15T03:42:28.625 INFO:teuthology.orchestra.run.smithi026.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-15T03:42:28.626 INFO:teuthology.orchestra.run.smithi026.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-15T03:42:28.657 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T03:42:28.659 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T03:42:28.660 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T03:42:28.660 INFO:teuthology.orchestra.run.smithi026.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-15T03:42:28.713 INFO:teuthology.orchestra.run.smithi026.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-15T03:42:28.713 INFO:teuthology.orchestra.run.smithi026.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-15T03:42:28.714 INFO:teuthology.orchestra.run.smithi026.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-15T03:42:28.715 INFO:teuthology.orchestra.run.smithi026.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-15T03:42:28.717 INFO:teuthology.orchestra.run.smithi026.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-15T03:42:28.717 INFO:teuthology.orchestra.run.smithi026.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-15T03:42:28.718 INFO:teuthology.orchestra.run.smithi026.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-15T03:42:28.720 INFO:teuthology.orchestra.run.smithi026.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-15T03:42:28.741 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T03:42:28.742 INFO:teuthology.orchestra.run.smithi026.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-15T03:42:28.746 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T03:42:28.747 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T03:42:28.748 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T03:42:28.749 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T03:42:28.751 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T03:42:28.753 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T03:42:28.819 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T03:42:28.819 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T03:42:28.823 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T03:42:28.823 INFO:teuthology.orchestra.run.smithi026.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-15T03:42:28.825 INFO:teuthology.orchestra.run.smithi026.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-15T03:42:28.826 INFO:teuthology.orchestra.run.smithi026.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-15T03:42:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-15T03:42:28.835 INFO:teuthology.orchestra.run.smithi026.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-15T03:42:28.836 INFO:teuthology.orchestra.run.smithi026.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-15T03:42:28.836 INFO:teuthology.orchestra.run.smithi026.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-15T03:42:28.837 INFO:teuthology.orchestra.run.smithi026.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-15T03:42:28.900 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T03:42:28.902 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T03:42:28.902 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T03:42:28.903 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T03:42:28.904 INFO:teuthology.orchestra.run.smithi026.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-15T03:42:28.907 INFO:teuthology.orchestra.run.smithi026.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-15T03:42:28.986 INFO:teuthology.orchestra.run.smithi026.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-15T03:42:28.988 INFO:teuthology.orchestra.run.smithi026.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-15T03:42:28.991 INFO:teuthology.orchestra.run.smithi026.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-15T03:42:28.993 INFO:teuthology.orchestra.run.smithi026.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-15T03:42:28.993 INFO:teuthology.orchestra.run.smithi026.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-15T03:42:28.995 INFO:teuthology.orchestra.run.smithi026.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-15T03:42:29.143 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T03:42:29.143 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T03:42:29.155 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T03:42:29.172 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T03:42:29.173 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T03:42:29.174 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T03:42:29.260 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T03:42:29.283 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T03:42:29.284 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T03:42:29.313 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T03:42:29.318 INFO:teuthology.orchestra.run.smithi026.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-15T03:42:29.321 INFO:teuthology.orchestra.run.smithi026.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1643-gd75f2382-1focal [3,564 kB] 2024-09-15T03:42:29.352 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T03:42:29.364 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T03:42:29.364 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T03:42:29.365 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T03:42:29.365 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T03:42:29.366 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T03:42:29.366 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T03:42:29.389 INFO:teuthology.orchestra.run.smithi026.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-15T03:42:29.392 INFO:teuthology.orchestra.run.smithi026.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-15T03:42:29.415 INFO:teuthology.orchestra.run.smithi026.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-15T03:42:29.415 INFO:teuthology.orchestra.run.smithi026.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-15T03:42:29.415 INFO:teuthology.orchestra.run.smithi026.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-15T03:42:29.416 INFO:teuthology.orchestra.run.smithi026.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-15T03:42:29.416 INFO:teuthology.orchestra.run.smithi026.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-15T03:42:29.417 INFO:teuthology.orchestra.run.smithi026.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-15T03:42:29.418 INFO:teuthology.orchestra.run.smithi026.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-15T03:42:29.423 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T03:42:29.424 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T03:42:29.453 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T03:42:29.453 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T03:42:29.481 INFO:teuthology.orchestra.run.smithi026.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-15T03:42:29.482 INFO:teuthology.orchestra.run.smithi026.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-15T03:42:29.504 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T03:42:29.512 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T03:42:29.513 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T03:42:29.516 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T03:42:29.518 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T03:42:29.521 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T03:42:29.523 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T03:42:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-15T03:42:29.566 INFO:teuthology.orchestra.run.smithi026.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-15T03:42:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-15T03:42:29.570 INFO:teuthology.orchestra.run.smithi026.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-15T03:42:29.571 INFO:teuthology.orchestra.run.smithi026.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-15T03:42:29.574 INFO:teuthology.orchestra.run.smithi026.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-15T03:42:29.576 INFO:teuthology.orchestra.run.smithi026.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-15T03:42:29.604 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T03:42:29.609 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T03:42:29.651 INFO:teuthology.orchestra.run.smithi026.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-15T03:42:29.656 INFO:teuthology.orchestra.run.smithi026.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-15T03:42:29.765 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T03:42:29.767 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T03:42:29.767 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T03:42:29.768 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T03:42:29.768 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T03:42:29.769 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T03:42:29.819 INFO:teuthology.orchestra.run.smithi026.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-15T03:42:29.821 INFO:teuthology.orchestra.run.smithi026.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-15T03:42:29.821 INFO:teuthology.orchestra.run.smithi026.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-15T03:42:29.821 INFO:teuthology.orchestra.run.smithi026.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-15T03:42:29.822 INFO:teuthology.orchestra.run.smithi026.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-15T03:42:29.822 INFO:teuthology.orchestra.run.smithi026.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-15T03:42:29.832 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T03:42:29.833 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T03:42:29.833 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T03:42:29.834 INFO:teuthology.orchestra.run.smithi026.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-15T03:42:29.835 INFO:teuthology.orchestra.run.smithi026.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-15T03:42:29.835 INFO:teuthology.orchestra.run.smithi026.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-15T03:42:29.847 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T03:42:29.905 INFO:teuthology.orchestra.run.smithi026.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-15T03:42:29.928 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T03:42:29.928 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T03:42:29.929 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T03:42:29.933 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T03:42:29.934 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T03:42:29.986 INFO:teuthology.orchestra.run.smithi026.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-15T03:42:29.986 INFO:teuthology.orchestra.run.smithi026.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-15T03:42:29.987 INFO:teuthology.orchestra.run.smithi026.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1643-gd75f2382-1focal [761 kB] 2024-09-15T03:42:29.987 INFO:teuthology.orchestra.run.smithi026.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-15T03:42:29.989 INFO:teuthology.orchestra.run.smithi026.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-15T03:42:29.990 INFO:teuthology.orchestra.run.smithi026.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-15T03:42:30.099 INFO:teuthology.orchestra.run.smithi026.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1643-gd75f2382-1focal [414 kB] 2024-09-15T03:42:30.157 INFO:teuthology.orchestra.run.smithi026.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1643-gd75f2382-1focal [22.6 kB] 2024-09-15T03:42:30.158 INFO:teuthology.orchestra.run.smithi026.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1643-gd75f2382-1focal [200 kB] 2024-09-15T03:42:30.163 INFO:teuthology.orchestra.run.smithi026.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1643-gd75f2382-1focal [49.1 kB] 2024-09-15T03:42:30.164 INFO:teuthology.orchestra.run.smithi026.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1643-gd75f2382-1focal [389 kB] 2024-09-15T03:42:30.220 INFO:teuthology.orchestra.run.smithi026.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1643-gd75f2382-1focal [5,637 kB] 2024-09-15T03:42:30.600 INFO:teuthology.orchestra.run.smithi026.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1643-gd75f2382-1focal [116 kB] 2024-09-15T03:42:30.601 INFO:teuthology.orchestra.run.smithi026.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1643-gd75f2382-1focal [479 kB] 2024-09-15T03:42:30.607 INFO:teuthology.orchestra.run.smithi026.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1643-gd75f2382-1focal [27.0 MB] 2024-09-15T03:42:30.727 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T03:42:30.731 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T03:42:30.732 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T03:42:31.141 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T03:42:31.409 INFO:teuthology.orchestra.run.smithi026.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1643-gd75f2382-1focal [5,178 kB] 2024-09-15T03:42:31.452 INFO:teuthology.orchestra.run.smithi026.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1643-gd75f2382-1focal [223 kB] 2024-09-15T03:42:31.454 INFO:teuthology.orchestra.run.smithi026.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1643-gd75f2382-1focal [157 kB] 2024-09-15T03:42:31.455 INFO:teuthology.orchestra.run.smithi026.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1643-gd75f2382-1focal [1,594 kB] 2024-09-15T03:42:31.478 INFO:teuthology.orchestra.run.smithi026.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1643-gd75f2382-1focal [4,684 kB] 2024-09-15T03:42:31.556 INFO:teuthology.orchestra.run.smithi026.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1643-gd75f2382-1focal [23.1 MB] 2024-09-15T03:42:31.781 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 201 MB in 4s (53.5 MB/s) 2024-09-15T03:42:31.932 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T03:42:31.989 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T03:42:31.993 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T03:42:32.009 INFO:teuthology.orchestra.run.smithi026.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1643-gd75f2382-1focal [3,940 B] 2024-09-15T03:42:32.010 INFO:teuthology.orchestra.run.smithi026.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1643-gd75f2382-1focal [871 kB] 2024-09-15T03:42:32.024 INFO:teuthology.orchestra.run.smithi026.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1643-gd75f2382-1focal [2,418 kB] 2024-09-15T03:42:32.042 INFO:teuthology.orchestra.run.smithi026.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1643-gd75f2382-1focal [81.4 kB] 2024-09-15T03:42:32.043 INFO:teuthology.orchestra.run.smithi026.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1643-gd75f2382-1focal [120 kB] 2024-09-15T03:42:32.044 INFO:teuthology.orchestra.run.smithi026.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1643-gd75f2382-1focal [6,653 kB] 2024-09-15T03:42:32.072 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:42:32.133 INFO:teuthology.orchestra.run.smithi026.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1643-gd75f2382-1focal [7,885 kB] 2024-09-15T03:42:32.263 INFO:teuthology.orchestra.run.smithi026.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1643-gd75f2382-1focal [4,064 B] 2024-09-15T03:42:32.263 INFO:teuthology.orchestra.run.smithi026.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1643-gd75f2382-1focal [53.4 MB] 2024-09-15T03:42:32.434 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:32.567 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:42:33.279 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:33.362 INFO:teuthology.orchestra.run.smithi026.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1643-gd75f2382-1focal [231 kB] 2024-09-15T03:42:33.366 INFO:teuthology.orchestra.run.smithi026.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1643-gd75f2382-1focal [21.9 kB] 2024-09-15T03:42:33.366 INFO:teuthology.orchestra.run.smithi026.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1643-gd75f2382-1focal [17.2 MB] 2024-09-15T03:42:33.380 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:42:33.678 INFO:teuthology.orchestra.run.smithi026.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1643-gd75f2382-1focal [80.7 kB] 2024-09-15T03:42:34.084 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T03:42:34.101 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:34.142 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:34.347 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 201 MB in 6s (33.2 MB/s) 2024-09-15T03:42:34.518 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-15T03:42:34.520 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-09-15T03:42:34.539 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:34.575 INFO:teuthology.orchestra.run.smithi026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-15T03:42:34.578 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:34.579 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-15T03:42:34.692 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:42:34.948 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T03:42:34.964 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:42:35.006 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:35.055 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../001-librbd1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:35.162 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librbd1 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:42:35.283 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T03:42:35.299 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:35.342 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:35.720 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T03:42:35.735 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:42:35.778 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:35.883 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../002-librados2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:36.008 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librados2 (17.2.7-1643-gd75f2382-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-15T03:42:36.106 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T03:42:36.126 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T03:42:36.156 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T03:42:36.358 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T03:42:36.376 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:36.416 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:36.654 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libcephfs2. 2024-09-15T03:42:36.670 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:36.704 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T03:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T03:42:36.861 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:42:37.073 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rados. 2024-09-15T03:42:37.090 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:37.131 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:37.247 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T03:42:37.263 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T03:42:37.306 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:42:37.493 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-15T03:42:37.513 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:42:37.551 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:37.584 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T03:42:37.602 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T03:42:37.633 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:42:37.803 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cephfs. 2024-09-15T03:42:37.819 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:37.853 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:38.037 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T03:42:38.053 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T03:42:38.095 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:42:38.189 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-15T03:42:38.205 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:42:38.247 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:38.456 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T03:42:38.473 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T03:42:38.514 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:42:38.567 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-prettytable. 2024-09-15T03:42:38.583 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-15T03:42:38.625 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-15T03:42:38.903 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rbd. 2024-09-15T03:42:38.922 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:38.934 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T03:42:38.953 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:42:38.961 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:39.028 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:42:39.264 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-15T03:42:39.282 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-15T03:42:39.347 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:42:39.571 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T03:42:39.587 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:42:39.629 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:42:39.667 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-15T03:42:39.685 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-15T03:42:39.729 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:42:40.016 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T03:42:40.032 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:42:40.074 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:42:40.086 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-15T03:42:40.103 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-15T03:42:40.144 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:42:40.419 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T03:42:40.437 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T03:42:40.469 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:42:40.522 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-15T03:42:40.539 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-15T03:42:40.572 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:42:40.838 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-09-15T03:42:40.857 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:40.858 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-15T03:42:40.878 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-15T03:42:40.896 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:40.917 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:42:41.413 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-15T03:42:41.428 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:42:41.505 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:42:41.733 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T03:42:41.752 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:41.973 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-15T03:42:41.993 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:42:42.032 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0. 2024-09-15T03:42:42.145 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T03:42:42.177 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T03:42:42.393 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-15T03:42:42.409 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-15T03:42:42.451 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:42:42.504 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T03:42:42.524 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:42.563 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:42.829 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-15T03:42:42.847 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-15T03:42:42.912 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:42:43.066 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-09-15T03:42:43.082 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:43.124 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:43.265 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librgw2. 2024-09-15T03:42:43.285 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../019-librgw2_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:43.386 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:44.135 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rgw. 2024-09-15T03:42:44.153 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:44.193 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:44.495 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liboath0. 2024-09-15T03:42:44.513 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-15T03:42:44.545 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-15T03:42:44.873 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libradosstriper1. 2024-09-15T03:42:44.891 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:44.957 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:45.451 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-common. 2024-09-15T03:42:45.468 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:45.509 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:45.649 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-09-15T03:42:45.668 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:45.715 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T03:42:46.610 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T03:42:46.738 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:42:47.047 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T03:42:47.063 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T03:42:47.105 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T03:42:47.457 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T03:42:47.474 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T03:42:47.516 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T03:42:47.794 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T03:42:47.812 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:42:47.852 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:48.090 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-base. 2024-09-15T03:42:48.107 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:48.159 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:48.265 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T03:42:48.282 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T03:42:48.323 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:42:48.651 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T03:42:48.669 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T03:42:48.709 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:42:49.080 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T03:42:49.086 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cryptography. 2024-09-15T03:42:49.095 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T03:42:49.105 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-15T03:42:49.138 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T03:42:49.144 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:42:49.458 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T03:42:49.476 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T03:42:49.514 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-openssl. 2024-09-15T03:42:49.515 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:42:49.533 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-15T03:42:49.572 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-15T03:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T03:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T03:42:49.892 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-packaging. 2024-09-15T03:42:49.910 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-15T03:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T03:42:49.950 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-15T03:42:50.197 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T03:42:50.211 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-15T03:42:50.216 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T03:42:50.228 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:42:50.269 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:50.271 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:42:50.574 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-09-15T03:42:50.593 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T03:42:50.633 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:42:50.674 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-15T03:42:50.693 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-15T03:42:50.732 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:42:50.986 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T03:42:51.002 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T03:42:51.035 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-15T03:42:51.044 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:42:51.052 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-15T03:42:51.094 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:42:51.372 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T03:42:51.387 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T03:42:51.422 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-natsort. 2024-09-15T03:42:51.430 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:42:51.441 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-15T03:42:51.480 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-15T03:42:51.758 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-09-15T03:42:51.774 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T03:42:51.816 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:42:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jwt. 2024-09-15T03:42:51.841 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-15T03:42:51.883 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:42:52.161 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T03:42:52.178 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-logutils. 2024-09-15T03:42:52.180 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T03:42:52.194 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-15T03:42:52.219 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:42:52.236 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-15T03:42:52.555 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-15T03:42:52.564 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T03:42:52.571 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-15T03:42:52.581 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T03:42:52.614 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:42:52.622 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T03:42:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-mako. 2024-09-15T03:42:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-15T03:42:52.990 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T03:42:53.006 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T03:42:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:42:53.052 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:42:53.336 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-15T03:42:53.357 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-15T03:42:53.394 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:42:53.402 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T03:42:53.418 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T03:42:53.485 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T03:42:53.706 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-15T03:42:53.725 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-15T03:42:53.764 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:42:53.838 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-09-15T03:42:53.854 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T03:42:53.896 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:42:54.060 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-webob. 2024-09-15T03:42:54.078 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-15T03:42:54.144 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:42:54.233 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T03:42:54.249 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T03:42:54.291 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:42:54.480 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-15T03:42:54.496 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-15T03:42:54.538 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:42:54.619 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T03:42:54.635 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T03:42:54.677 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:42:54.857 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-bs4. 2024-09-15T03:42:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-15T03:42:54.916 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-15T03:42:54.989 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T03:42:55.004 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T03:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T03:42:55.277 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-waitress. 2024-09-15T03:42:55.294 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-15T03:42:55.364 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:42:55.392 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T03:42:55.407 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T03:42:55.450 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:42:55.689 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-tempita. 2024-09-15T03:42:55.709 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-15T03:42:55.747 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-15T03:42:55.825 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T03:42:55.841 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T03:42:55.883 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:42:56.075 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-paste. 2024-09-15T03:42:56.094 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-15T03:42:56.133 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:42:56.272 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T03:42:56.292 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:56.330 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:56.492 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-15T03:42:56.510 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-15T03:42:56.551 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:42:56.717 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T03:42:56.733 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:56.775 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:56.854 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-15T03:42:56.871 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-15T03:42:56.912 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:42:57.232 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-webtest. 2024-09-15T03:42:57.248 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-15T03:42:57.277 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T03:42:57.289 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-15T03:42:57.296 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:57.361 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:57.567 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pecan. 2024-09-15T03:42:57.586 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-15T03:42:57.626 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:42:57.981 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-15T03:42:58.001 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-15T03:42:58.039 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:42:58.206 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T03:42:58.222 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:58.264 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:58.409 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-15T03:42:58.425 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:58.467 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:42:58.845 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr. 2024-09-15T03:42:58.863 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:42:58.903 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:00.100 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mon. 2024-09-15T03:43:00.119 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:00.159 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:00.642 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-09-15T03:43:00.658 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:00.700 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:01.037 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T03:43:01.039 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-osd. 2024-09-15T03:43:01.053 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:01.057 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:01.095 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:01.097 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:01.557 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T03:43:01.573 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:01.615 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:02.176 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-09-15T03:43:02.193 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:02.234 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:02.587 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T03:43:02.605 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T03:43:02.645 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:43:02.973 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T03:43:02.992 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:03.258 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph. 2024-09-15T03:43:03.275 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../053-ceph_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:03.308 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:03.418 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T03:43:03.434 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T03:43:03.476 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T03:43:03.627 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-fuse. 2024-09-15T03:43:03.643 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:03.685 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:03.729 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T03:43:03.745 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T03:43:03.803 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:43:04.089 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T03:43:04.106 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T03:43:04.113 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mds. 2024-09-15T03:43:04.132 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:04.148 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:43:04.172 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:04.517 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-09-15T03:43:04.536 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T03:43:04.594 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T03:43:04.658 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package cephadm. 2024-09-15T03:43:04.677 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../056-cephadm_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:04.708 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:04.920 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T03:43:04.937 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:04.978 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:05.104 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jinja2. 2024-09-15T03:43:05.120 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-15T03:43:05.162 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:43:05.514 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-15T03:43:05.534 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:05.572 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:05.917 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-15T03:43:05.935 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-15T03:43:05.975 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-15T03:43:06.245 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libjs-underscore. 2024-09-15T03:43:06.263 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-15T03:43:06.295 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:43:06.342 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T03:43:06.358 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T03:43:06.400 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:43:06.556 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-15T03:43:06.573 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-15T03:43:06.647 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:43:06.737 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T03:43:06.753 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T03:43:06.794 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:43:06.950 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-routes. 2024-09-15T03:43:06.969 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-15T03:43:07.008 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-15T03:43:07.149 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T03:43:07.166 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T03:43:07.207 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T03:43:07.369 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-15T03:43:07.387 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:07.427 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:08.474 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T03:43:08.493 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T03:43:08.532 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:08.799 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-decorator. 2024-09-15T03:43:08.816 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-15T03:43:08.857 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:43:09.103 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T03:43:09.119 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T03:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T03:43:09.302 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-15T03:43:09.322 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-15T03:43:09.360 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:43:09.556 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T03:43:09.576 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T03:43:09.639 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:09.705 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-scipy. 2024-09-15T03:43:09.723 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-15T03:43:09.763 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-15T03:43:10.339 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T03:43:10.355 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:10.421 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:11.090 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-15T03:43:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-15T03:43:11.148 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:11.559 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T03:43:11.576 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T03:43:11.609 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T03:43:11.676 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-joblib. 2024-09-15T03:43:11.694 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-15T03:43:11.735 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-15T03:43:11.903 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T03:43:11.921 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T03:43:11.961 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:43:12.138 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-sklearn. 2024-09-15T03:43:12.156 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-15T03:43:12.196 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:12.323 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T03:43:12.339 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T03:43:12.381 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:43:12.734 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T03:43:12.750 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T03:43:12.792 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T03:43:12.880 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-15T03:43:12.896 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:12.938 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:13.120 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T03:43:13.141 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T03:43:13.178 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T03:43:13.499 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T03:43:13.519 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T03:43:13.549 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T03:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T03:43:13.935 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:43:14.057 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cachetools. 2024-09-15T03:43:14.074 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-15T03:43:14.116 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-15T03:43:14.297 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T03:43:14.314 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T03:43:14.397 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:43:14.435 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-15T03:43:14.452 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-15T03:43:14.493 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:43:14.821 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-15T03:43:14.839 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-15T03:43:14.880 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:43:15.075 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T03:43:15.096 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T03:43:15.159 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:43:15.471 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T03:43:15.487 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T03:43:15.545 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:43:15.557 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rsa. 2024-09-15T03:43:15.577 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-15T03:43:15.615 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-15T03:43:15.902 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-google-auth. 2024-09-15T03:43:15.922 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-15T03:43:15.960 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-15T03:43:16.248 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T03:43:16.264 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:16.297 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-blinker. 2024-09-15T03:43:16.306 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:16.313 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-15T03:43:16.355 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:43:16.658 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-15T03:43:16.667 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T03:43:16.675 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-15T03:43:16.684 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T03:43:16.726 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:43:16.732 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:43:17.061 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-15T03:43:17.078 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-15T03:43:17.087 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T03:43:17.108 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:43:17.119 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:43:17.145 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:17.432 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-09-15T03:43:17.448 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:43:17.456 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-websocket. 2024-09-15T03:43:17.472 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-15T03:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:17.514 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:43:17.776 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-09-15T03:43:17.794 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T03:43:17.834 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T03:43:17.851 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-15T03:43:17.870 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-15T03:43:17.917 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:43:18.197 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T03:43:18.214 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T03:43:18.255 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:43:18.575 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-15T03:43:18.593 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:18.614 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-09-15T03:43:18.633 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:18.633 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:18.672 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:18.953 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-15T03:43:18.973 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-15T03:43:19.036 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:43:19.389 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-15T03:43:19.406 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:43:19.447 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:19.736 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package jq. 2024-09-15T03:43:19.755 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-15T03:43:19.795 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:20.032 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package socat. 2024-09-15T03:43:20.049 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-15T03:43:20.089 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-15T03:43:20.418 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package xmlstarlet. 2024-09-15T03:43:20.438 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-15T03:43:20.476 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:43:20.873 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-test. 2024-09-15T03:43:20.889 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:20.931 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:23.310 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T03:43:23.326 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:23.368 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:23.731 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T03:43:23.751 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:43:23.792 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:24.337 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T03:43:24.354 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:24.395 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:24.807 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T03:43:24.825 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T03:43:24.883 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T03:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:25.644 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-volume. 2024-09-15T03:43:25.661 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1643-gd75f2382-1focal_all.deb ... 2024-09-15T03:43:25.702 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:26.065 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-15T03:43:26.082 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:43:26.119 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:26.656 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-15T03:43:26.673 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:26.715 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:26.992 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T03:43:27.014 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T03:43:27.058 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T03:43:27.093 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package nvme-cli. 2024-09-15T03:43:27.109 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-15T03:43:27.170 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:43:27.428 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T03:43:27.446 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T03:43:27.495 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:43:27.533 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-15T03:43:27.551 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-15T03:43:27.591 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:27.823 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-attr. 2024-09-15T03:43:27.842 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T03:43:27.881 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T03:43:28.242 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T03:43:28.259 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T03:43:28.300 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:43:28.628 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T03:43:28.646 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T03:43:28.686 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T03:43:29.064 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T03:43:29.082 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T03:43:29.122 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:43:29.246 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-15T03:43:29.265 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-15T03:43:29.304 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-15T03:43:29.585 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T03:43:29.605 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T03:43:29.640 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-15T03:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T03:43:29.657 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-15T03:43:29.732 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:43:29.987 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T03:43:30.004 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T03:43:30.035 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-attr. 2024-09-15T03:43:30.044 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T03:43:30.053 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-15T03:43:30.093 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-15T03:43:30.365 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T03:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T03:43:30.422 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:43:30.445 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-webencodings. 2024-09-15T03:43:30.464 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-15T03:43:30.504 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-09-15T03:43:30.790 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-html5lib. 2024-09-15T03:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T03:43:30.810 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-15T03:43:30.833 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:43:30.848 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-15T03:43:31.178 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T03:43:31.195 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T03:43:31.218 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-15T03:43:31.236 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-15T03:43:31.236 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:43:31.276 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:43:31.564 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T03:43:31.582 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T03:43:31.622 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:43:31.746 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pastescript. 2024-09-15T03:43:31.765 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-15T03:43:31.804 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-15T03:43:31.950 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T03:43:31.967 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T03:43:32.008 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T03:43:32.165 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pluggy. 2024-09-15T03:43:32.183 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-15T03:43:32.232 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-15T03:43:32.370 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T03:43:32.387 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T03:43:32.428 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:43:32.551 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-psutil. 2024-09-15T03:43:32.569 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-15T03:43:32.612 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:43:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T03:43:32.731 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T03:43:32.772 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:43:32.971 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-py. 2024-09-15T03:43:32.989 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-15T03:43:33.029 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:43:33.382 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-15T03:43:33.400 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-15T03:43:33.440 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:43:33.530 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-09-15T03:43:33.549 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:33.587 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:33.743 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-15T03:43:33.760 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-15T03:43:33.801 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:43:34.129 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pytest. 2024-09-15T03:43:34.150 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-15T03:43:34.187 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-15T03:43:34.549 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-simplejson. 2024-09-15T03:43:34.566 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-15T03:43:34.599 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:43:34.843 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-15T03:43:34.863 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-15T03:43:34.901 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:43:35.307 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T03:43:35.326 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:35.365 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:35.580 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package radosgw. 2024-09-15T03:43:35.600 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../108-radosgw_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:35.663 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:35.738 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T03:43:36.029 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:43:36.155 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:43:36.281 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:43:36.507 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:37.096 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-09-15T03:43:37.258 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:37.384 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:43:37.442 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package rbd-fuse. 2024-09-15T03:43:37.460 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1643-gd75f2382-1focal_amd64.deb ... 2024-09-15T03:43:37.500 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:37.743 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T03:43:37.799 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-15T03:43:38.068 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:38.108 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-15T03:43:38.204 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T03:43:38.208 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-15T03:43:38.309 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-15T03:43:38.331 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:43:38.503 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:38.675 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T03:43:39.085 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:43:39.089 INFO:teuthology.orchestra.run.smithi026.stdout:Adding system user cephadm....done 2024-09-15T03:43:39.276 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:39.377 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-15T03:43:39.378 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T03:43:39.671 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:43:39.737 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-15T03:43:39.972 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:43:40.021 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-15T03:43:40.148 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-15T03:43:40.256 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:40.267 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-15T03:43:40.532 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:40.602 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-15T03:43:40.691 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:43:40.996 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-15T03:43:41.001 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:43:41.277 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:43:41.280 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-15T03:43:41.428 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:43:41.582 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-15T03:43:41.810 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T03:43:41.866 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-15T03:43:42.105 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:43:42.134 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-ceph-argparse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:42.381 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:43:42.410 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:42.529 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-15T03:43:42.724 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:43:42.847 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-15T03:43:43.009 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:43:43.140 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-15T03:43:43.291 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-15T03:43:43.368 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T03:43:43.677 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T03:43:43.684 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-15T03:43:43.960 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-15T03:43:43.970 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:43:44.096 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:43:44.222 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:43:44.228 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-15T03:43:44.515 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:43:44.587 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-15T03:43:44.880 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-15T03:43:44.900 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:43:45.231 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-15T03:43:45.259 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:43:45.549 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-15T03:43:45.636 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T03:43:45.859 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-15T03:43:45.960 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-15T03:43:46.270 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:46.319 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-15T03:43:46.714 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-15T03:43:46.873 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T03:43:47.224 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:43:47.238 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-15T03:43:47.375 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:43:47.501 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T03:43:47.598 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-15T03:43:47.794 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:43:47.975 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-15T03:43:48.076 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:48.120 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:43:48.263 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:43:48.369 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:43:48.389 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:43:48.504 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-15T03:43:48.715 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:43:48.863 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-15T03:43:48.998 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-15T03:43:49.008 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T03:43:49.149 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-15T03:43:49.293 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:43:49.442 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-15T03:43:49.444 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:43:49.720 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T03:43:49.735 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:43:49.836 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-15T03:43:49.954 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-15T03:43:50.046 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:43:50.297 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-15T03:43:50.355 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:43:50.581 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-15T03:43:50.799 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T03:43:50.883 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-15T03:43:51.009 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-15T03:43:51.066 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:43:51.192 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:43:51.276 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-15T03:43:51.501 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:43:51.553 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-15T03:43:51.837 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-15T03:43:51.869 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:43:51.995 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:43:52.130 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:43:52.232 INFO:teuthology.orchestra.run.smithi026.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-15T03:43:52.456 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:43:52.498 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-15T03:43:52.649 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-15T03:43:52.749 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:52.866 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T03:43:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-15T03:43:53.310 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-15T03:43:53.411 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-15T03:43:53.529 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-15T03:43:53.863 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-15T03:43:54.148 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librados2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:54.282 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-15T03:43:54.339 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:54.464 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:54.590 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T03:43:54.959 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T03:43:55.251 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:43:55.527 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:43:55.687 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T03:43:55.829 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:55.955 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:56.055 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:56.081 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-15T03:43:56.181 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:43:56.491 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:56.508 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-15T03:43:56.608 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:43:56.810 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-15T03:43:56.927 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:57.094 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-15T03:43:57.220 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-15T03:43:57.579 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libcephfs2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:57.697 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-15T03:43:57.973 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:58.024 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libradosstriper1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:58.124 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:43:58.150 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-15T03:43:58.250 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T03:43:58.376 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:43:58.518 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-15T03:43:58.527 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T03:43:58.820 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:43:58.971 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T03:43:59.272 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:59.423 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:59.573 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librbd1 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:43:59.646 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:43:59.646 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:43:59.724 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-15T03:43:59.850 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-15T03:43:59.976 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-15T03:44:00.075 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:00.102 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-15T03:44:00.369 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:44:00.411 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-15T03:44:00.537 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-15T03:44:00.697 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T03:44:00.847 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-modules-core (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:00.964 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:01.204 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:44:01.204 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-15T03:44:01.340 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:44:01.675 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libcephfs-dev (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:01.701 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:01.827 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:01.903 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-15T03:44:01.987 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:02.121 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:02.246 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-15T03:44:02.247 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T03:44:02.580 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-15T03:44:02.582 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T03:44:02.892 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:02.941 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rados (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:03.034 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:03.067 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librgw2 (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:03.193 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:44:03.202 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rbd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:03.328 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up rbd-fuse (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:03.462 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-15T03:44:03.747 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-15T03:44:04.056 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:04.114 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:04.176 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cephfs (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:04.302 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-15T03:44:04.683 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-09-15T03:44:05.093 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-09-15T03:44:05.230 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-common (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:05.286 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-09-15T03:44:05.311 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T03:44:05.459 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T03:44:05.795 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T03:44:05.917 INFO:teuthology.orchestra.run.smithi026.stdout:Adding group ceph....done 2024-09-15T03:44:06.248 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:06.301 INFO:teuthology.orchestra.run.smithi026.stdout:Adding system user ceph....done 2024-09-15T03:44:06.374 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:06.469 INFO:teuthology.orchestra.run.smithi026.stdout:Setting system user ceph properties....done 2024-09-15T03:44:06.495 INFO:teuthology.orchestra.run.smithi026.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-15T03:44:06.651 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-15T03:44:06.681 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:44:06.681 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:44:06.990 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-15T03:44:07.335 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:07.457 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-test (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:07.583 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up radosgw (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:07.704 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T03:44:07.860 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:44:07.860 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-15T03:44:08.153 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:08.388 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:44:08.388 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:44:08.552 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-base (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:08.822 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:08.921 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-15T03:44:09.055 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:44:09.055 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:44:09.388 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mds (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:09.490 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:09.608 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:44:09.609 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-15T03:44:09.962 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:44:09.962 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:44:10.056 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:10.310 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:44:10.310 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-15T03:44:10.425 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:10.550 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:10.718 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:10.766 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-osd (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:10.940 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:44:10.940 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:44:11.286 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:44:11.287 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-15T03:44:11.386 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:11.562 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:11.696 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:11.743 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:11.869 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:12.011 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mon (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:12.035 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:12.217 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T03:44:12.245 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:44:12.245 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-15T03:44:12.394 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T03:44:12.688 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-cephadm (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:12.703 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T03:44:12.831 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:12.957 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-dashboard (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:13.125 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-volume (17.2.7-1643-gd75f2382-1focal) ... 2024-09-15T03:44:13.260 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-15T03:44:13.412 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-15T03:44:13.697 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-15T03:44:15.514 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T03:44:16.133 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T03:44:16.139 DEBUG:teuthology.parallel:result is None 2024-09-15T03:44:16.574 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-15T03:44:17.184 INFO:teuthology.orchestra.run.smithi026.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-15T03:44:17.188 DEBUG:teuthology.parallel:result is None 2024-09-15T03:44:17.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:44:17.330 DEBUG:teuthology.orchestra.run.smithi026:> dpkg-query -W -f '${Version}' ceph 2024-09-15T03:44:17.353 INFO:teuthology.orchestra.run.smithi026.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T03:44:17.354 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:44:17.354 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T03:44:17.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-15T03:44:17.497 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-09-15T03:44:17.520 INFO:teuthology.orchestra.run.smithi122.stdout:17.2.7-1643-gd75f2382-1focal 2024-09-15T03:44:17.520 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643-gd75f2382-1focal 2024-09-15T03:44:17.520 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643-gd75f2382-1focal is installed. 2024-09-15T03:44:17.522 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T03:44:17.522 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:17.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T03:44:17.539 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:17.539 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T03:44:17.576 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T03:44:17.576 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:17.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T03:44:17.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T03:44:17.606 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:17.606 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T03:44:17.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T03:44:17.631 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T03:44:17.631 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:17.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T03:44:17.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T03:44:17.697 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:17.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T03:44:17.712 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T03:44:17.722 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T03:44:17.722 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:17.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T03:44:17.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T03:44:17.753 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:17.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T03:44:17.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T03:44:17.825 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T03:44:17.900 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T03:44:17.901 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-09-15T03:44:17.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-15T03:44:17.918 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T03:44:17.918 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T03:44:17.921 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T03:44:17.931 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T03:44:17.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T03:44:17.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T03:44:18.001 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T03:44:18.001 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-15T03:44:18.001 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7905302', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 2126, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'thrash_delay': 1, 'thrash_many': True}}, {'rados': {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_00:54:39', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T03:44:18.002 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T03:44:18.004 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T03:44:18.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T03:44:18.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T03:44:18.071 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:18.071 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T03:44:18.078 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T03:44:18.078 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-09-15T03:44:18.085 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T03:44:18.085 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.085 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-09-15T03:44:18.086 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.086 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-15 03:40:43.068226909 +0000 2024-09-15T03:44:18.086 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-15 03:40:42.840232156 +0000 2024-09-15T03:44:18.086 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-15 03:40:42.840232156 +0000 2024-09-15T03:44:18.086 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-09-15T03:44:18.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T03:44:18.142 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-15T03:44:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-15T03:44:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000306167 s, 1.7 MB/s 2024-09-15T03:44:18.144 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T03:44:18.152 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-09-15T03:44:18.158 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 376 Links: 1 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-15 03:40:43.468217703 +0000 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-15 03:40:43.256222581 +0000 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-15 03:40:43.256222581 +0000 2024-09-15T03:44:18.159 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-09-15T03:44:18.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T03:44:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-15T03:44:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-15T03:44:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000297603 s, 1.7 MB/s 2024-09-15T03:44:18.220 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T03:44:18.228 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 382 Links: 1 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-15 03:40:43.900207763 +0000 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-15 03:40:43.692212549 +0000 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-15 03:40:43.692212549 +0000 2024-09-15T03:44:18.234 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-09-15T03:44:18.235 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T03:44:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-15T03:44:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-15T03:44:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000304787 s, 1.7 MB/s 2024-09-15T03:44:18.291 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T03:44:18.299 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 390 Links: 1 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-09-15 03:40:44.320198097 +0000 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-09-15 03:40:44.092203344 +0000 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-09-15 03:40:44.092203344 +0000 2024-09-15T03:44:18.305 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2024-09-15T03:44:18.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T03:44:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-09-15T03:44:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-09-15T03:44:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000263466 s, 1.9 MB/s 2024-09-15T03:44:18.321 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T03:44:18.372 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:18.372 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:18.372 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T03:44:18.379 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T03:44:18.379 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-15T03:44:18.386 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 368 Links: 1 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 03:40:51.090252376 +0000 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 03:40:50.878265991 +0000 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 03:40:50.878265991 +0000 2024-09-15T03:44:18.387 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-15T03:44:18.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T03:44:18.400 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T03:44:18.400 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T03:44:18.400 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000427123 s, 1.2 MB/s 2024-09-15T03:44:18.401 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T03:44:18.408 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-15T03:44:18.413 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 03:40:51.502226398 +0000 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 03:40:51.290239766 +0000 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 03:40:51.290239766 +0000 2024-09-15T03:44:18.414 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-15T03:44:18.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T03:44:18.468 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T03:44:18.469 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T03:44:18.469 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000393118 s, 1.3 MB/s 2024-09-15T03:44:18.470 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T03:44:18.477 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-15T03:44:18.483 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T03:44:18.483 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.483 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 384 Links: 1 2024-09-15T03:44:18.484 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.484 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 03:40:51.926199666 +0000 2024-09-15T03:44:18.484 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 03:40:51.722212528 +0000 2024-09-15T03:44:18.484 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 03:40:51.722212528 +0000 2024-09-15T03:44:18.484 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-15T03:44:18.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T03:44:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T03:44:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T03:44:18.541 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000410174 s, 1.2 MB/s 2024-09-15T03:44:18.542 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T03:44:18.549 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 6h/6d Inode: 390 Links: 1 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 03:40:52.382170915 +0000 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 03:40:52.158185038 +0000 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 03:40:52.158185038 +0000 2024-09-15T03:44:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-15T03:44:18.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T03:44:18.612 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T03:44:18.613 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T03:44:18.613 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000365038 s, 1.4 MB/s 2024-09-15T03:44:18.614 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T03:44:18.621 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:18.622 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T03:44:18.622 INFO:tasks.ceph:Generating config... 2024-09-15T03:44:18.623 INFO:tasks.ceph:[client] debug monc = 20 2024-09-15T03:44:18.623 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T03:44:18.623 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T03:44:18.623 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-15T03:44:18.623 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-15T03:44:18.623 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T03:44:18.623 INFO:tasks.ceph:[global] ms type = async 2024-09-15T03:44:18.623 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T03:44:18.623 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T03:44:18.623 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T03:44:18.623 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T03:44:18.623 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] paxos min = 10 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-09-15T03:44:18.624 INFO:tasks.ceph:[mon] paxos trim min = 10 2024-09-15T03:44:18.624 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-15T03:44:18.624 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-15T03:44:18.624 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-15T03:44:18.624 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T03:44:18.625 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T03:44:18.626 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T03:44:18.626 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T03:44:18.626 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T03:44:18.626 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T03:44:18.626 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T03:44:18.626 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T03:44:18.626 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T03:44:18.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:18.661 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T03:44:18.663 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T03:44:18.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T03:44:18.756 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.26:6789]'), ('mon.b', '[v1:172.21.15.26:6790]'), ('mon.c', '[v1:172.21.15.26:6791]'), ('mon.d', '[v1:172.21.15.26:6792]'), ('mon.e', '[v1:172.21.15.26:6793]'), ('mon.f', '[v1:172.21.15.122:6789]'), ('mon.g', '[v1:172.21.15.122:6790]'), ('mon.h', '[v1:172.21.15.122:6791]'), ('mon.i', '[v1:172.21.15.122:6792]')] 2024-09-15T03:44:18.757 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.26:6789],[v1:172.21.15.26:6790],[v1:172.21.15.26:6791],[v1:172.21.15.26:6792],[v1:172.21.15.26:6793],[v1:172.21.15.122:6789],[v1:172.21.15.122:6790],[v1:172.21.15.122:6791],[v1:172.21.15.122:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-09-15T03:44:18.757 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:18.757 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T03:44:18.762 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.26:6789]' --addv b '[v1:172.21.15.26:6790]' --addv c '[v1:172.21.15.26:6791]' --addv d '[v1:172.21.15.26:6792]' --addv e '[v1:172.21.15.26:6793]' --addv f '[v1:172.21.15.122:6789]' --addv g '[v1:172.21.15.122:6790]' --addv h '[v1:172.21.15.122:6791]' --addv i '[v1:172.21.15.122:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:18.867 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:18.964 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = octopus 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 15 (octopus) 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 3 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:1: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:18.965 INFO:teuthology.orchestra.run.smithi026.stdout:2: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:3: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:4: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:5: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:6: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:7: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:8: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:18.966 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-09-15T03:44:18.967 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T03:44:18.975 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4d3becf5-10bb-4b2a-818b-afafac96b25a... 2024-09-15T03:44:18.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T03:44:18.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-09-15T03:44:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T03:44:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "[v1:172.21.15.26:6789],[v1:172.21.15.26:6790],[v1:172.21.15.26:6791],[v1:172.21.15.26:6792],[v1:172.21.15.26:6793],[v1:172.21.15.122:6789],[v1:172.21.15.122:6790],[v1:172.21.15.122:6791],[v1:172.21.15.122:6792]" 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: mon client directed command retry = 5 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: mon election default strategy = 3 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: ms bind msgr2 = False 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: ms inject socket failures = 5000 2024-09-15T03:44:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: ms type = async 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-09-15T03:44:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: bdev async discard = True 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: bdev enable discard = True 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = stupid 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore zero block detection = True 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 20 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 20 2024-09-15T03:44:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 10 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify cached snaps = True 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify missing on start = True 2024-09-15T03:44:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock override recovery settings = True 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: osd op queue = debug_random 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: osd op queue cut off = debug_random 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-15T03:44:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-09-15T03:44:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-09-15T03:44:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T03:44:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: mon min osdmap epochs = 25 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: mon osdmap full prune interval = 2 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: mon osdmap full prune min = 15 2024-09-15T03:44:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: mon osdmap full prune txsize = 2 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: mon scrub inject crc mismatch = 0.01 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: mon scrub inject missing keys = 0.05 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: mon scrub interval = 300 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: paxos min = 10 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: paxos service trim min = 5 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: paxos trim min = 10 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:19.030 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: debug monc = 20 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-09-15T03:44:19.031 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.d] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.e] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.f] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.g] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.h] 2024-09-15T03:44:19.032 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.i] 2024-09-15T03:44:19.034 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-15T03:44:19.035 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.036 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T03:44:19.037 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "[v1:172.21.15.26:6789],[v1:172.21.15.26:6790],[v1:172.21.15.26:6791],[v1:172.21.15.26:6792],[v1:172.21.15.26:6793],[v1:172.21.15.122:6789],[v1:172.21.15.122:6790],[v1:172.21.15.122:6791],[v1:172.21.15.122:6792]" 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: mon client directed command retry = 5 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: mon election default strategy = 3 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: ms bind msgr2 = False 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject socket failures = 5000 2024-09-15T03:44:19.038 INFO:teuthology.orchestra.run.smithi122.stdout: ms type = async 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-15T03:44:19.039 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-15T03:44:19.040 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bdev async discard = True 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bdev enable discard = True 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore allocator = stupid 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore zero block detection = True 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 20 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 20 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-15T03:44:19.041 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 10 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify cached snaps = True 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify missing on start = True 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock override recovery settings = True 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd op queue = debug_random 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: osd op queue cut off = debug_random 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-15T03:44:19.042 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-15T03:44:19.043 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.044 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon min osdmap epochs = 25 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon osdmap full prune interval = 2 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon osdmap full prune min = 15 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon osdmap full prune txsize = 2 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon scrub inject crc mismatch = 0.01 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon scrub inject missing keys = 0.05 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: mon scrub interval = 300 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: paxos min = 10 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: paxos service trim min = 5 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: paxos trim min = 10 2024-09-15T03:44:19.045 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: debug monc = 20 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.d] 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.e] 2024-09-15T03:44:19.046 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.f] 2024-09-15T03:44:19.047 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.g] 2024-09-15T03:44:19.047 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.h] 2024-09-15T03:44:19.047 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.i] 2024-09-15T03:44:19.047 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T03:44:19.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T03:44:19.127 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T03:44:19.127 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:19.127 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T03:44:19.134 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:19.134 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T03:44:19.140 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T03:44:19.140 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:19.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T03:44:19.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T03:44:19.210 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:19.210 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:19.216 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T03:44:19.216 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:19.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T03:44:19.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T03:44:19.243 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:19.243 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:19.250 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T03:44:19.250 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T03:44:19.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T03:44:19.290 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T03:44:19.293 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T03:44:19.293 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T03:44:19.293 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T03:44:19.374 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T03:44:19.386 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T03:44:19.387 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-15T03:44:19.387 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T03:44:19.404 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:19.404 INFO:tasks.ceph:role: osd.0 2024-09-15T03:44:19.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T03:44:19.404 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T03:44:19.491 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:44:19.491 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:44:19.491 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:44:19.491 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 2024-09-15T03:44:19.491 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:44:19.492 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:44:19.492 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:44:19.492 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:44:19.492 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:44:19.492 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:44:19.493 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-15T03:44:19.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T03:44:19.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T03:44:19.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:44:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:44:19.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T03:44:19.580 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:19.581 INFO:tasks.ceph:role: osd.1 2024-09-15T03:44:19.581 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T03:44:19.581 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T03:44:19.622 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:44:19.622 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:44:19.623 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:44:19.624 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-15T03:44:19.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T03:44:19.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T03:44:19.653 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:44:19.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:44:19.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T03:44:19.713 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:19.713 INFO:tasks.ceph:role: osd.2 2024-09-15T03:44:19.713 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T03:44:19.713 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T03:44:19.756 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:44:19.756 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:44:19.756 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:44:19.757 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:44:19.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-09-15T03:44:19.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T03:44:19.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T03:44:19.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:44:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:44:19.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:19.844+0000 7ff329961540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T03:44:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:19.844+0000 7ff329961540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T03:44:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:19.844+0000 7ff329961540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T03:44:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:19.844+0000 7ff329961540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T03:44:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:19.844+0000 7ff329961540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-15T03:44:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:19.844+0000 7ff329961540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T03:44:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:20.396+0000 7ff329961540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:21.176+0000 7ff329961540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:21.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:21.956+0000 7ff329961540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:22.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T03:44:22.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:22.852+0000 7fbc1141a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T03:44:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:22.852+0000 7fbc1141a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T03:44:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:22.852+0000 7fbc1141a540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T03:44:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:22.852+0000 7fbc1141a540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T03:44:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:22.852+0000 7fbc1141a540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-15T03:44:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:22.852+0000 7fbc1141a540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T03:44:23.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:23.412+0000 7fbc1141a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:24.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:24.192+0000 7fbc1141a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:24.976+0000 7fbc1141a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:25.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T03:44:25.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:25.928+0000 7fa3914a7540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T03:44:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:25.928+0000 7fa3914a7540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T03:44:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:25.928+0000 7fa3914a7540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T03:44:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:25.928+0000 7fa3914a7540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T03:44:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:25.928+0000 7fa3914a7540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-15T03:44:25.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:25.928+0000 7fa3914a7540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T03:44:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:26.484+0000 7fa3914a7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:27.260+0000 7fa3914a7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:28.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:28.044+0000 7fa3914a7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:28.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T03:44:28.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T03:44:28.921 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:28.921 INFO:tasks.ceph:role: osd.3 2024-09-15T03:44:28.921 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T03:44:28.921 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T03:44:29.446 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:44:29.446 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:44:29.446 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:44:29.446 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 2024-09-15T03:44:29.446 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:44:29.446 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:44:29.447 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:44:29.447 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:44:29.447 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:44:29.447 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:44:29.448 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T03:44:29.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-15T03:44:29.534 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T03:44:29.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:44:29.545 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:44:29.545 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T03:44:29.603 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:29.603 INFO:tasks.ceph:role: osd.4 2024-09-15T03:44:29.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T03:44:29.603 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:44:30.119 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:44:30.121 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T03:44:30.121 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-15T03:44:30.159 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T03:44:30.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:44:30.168 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:44:30.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T03:44:30.179 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-15T03:44:30.179 INFO:tasks.ceph:role: osd.5 2024-09-15T03:44:30.179 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T03:44:30.179 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T03:44:30.682 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T03:44:30.683 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-15T03:44:30.683 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T03:44:30.683 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T03:44:30.684 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T03:44:30.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-15T03:44:30.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T03:44:30.746 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-15T03:44:30.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T03:44:30.747 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:30.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:30.821+0000 7f0cdc011540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T03:44:30.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:30.825+0000 7f0cdc011540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T03:44:30.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:30.825+0000 7f0cdc011540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T03:44:30.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:30.825+0000 7f0cdc011540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T03:44:30.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:30.825+0000 7f0cdc011540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-15T03:44:30.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:30.825+0000 7f0cdc011540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T03:44:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:31.385+0000 7f0cdc011540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:32.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:32.169+0000 7f0cdc011540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:32.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:32.949+0000 7f0cdc011540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:33.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T03:44:33.814 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:33.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:33.897+0000 7fbe47116540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T03:44:33.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:33.901+0000 7fbe47116540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T03:44:33.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:33.901+0000 7fbe47116540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T03:44:33.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:33.901+0000 7fbe47116540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T03:44:33.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:33.901+0000 7fbe47116540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-15T03:44:33.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:33.901+0000 7fbe47116540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T03:44:34.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:34.453+0000 7fbe47116540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:35.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:35.237+0000 7fbe47116540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.017+0000 7fbe47116540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:36.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T03:44:36.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:36.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.905+0000 7f61d4816540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T03:44:36.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.905+0000 7f61d4816540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T03:44:36.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.905+0000 7f61d4816540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T03:44:36.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.905+0000 7f61d4816540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T03:44:36.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.905+0000 7f61d4816540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-15T03:44:36.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:36.909+0000 7f61d4816540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T03:44:37.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:37.465+0000 7f61d4816540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:38.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:38.245+0000 7f61d4816540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:39.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:39.025+0000 7f61d4816540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-15T03:44:39.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T03:44:39.937 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T03:44:39.937 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:39.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T03:44:39.954 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:39.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T03:44:40.016 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:40.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T03:44:40.029 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:40.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T03:44:40.046 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:40.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T03:44:40.056 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:40.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T03:44:40.112 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:40.112 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T03:44:40.125 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:40.126 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T03:44:40.132 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T03:44:40.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T03:44:40.139 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T03:44:40.177 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-09-15T03:44:40.177 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQATWOZmE6GgMhAAY5qrvZTLm31LqXCqA3esRw== 2024-09-15T03:44:40.177 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-09-15T03:44:40.177 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAWWOZmH/swMxAAEi1Cq3l/w64KtZKBIL2XzQ== 2024-09-15T03:44:40.177 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAZWOZmeRWxNxAAtwLSHBtwk7nN+74kyWUOcw== 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQATWOZmhUVVERAA4j+P7gZGPwvNxfiUA4RFTw== 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAeWOZmnAg+MRAA22z8DcKITj7Y+B6HQ4cL4g== 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAhWOZmWMPNNRAApSwXP9NJ124DclfHC0LdNg== 2024-09-15T03:44:40.178 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-09-15T03:44:40.179 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQAkWOZmyRk7NhAADdaTa7er76Kgyu5hH+LBtw== 2024-09-15T03:44:40.179 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-09-15T03:44:40.179 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQATWOZm6SREFhAAqsoGREL2y3FDoh5XBJBwDg== 2024-09-15T03:44:40.188 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-09-15T03:44:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQATWOZmE6GgMhAAY5qrvZTLm31LqXCqA3esRw== 2024-09-15T03:44:40.188 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAWWOZmH/swMxAAEi1Cq3l/w64KtZKBIL2XzQ== 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAZWOZmeRWxNxAAtwLSHBtwk7nN+74kyWUOcw== 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQATWOZmhUVVERAA4j+P7gZGPwvNxfiUA4RFTw== 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAeWOZmnAg+MRAA22z8DcKITj7Y+B6HQ4cL4g== 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAhWOZmWMPNNRAApSwXP9NJ124DclfHC0LdNg== 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-09-15T03:44:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAkWOZmyRk7NhAADdaTa7er76Kgyu5hH+LBtw== 2024-09-15T03:44:40.190 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-15T03:44:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQATWOZm6SREFhAAqsoGREL2y3FDoh5XBJBwDg== 2024-09-15T03:44:40.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.222 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.354 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T03:44:40.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T03:44:40.446 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.449 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.534 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T03:44:40.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T03:44:40.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T03:44:40.673 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T03:44:40.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T03:44:40.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:40.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T03:44:40.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T03:44:40.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:40.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T03:44:40.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T03:44:40.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:40.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T03:44:40.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-09-15T03:44:41.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:41.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-09-15T03:44:41.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-09-15T03:44:41.124 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:41.175 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-09-15T03:44:41.187 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-09-15T03:44:41.205 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:41.259 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-09-15T03:44:41.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-09-15T03:44:41.289 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:41.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-09-15T03:44:41.352 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-09-15T03:44:41.363 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:41.417 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-09-15T03:44:41.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-09-15T03:44:41.440 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T03:44:41.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-09-15T03:44:41.504 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:41.506 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T03:44:41.552 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T03:44:41.552 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T03:44:41.552 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T03:44:41.555 INFO:tasks.ceph.mon.a:Started 2024-09-15T03:44:41.555 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T03:44:41.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T03:44:41.557 INFO:tasks.ceph.mon.b:Started 2024-09-15T03:44:41.557 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T03:44:41.557 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T03:44:41.560 INFO:tasks.ceph.mon.c:Started 2024-09-15T03:44:41.560 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-15T03:44:41.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-15T03:44:41.562 INFO:tasks.ceph.mon.d:Started 2024-09-15T03:44:41.562 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-15T03:44:41.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-15T03:44:41.565 INFO:tasks.ceph.mon.e:Started 2024-09-15T03:44:41.565 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T03:44:41.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T03:44:41.568 INFO:tasks.ceph.mon.f:Started 2024-09-15T03:44:41.568 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T03:44:41.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T03:44:41.571 INFO:tasks.ceph.mon.g:Started 2024-09-15T03:44:41.571 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-15T03:44:41.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-15T03:44:41.573 INFO:tasks.ceph.mon.h:Started 2024-09-15T03:44:41.573 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T03:44:41.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T03:44:41.576 INFO:tasks.ceph.mon.i:Started 2024-09-15T03:44:41.576 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T03:44:41.577 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T03:44:41.577 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T03:44:41.580 INFO:tasks.ceph.mgr.x:Started 2024-09-15T03:44:41.580 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T03:44:41.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T03:44:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient: get_monmap_and_config 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient: build_initial_monmap 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient: monmap: 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:41.689221+0000 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:41.689221+0000 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 1 Processor -- start 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 1 -- start start 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient: init 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient: _add_conns ranks=[0,6,1,3,4,7,2,8,5] 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient(hunting): picked mon.noname-a con 0x7f063009bfd0 addr v1:172.21.15.26:6789/0 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient(hunting): picked mon.noname-d con 0x7f063009a000 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient(hunting): picked mon.noname-f con 0x7f063009d7b0 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063004ca10 con 0x7f063009bfd0 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0630099900 con 0x7f063009d7b0 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063009c3a0 con 0x7f063009a000 2024-09-15T03:44:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f0636021700 10 monclient(hunting): authenticate will time out at 826.118938s 2024-09-15T03:44:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f063501f700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f063009bfd0 0x7f063009c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40594/0 (socket says 172.21.15.26:40594) 2024-09-15T03:44:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:41.684+0000 7f063501f700 1 -- 172.21.15.26:0/224523357 learned_addr learned my addr 172.21.15.26:0/224523357 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient(hunting): tick 2024-09-15T03:44:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 monclient(hunting): continuing hunt 2024-09-15T03:44:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:44:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 -- 172.21.15.26:0/224523357 >> v1:172.21.15.26:6792/0 conn(0x7f063009a000 legacy=0x7f063009d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 -- 172.21.15.26:0/224523357 >> v1:172.21.15.122:6789/0 conn(0x7f063009d7b0 legacy=0x7f06301377c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 -- 172.21.15.26:0/224523357 >> v1:172.21.15.26:6789/0 conn(0x7f063009bfd0 legacy=0x7f063009c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient: _add_conns ranks=[7,2,8,4,3,5] 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient(hunting): picked mon.noname-i con 0x7f0614004940 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient(hunting): picked mon.noname-b con 0x7f06140085c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient(hunting): picked mon.noname-e con 0x7f061400c2e0 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 -- 172.21.15.26:0/224523357 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0630098ad0 con 0x7f06140085c0 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 -- 172.21.15.26:0/224523357 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0614011100 con 0x7f0614004940 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 1 -- 172.21.15.26:0/224523357 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0614012350 con 0x7f061400c2e0 2024-09-15T03:44:44.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:44.684+0000 7f0626ffd700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051339109 0 0) 0x7f0630098ad0 con 0x7f06140085c0 2024-09-15T03:44:47.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient(hunting): my global_id is 4115 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061c003570 con 0x7f06140085c0 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135077142 0 0) 0x7f0614012350 con 0x7f061400c2e0 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient(hunting): my global_id is 4112 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0630098ad0 con 0x7f061400c2e0 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1326873579 0 0) 0x7f061c003570 con 0x7f06140085c0 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 >> v1:172.21.15.26:6793/0 conn(0x7f061400c2e0 legacy=0x7f061400e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 >> v1:172.21.15.122:6792/0 conn(0x7f0614004940 legacy=0x7f0614006d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 monclient: found mon.noname-b 2024-09-15T03:44:47.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614013600 con 0x7f06140085c0 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c0017f0 con 0x7f06140085c0 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0636021700 5 monclient: authenticate success, global_id 4115 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0635820700 1 -- 172.21.15.26:0/224523357 reap_dead start 2024-09-15T03:44:47.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0636021700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.156+0000 7f0627fff700 10 monclient: dump: 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.168 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f062c001a50 con 0x7f06140085c0 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0627fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0627fff700 1 -- 172.21.15.26:0/224523357 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c001f80 con 0x7f06140085c0 2024-09-15T03:44:47.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0627fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0627fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0627fff700 10 monclient: dump: 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.170 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 4 set_mon_vals no callback set 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: shutdown 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- 172.21.15.26:0/224523357 >> v1:172.21.15.26:6790/0 conn(0x7f06140085c0 legacy=0x7f061400aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- 172.21.15.26:0/224523357 shutdown_connections 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- 172.21.15.26:0/224523357 >> 172.21.15.26:0/224523357 conn(0x7f0630063710 msgr2=0x7f0630063b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- 172.21.15.26:0/224523357 shutdown_connections 2024-09-15T03:44:47.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- 172.21.15.26:0/224523357 wait complete. 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: monmap: 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.172 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 Processor -- start 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- start start 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: init 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient: _add_conns ranks=[3,4,6,0,8,7,5,2,1] 2024-09-15T03:44:47.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient(hunting): picked mon.noname-g con 0x7f06140085c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient(hunting): picked mon.noname-c con 0x7f0630124a50 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient(hunting): picked mon.noname-d con 0x7f063014cbd0 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063009dc10 con 0x7f06140085c0 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063006e3e0 con 0x7f0630124a50 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0630124e20 con 0x7f063014cbd0 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f0636021700 10 monclient(hunting): authenticate will time out at 831.595691s 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f063481e700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f0630124a50 0x7f063014b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35608/0 (socket says 172.21.15.26:35608) 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.160+0000 7f063481e700 1 -- 172.21.15.26:0/2103589176 learned_addr learned my addr 172.21.15.26:0/2103589176 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792863431 0 0) 0x7f063006e3e0 con 0x7f0630124a50 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 10 monclient(hunting): my global_id is 4108 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0614011880 con 0x7f0630124a50 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696372818 0 0) 0x7f0630124e20 con 0x7f063014cbd0 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 10 monclient(hunting): my global_id is 4119 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.164+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063006e3e0 con 0x7f063014cbd0 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 663233286 0 0) 0x7f0614011880 con 0x7f0630124a50 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0630124e20 con 0x7f0630124a50 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c001f00 con 0x7f0630124a50 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): dump: 2024-09-15T03:44:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649280780 0 0) 0x7f063009dc10 con 0x7f06140085c0 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): my global_id is 4116 2024-09-15T03:44:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0614011880 con 0x7f06140085c0 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1052393445 0 0) 0x7f063006e3e0 con 0x7f063014cbd0 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f063009dc10 con 0x7f063014cbd0 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0620002d40 con 0x7f063014cbd0 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient(hunting): dump: 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 835398652 0 0) 0x7f0630124e20 con 0x7f0630124a50 2024-09-15T03:44:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 >> v1:172.21.15.26:6792/0 conn(0x7f063014cbd0 legacy=0x7f063014efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 >> v1:172.21.15.122:6790/0 conn(0x7f06140085c0 legacy=0x7f0630139f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 monclient: found mon.c 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630147090 con 0x7f0630124a50 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 5 monclient: authenticate success, global_id 4108 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 10 monclient: _renew_subs 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0630125ae0 con 0x7f0630124a50 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 10 monclient: _renew_subs 2024-09-15T03:44:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f06301261a0 con 0x7f0630124a50 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f062c0023d0 con 0x7f0630124a50 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.168+0000 7f06257fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06277fe700 4 set_mon_vals no callback set 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c002900 con 0x7f0630124a50 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06257fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06257fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06257fa700 10 monclient: dump: 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f062c005730 con 0x7f0630124a50 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1747837031 0 0) 0x7f062c006480 con 0x7f0630124a50 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f0636021700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f0636021700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f0636021700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:44:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.172+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100050b0 con 0x7f0630124a50 2024-09-15T03:44:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.176+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f062c005a90 con 0x7f0630124a50 2024-09-15T03:44:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.176+0000 7f06257fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.176+0000 7f06257fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:47.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.248+0000 7f0636021700 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T03:44:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.248+0000 7f0636021700 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T03:44:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.248+0000 7f0636021700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:44:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.252+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0610005360 con 0x7f0630124a50 2024-09-15T03:44:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f06257fa700 1 -- 172.21.15.26:0/2103589176 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (unknown 2277712955 0 0) 0x7f062c006690 con 0x7f0630124a50 2024-09-15T03:44:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f06257fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-09-15T03:44:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f06257fa700 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-09-15T03:44:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 10 monclient: shutdown 2024-09-15T03:44:47.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 >> v1:172.21.15.26:6791/0 conn(0x7f0630124a50 legacy=0x7f063014b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 shutdown_connections 2024-09-15T03:44:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 >> 172.21.15.26:0/2103589176 conn(0x7f0630063710 msgr2=0x7f063009fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 shutdown_connections 2024-09-15T03:44:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:47.256+0000 7f0636021700 1 -- 172.21.15.26:0/2103589176 wait complete. 2024-09-15T03:44:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-09-15T03:44:47.275 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T03:44:47.275 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:47.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T03:44:47.284 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:47.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T03:44:47.294 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-09-15T03:44:47.294 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T03:44:47.305 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:47.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T03:44:47.319 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:47.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T03:44:47.369 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:47.365+0000 7fe519da4000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T03:44:47.381 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T03:44:47.381 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T03:44:47.391 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new b65c9d59-0082-4f20-9069-40cf97297ffc 0 2024-09-15T03:44:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: get_monmap_and_config 2024-09-15T03:44:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: monmap: 2024-09-15T03:44:47.499 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:47.491461+0000 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:47.491461+0000 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.500 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 Processor -- start 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- start start 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: init 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: _add_conns ranks=[1,0,3,8,6,5,2,4,7] 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient(hunting): picked mon.noname-f con 0x7f43f8132a10 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient(hunting): picked mon.noname-a con 0x7f43f809fab0 addr v1:172.21.15.26:6789/0 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient(hunting): picked mon.noname-g con 0x7f43f809fe80 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f804ca10 con 0x7f43f809fab0 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f8068d20 con 0x7f43f8132a10 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80a6e90 con 0x7f43f809fe80 2024-09-15T03:44:47.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient(hunting): authenticate will time out at 821.509997s 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f77fe700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f43f809fe80 0x7f43f813b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.122:57402/0 (socket says 172.21.15.122:57402) 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f77fe700 1 -- 172.21.15.122:0/1347073377 learned_addr learned my addr 172.21.15.122:0/1347073377 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551783910 0 0) 0x7f43f80a6e90 con 0x7f43f809fe80 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient(hunting): my global_id is 4125 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d8003570 con 0x7f43f809fe80 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929570051 0 0) 0x7f43f8068d20 con 0x7f43f8132a10 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient(hunting): my global_id is 4105 2024-09-15T03:44:47.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f80a6e90 con 0x7f43f8132a10 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707287541 0 0) 0x7f43d8003570 con 0x7f43f809fe80 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 >> v1:172.21.15.122:6789/0 conn(0x7f43f8132a10 legacy=0x7f43f8132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 >> v1:172.21.15.26:6789/0 conn(0x7f43f809fab0 legacy=0x7f43f8137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 monclient: found mon.noname-g 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f80a0f00 con 0x7f43f809fe80 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 5 monclient: authenticate success, global_id 4125 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43e80017b0 con 0x7f43f809fe80 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: dump: 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.504 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43e80019d0 con 0x7f43f809fe80 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 4 set_mon_vals no callback set 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 1 -- 172.21.15.122:0/1347073377 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43e80048f0 con 0x7f43f809fe80 2024-09-15T03:44:47.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f5ffb700 10 monclient: dump: 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.506 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 10 monclient: shutdown 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- 172.21.15.122:0/1347073377 >> v1:172.21.15.122:6790/0 conn(0x7f43f809fe80 legacy=0x7f43f813b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- 172.21.15.122:0/1347073377 shutdown_connections 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.489+0000 7f43f7fff700 1 -- 172.21.15.122:0/1347073377 >> 172.21.15.122:0/1347073377 conn(0x7f43f806e730 msgr2=0x7f43f80a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- 172.21.15.122:0/1347073377 shutdown_connections 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- 172.21.15.122:0/1347073377 wait complete. 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: monmap: 2024-09-15T03:44:47.507 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 0.000000 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:created 0.000000 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 1 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.508 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 Processor -- start 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- start start 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: init 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: _add_conns ranks=[7,3,2,1,0,6,8,5,4] 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient(hunting): picked mon.noname-i con 0x7f43f809fab0 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient(hunting): picked mon.noname-g con 0x7f43f809fe80 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient(hunting): picked mon.noname-b con 0x7f43f8132a10 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f80a1a10 con 0x7f43f8132a10 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f81205e0 con 0x7f43f809fe80 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f8120300 con 0x7f43f809fab0 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient(hunting): authenticate will time out at 821.514016s 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f6ffd700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f43f809fab0 0x7f43f8142170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.122:45704/0 (socket says 172.21.15.122:45704) 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f6ffd700 1 -- 172.21.15.122:0/1057384965 learned_addr learned my addr 172.21.15.122:0/1057384965 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f67fc700 1 --1- 172.21.15.122:0/1057384965 >> v1:172.21.15.122:6790/0 conn(0x7f43f809fe80 0x7f43f8146e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.122:57418/0 (socket says 172.21.15.122:57418) 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646779530 0 0) 0x7f43f8120300 con 0x7f43f809fab0 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): my global_id is 4138 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43dc003570 con 0x7f43f809fab0 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252629325 0 0) 0x7f43f81205e0 con 0x7f43f809fe80 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): my global_id is 4134 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f8120300 con 0x7f43f809fe80 2024-09-15T03:44:47.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1379892563 0 0) 0x7f43dc003570 con 0x7f43f809fab0 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43f81205e0 con 0x7f43f809fab0 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934914902 0 0) 0x7f43f80a1a10 con 0x7f43f8132a10 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): my global_id is 4133 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43dc003570 con 0x7f43f8132a10 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43f0001a40 con 0x7f43f809fab0 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): dump: 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.511 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:47.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 648872469 0 0) 0x7f43f8120300 con 0x7f43f809fe80 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43f80a1a10 con 0x7f43f809fe80 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43e0001dc0 con 0x7f43f809fe80 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient(hunting): dump: 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.513 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 674575945 0 0) 0x7f43f81205e0 con 0x7f43f809fab0 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 >> v1:172.21.15.122:6790/0 conn(0x7f43f809fe80 legacy=0x7f43f8146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 >> v1:172.21.15.26:6790/0 conn(0x7f43f8132a10 legacy=0x7f43f814a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 monclient: found mon.i 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f811f0c0 con 0x7f43f809fab0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43eeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 5 monclient: authenticate success, global_id 4138 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: _renew_subs 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f8120020 con 0x7f43f809fab0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: _renew_subs 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.493+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f811fca0 con 0x7f43f809fab0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43f0002070 con 0x7f43f809fab0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43f0004e00 con 0x7f43f809fab0 2024-09-15T03:44:47.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43f57fa700 4 set_mon_vals no callback set 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 10 monclient: dump: 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.516 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f43f00051e0 con 0x7f43f809fab0 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (unknown 2888805938 0 0) 0x7f43f0006130 con 0x7f43f809fab0 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43f7fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43f7fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43f7fff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.497+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d00050b0 con 0x7f43f809fab0 2024-09-15T03:44:47.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.501+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f43f00057c0 con 0x7f43f809fab0 2024-09-15T03:44:47.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.501+0000 7f43eeffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.501+0000 7f43eeffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:47.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.573+0000 7f43f7fff700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "b65c9d59-0082-4f20-9069-40cf97297ffc", "id": 0}] 2024-09-15T03:44:47.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.573+0000 7f43f7fff700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "b65c9d59-0082-4f20-9069-40cf97297ffc", "id": 0}] 2024-09-15T03:44:47.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.573+0000 7f43f7fff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.573+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "osd new", "uuid": "b65c9d59-0082-4f20-9069-40cf97297ffc", "id": 0} v 0) v1 -- 0x7f43d00058e0 con 0x7f43f809fab0 2024-09-15T03:44:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43eeffd700 1 -- 172.21.15.122:0/1057384965 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b65c9d59-0082-4f20-9069-40cf97297ffc", "id": 0}]=0 v3) v1 ==== 112+0+2 (unknown 1957450844 0 732707515) 0x7f43f0006360 con 0x7f43f809fab0 2024-09-15T03:44:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43eeffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "b65c9d59-0082-4f20-9069-40cf97297ffc", "id": 0}] 2024-09-15T03:44:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43eeffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:47.584 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-09-15T03:44:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 10 monclient: shutdown 2024-09-15T03:44:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 >> v1:172.21.15.122:6792/0 conn(0x7f43f809fab0 legacy=0x7f43f8142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 shutdown_connections 2024-09-15T03:44:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 >> 172.21.15.122:0/1057384965 conn(0x7f43f806e730 msgr2=0x7f43f8067910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 shutdown_connections 2024-09-15T03:44:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.581+0000 7f43f7fff700 1 -- 172.21.15.122:0/1057384965 wait complete. 2024-09-15T03:44:47.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 57399273-1f9a-4dfb-bef9-0ced7ef1a7c1 1 2024-09-15T03:44:47.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.697+0000 7f5e318f5700 10 monclient: get_monmap_and_config 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.697+0000 7f5e318f5700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.697+0000 7f5e318f5700 10 monclient: monmap: 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:47.700799+0000 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:47.700799+0000 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.701 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.702 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.702 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.702 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.702 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.702 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.702 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.697+0000 7f5e318f5700 1 Processor -- start 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.697+0000 7f5e318f5700 1 -- start start 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.697+0000 7f5e318f5700 10 monclient: init 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: _add_conns ranks=[8,4,7,1,0,5,6,3,2] 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient(hunting): picked mon.noname-e con 0x7f5e2c099b30 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient(hunting): picked mon.noname-c con 0x7f5e2c09e930 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient(hunting): picked mon.noname-i con 0x7f5e2c135b30 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c04ca10 con 0x7f5e2c09e930 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c1305f0 con 0x7f5e2c135b30 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c098bd0 con 0x7f5e2c099b30 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient(hunting): authenticate will time out at 821.720176s 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e310f4700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f5e2c135b30 0x7f5e2c137f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.122:45710/0 (socket says 172.21.15.122:45710) 2024-09-15T03:44:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e310f4700 1 -- 172.21.15.122:0/909458391 learned_addr learned my addr 172.21.15.122:0/909458391 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963926396 0 0) 0x7f5e2c1305f0 con 0x7f5e2c135b30 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient(hunting): my global_id is 4147 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e1c003570 con 0x7f5e2c135b30 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215597394 0 0) 0x7f5e1c003570 con 0x7f5e2c135b30 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 >> v1:172.21.15.26:6793/0 conn(0x7f5e2c099b30 legacy=0x7f5e2c12f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 >> v1:172.21.15.26:6791/0 conn(0x7f5e2c09e930 legacy=0x7f5e2c12fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 monclient: found mon.noname-i 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c098eb0 con 0x7f5e2c135b30 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5e24001b30 con 0x7f5e2c135b30 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 5 monclient: authenticate success, global_id 4147 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: dump: 2024-09-15T03:44:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5e24001e50 con 0x7f5e2c135b30 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 1 -- 172.21.15.122:0/909458391 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5e24004fb0 con 0x7f5e2c135b30 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e2b7fe700 10 monclient: dump: 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.710 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 4 set_mon_vals no callback set 2024-09-15T03:44:47.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: shutdown 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- 172.21.15.122:0/909458391 >> v1:172.21.15.122:6792/0 conn(0x7f5e2c135b30 legacy=0x7f5e2c137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- 172.21.15.122:0/909458391 shutdown_connections 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- 172.21.15.122:0/909458391 >> 172.21.15.122:0/909458391 conn(0x7f5e2c0a3800 msgr2=0x7f5e2c09fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- 172.21.15.122:0/909458391 shutdown_connections 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- 172.21.15.122:0/909458391 wait complete. 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: monmap: 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 0.000000 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:created 0.000000 2024-09-15T03:44:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 1 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.713 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 Processor -- start 2024-09-15T03:44:47.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 1 -- start start 2024-09-15T03:44:47.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.701+0000 7f5e318f5700 10 monclient: init 2024-09-15T03:44:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _add_conns ranks=[6,5,2,0,7,3,8,4,1] 2024-09-15T03:44:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient(hunting): picked mon.noname-d con 0x7f5e2c099b30 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient(hunting): picked mon.noname-h con 0x7f5e2c09e930 addr v1:172.21.15.122:6791/0 2024-09-15T03:44:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient(hunting): picked mon.noname-b con 0x7f5e2c135b30 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:47.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c06cb30 con 0x7f5e2c135b30 2024-09-15T03:44:47.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c098720 con 0x7f5e2c09e930 2024-09-15T03:44:47.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c06c640 con 0x7f5e2c099b30 2024-09-15T03:44:47.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient(hunting): authenticate will time out at 821.724265s 2024-09-15T03:44:47.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e2bfff700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f5e2c09e930 0x7f5e2c063a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.122:44544/0 (socket says 172.21.15.122:44544) 2024-09-15T03:44:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e2bfff700 1 -- 172.21.15.122:0/1715680928 learned_addr learned my addr 172.21.15.122:0/1715680928 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090718311 0 0) 0x7f5e2c098720 con 0x7f5e2c09e930 2024-09-15T03:44:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): my global_id is 4109 2024-09-15T03:44:47.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e10003570 con 0x7f5e2c09e930 2024-09-15T03:44:47.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230356103 0 0) 0x7f5e2c06cb30 con 0x7f5e2c135b30 2024-09-15T03:44:47.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): my global_id is 4142 2024-09-15T03:44:47.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e2c098720 con 0x7f5e2c135b30 2024-09-15T03:44:47.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516453369 0 0) 0x7f5e2c06c640 con 0x7f5e2c099b30 2024-09-15T03:44:47.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): my global_id is 4128 2024-09-15T03:44:47.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e2c06cb30 con 0x7f5e2c099b30 2024-09-15T03:44:47.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2046022963 0 0) 0x7f5e10003570 con 0x7f5e2c09e930 2024-09-15T03:44:47.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5e2c06c640 con 0x7f5e2c09e930 2024-09-15T03:44:47.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5e18001a60 con 0x7f5e2c09e930 2024-09-15T03:44:47.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:44:47.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): dump: 2024-09-15T03:44:47.729 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.729 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.729 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.729 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.729 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.730 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.730 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.730 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.730 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.730 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.731 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.731 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.731 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.731 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.731 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.731 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:47.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1073764195 0 0) 0x7f5e2c06c640 con 0x7f5e2c09e930 2024-09-15T03:44:47.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 >> v1:172.21.15.26:6792/0 conn(0x7f5e2c099b30 legacy=0x7f5e2c063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 >> v1:172.21.15.26:6790/0 conn(0x7f5e2c135b30 legacy=0x7f5e2c147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 monclient: found mon.h 2024-09-15T03:44:47.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:44:47.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c06e090 con 0x7f5e2c09e930 2024-09-15T03:44:47.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 5 monclient: authenticate success, global_id 4109 2024-09-15T03:44:47.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _renew_subs 2024-09-15T03:44:47.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:44:47.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c064160 con 0x7f5e2c09e930 2024-09-15T03:44:47.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _renew_subs 2024-09-15T03:44:47.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:44:47.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5e18001d20 con 0x7f5e2c09e930 2024-09-15T03:44:47.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c06cd00 con 0x7f5e2c09e930 2024-09-15T03:44:47.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5e180025b0 con 0x7f5e2c09e930 2024-09-15T03:44:47.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:44:47.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 10 monclient: dump: 2024-09-15T03:44:47.736 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.736 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.736 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.736 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.737 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.737 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.737 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.737 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.737 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.737 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.738 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f5e18002990 con 0x7f5e2c09e930 2024-09-15T03:44:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e2affd700 4 set_mon_vals no callback set 2024-09-15T03:44:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (unknown 1027044621 0 0) 0x7f5e180063c0 con 0x7f5e2c09e930 2024-09-15T03:44:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:44:47.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.705+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c04bbf0 con 0x7f5e2c09e930 2024-09-15T03:44:47.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.709+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f5e180058c0 con 0x7f5e2c09e930 2024-09-15T03:44:47.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.709+0000 7f5e28ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.709+0000 7f5e28ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.805+0000 7f5e318f5700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "57399273-1f9a-4dfb-bef9-0ced7ef1a7c1", "id": 1}] 2024-09-15T03:44:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.805+0000 7f5e318f5700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "57399273-1f9a-4dfb-bef9-0ced7ef1a7c1", "id": 1}] 2024-09-15T03:44:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.805+0000 7f5e318f5700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:44:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.805+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "osd new", "uuid": "57399273-1f9a-4dfb-bef9-0ced7ef1a7c1", "id": 1} v 0) v1 -- 0x7f5e2c067d50 con 0x7f5e2c09e930 2024-09-15T03:44:47.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e28ff9700 1 -- 172.21.15.122:0/1715680928 <== mon.5 v1:172.21.15.122:6791/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57399273-1f9a-4dfb-bef9-0ced7ef1a7c1", "id": 1}]=0 v4) v1 ==== 112+0+2 (unknown 370452313 0 940483276) 0x7f5e18006600 con 0x7f5e2c09e930 2024-09-15T03:44:47.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e28ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "57399273-1f9a-4dfb-bef9-0ced7ef1a7c1", "id": 1}] 2024-09-15T03:44:47.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e28ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:47.817 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-15T03:44:47.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 10 monclient: shutdown 2024-09-15T03:44:47.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 >> v1:172.21.15.122:6791/0 conn(0x7f5e2c09e930 legacy=0x7f5e2c063a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 shutdown_connections 2024-09-15T03:44:47.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 >> 172.21.15.122:0/1715680928 conn(0x7f5e2c0a3800 msgr2=0x7f5e2c12bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 shutdown_connections 2024-09-15T03:44:47.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.813+0000 7f5e318f5700 1 -- 172.21.15.122:0/1715680928 wait complete. 2024-09-15T03:44:47.830 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new e72df2db-c0f8-48fe-84a2-7de252df9821 2 2024-09-15T03:44:47.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient: get_monmap_and_config 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient: monmap: 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:47.931778+0000 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:47.931778+0000 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.932 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.933 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.933 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.933 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.933 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.933 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.933 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 1 Processor -- start 2024-09-15T03:44:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 1 -- start start 2024-09-15T03:44:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient: init 2024-09-15T03:44:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient: _add_conns ranks=[1,2,0,4,5,8,3,6,7] 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient(hunting): picked mon.noname-f con 0x7ff804070f80 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient(hunting): picked mon.noname-b con 0x7ff80406bce0 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient(hunting): picked mon.noname-a con 0x7ff804136050 addr v1:172.21.15.26:6789/0 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80404ca10 con 0x7ff804136050 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80406b300 con 0x7ff804070f80 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80406a5b0 con 0x7ff80406bce0 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff809f55700 10 monclient(hunting): authenticate will time out at 821.951548s 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff8037fe700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff804070f80 0x7ff804069790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:57086/0 (socket says 172.21.15.122:57086) 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff8037fe700 1 -- 172.21.15.122:0/3726462529 learned_addr learned my addr 172.21.15.122:0/3726462529 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238120753 0 0) 0x7ff80406b300 con 0x7ff804070f80 2024-09-15T03:44:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff8027fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff8027fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.929+0000 7ff8027fc700 10 monclient(hunting): my global_id is 4114 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ec003570 con 0x7ff804070f80 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746233180 0 0) 0x7ff7ec003570 con 0x7ff804070f80 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 >> v1:172.21.15.26:6790/0 conn(0x7ff80406bce0 legacy=0x7ff804069ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 >> v1:172.21.15.26:6789/0 conn(0x7ff804136050 legacy=0x7ff804138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 monclient: found mon.noname-f 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80406b020 con 0x7ff804070f80 2024-09-15T03:44:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 5 monclient: authenticate success, global_id 4114 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7f40017b0 con 0x7ff804070f80 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: dump: 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7f4001ad0 con 0x7ff804070f80 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 4 set_mon_vals no callback set 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 1 -- 172.21.15.122:0/3726462529 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7f4004c30 con 0x7ff804070f80 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff8027fc700 10 monclient: dump: 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 10 monclient: shutdown 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 -- 172.21.15.122:0/3726462529 >> v1:172.21.15.122:6789/0 conn(0x7ff804070f80 legacy=0x7ff804069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 -- 172.21.15.122:0/3726462529 shutdown_connections 2024-09-15T03:44:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 -- 172.21.15.122:0/3726462529 >> 172.21.15.122:0/3726462529 conn(0x7ff804063e10 msgr2=0x7ff8040a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 -- 172.21.15.122:0/3726462529 shutdown_connections 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 -- 172.21.15.122:0/3726462529 wait complete. 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 10 monclient: build_initial_monmap 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 10 monclient: monmap: 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 0.000000 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:created 0.000000 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 1 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:47.944 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:47.945 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 Processor -- start 2024-09-15T03:44:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 1 -- start start 2024-09-15T03:44:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.933+0000 7ff809f55700 10 monclient: init 2024-09-15T03:44:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _add_conns ranks=[2,7,6,1,0,8,3,5,4] 2024-09-15T03:44:47.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient(hunting): picked mon.noname-b con 0x7ff80406bce0 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient(hunting): picked mon.noname-i con 0x7ff804070f80 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient(hunting): picked mon.noname-d con 0x7ff804136050 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80406aa40 con 0x7ff80406bce0 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80409cce0 con 0x7ff804136050 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80411f730 con 0x7ff804070f80 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient(hunting): authenticate will time out at 821.956297s 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff802ffd700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7ff804070f80 0x7ff80409d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.122:45712/0 (socket says 172.21.15.122:45712) 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff802ffd700 1 -- 172.21.15.122:0/1156814051 learned_addr learned my addr 172.21.15.122:0/1156814051 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661181996 0 0) 0x7ff80411f730 con 0x7ff804070f80 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): my global_id is 4156 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7f0003570 con 0x7ff804070f80 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620388226 0 0) 0x7ff80406aa40 con 0x7ff80406bce0 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): my global_id is 4160 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80411f730 con 0x7ff80406bce0 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674499523 0 0) 0x7ff80409cce0 con 0x7ff804136050 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): my global_id is 4137 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80406aa40 con 0x7ff804136050 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 125003846 0 0) 0x7ff7f0003570 con 0x7ff804070f80 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff80409cce0 con 0x7ff804070f80 2024-09-15T03:44:47.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7f8001a90 con 0x7ff804070f80 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): dump: 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.952 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2580537200 0 0) 0x7ff80409cce0 con 0x7ff804070f80 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 >> v1:172.21.15.26:6792/0 conn(0x7ff804136050 legacy=0x7ff80414a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 >> v1:172.21.15.26:6790/0 conn(0x7ff80406bce0 legacy=0x7ff80409d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 monclient: found mon.i 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:47.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80409c330 con 0x7ff804070f80 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: _finish_auth 0 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 5 monclient: authenticate success, global_id 4156 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _renew_subs 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80409e050 con 0x7ff804070f80 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _renew_subs 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8041190a0 con 0x7ff804070f80 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7f80021b0 con 0x7ff804070f80 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff8017fa700 4 set_mon_vals no callback set 2024-09-15T03:44:47.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7f80026e0 con 0x7ff804070f80 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 10 monclient: dump: 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7ff7f8002ac0 con 0x7ff804070f80 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (unknown 196564875 0 0) 0x7ff7f8002d70 con 0x7ff804070f80 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.937+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e40050b0 con 0x7ff804070f80 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.941+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7ff7f80058c0 con 0x7ff804070f80 2024-09-15T03:44:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.941+0000 7ff7eb7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:47.941+0000 7ff7eb7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:48.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.029+0000 7ff809f55700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "e72df2db-c0f8-48fe-84a2-7de252df9821", "id": 2}] 2024-09-15T03:44:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.029+0000 7ff809f55700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "e72df2db-c0f8-48fe-84a2-7de252df9821", "id": 2}] 2024-09-15T03:44:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.029+0000 7ff809f55700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.029+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "osd new", "uuid": "e72df2db-c0f8-48fe-84a2-7de252df9821", "id": 2} v 0) v1 -- 0x7ff7e40058e0 con 0x7ff804070f80 2024-09-15T03:44:48.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff7eb7fe700 1 -- 172.21.15.122:0/1156814051 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e72df2db-c0f8-48fe-84a2-7de252df9821", "id": 2}]=0 v5) v1 ==== 112+0+2 (unknown 3961225515 0 216599125) 0x7ff7f8006020 con 0x7ff804070f80 2024-09-15T03:44:48.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff7eb7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "e72df2db-c0f8-48fe-84a2-7de252df9821", "id": 2}] 2024-09-15T03:44:48.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff7eb7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:48.043 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 10 monclient: shutdown 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 >> v1:172.21.15.122:6792/0 conn(0x7ff804070f80 legacy=0x7ff80409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 shutdown_connections 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 >> 172.21.15.122:0/1156814051 conn(0x7ff804063e10 msgr2=0x7ff8040a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 shutdown_connections 2024-09-15T03:44:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.041+0000 7ff809f55700 1 -- 172.21.15.122:0/1156814051 wait complete. 2024-09-15T03:44:48.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new f50cfade-e9a7-4aaa-8816-2172cb4eace1 3 2024-09-15T03:44:48.133 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.129+0000 7fe519da4000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T03:44:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.145+0000 7fd63d416700 10 monclient: get_monmap_and_config 2024-09-15T03:44:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.145+0000 7fd63d416700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.145+0000 7fd63d416700 10 monclient: monmap: 2024-09-15T03:44:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:48.149451+0000 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:48.149451+0000 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.150 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.145+0000 7fd63d416700 1 Processor -- start 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.145+0000 7fd63d416700 1 -- start start 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.145+0000 7fd63d416700 10 monclient: init 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: _add_conns ranks=[3,1,7,5,4,8,0,2,6] 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient(hunting): picked mon.noname-g con 0x7fd6380a3890 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:48.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient(hunting): picked mon.noname-f con 0x7fd638099360 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient(hunting): picked mon.noname-i con 0x7fd638138ee0 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63804ca10 con 0x7fd638099360 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63806e6a0 con 0x7fd6380a3890 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63809e190 con 0x7fd638138ee0 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient(hunting): authenticate will time out at 822.168195s 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd636ffd700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fd6380a3890 0x7fd6381331b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.122:57428/0 (socket says 172.21.15.122:57428) 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd6367fc700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd638099360 0x7fd638099730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:57096/0 (socket says 172.21.15.122:57096) 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd636ffd700 1 -- 172.21.15.122:0/3535056709 learned_addr learned my addr 172.21.15.122:0/3535056709 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769315907 0 0) 0x7fd63809e190 con 0x7fd638138ee0 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): my global_id is 4165 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd620003570 con 0x7fd638138ee0 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761560569 0 0) 0x7fd63804ca10 con 0x7fd638099360 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): my global_id is 4123 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63809e190 con 0x7fd638099360 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034693214 0 0) 0x7fd63806e6a0 con 0x7fd6380a3890 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient(hunting): my global_id is 4143 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63804ca10 con 0x7fd6380a3890 2024-09-15T03:44:48.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1034401170 0 0) 0x7fd620003570 con 0x7fd638138ee0 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 >> v1:172.21.15.122:6790/0 conn(0x7fd6380a3890 legacy=0x7fd6381331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 >> v1:172.21.15.122:6789/0 conn(0x7fd638099360 legacy=0x7fd638099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 monclient: found mon.noname-i 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6381338c0 con 0x7fd638138ee0 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd630001a60 con 0x7fd638138ee0 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 5 monclient: authenticate success, global_id 4165 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: dump: 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd630001c50 con 0x7fd638138ee0 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 1 -- 172.21.15.122:0/3535056709 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd630002180 con 0x7fd638138ee0 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 4 set_mon_vals no callback set 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd635ffb700 10 monclient: dump: 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: shutdown 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- 172.21.15.122:0/3535056709 >> v1:172.21.15.122:6792/0 conn(0x7fd638138ee0 legacy=0x7fd63813b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- 172.21.15.122:0/3535056709 shutdown_connections 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- 172.21.15.122:0/3535056709 >> 172.21.15.122:0/3535056709 conn(0x7fd638071350 msgr2=0x7fd6380a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- 172.21.15.122:0/3535056709 shutdown_connections 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- 172.21.15.122:0/3535056709 wait complete. 2024-09-15T03:44:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: monmap: 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 0.000000 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:created 0.000000 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 1 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 Processor -- start 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 1 -- start start 2024-09-15T03:44:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.149+0000 7fd63d416700 10 monclient: init 2024-09-15T03:44:48.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _add_conns ranks=[3,7,8,4,1,2,6,0,5] 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient(hunting): picked mon.noname-g con 0x7fd638099360 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient(hunting): picked mon.noname-i con 0x7fd638138ee0 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient(hunting): picked mon.noname-e con 0x7fd6381316b0 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63809ec80 con 0x7fd638099360 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63809e8c0 con 0x7fd638138ee0 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63812fd70 con 0x7fd6381316b0 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient(hunting): authenticate will time out at 822.172564s 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd636ffd700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fd638099360 0x7fd63812ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.122:57434/0 (socket says 172.21.15.122:57434) 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd636ffd700 1 -- 172.21.15.122:0/2250620604 learned_addr learned my addr 172.21.15.122:0/2250620604 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327331412 0 0) 0x7fd63809ec80 con 0x7fd638099360 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): my global_id is 4152 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd624003570 con 0x7fd638099360 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165844692 0 0) 0x7fd63809e8c0 con 0x7fd638138ee0 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): my global_id is 4174 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63809ec80 con 0x7fd638138ee0 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898138771 0 0) 0x7fd63812fd70 con 0x7fd6381316b0 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): my global_id is 4130 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63809e8c0 con 0x7fd6381316b0 2024-09-15T03:44:48.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2916931517 0 0) 0x7fd624003570 con 0x7fd638099360 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd63812fd70 con 0x7fd638099360 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd6280019a0 con 0x7fd638099360 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): dump: 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.169 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 530089677 0 0) 0x7fd63809ec80 con 0x7fd638138ee0 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd624003570 con 0x7fd638138ee0 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd62c001440 con 0x7fd638138ee0 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient(hunting): dump: 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.171 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3838223525 0 0) 0x7fd63812fd70 con 0x7fd638099360 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 >> v1:172.21.15.26:6793/0 conn(0x7fd6381316b0 legacy=0x7fd63814a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 >> v1:172.21.15.122:6792/0 conn(0x7fd638138ee0 legacy=0x7fd63812f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 monclient: found mon.g 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638130ff0 con 0x7fd638099360 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 5 monclient: authenticate success, global_id 4152 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _renew_subs 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638131430 con 0x7fd638099360 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _renew_subs 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd638130ae0 con 0x7fd638099360 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6280048b0 con 0x7fd638099360 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd628004d60 con 0x7fd638099360 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd6357fa700 4 set_mon_vals no callback set 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 10 monclient: dump: 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fd6280051c0 con 0x7fd638099360 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 8 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (unknown 880745263 0 0) 0x7fd628006430 con 0x7fd638099360 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.153+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6180050b0 con 0x7fd638099360 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.157+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fd628005750 con 0x7fd638099360 2024-09-15T03:44:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.157+0000 7fd61effd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.157+0000 7fd61effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:48.197 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.193+0000 7fe519da4000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T03:44:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.241+0000 7fd63d416700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "f50cfade-e9a7-4aaa-8816-2172cb4eace1", "id": 3}] 2024-09-15T03:44:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.241+0000 7fd63d416700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "f50cfade-e9a7-4aaa-8816-2172cb4eace1", "id": 3}] 2024-09-15T03:44:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.241+0000 7fd63d416700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.241+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 --> v1:172.21.15.122:6790/0 -- mon_command({"prefix": "osd new", "uuid": "f50cfade-e9a7-4aaa-8816-2172cb4eace1", "id": 3} v 0) v1 -- 0x7fd618005360 con 0x7fd638099360 2024-09-15T03:44:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.249+0000 7fd61effd700 1 -- 172.21.15.122:0/2250620604 <== mon.3 v1:172.21.15.122:6790/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f50cfade-e9a7-4aaa-8816-2172cb4eace1", "id": 3}]=0 v6) v1 ==== 112+0+2 (unknown 619304404 0 525046306) 0x7fd628005ac0 con 0x7fd638099360 2024-09-15T03:44:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.249+0000 7fd61effd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "f50cfade-e9a7-4aaa-8816-2172cb4eace1", "id": 3}] 2024-09-15T03:44:48.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.249+0000 7fd61effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:48.254 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-09-15T03:44:48.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 10 monclient: shutdown 2024-09-15T03:44:48.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 >> v1:172.21.15.122:6790/0 conn(0x7fd638099360 legacy=0x7fd63812ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 shutdown_connections 2024-09-15T03:44:48.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 >> 172.21.15.122:0/2250620604 conn(0x7fd638071350 msgr2=0x7fd6380a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 shutdown_connections 2024-09-15T03:44:48.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.253+0000 7fd63d416700 1 -- 172.21.15.122:0/2250620604 wait complete. 2024-09-15T03:44:48.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new c473a258-e2aa-45ab-8209-98faa4e1a34d 4 2024-09-15T03:44:48.315 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.313+0000 7fe519da4000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: get_monmap_and_config 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: monmap: 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:48.367026+0000 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:48.367026+0000 2024-09-15T03:44:48.367 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.368 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 Processor -- start 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- start start 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: init 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: _add_conns ranks=[7,5,3,0,2,6,1,8,4] 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient(hunting): picked mon.noname-i con 0x7f8810132a10 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient(hunting): picked mon.noname-h con 0x7f881009fab0 addr v1:172.21.15.122:6791/0 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient(hunting): picked mon.noname-g con 0x7f881009fe80 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881004ca10 con 0x7f881009fe80 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8810068d20 con 0x7f881009fab0 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88100a6e90 con 0x7f8810132a10 2024-09-15T03:44:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient(hunting): authenticate will time out at 822.385626s 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880f7fe700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f8810132a10 0x7f8810132de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.122:45742/0 (socket says 172.21.15.122:45742) 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880f7fe700 1 -- 172.21.15.122:0/2495043198 learned_addr learned my addr 172.21.15.122:0/2495043198 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880effd700 1 --1- 172.21.15.122:0/2495043198 >> v1:172.21.15.122:6791/0 conn(0x7f881009fab0 0x7f8810137d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.122:44548/0 (socket says 172.21.15.122:44548) 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836962575 0 0) 0x7f881004ca10 con 0x7f881009fe80 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): my global_id is 4161 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f8003570 con 0x7f881009fe80 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724854389 0 0) 0x7f8810068d20 con 0x7f881009fab0 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): my global_id is 4118 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f881004ca10 con 0x7f881009fab0 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477890381 0 0) 0x7f88100a6e90 con 0x7f8810132a10 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient(hunting): my global_id is 4183 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8810068d20 con 0x7f8810132a10 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3595642492 0 0) 0x7f87f8003570 con 0x7f881009fe80 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 >> v1:172.21.15.122:6792/0 conn(0x7f8810132a10 legacy=0x7f8810132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 >> v1:172.21.15.122:6791/0 conn(0x7f881009fab0 legacy=0x7f8810137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 monclient: found mon.noname-g 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100a0f00 con 0x7f881009fe80 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 5 monclient: authenticate success, global_id 4161 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f88000017b0 con 0x7f881009fe80 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: dump: 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8800001a20 con 0x7f881009fe80 2024-09-15T03:44:48.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 1 -- 172.21.15.122:0/2495043198 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8800001c10 con 0x7f881009fe80 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 4 set_mon_vals no callback set 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f880e7fc700 10 monclient: dump: 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.381 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 10 monclient: shutdown 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- 172.21.15.122:0/2495043198 >> v1:172.21.15.122:6790/0 conn(0x7f881009fe80 legacy=0x7f881013b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- 172.21.15.122:0/2495043198 shutdown_connections 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- 172.21.15.122:0/2495043198 >> 172.21.15.122:0/2495043198 conn(0x7f881006e730 msgr2=0x7f88100a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- 172.21.15.122:0/2495043198 shutdown_connections 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.365+0000 7f8814b13700 1 -- 172.21.15.122:0/2495043198 wait complete. 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: monmap: 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 0.000000 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:created 0.000000 2024-09-15T03:44:48.383 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 1 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 Processor -- start 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 -- start start 2024-09-15T03:44:48.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: init 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: _add_conns ranks=[6,2,7,5,8,0,4,1,3] 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient(hunting): picked mon.noname-d con 0x7f881009fab0 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient(hunting): picked mon.noname-b con 0x7f881009fe80 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient(hunting): picked mon.noname-i con 0x7f8810132a10 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881009cce0 con 0x7f881009fe80 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881011fe90 con 0x7f881009fab0 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88101429a0 con 0x7f8810132a10 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient(hunting): authenticate will time out at 822.389499s 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f880ffff700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f8810132a10 0x7f881014a590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.122:45748/0 (socket says 172.21.15.122:45748) 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f880ffff700 1 -- 172.21.15.122:0/481427665 learned_addr learned my addr 172.21.15.122:0/481427665 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:48.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101962898 0 0) 0x7f88101429a0 con 0x7f8810132a10 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): my global_id is 4192 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87fc003570 con 0x7f8810132a10 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584101984 0 0) 0x7f881011fe90 con 0x7f881009fab0 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): my global_id is 4146 2024-09-15T03:44:48.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88101429a0 con 0x7f881009fab0 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719545037 0 0) 0x7f881009cce0 con 0x7f881009fe80 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): my global_id is 4169 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f881011fe90 con 0x7f881009fe80 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1544526434 0 0) 0x7f87fc003570 con 0x7f8810132a10 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f881009cce0 con 0x7f8810132a10 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8800002220 con 0x7f8810132a10 2024-09-15T03:44:48.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): dump: 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.388 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3146249744 0 0) 0x7f881009cce0 con 0x7f8810132a10 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 >> v1:172.21.15.26:6792/0 conn(0x7f881009fab0 legacy=0x7f881012fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 >> v1:172.21.15.26:6790/0 conn(0x7f881009fe80 legacy=0x7f8810146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 monclient: found mon.i 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881011fbb0 con 0x7f8810132a10 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 5 monclient: authenticate success, global_id 4192 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: _renew_subs 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881011e0a0 con 0x7f8810132a10 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: _renew_subs 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8810130170 con 0x7f8810132a10 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8800001bb0 con 0x7f8810132a10 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8800002a80 con 0x7f8810132a10 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f880dffb700 4 set_mon_vals no callback set 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 10 monclient: dump: 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.391 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f88000057d0 con 0x7f8810132a10 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (unknown 2179620005 0 0) 0x7f8800006370 con 0x7f8810132a10 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.369+0000 7f8814b13700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.373+0000 7f8814b13700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.373+0000 7f8814b13700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.373+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f00050b0 con 0x7f8810132a10 2024-09-15T03:44:48.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.373+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f8800005a80 con 0x7f8810132a10 2024-09-15T03:44:48.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.373+0000 7f87f77fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.373+0000 7f87f77fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.449+0000 7f8814b13700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c473a258-e2aa-45ab-8209-98faa4e1a34d", "id": 4}] 2024-09-15T03:44:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.449+0000 7f8814b13700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c473a258-e2aa-45ab-8209-98faa4e1a34d", "id": 4}] 2024-09-15T03:44:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.449+0000 7f8814b13700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.449+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "osd new", "uuid": "c473a258-e2aa-45ab-8209-98faa4e1a34d", "id": 4} v 0) v1 -- 0x7f87f00058e0 con 0x7f8810132a10 2024-09-15T03:44:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.457+0000 7f87f77fe700 1 -- 172.21.15.122:0/481427665 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c473a258-e2aa-45ab-8209-98faa4e1a34d", "id": 4}]=0 v7) v1 ==== 112+0+2 (unknown 1230603945 0 1697012583) 0x7f8800005d80 con 0x7f8810132a10 2024-09-15T03:44:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.457+0000 7f87f77fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c473a258-e2aa-45ab-8209-98faa4e1a34d", "id": 4}] 2024-09-15T03:44:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.457+0000 7f87f77fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:48.462 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 10 monclient: shutdown 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 >> v1:172.21.15.122:6792/0 conn(0x7f8810132a10 legacy=0x7f881014a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 shutdown_connections 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 >> 172.21.15.122:0/481427665 conn(0x7f881006e730 msgr2=0x7f88100681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 shutdown_connections 2024-09-15T03:44:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.461+0000 7f8814b13700 1 -- 172.21.15.122:0/481427665 wait complete. 2024-09-15T03:44:48.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new c49841a6-c038-49c5-9b02-b124380dad0f 5 2024-09-15T03:44:48.559 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.557+0000 7fe519da4000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T03:44:48.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.577+0000 7f45e6302700 10 monclient: get_monmap_and_config 2024-09-15T03:44:48.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.577+0000 7f45e6302700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.577+0000 7f45e6302700 10 monclient: monmap: 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:48.582155+0000 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:48.582155+0000 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.583 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.584 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.584 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.584 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 Processor -- start 2024-09-15T03:44:48.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- start start 2024-09-15T03:44:48.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: init 2024-09-15T03:44:48.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: _add_conns ranks=[2,3,4,8,5,0,7,1,6] 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient(hunting): picked mon.noname-b con 0x7f45e00a03b0 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient(hunting): picked mon.noname-g con 0x7f45e0099410 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient(hunting): picked mon.noname-c con 0x7f45e0138ed0 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e004ca10 con 0x7f45e00a03b0 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e0133890 con 0x7f45e0099410 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e00a0780 con 0x7f45e0138ed0 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e4aff700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f45e0099410 0x7f45e00997e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.122:57452/0 (socket says 172.21.15.122:57452) 2024-09-15T03:44:48.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e4aff700 1 -- 172.21.15.122:0/1491620268 learned_addr learned my addr 172.21.15.122:0/1491620268 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient(hunting): authenticate will time out at 822.601079s 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917707647 0 0) 0x7f45e0133890 con 0x7f45e0099410 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient(hunting): my global_id is 4170 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45cc003570 con 0x7f45e0099410 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146960798 0 0) 0x7f45e00a0780 con 0x7f45e0138ed0 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient(hunting): my global_id is 4126 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e0133890 con 0x7f45e0138ed0 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2974667051 0 0) 0x7f45cc003570 con 0x7f45e0099410 2024-09-15T03:44:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 >> v1:172.21.15.26:6791/0 conn(0x7f45e0138ed0 legacy=0x7f45e013b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 >> v1:172.21.15.26:6790/0 conn(0x7f45e00a03b0 legacy=0x7f45e0133180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 monclient: found mon.noname-g 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e0099ef0 con 0x7f45e0099410 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45d00017b0 con 0x7f45e0099410 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 5 monclient: authenticate success, global_id 4170 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:44:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: dump: 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.595 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f45d0001a10 con 0x7f45e0099410 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 1 -- 172.21.15.122:0/1491620268 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45d0004c30 con 0x7f45e0099410 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 4 set_mon_vals no callback set 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45d7fff700 10 monclient: dump: 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: shutdown 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- 172.21.15.122:0/1491620268 >> v1:172.21.15.122:6790/0 conn(0x7f45e0099410 legacy=0x7f45e00997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- 172.21.15.122:0/1491620268 shutdown_connections 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- 172.21.15.122:0/1491620268 >> 172.21.15.122:0/1491620268 conn(0x7f45e00a3800 msgr2=0x7f45e009fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- 172.21.15.122:0/1491620268 shutdown_connections 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 1 -- 172.21.15.122:0/1491620268 wait complete. 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.581+0000 7f45e6302700 10 monclient: monmap: 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 0 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:48.598 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 0.000000 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:created 0.000000 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 1 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.599 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 Processor -- start 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- start start 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: init 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _add_conns ranks=[2,7,0,3,1,6,8,5,4] 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient(hunting): picked mon.noname-b con 0x7f45e0099410 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient(hunting): picked mon.noname-i con 0x7f45e00a03b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient(hunting): picked mon.noname-a con 0x7f45e0138ed0 addr v1:172.21.15.26:6789/0 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e009ed60 con 0x7f45e0138ed0 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e009ea90 con 0x7f45e0099410 2024-09-15T03:44:48.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e0132770 con 0x7f45e00a03b0 2024-09-15T03:44:48.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient(hunting): authenticate will time out at 822.604941s 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e4aff700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f45e00a03b0 0x7f45e012f4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.122:45762/0 (socket says 172.21.15.122:45762) 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e4aff700 1 -- 172.21.15.122:0/263468554 learned_addr learned my addr 172.21.15.122:0/263468554 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302758816 0 0) 0x7f45e0132770 con 0x7f45e00a03b0 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): my global_id is 4201 2024-09-15T03:44:48.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c4003570 con 0x7f45e00a03b0 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080076948 0 0) 0x7f45e009ea90 con 0x7f45e0099410 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): my global_id is 4187 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e0132770 con 0x7f45e0099410 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038175209 0 0) 0x7f45e009ed60 con 0x7f45e0138ed0 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): my global_id is 4122 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e009ea90 con 0x7f45e0138ed0 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1858876423 0 0) 0x7f45c4003570 con 0x7f45e00a03b0 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45e009ed60 con 0x7f45e00a03b0 2024-09-15T03:44:48.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45d0004ca0 con 0x7f45e00a03b0 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): dump: 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.604 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4276092299 0 0) 0x7f45e009ed60 con 0x7f45e00a03b0 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 >> v1:172.21.15.26:6790/0 conn(0x7f45e0099410 legacy=0x7f45e012ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 >> v1:172.21.15.26:6789/0 conn(0x7f45e0138ed0 legacy=0x7f45e014a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 monclient: found mon.i 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e012fbb0 con 0x7f45e00a03b0 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 5 monclient: authenticate success, global_id 4201 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _renew_subs 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e0132400 con 0x7f45e00a03b0 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _renew_subs 2024-09-15T03:44:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e0130200 con 0x7f45e00a03b0 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f45d0004f00 con 0x7f45e00a03b0 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d6ffd700 4 set_mon_vals no callback set 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45d00053b0 con 0x7f45e00a03b0 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 10 monclient: dump: 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:epoch 1 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:election_strategy: 3 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f45d0005790 con 0x7f45e00a03b0 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (unknown 1854984604 0 0) 0x7f45d0005a40 con 0x7f45e00a03b0 2024-09-15T03:44:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.585+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c00050b0 con 0x7f45e00a03b0 2024-09-15T03:44:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.589+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f45d0005cd0 con 0x7f45e00a03b0 2024-09-15T03:44:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.589+0000 7f45d57fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.589+0000 7f45d57fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:48.610 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.605+0000 7fe519da4000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T03:44:48.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.665+0000 7f45e6302700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c49841a6-c038-49c5-9b02-b124380dad0f", "id": 5}] 2024-09-15T03:44:48.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.665+0000 7f45e6302700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c49841a6-c038-49c5-9b02-b124380dad0f", "id": 5}] 2024-09-15T03:44:48.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.665+0000 7f45e6302700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:44:48.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.665+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "osd new", "uuid": "c49841a6-c038-49c5-9b02-b124380dad0f", "id": 5} v 0) v1 -- 0x7f45c00058e0 con 0x7f45e00a03b0 2024-09-15T03:44:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.677+0000 7f45d57fa700 1 -- 172.21.15.122:0/263468554 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c49841a6-c038-49c5-9b02-b124380dad0f", "id": 5}]=0 v8) v1 ==== 112+0+2 (unknown 3988879764 0 1988412176) 0x7f45d00061f0 con 0x7f45e00a03b0 2024-09-15T03:44:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.677+0000 7f45d57fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c49841a6-c038-49c5-9b02-b124380dad0f", "id": 5}] 2024-09-15T03:44:48.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.677+0000 7f45d57fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:48.681 INFO:teuthology.orchestra.run.smithi122.stdout:5 2024-09-15T03:44:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.677+0000 7f45e6302700 10 monclient: shutdown 2024-09-15T03:44:48.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.677+0000 7f45e6302700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.677+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 >> v1:172.21.15.122:6792/0 conn(0x7f45e00a03b0 legacy=0x7f45e012f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.681+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 shutdown_connections 2024-09-15T03:44:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.681+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 >> 172.21.15.122:0/263468554 conn(0x7f45e00a3800 msgr2=0x7f45e00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.681+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 shutdown_connections 2024-09-15T03:44:48.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T03:44:48.681+0000 7f45e6302700 1 -- 172.21.15.122:0/263468554 wait complete. 2024-09-15T03:44:48.697 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T03:44:48.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T03:44:48.701 INFO:tasks.ceph.osd.0:Started 2024-09-15T03:44:48.702 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T03:44:48.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T03:44:48.704 INFO:tasks.ceph.osd.1:Started 2024-09-15T03:44:48.704 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T03:44:48.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T03:44:48.707 INFO:tasks.ceph.osd.2:Started 2024-09-15T03:44:48.708 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T03:44:48.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T03:44:48.710 INFO:tasks.ceph.osd.3:Started 2024-09-15T03:44:48.710 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T03:44:48.710 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T03:44:48.712 INFO:tasks.ceph.osd.4:Started 2024-09-15T03:44:48.712 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T03:44:48.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T03:44:48.714 INFO:tasks.ceph.osd.5:Started 2024-09-15T03:44:48.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:44:48.801 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.797+0000 7fe519da4000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.816+0000 7f350d1fa700 10 monclient: get_monmap_and_config 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.816+0000 7f350d1fa700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.816+0000 7f350d1fa700 10 monclient: monmap: 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:48.823277+0000 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:48.823277+0000 2024-09-15T03:44:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 Processor -- start 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- start start 2024-09-15T03:44:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient: init 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient: _add_conns ranks=[1,7,2,3,6,4,0,5,8] 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient(hunting): picked mon.noname-f con 0x7f3508097a60 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient(hunting): picked mon.noname-i con 0x7f3508097e30 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient(hunting): picked mon.noname-b con 0x7f35081391c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350804ca10 con 0x7f3508097a60 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3508133990 con 0x7f35081391c0 2024-09-15T03:44:48.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350809ca70 con 0x7f3508097e30 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient(hunting): authenticate will time out at 833.253553s 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350759e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f35081391c0 0x7f350813b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54054/0 (socket says 172.21.15.26:54054) 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350759e700 1 -- 172.21.15.26:0/2312236366 learned_addr learned my addr 172.21.15.26:0/2312236366 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350659c700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f3508097e30 0x7f3508098200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:47216/0 (socket says 172.21.15.26:47216) 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126296411 0 0) 0x7f3508133990 con 0x7f35081391c0 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): my global_id is 4196 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e8003570 con 0x7f35081391c0 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009029285 0 0) 0x7f350804ca10 con 0x7f3508097a60 2024-09-15T03:44:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): my global_id is 4150 2024-09-15T03:44:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3508133990 con 0x7f3508097a60 2024-09-15T03:44:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764310001 0 0) 0x7f350809ca70 con 0x7f3508097e30 2024-09-15T03:44:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient(hunting): my global_id is 4210 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f350804ca10 con 0x7f3508097e30 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185004890 0 0) 0x7f34e8003570 con 0x7f35081391c0 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 >> v1:172.21.15.122:6792/0 conn(0x7f3508097e30 legacy=0x7f3508098200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 >> v1:172.21.15.122:6789/0 conn(0x7f3508097a60 legacy=0x7f3508133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 monclient: found mon.noname-b 2024-09-15T03:44:48.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350809ccf0 con 0x7f35081391c0 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34f80017b0 con 0x7f35081391c0 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 5 monclient: authenticate success, global_id 4196 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: dump: 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:48.840 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34f8001a10 con 0x7f35081391c0 2024-09-15T03:44:48.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 4 set_mon_vals no callback set 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 1 -- 172.21.15.26:0/2312236366 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34f8004c30 con 0x7f35081391c0 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f3505d9b700 10 monclient: dump: 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:48.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 10 monclient: shutdown 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- 172.21.15.26:0/2312236366 >> v1:172.21.15.26:6790/0 conn(0x7f35081391c0 legacy=0x7f350813b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- 172.21.15.26:0/2312236366 shutdown_connections 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- 172.21.15.26:0/2312236366 >> 172.21.15.26:0/2312236366 conn(0x7f3508067800 msgr2=0x7f3508063ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- 172.21.15.26:0/2312236366 shutdown_connections 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.820+0000 7f350d1fa700 1 -- 172.21.15.26:0/2312236366 wait complete. 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: build_initial_monmap 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: monmap: 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:44:48.844 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:48.845 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 Processor -- start 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 -- start start 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: init 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: _add_conns ranks=[1,5,2,7,3,6,4,0,8] 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient(hunting): picked mon.noname-f con 0x7f3508097a60 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient(hunting): picked mon.noname-h con 0x7f3508097e30 addr v1:172.21.15.122:6791/0 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient(hunting): picked mon.noname-b con 0x7f35081391c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:48.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3508134570 con 0x7f3508097a60 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3508137600 con 0x7f35081391c0 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35081378e0 con 0x7f3508097e30 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient(hunting): authenticate will time out at 833.257175s 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350759e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f35081391c0 0x7f3508149dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54058/0 (socket says 172.21.15.26:54058) 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350759e700 1 -- 172.21.15.26:0/3793421531 learned_addr learned my addr 172.21.15.26:0/3793421531 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245085577 0 0) 0x7f3508137600 con 0x7f35081391c0 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): my global_id is 4205 2024-09-15T03:44:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34ec003570 con 0x7f35081391c0 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425254952 0 0) 0x7f3508134570 con 0x7f3508097a60 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): my global_id is 4159 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3508137600 con 0x7f3508097a60 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2347469122 0 0) 0x7f34ec003570 con 0x7f35081391c0 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3508134570 con 0x7f35081391c0 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34f8001770 con 0x7f35081391c0 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): dump: 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:48.848 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.849 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922818725 0 0) 0x7f35081378e0 con 0x7f3508097e30 2024-09-15T03:44:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient(hunting): my global_id is 4163 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34ec003570 con 0x7f3508097e30 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 193813407 0 0) 0x7f3508134570 con 0x7f35081391c0 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 >> v1:172.21.15.122:6791/0 conn(0x7f3508097e30 legacy=0x7f35081349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 >> v1:172.21.15.122:6789/0 conn(0x7f3508097a60 legacy=0x7f35081382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: _finish_hunting 0 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 monclient: found mon.b 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508137bc0 con 0x7f35081391c0 2024-09-15T03:44:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: _finish_auth 0 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 5 monclient: authenticate success, global_id 4205 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: _renew_subs 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35081367f0 con 0x7f35081391c0 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: _renew_subs 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35081350b0 con 0x7f35081391c0 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34f8004be0 con 0x7f35081391c0 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:48.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34f8005110 con 0x7f35081391c0 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f350559a700 4 set_mon_vals no callback set 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 10 monclient: dump: 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.853 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f34f8005570 con 0x7f35081391c0 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.824+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (unknown 2693500699 0 0) 0x7f34f8006970 con 0x7f35081391c0 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.828+0000 7f350d1fa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.828+0000 7f350d1fa700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.828+0000 7f350d1fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.828+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00050b0 con 0x7f35081391c0 2024-09-15T03:44:48.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.832+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f34f8006b30 con 0x7f35081391c0 2024-09-15T03:44:48.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.832+0000 7f34feffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:48.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.832+0000 7f34feffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:48.864 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.861+0000 7fe519da4000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T03:44:48.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.896+0000 7f350d1fa700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:48.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:48.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:48.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34e0005360 con 0x7f35081391c0 2024-09-15T03:44:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f34feffd700 1 -- 172.21.15.26:0/3793421531 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5560 (unknown 150273500 0 1884421512) 0x7f34f8005b10 con 0x7f35081391c0 2024-09-15T03:44:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f34feffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f34feffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:48.907 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:44:48.907 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T03:44:48.670274+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:44:48.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 10 monclient: shutdown 2024-09-15T03:44:48.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 >> v1:172.21.15.26:6790/0 conn(0x7f35081391c0 legacy=0x7f3508149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 shutdown_connections 2024-09-15T03:44:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 >> 172.21.15.26:0/3793421531 conn(0x7f3508067800 msgr2=0x7f35080a05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 shutdown_connections 2024-09-15T03:44:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:48.900+0000 7f350d1fa700 1 -- 172.21.15.26:0/3793421531 wait complete. 2024-09-15T03:44:48.924 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T03:44:48.924 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T03:44:48.925 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:48.921+0000 7fe519da4000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T03:44:49.303 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:49.301+0000 7fe519da4000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T03:44:49.309 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-15T03:44:49.305+0000 7fdd8d5e1540 -1 Falling back to public interface 2024-09-15T03:44:49.310 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-15T03:44:49.304+0000 7fb7141ae540 -1 Falling back to public interface 2024-09-15T03:44:49.312 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T03:44:49.309+0000 7f8eb6d94540 -1 Falling back to public interface 2024-09-15T03:44:49.323 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-09-15T03:44:49.316+0000 7f522b9dc540 -1 Falling back to public interface 2024-09-15T03:44:49.323 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T03:44:49.321+0000 7f126ba6a540 -1 Falling back to public interface 2024-09-15T03:44:49.359 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-09-15T03:44:49.352+0000 7f54db35c540 -1 Falling back to public interface 2024-09-15T03:44:49.417 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:49.412+0000 7fe519da4000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T03:44:49.529 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T03:44:49.631 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.624+0000 7f9aba1e5700 10 monclient: get_monmap_and_config 2024-09-15T03:44:49.631 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.624+0000 7f9aba1e5700 10 monclient: build_initial_monmap 2024-09-15T03:44:49.631 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.624+0000 7f9aba1e5700 10 monclient: monmap: 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:epoch 0 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:49.630528+0000 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:49.630528+0000 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:49.632 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:49.633 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:49.633 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:49.633 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:49.633 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:49.638 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.624+0000 7f9aba1e5700 1 Processor -- start 2024-09-15T03:44:49.638 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.624+0000 7f9aba1e5700 1 -- start start 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.624+0000 7f9aba1e5700 10 monclient: init 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: _add_conns ranks=[8,1,7,4,6,5,0,3,2] 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient(hunting): picked mon.noname-e con 0x7f9ab4099350 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient(hunting): picked mon.noname-f con 0x7f9ab4099720 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient(hunting): picked mon.noname-i con 0x7f9ab4099af0 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab404c150 con 0x7f9ab4099720 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab4133ad0 con 0x7f9ab4099af0 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40a0f70 con 0x7f9ab4099350 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:49.639 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient(hunting): authenticate will time out at 834.060167s 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9ab91e3700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f9ab4099350 0x7f9ab41333c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:59034/0 (socket says 172.21.15.26:59034) 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9ab91e3700 1 -- 172.21.15.26:0/1062112483 learned_addr learned my addr 172.21.15.26:0/1062112483 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578244964 0 0) 0x7f9ab40a0f70 con 0x7f9ab4099350 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): my global_id is 4166 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa0003570 con 0x7f9ab4099350 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676449843 0 0) 0x7f9ab4133ad0 con 0x7f9ab4099af0 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): my global_id is 4219 2024-09-15T03:44:49.640 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab40a0f70 con 0x7f9ab4099af0 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572074581 0 0) 0x7f9ab404c150 con 0x7f9ab4099720 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient(hunting): my global_id is 4168 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab4133ad0 con 0x7f9ab4099720 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030400978 0 0) 0x7f9aa0003570 con 0x7f9ab4099350 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 >> v1:172.21.15.122:6792/0 conn(0x7f9ab4099af0 legacy=0x7f9ab413b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 >> v1:172.21.15.122:6789/0 conn(0x7f9ab4099720 legacy=0x7f9ab4137f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: _finish_hunting 0 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 monclient: found mon.noname-e 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:44:49.641 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40a13b0 con 0x7f9ab4099350 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: _finish_auth 0 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 5 monclient: authenticate success, global_id 4166 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9aac001b30 con 0x7f9ab4099350 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: dump: 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.642 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:49.643 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9aac001e10 con 0x7f9ab4099350 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 4 set_mon_vals no callback set 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 1 -- 172.21.15.26:0/1062112483 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9aac004fb0 con 0x7f9ab4099350 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aabfff700 10 monclient: dump: 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.644 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:49.645 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: shutdown 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- 172.21.15.26:0/1062112483 >> v1:172.21.15.26:6793/0 conn(0x7f9ab4099350 legacy=0x7f9ab41333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- 172.21.15.26:0/1062112483 shutdown_connections 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- 172.21.15.26:0/1062112483 >> 172.21.15.26:0/1062112483 conn(0x7f9ab40a6bb0 msgr2=0x7f9ab40a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- 172.21.15.26:0/1062112483 shutdown_connections 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- 172.21.15.26:0/1062112483 wait complete. 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: build_initial_monmap 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: monmap: 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:epoch 0 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:last_changed 0.000000 2024-09-15T03:44:49.646 INFO:teuthology.misc.health.smithi026.stderr:created 0.000000 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 1 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:49.647 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 Processor -- start 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 1 -- start start 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.628+0000 7f9aba1e5700 10 monclient: init 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _add_conns ranks=[7,0,8,2,6,3,1,4,5] 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient(hunting): picked mon.noname-i con 0x7f9ab4099350 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient(hunting): picked mon.noname-a con 0x7f9ab4099720 addr v1:172.21.15.26:6789/0 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient(hunting): picked mon.noname-e con 0x7f9ab4099af0 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40a1da0 con 0x7f9ab4099720 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab411e6c0 con 0x7f9ab4099350 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40a18e0 con 0x7f9ab4099af0 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient(hunting): authenticate will time out at 834.064172s 2024-09-15T03:44:49.648 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9ab89e2700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ab4099720 0x7f9ab4130060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58834/0 (socket says 172.21.15.26:58834) 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9ab99e4700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f9ab4099af0 0x7f9ab414a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:59044/0 (socket says 172.21.15.26:59044) 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9ab89e2700 1 -- 172.21.15.26:0/3985992318 learned_addr learned my addr 172.21.15.26:0/3985992318 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512751940 0 0) 0x7f9ab40a18e0 con 0x7f9ab4099af0 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): my global_id is 4175 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a98003570 con 0x7f9ab4099af0 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868859989 0 0) 0x7f9ab4133ad0 con 0x7f9ab4099720 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): my global_id is 4140 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab40a18e0 con 0x7f9ab4099720 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706105328 0 0) 0x7f9ab411e6c0 con 0x7f9ab4099350 2024-09-15T03:44:49.649 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): my global_id is 4228 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab4133ad0 con 0x7f9ab4099350 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 153977864 0 0) 0x7f9a98003570 con 0x7f9ab4099af0 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ab411e6c0 con 0x7f9ab4099af0 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ab0001980 con 0x7f9ab4099af0 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): dump: 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.650 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:49.651 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2802092924 0 0) 0x7f9ab411e6c0 con 0x7f9ab4099af0 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 >> v1:172.21.15.122:6792/0 conn(0x7f9ab4099350 legacy=0x7f9ab4141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 >> v1:172.21.15.26:6789/0 conn(0x7f9ab4099720 legacy=0x7f9ab4130060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: _finish_hunting 0 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 monclient: found mon.e 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab41201d0 con 0x7f9ab4099af0 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: _finish_auth 0 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 5 monclient: authenticate success, global_id 4175 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _renew_subs 2024-09-15T03:44:49.652 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab41204b0 con 0x7f9ab4099af0 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _renew_subs 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab411f270 con 0x7f9ab4099af0 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9ab0001ea0 con 0x7f9ab4099af0 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ab0002db0 con 0x7f9ab4099af0 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aab7fe700 4 set_mon_vals no callback set 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 10 monclient: dump: 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:49.653 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:49.654 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:49.655 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:49.655 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f9ab00026d0 con 0x7f9ab4099af0 2024-09-15T03:44:49.655 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (unknown 2693500699 0 0) 0x7f9ab0006510 con 0x7f9ab4099af0 2024-09-15T03:44:49.655 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:49.655 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:49.656 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:49.656 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.632+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a940050b0 con 0x7f9ab4099af0 2024-09-15T03:44:49.656 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.636+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f9ab0005810 con 0x7f9ab4099af0 2024-09-15T03:44:49.656 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.636+0000 7f9aa97fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:49.656 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.636+0000 7f9aa97fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:49.709 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:49.710 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:49.710 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:49.710 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a94005360 con 0x7f9ab4099af0 2024-09-15T03:44:49.710 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:49.704+0000 7fe519da4000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T03:44:49.711 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aa97fa700 1 -- 172.21.15.26:0/3985992318 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5560 (unknown 150273500 0 1884421512) 0x7f9ab0005f20 con 0x7f9ab4099af0 2024-09-15T03:44:49.711 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aa97fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:49.711 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aa97fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:49.711 INFO:teuthology.misc.health.smithi026.stdout: 2024-09-15T03:44:49.711 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":8,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T03:44:48.670274+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:44:49.712 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 10 monclient: shutdown 2024-09-15T03:44:49.712 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:49.712 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 >> v1:172.21.15.26:6793/0 conn(0x7f9ab4099af0 legacy=0x7f9ab414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:49.712 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 shutdown_connections 2024-09-15T03:44:49.712 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 >> 172.21.15.26:0/3985992318 conn(0x7f9ab40a6bb0 msgr2=0x7f9ab4067a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:49.712 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 shutdown_connections 2024-09-15T03:44:49.713 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:49.704+0000 7f9aba1e5700 1 -- 172.21.15.26:0/3985992318 wait complete. 2024-09-15T03:44:49.722 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-15T03:44:49.874 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:49.868+0000 7fe519da4000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T03:44:49.941 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:49.936+0000 7fe519da4000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T03:44:49.989 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:49.984+0000 7fe519da4000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T03:44:50.041 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:50.036+0000 7fe519da4000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T03:44:50.200 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:50.196+0000 7fe519da4000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T03:44:50.376 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:50.372+0000 7fe519da4000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T03:44:50.447 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:50.444+0000 7fe519da4000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T03:44:50.623 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:50.620+0000 7fe519da4000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T03:44:50.695 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:50.692+0000 7fe519da4000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T03:44:51.423 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:51.420+0000 7fe519da4000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T03:44:52.465 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.460+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T03:44:52.467 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T03:44:52.468 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.g 2024-09-15T03:44:52.468 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.i 2024-09-15T03:44:52.468 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.e 2024-09-15T03:44:52.469 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.d 2024-09-15T03:44:52.469 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.h 2024-09-15T03:44:52.469 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T03:44:52.464+0000 7fe502e4f700 -1 mgr.server handle_report got status from non-daemon mon.f 2024-09-15T03:44:52.908 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T03:44:52.904+0000 7f8eb6d94540 -1 osd.5 0 log_to_monitors true 2024-09-15T03:44:52.982 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T03:44:52.976+0000 7f126ba6a540 -1 osd.4 0 log_to_monitors true 2024-09-15T03:44:53.073 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-15T03:44:53.068+0000 7fb7141ae540 -1 osd.1 0 log_to_monitors true 2024-09-15T03:44:53.085 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-15T03:44:53.080+0000 7fdd8d5e1540 -1 osd.3 0 log_to_monitors true 2024-09-15T03:44:53.136 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-09-15T03:44:53.128+0000 7f522b9dc540 -1 osd.0 0 log_to_monitors true 2024-09-15T03:44:53.200 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-09-15T03:44:53.192+0000 7f54db35c540 -1 osd.2 0 log_to_monitors true 2024-09-15T03:44:54.536 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-15T03:44:54.528+0000 7fb70f94e700 -1 osd.1 0 waiting for initial osdmap 2024-09-15T03:44:54.536 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-09-15T03:44:54.528+0000 7f52289a1700 -1 osd.0 0 waiting for initial osdmap 2024-09-15T03:44:54.539 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-15T03:44:54.536+0000 7fdd88d81700 -1 osd.3 0 waiting for initial osdmap 2024-09-15T03:44:54.542 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-09-15T03:44:54.536+0000 7fb7086f1700 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:44:54.542 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-09-15T03:44:54.536+0000 7f5220720700 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:44:54.544 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-09-15T03:44:54.540+0000 7f54d8321700 -1 osd.2 0 waiting for initial osdmap 2024-09-15T03:44:54.545 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T03:44:54.540+0000 7f1268a2f700 -1 osd.4 0 waiting for initial osdmap 2024-09-15T03:44:54.545 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T03:44:54.540+0000 7f8eb2534700 -1 osd.5 0 waiting for initial osdmap 2024-09-15T03:44:54.549 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T03:44:54.544+0000 7f8eab2d7700 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:44:54.550 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-15T03:44:54.544+0000 7fdd81b24700 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:44:54.550 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-09-15T03:44:54.544+0000 7f54d00a0700 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:44:54.550 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T03:44:54.544+0000 7f125ffad700 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T03:44:56.326 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: get_monmap_and_config 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: build_initial_monmap 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: monmap: 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:epoch 0 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:56.427967+0000 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:56.427967+0000 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.429 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:56.430 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.434 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 1 Processor -- start 2024-09-15T03:44:56.434 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 1 -- start start 2024-09-15T03:44:56.434 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: init 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: _add_conns ranks=[7,4,6,1,5,3,0,8,2] 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient(hunting): picked mon.noname-i con 0x7ff48c0a1d00 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient(hunting): picked mon.noname-c con 0x7ff48c1333f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient(hunting): picked mon.noname-d con 0x7ff48c133820 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c05f2e0 con 0x7ff48c1333f0 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0a73f0 con 0x7ff48c133820 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0a20d0 con 0x7ff48c0a1d00 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient(hunting): authenticate will time out at 840.857510s 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff490fe4700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ff48c133820 0x7ff48c13b740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:56262/0 (socket says 172.21.15.26:56262) 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48b7fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7ff48c1333f0 0x7ff48c138010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35636/0 (socket says 172.21.15.26:35636) 2024-09-15T03:44:56.435 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff490fe4700 1 -- 172.21.15.26:0/511651552 learned_addr learned my addr 172.21.15.26:0/511651552 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725455694 0 0) 0x7ff48c0a73f0 con 0x7ff48c133820 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): my global_id is 4191 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff46c003570 con 0x7ff48c133820 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452106811 0 0) 0x7ff48c0a20d0 con 0x7ff48c0a1d00 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): my global_id is 4309 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0a73f0 con 0x7ff48c0a1d00 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841785234 0 0) 0x7ff48c05f2e0 con 0x7ff48c1333f0 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.436 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient(hunting): my global_id is 4180 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0a20d0 con 0x7ff48c1333f0 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164763769 0 0) 0x7ff48c0a20d0 con 0x7ff48c1333f0 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 >> v1:172.21.15.122:6792/0 conn(0x7ff48c0a1d00 legacy=0x7ff48c132ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 >> v1:172.21.15.26:6792/0 conn(0x7ff48c133820 legacy=0x7ff48c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: _finish_hunting 0 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 monclient: found mon.noname-c 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0a13f0 con 0x7ff48c1333f0 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: _finish_auth 0 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:56.437 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 5 monclient: authenticate success, global_id 4180 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff484001750 con 0x7ff48c1333f0 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: dump: 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.438 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff4840019b0 con 0x7ff48c1333f0 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 1 -- 172.21.15.26:0/511651552 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff484004830 con 0x7ff48c1333f0 2024-09-15T03:44:56.439 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff492a47700 4 set_mon_vals no callback set 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.424+0000 7ff48affd700 10 monclient: dump: 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.440 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient: shutdown 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- 172.21.15.26:0/511651552 >> v1:172.21.15.26:6791/0 conn(0x7ff48c1333f0 legacy=0x7ff48c138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- 172.21.15.26:0/511651552 shutdown_connections 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- 172.21.15.26:0/511651552 >> 172.21.15.26:0/511651552 conn(0x7ff48c064580 msgr2=0x7ff48c0a64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- 172.21.15.26:0/511651552 shutdown_connections 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- 172.21.15.26:0/511651552 wait complete. 2024-09-15T03:44:56.441 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient: build_initial_monmap 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient: monmap: 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:epoch 0 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:last_changed 0.000000 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:created 0.000000 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 1 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:56.442 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 Processor -- start 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- start start 2024-09-15T03:44:56.443 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient: init 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient: _add_conns ranks=[0,8,6,5,4,1,3,2,7] 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient(hunting): picked mon.noname-a con 0x7ff48c0a1d00 addr v1:172.21.15.26:6789/0 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient(hunting): picked mon.noname-e con 0x7ff48c1333f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient(hunting): picked mon.noname-d con 0x7ff48c133820 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c099a80 con 0x7ff48c0a1d00 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c120060 con 0x7ff48c133820 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c142af0 con 0x7ff48c1333f0 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff492a47700 10 monclient(hunting): authenticate will time out at 840.862805s 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff48b7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7ff48c1333f0 0x7ff48c1423e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:59074/0 (socket says 172.21.15.26:59074) 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff48bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff48c0a1d00 0x7ff48c141cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58846/0 (socket says 172.21.15.26:58846) 2024-09-15T03:44:56.447 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff48b7fe700 1 -- 172.21.15.26:0/3874435882 learned_addr learned my addr 172.21.15.26:0/3874435882 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268349006 0 0) 0x7ff48c120060 con 0x7ff48c133820 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): my global_id is 4200 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff470003570 con 0x7ff48c133820 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810556056 0 0) 0x7ff48c142af0 con 0x7ff48c1333f0 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): my global_id is 4220 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c120060 con 0x7ff48c1333f0 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926249630 0 0) 0x7ff48c099a80 con 0x7ff48c0a1d00 2024-09-15T03:44:56.448 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 10 monclient(hunting): my global_id is 4158 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.428+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c142af0 con 0x7ff48c0a1d00 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2287235322 0 0) 0x7ff48c120060 con 0x7ff48c1333f0 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff48c099a80 con 0x7ff48c1333f0 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1457649020 0 0) 0x7ff48c142af0 con 0x7ff48c0a1d00 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff48c120060 con 0x7ff48c0a1d00 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff484001710 con 0x7ff48c1333f0 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): dump: 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:56.449 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.450 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 247742114 0 0) 0x7ff470003570 con 0x7ff48c133820 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff48c142af0 con 0x7ff48c133820 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff474001a50 con 0x7ff48c0a1d00 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): dump: 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.451 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff47c001c10 con 0x7ff48c133820 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.452 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient(hunting): dump: 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.453 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 380269358 0 0) 0x7ff48c099a80 con 0x7ff48c1333f0 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 >> v1:172.21.15.26:6792/0 conn(0x7ff48c133820 legacy=0x7ff48c149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 >> v1:172.21.15.26:6789/0 conn(0x7ff48c0a1d00 legacy=0x7ff48c141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: _finish_hunting 0 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 monclient: found mon.e 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c122e70 con 0x7ff48c1333f0 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: _finish_auth 0 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:56.454 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 5 monclient: authenticate success, global_id 4220 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 monclient: _renew_subs 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0a2410 con 0x7ff48c1333f0 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 monclient: _renew_subs 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c121b20 con 0x7ff48c1333f0 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff484004ea0 con 0x7ff48c1333f0 2024-09-15T03:44:56.455 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff484005350 con 0x7ff48c1333f0 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff48a7fc700 4 set_mon_vals no callback set 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 10 monclient: dump: 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:epoch 1 2024-09-15T03:44:56.456 INFO:teuthology.misc.health.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.457 INFO:teuthology.misc.health.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr: 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4640050b0 con 0x7ff48c1333f0 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 4) v1 ==== 43036+0+0 (unknown 1105270920 0 0) 0x7ff484005530 con 0x7ff48c1333f0 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(11..11 src has 1..11) v4 ==== 2763+0+0 (unknown 3748725937 0 0) 0x7ff484043c50 con 0x7ff48c1333f0 2024-09-15T03:44:56.458 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.432+0000 7ff48bfff700 10 monclient: get_auth_request con 0x7ff4700380f0 auth_method 0 2024-09-15T03:44:56.459 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.440+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ff484014d40 con 0x7ff48c1333f0 2024-09-15T03:44:56.459 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.440+0000 7ff483fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:56.459 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.440+0000 7ff483fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:56.624 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.616+0000 7ff492a47700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:56.624 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.616+0000 7ff492a47700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:56.624 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.616+0000 7ff492a47700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:44:56.624 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.616+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff464005360 con 0x7ff48c1333f0 2024-09-15T03:44:56.624 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff483fff700 1 -- 172.21.15.26:0/3874435882 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+6910 (unknown 4250787923 0 713541818) 0x7ff484014d40 con 0x7ff48c1333f0 2024-09-15T03:44:56.625 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff483fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:44:56.625 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff483fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:44:56.625 INFO:teuthology.misc.health.smithi026.stdout: 2024-09-15T03:44:56.625 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":12,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T03:44:56.553727+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:44:56.627 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 >> v1:172.21.15.122:6812/20895 conn(0x7ff4700380f0 legacy=0x7ff47003a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.627 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 10 monclient: shutdown 2024-09-15T03:44:56.627 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:56.627 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 >> v1:172.21.15.26:6793/0 conn(0x7ff48c1333f0 legacy=0x7ff48c1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.628 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 shutdown_connections 2024-09-15T03:44:56.628 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 >> 172.21.15.26:0/3874435882 conn(0x7ff48c064580 msgr2=0x7ff48c09fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:56.628 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 shutdown_connections 2024-09-15T03:44:56.628 INFO:teuthology.misc.health.smithi026.stderr:2024-09-15T03:44:56.620+0000 7ff492a47700 1 -- 172.21.15.26:0/3874435882 wait complete. 2024-09-15T03:44:56.637 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-15T03:44:56.637 INFO:tasks.ceph:Creating RBD pool 2024-09-15T03:44:56.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.740+0000 7fc715a89700 10 monclient: get_monmap_and_config 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.740+0000 7fc715a89700 10 monclient: build_initial_monmap 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.740+0000 7fc715a89700 10 monclient: monmap: 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:56.746873+0000 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:56.746873+0000 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:56.749 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 1 Processor -- start 2024-09-15T03:44:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 1 -- start start 2024-09-15T03:44:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient: init 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient: _add_conns ranks=[3,7,1,2,5,0,4,8,6] 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient(hunting): picked mon.noname-g con 0x7fc7100990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient(hunting): picked mon.noname-i con 0x7fc7100a1310 addr v1:172.21.15.122:6792/0 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient(hunting): picked mon.noname-f con 0x7fc7101335a0 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71004ca10 con 0x7fc7101335a0 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7100a1b20 con 0x7fc7100990c0 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7100a10b0 con 0x7fc7100a1310 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient(hunting): authenticate will time out at 841.177285s 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc707fff700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fc7100a1310 0x7fc710099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:55854/0 (socket says 172.21.15.26:55854) 2024-09-15T03:44:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc707fff700 1 -- 172.21.15.26:0/1557447982 learned_addr learned my addr 172.21.15.26:0/1557447982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70ffff700 1 --1- 172.21.15.26:0/1557447982 >> v1:172.21.15.122:6789/0 conn(0x7fc7101335a0 0x7fc71013afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:35956/0 (socket says 172.21.15.26:35956) 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140852163 0 0) 0x7fc7100a10b0 con 0x7fc7100a1310 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): my global_id is 4318 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6fc003570 con 0x7fc7100a1310 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213772614 0 0) 0x7fc7100a1b20 con 0x7fc7100990c0 2024-09-15T03:44:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): my global_id is 4242 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7100a10b0 con 0x7fc7100990c0 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146266756 0 0) 0x7fc71004ca10 con 0x7fc7101335a0 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient(hunting): my global_id is 4249 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7100a1b20 con 0x7fc7101335a0 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4154309688 0 0) 0x7fc6fc003570 con 0x7fc7100a1310 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 >> v1:172.21.15.122:6790/0 conn(0x7fc7100990c0 legacy=0x7fc710132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 >> v1:172.21.15.122:6789/0 conn(0x7fc7101335a0 legacy=0x7fc71013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: _finish_hunting 0 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 monclient: found mon.noname-i 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710099ba0 con 0x7fc7100a1310 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: _finish_auth 0 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 5 monclient: authenticate success, global_id 4318 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc6f80017b0 con 0x7fc7100a1310 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:44:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: dump: 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc6f8001a90 con 0x7fc7100a1310 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 10 monclient: get_monmap_and_config success 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc715a89700 4 set_mon_vals no callback set 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 1 -- 172.21.15.26:0/1557447982 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc6f8004c30 con 0x7fc7100a1310 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.744+0000 7fc70effd700 10 monclient: dump: 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient: shutdown 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- 172.21.15.26:0/1557447982 >> v1:172.21.15.122:6792/0 conn(0x7fc7100a1310 legacy=0x7fc710099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- 172.21.15.26:0/1557447982 shutdown_connections 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- 172.21.15.26:0/1557447982 >> 172.21.15.26:0/1557447982 conn(0x7fc71006e370 msgr2=0x7fc7100a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- 172.21.15.26:0/1557447982 shutdown_connections 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- 172.21.15.26:0/1557447982 wait complete. 2024-09-15T03:44:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient: build_initial_monmap 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient: monmap: 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 Processor -- start 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- start start 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient: init 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient: _reopen_session rank -1 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient: _add_conns ranks=[8,6,4,5,3,0,1,7,2] 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient(hunting): picked mon.noname-e con 0x7fc7100990c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient(hunting): picked mon.noname-d con 0x7fc7100a1310 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient(hunting): picked mon.noname-c con 0x7fc7101335a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:56.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71009a010 con 0x7fc7101335a0 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7100a1ce0 con 0x7fc7100a1310 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc710120cf0 con 0x7fc7100990c0 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient(hunting): _renew_subs 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc715a89700 10 monclient(hunting): authenticate will time out at 841.182879s 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc70ffff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fc7101335a0 0x7fc71014a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35748/0 (socket says 172.21.15.26:35748) 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc70f7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fc7100990c0 0x7fc710131b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48722/0 (socket says 172.21.15.26:48722) 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc707fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fc7100a1310 0x7fc7101470b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54944/0 (socket says 172.21.15.26:54944) 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc70f7fe700 1 -- 172.21.15.26:0/994473262 learned_addr learned my addr 172.21.15.26:0/994473262 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626256827 0 0) 0x7fc710120cf0 con 0x7fc7100990c0 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): my global_id is 4229 2024-09-15T03:44:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f0003570 con 0x7fc7100990c0 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669082247 0 0) 0x7fc7100a1ce0 con 0x7fc7100a1310 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): my global_id is 4209 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc710120cf0 con 0x7fc7100a1310 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437517965 0 0) 0x7fc71009a010 con 0x7fc7101335a0 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 10 monclient(hunting): my global_id is 4189 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.748+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7100a1ce0 con 0x7fc7101335a0 2024-09-15T03:44:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2930562797 0 0) 0x7fc710120cf0 con 0x7fc7100a1310 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc71009a010 con 0x7fc7100a1310 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc6f8004b70 con 0x7fc7100a1310 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient(hunting): dump: 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.770 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:56.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2653594504 0 0) 0x7fc71009a010 con 0x7fc7100a1310 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 >> v1:172.21.15.26:6793/0 conn(0x7fc7100990c0 legacy=0x7fc710131b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 >> v1:172.21.15.26:6791/0 conn(0x7fc7101335a0 legacy=0x7fc71014a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: _finish_hunting 0 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 monclient: found mon.d 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710120870 con 0x7fc7100a1310 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: _finish_auth 0 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 5 monclient: authenticate success, global_id 4209 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 monclient: _renew_subs 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:44:56.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc710121c50 con 0x7fc7100a1310 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc6f8004f00 con 0x7fc7100a1310 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc6f80053b0 con 0x7fc7100a1310 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 monclient: _renew_subs 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc710134220 con 0x7fc7100a1310 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc70e7fc700 4 set_mon_vals no callback set 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 10 monclient: dump: 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:56.774 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:44:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71004bbf0 con 0x7fc7100a1310 2024-09-15T03:44:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 4) v1 ==== 43036+0+0 (unknown 1105270920 0 0) 0x7fc6f800fd40 con 0x7fc7100a1310 2024-09-15T03:44:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(12..12 src has 1..12) v4 ==== 2763+0+0 (unknown 1004204906 0 0) 0x7fc6f8043b20 con 0x7fc7100a1310 2024-09-15T03:44:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.752+0000 7fc70f7fe700 10 monclient: get_auth_request con 0x7fc6f0037580 auth_method 0 2024-09-15T03:44:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.760+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fc6f8014d00 con 0x7fc7100a1310 2024-09-15T03:44:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.760+0000 7fc7077fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:44:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.760+0000 7fc7077fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:44:56.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.928+0000 7fc715a89700 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T03:44:56.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.928+0000 7fc715a89700 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T03:44:56.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.928+0000 7fc715a89700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:44:56.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:56.928+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc71006fac0 con 0x7fc7100a1310 2024-09-15T03:44:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.584+0000 7fc7077fe700 1 -- 172.21.15.26:0/994473262 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (unknown 2984082091 0 0) 0x7fc6f8014d00 con 0x7fc7100a1310 2024-09-15T03:44:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.584+0000 7fc7077fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-09-15T03:44:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.584+0000 7fc7077fe700 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 >> v1:172.21.15.122:6812/20895 conn(0x7fc6f0037580 legacy=0x7fc6f0039a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 10 monclient: shutdown 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 >> v1:172.21.15.26:6792/0 conn(0x7fc7100a1310 legacy=0x7fc7101470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 shutdown_connections 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 >> 172.21.15.26:0/994473262 conn(0x7fc71006e370 msgr2=0x7fc7100a4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 shutdown_connections 2024-09-15T03:44:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.588+0000 7fc715a89700 1 -- 172.21.15.26:0/994473262 wait complete. 2024-09-15T03:44:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'rbd' created 2024-09-15T03:44:57.605 DEBUG:teuthology.orchestra.run.smithi026:> rbd --cluster ceph pool init rbd 2024-09-15T03:44:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.636+0000 7f06789334c0 10 monclient: get_monmap_and_config 2024-09-15T03:44:57.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.636+0000 7f06789334c0 10 monclient: build_initial_monmap 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.636+0000 7f06789334c0 10 monclient: monmap: 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:57.643627+0000 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:57.643627+0000 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:57.645 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:57.646 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 1 Processor -- start 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 1 -- start start 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient: init 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient: _reopen_session rank -1 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient: _add_conns ranks=[4,1,6,0,2,5,3,7,8] 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient(hunting): picked mon.noname-c con 0x55f101daf430 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient(hunting): picked mon.noname-f con 0x55f101d95d10 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient(hunting): picked mon.noname-d con 0x55f101d960e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55f101cd1660 con 0x55f101d95d10 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55f101caa7f0 con 0x55f101daf430 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55f101cb7660 con 0x55f101d960e0 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient(hunting): _renew_subs 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient(hunting): authenticate will time out at 842.073203s 2024-09-15T03:44:57.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0677469700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x55f101d960e0 0x55f101e4f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54956/0 (socket says 172.21.15.26:54956) 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0677469700 1 -- 172.21.15.26:0/4286764128 learned_addr learned my addr 172.21.15.26:0/4286764128 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0676c68700 1 --1- 172.21.15.26:0/4286764128 >> v1:172.21.15.26:6791/0 conn(0x55f101daf430 0x55f101daf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35764/0 (socket says 172.21.15.26:35764) 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169676929 0 0) 0x55f101cb7660 con 0x55f101d960e0 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): my global_id is 4218 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f065c001e10 con 0x55f101d960e0 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793411658 0 0) 0x55f101caa7f0 con 0x55f101daf430 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): my global_id is 4198 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f101cb7660 con 0x55f101daf430 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901341809 0 0) 0x55f101cd1660 con 0x55f101d95d10 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient(hunting): my global_id is 4258 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f101caa7f0 con 0x55f101d95d10 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3208354351 0 0) 0x7f065c001e10 con 0x55f101d960e0 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 >> v1:172.21.15.26:6791/0 conn(0x55f101daf430 legacy=0x55f101daf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 >> v1:172.21.15.122:6789/0 conn(0x55f101d95d10 legacy=0x55f101e4bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: _finish_hunting 0 2024-09-15T03:44:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 monclient: found mon.noname-d 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f101caab80 con 0x55f101d960e0 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: _finish_auth 0 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 5 monclient: authenticate success, global_id 4218 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0668001b30 con 0x55f101d960e0 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:44:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: dump: 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0668001dc0 con 0x55f101d960e0 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 1 -- 172.21.15.26:0/4286764128 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0668003850 con 0x55f101d960e0 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 10 monclient: get_monmap_and_config success 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f06789334c0 4 set_mon_vals no callback set 2024-09-15T03:44:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.640+0000 7f0675c66700 10 monclient: dump: 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:57.656 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient: shutdown 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- 172.21.15.26:0/4286764128 >> v1:172.21.15.26:6792/0 conn(0x55f101d960e0 legacy=0x55f101e4f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- 172.21.15.26:0/4286764128 shutdown_connections 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- 172.21.15.26:0/4286764128 >> 172.21.15.26:0/4286764128 conn(0x55f101db5c40 msgr2=0x55f101db6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- 172.21.15.26:0/4286764128 shutdown_connections 2024-09-15T03:44:57.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- 172.21.15.26:0/4286764128 wait complete. 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient: build_initial_monmap 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient: monmap: 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:44:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 Processor -- start 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- start start 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient: init 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient: _reopen_session rank -1 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient: _add_conns ranks=[2,4,1,6,5,0,8,3,7] 2024-09-15T03:44:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient(hunting): picked mon.noname-b con 0x55f101d95d10 addr v1:172.21.15.26:6790/0 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient(hunting): picked mon.noname-c con 0x55f101d960e0 addr v1:172.21.15.26:6791/0 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 10 monclient(hunting): picked mon.noname-f con 0x55f101daf430 addr v1:172.21.15.122:6789/0 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.644+0000 7f06789334c0 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55f101caa5e0 con 0x55f101daf430 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55f101e36910 con 0x55f101d95d10 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x55f101e516e0 con 0x55f101d960e0 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 10 monclient(hunting): _renew_subs 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 10 monclient(hunting): authenticate will time out at 842.080075s 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0676467700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x55f101d960e0 0x55f101e5a220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35776/0 (socket says 172.21.15.26:35776) 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0676467700 1 -- 172.21.15.26:0/3621292014 learned_addr learned my addr 172.21.15.26:0/3621292014 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543709594 0 0) 0x55f101e36910 con 0x55f101d95d10 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): my global_id is 4241 2024-09-15T03:44:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0658001e10 con 0x55f101d95d10 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878168475 0 0) 0x55f101e516e0 con 0x55f101d960e0 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): my global_id is 4207 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f101e36910 con 0x55f101d960e0 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320496953 0 0) 0x55f101caa5e0 con 0x55f101daf430 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): my global_id is 4267 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f101e516e0 con 0x55f101daf430 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3297874889 0 0) 0x7f0658001e10 con 0x55f101d95d10 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x55f101caa5e0 con 0x55f101d95d10 2024-09-15T03:44:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f066c0019c0 con 0x55f101d95d10 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): dump: 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3101531865 0 0) 0x55f101e36910 con 0x55f101d960e0 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0658001e10 con 0x55f101d960e0 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0664002af0 con 0x55f101d960e0 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): dump: 2024-09-15T03:44:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1156858984 0 0) 0x55f101e516e0 con 0x55f101daf430 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x55f101e36910 con 0x55f101daf430 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f06680016f0 con 0x55f101daf430 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient(hunting): dump: 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 585834235 0 0) 0x55f101caa5e0 con 0x55f101d95d10 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 >> v1:172.21.15.26:6791/0 conn(0x55f101d960e0 legacy=0x55f101e5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 >> v1:172.21.15.122:6789/0 conn(0x55f101daf430 legacy=0x55f101e5d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:44:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 monclient: found mon.b 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f101e36350 con 0x55f101d95d10 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: _finish_auth 0 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 5 monclient: authenticate success, global_id 4241 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 10 monclient: _renew_subs 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f101e36070 con 0x55f101d95d10 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 10 monclient: _renew_subs 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x55f101e33e70 con 0x55f101d95d10 2024-09-15T03:44:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f066c001d90 con 0x55f101d95d10 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f066c0036c0 con 0x55f101d95d10 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 10 monclient: dump: 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:44:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0675465700 4 set_mon_vals no callback set 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.648+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 4) v1 ==== 43036+0+0 (unknown 1105270920 0 0) 0x7f066c00e0d0 con 0x55f101d95d10 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.652+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(13..13 src has 1..13) v4 ==== 3480+0+0 (unknown 3644392469 0 0) 0x7f066c042450 con 0x55f101d95d10 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.652+0000 7f06789334c0 10 start_mon_command cmd=[{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.652+0000 7f06789334c0 10 monclient: _send_command 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T03:44:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.652+0000 7f06789334c0 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.652+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x55f101e353b0 con 0x55f101d95d10 2024-09-15T03:44:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:57.652+0000 7f0676467700 10 monclient: get_auth_request con 0x7f06580367b0 auth_method 0 2024-09-15T03:44:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.591+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 141+0+0 (unknown 2078359144 0 0) 0x7f066c0130f0 con 0x55f101d95d10 2024-09-15T03:44:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.591+0000 7f0662ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}] 2024-09-15T03:44:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.591+0000 7f0662ffd700 10 monclient: _finish_command 1 = system:0 enabled application 'rbd' on pool 'rbd' 2024-09-15T03:44:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.591+0000 7f06789334c0 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:58.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.591+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x55f101cabd30 con 0x55f101d95d10 2024-09-15T03:44:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (unknown 1566986600 0 0) 0x7f066c0130f0 con 0x55f101d95d10 2024-09-15T03:44:58.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0662ffd700 10 monclient: handle_get_version_reply finishing 1 version 14 2024-09-15T03:44:58.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0675465700 10 monclient: _renew_subs 2024-09-15T03:44:58.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0675465700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:58.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0675465700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=14}) v3 -- 0x7f0650002a70 con 0x55f101d95d10 2024-09-15T03:44:58.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 11 ==== osd_map(14..14 src has 1..14) v4 ==== 1020+0+0 (unknown 4065773812 0 0) 0x7f066c0132b0 con 0x55f101d95d10 2024-09-15T03:44:58.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6800/21130 -- osd_op(unknown.0.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) v8 -- 0x55f101e63080 con 0x55f101e5f580 2024-09-15T03:44:58.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.599+0000 7f0677469700 10 monclient: get_auth_request con 0x55f101e5f580 auth_method 0 2024-09-15T03:44:58.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.607+0000 7f0677469700 1 -- 172.21.15.26:0/3621292014 <== osd.1 v1:172.21.15.26:6800/21130 1 ==== osd_op_reply(1 rbd_trash [create] v14'1 uv1 ondisk = 0) v8 ==== 153+0+0 (unknown 1318299302 0 0) 0x7f0668001b60 con 0x55f101e5f580 2024-09-15T03:44:58.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.122:6808/21254 -- osd_op(unknown.0.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x55f101eb4180 con 0x55f101eb0590 2024-09-15T03:44:58.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f0676c68700 10 monclient: get_auth_request con 0x55f101eb0590 auth_method 0 2024-09-15T03:44:58.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f0676c68700 1 -- 172.21.15.26:0/3621292014 <== osd.4 v1:172.21.15.122:6808/21254 1 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 588695289 0 0) 0x7f066c00f390 con 0x55f101eb0590 2024-09-15T03:44:58.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.122:6808/21254 -- osd_op(unknown.0.0:3 1.2 1:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) v8 -- 0x55f101eaf020 con 0x55f101eb0590 2024-09-15T03:44:58.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f0676c68700 1 -- 172.21.15.26:0/3621292014 <== osd.4 v1:172.21.15.122:6808/21254 2 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (unknown 575028469 0 0) 0x7f066c00f640 con 0x55f101eb0590 2024-09-15T03:44:58.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f0674c64700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:58.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:58.611+0000 7f0674c64700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- pool_op(create unmanaged snap pool 1 tid 4 name v0) v4 -- 0x7f0654001ee0 con 0x55f101d95d10 2024-09-15T03:44:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.715+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 12 ==== pool_op_reply(tid 4 (0) Success v15) v1 ==== 55+0+0 (unknown 3846783585 0 0) 0x7f066c012f10 con 0x55f101d95d10 2024-09-15T03:44:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.715+0000 7f0662ffd700 10 monclient: _renew_subs 2024-09-15T03:44:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.715+0000 7f0662ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:59.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.715+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=15}) v3 -- 0x55f101caa5e0 con 0x55f101d95d10 2024-09-15T03:44:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.715+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 13 ==== osd_map(15..15 src has 1..15) v4 ==== 988+0+0 (unknown 1638154819 0 0) 0x7f066c013740 con 0x55f101d95d10 2024-09-15T03:44:59.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.715+0000 7f0675465700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.122:6808/21254 -- osd_op(unknown.0.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x7f0650005580 con 0x55f101eb0590 2024-09-15T03:44:59.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.719+0000 7f0676c68700 1 -- 172.21.15.26:0/3621292014 <== osd.4 v1:172.21.15.122:6808/21254 3 ==== osd_op_reply(5 rbd_info [create,write 0~8] v15'1 uv1 ondisk = 0) v8 ==== 194+0+0 (unknown 453491686 0 0) 0x7f066c00f640 con 0x55f101eb0590 2024-09-15T03:44:59.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.719+0000 7f0674c64700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:44:59.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:44:59.719+0000 7f0674c64700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- pool_op(delete unmanaged snap pool 1 tid 6 name v15) v4 -- 0x7f06540033c0 con 0x55f101d95d10 2024-09-15T03:45:00.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.335+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 14 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f066c013b20 con 0x55f101d95d10 2024-09-15T03:45:00.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.643+0000 7f0661ffb700 10 monclient: tick 2024-09-15T03:45:00.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.643+0000 7f0661ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:00.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.643+0000 7f0661ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 15 ==== pool_op_reply(tid 6 (0) Success v16) v1 ==== 43+0+0 (unknown 1302770963 0 0) 0x7f066c041dd0 con 0x55f101d95d10 2024-09-15T03:45:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0662ffd700 10 monclient: _renew_subs 2024-09-15T03:45:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0662ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=16}) v3 -- 0x7f0658001e10 con 0x55f101d95d10 2024-09-15T03:45:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0662ffd700 1 -- 172.21.15.26:0/3621292014 <== mon.2 v1:172.21.15.26:6790/0 16 ==== osd_map(16..16 src has 1..16) v4 ==== 646+0+0 (unknown 3506949002 0 0) 0x7f066c040f90 con 0x55f101d95d10 2024-09-15T03:45:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0675465700 1 -- 172.21.15.26:0/3621292014 --> v1:172.21.15.122:6808/21254 -- osd_op(unknown.0.0:7 1.2 1:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7f0650005930 con 0x55f101eb0590 2024-09-15T03:45:00.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0676c68700 1 -- 172.21.15.26:0/3621292014 <== osd.4 v1:172.21.15.122:6808/21254 4 ==== osd_op_reply(7 rbd_info [write 0~19] v16'2 uv2 ondisk = 0) v8 ==== 152+0+0 (unknown 1471445399 0 0) 0x7f066c00f640 con 0x55f101eb0590 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 >> v1:172.21.15.26:6800/21130 conn(0x55f101e5f580 legacy=0x55f101e61960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 >> v1:172.21.15.122:6808/21254 conn(0x55f101eb0590 legacy=0x55f101eb2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 >> v1:172.21.15.122:6812/20895 conn(0x7f06580367b0 legacy=0x7f0658038c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 10 monclient: shutdown 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 >> v1:172.21.15.26:6790/0 conn(0x55f101d95d10 legacy=0x55f101e50fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f0677469700 1 -- 172.21.15.26:0/3621292014 reap_dead start 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 shutdown_connections 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 >> 172.21.15.26:0/3621292014 conn(0x55f101db5c40 msgr2=0x55f101db6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 shutdown_connections 2024-09-15T03:45:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.743+0000 7f06789334c0 1 -- 172.21.15.26:0/3621292014 wait complete. 2024-09-15T03:45:00.755 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T03:45:00.755 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T03:45:00.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T03:45:00.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T03:45:00.756 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T03:45:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient: get_monmap_and_config 2024-09-15T03:45:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient: build_initial_monmap 2024-09-15T03:45:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient: monmap: 2024-09-15T03:45:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:00.857864+0000 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:00.857864+0000 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:00.860 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:00.861 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 1 Processor -- start 2024-09-15T03:45:00.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 1 -- start start 2024-09-15T03:45:00.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient: init 2024-09-15T03:45:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient: _add_conns ranks=[5,8,1,7,0,4,3,6,2] 2024-09-15T03:45:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient(hunting): picked mon.noname-h con 0x7ff0100a4b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient(hunting): picked mon.noname-e con 0x7ff01009aca0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient(hunting): picked mon.noname-f con 0x7ff010136ed0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01004ca10 con 0x7ff010136ed0 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01009cc60 con 0x7ff0100a4b40 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0100a02a0 con 0x7ff01009aca0 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff015e6a700 10 monclient(hunting): authenticate will time out at 845.287202s 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff00ffff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7ff01009aca0 0x7ff0101357b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48736/0 (socket says 172.21.15.26:48736) 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff00ffff700 1 -- 172.21.15.26:0/1148605595 learned_addr learned my addr 172.21.15.26:0/1148605595 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002274748 0 0) 0x7ff0100a02a0 con 0x7ff01009aca0 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff00f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff00f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.851+0000 7ff00f7fe700 10 monclient(hunting): my global_id is 4247 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff8003570 con 0x7ff01009aca0 2024-09-15T03:45:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471037139 0 0) 0x7ff01004ca10 con 0x7ff010136ed0 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient(hunting): my global_id is 4285 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0100a02a0 con 0x7ff010136ed0 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127882573 0 0) 0x7ff01009cc60 con 0x7ff0100a4b40 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient(hunting): my global_id is 4226 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff01004ca10 con 0x7ff0100a4b40 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707528872 0 0) 0x7feff8003570 con 0x7ff01009aca0 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 >> v1:172.21.15.122:6791/0 conn(0x7ff0100a4b40 legacy=0x7ff01009dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 >> v1:172.21.15.122:6789/0 conn(0x7ff010136ed0 legacy=0x7ff0101392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 monclient: found mon.noname-e 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01009b4a0 con 0x7ff01009aca0 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff000001b30 con 0x7ff01009aca0 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 5 monclient: authenticate success, global_id 4247 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: dump: 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff000001da0 con 0x7ff01009aca0 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 1 -- 172.21.15.26:0/1148605595 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff000004fb0 con 0x7ff01009aca0 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 4 set_mon_vals no callback set 2024-09-15T03:45:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff00f7fe700 10 monclient: dump: 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 10 monclient: shutdown 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 1 -- 172.21.15.26:0/1148605595 >> v1:172.21.15.26:6793/0 conn(0x7ff01009aca0 legacy=0x7ff0101357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 1 -- 172.21.15.26:0/1148605595 shutdown_connections 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 1 -- 172.21.15.26:0/1148605595 >> 172.21.15.26:0/1148605595 conn(0x7ff010063dc0 msgr2=0x7ff0100a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 1 -- 172.21.15.26:0/1148605595 shutdown_connections 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 1 -- 172.21.15.26:0/1148605595 wait complete. 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 10 monclient: build_initial_monmap 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.855+0000 7ff015e6a700 10 monclient: monmap: 2024-09-15T03:45:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:00.873 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 Processor -- start 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 -- start start 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: init 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: _add_conns ranks=[2,0,7,3,8,4,5,6,1] 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient(hunting): picked mon.noname-b con 0x7ff01009aca0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient(hunting): picked mon.noname-a con 0x7ff0100a4b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient(hunting): picked mon.noname-i con 0x7ff010136ed0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01006e5f0 con 0x7ff0100a4b40 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01006e190 con 0x7ff01009aca0 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff010069400 con 0x7ff010136ed0 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient(hunting): authenticate will time out at 845.293959s 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff0100a4b40 0x7ff010135400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52984/0 (socket says 172.21.15.26:52984) 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00ffff700 1 -- 172.21.15.26:0/4143328172 learned_addr learned my addr 172.21.15.26:0/4143328172 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536190906 0 0) 0x7ff01006e5f0 con 0x7ff0100a4b40 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): my global_id is 4194 2024-09-15T03:45:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feffc003570 con 0x7ff0100a4b40 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497638647 0 0) 0x7ff01006e190 con 0x7ff01009aca0 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): my global_id is 4259 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff01006e5f0 con 0x7ff01009aca0 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153221769 0 0) 0x7ff010069400 con 0x7ff010136ed0 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): my global_id is 4336 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff01006e190 con 0x7ff010136ed0 2024-09-15T03:45:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 409384866 0 0) 0x7feffc003570 con 0x7ff0100a4b40 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff010069400 con 0x7ff0100a4b40 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff000001bb0 con 0x7ff0100a4b40 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): dump: 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2049506564 0 0) 0x7ff01006e5f0 con 0x7ff01009aca0 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feffc003570 con 0x7ff01009aca0 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff004001a60 con 0x7ff01009aca0 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): dump: 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3775993330 0 0) 0x7ff01006e190 con 0x7ff010136ed0 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff01006e5f0 con 0x7ff010136ed0 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff008001900 con 0x7ff010136ed0 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient(hunting): dump: 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3956876345 0 0) 0x7feffc003570 con 0x7ff01009aca0 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 >> v1:172.21.15.122:6792/0 conn(0x7ff010136ed0 legacy=0x7ff01014a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 >> v1:172.21.15.26:6789/0 conn(0x7ff0100a4b40 legacy=0x7ff010135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient: _finish_hunting 0 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 monclient: found mon.b 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010069120 con 0x7ff01009aca0 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient: _finish_auth 0 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 5 monclient: authenticate success, global_id 4259 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: _renew_subs 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff010067c60 con 0x7ff01009aca0 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: _renew_subs 2024-09-15T03:45:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff010067ee0 con 0x7ff01009aca0 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff004002d60 con 0x7ff01009aca0 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.859+0000 7ff00cff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff0040022c0 con 0x7ff01009aca0 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00effd700 4 set_mon_vals no callback set 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00cff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00cff9700 10 monclient: dump: 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff004002420 con 0x7ff01009aca0 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7ff0040412f0 con 0x7ff01009aca0 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff015e6a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff015e6a700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff015e6a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff00050b0 con 0x7ff01009aca0 2024-09-15T03:45:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.863+0000 7ff00ffff700 10 monclient: get_auth_request con 0x7feffc038020 auth_method 0 2024-09-15T03:45:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.871+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ff004011e60 con 0x7ff01009aca0 2024-09-15T03:45:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.871+0000 7ff00cff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:00.871+0000 7ff00cff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.075+0000 7ff015e6a700 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-09-15T03:45:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.075+0000 7ff015e6a700 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-15T03:45:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.075+0000 7ff015e6a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.075+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7feff0005360 con 0x7ff01009aca0 2024-09-15T03:45:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.079+0000 7ff00cff9700 1 -- 172.21.15.26:0/4143328172 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+83787 (unknown 481731318 0 2777897280) 0x7ff004011e60 con 0x7ff01009aca0 2024-09-15T03:45:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.079+0000 7ff00cff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-09-15T03:45:01.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.079+0000 7ff00cff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:01.089 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:01.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 >> v1:172.21.15.122:6812/20895 conn(0x7feffc038020 legacy=0x7feffc03a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 10 monclient: shutdown 2024-09-15T03:45:01.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 >> v1:172.21.15.26:6790/0 conn(0x7ff01009aca0 legacy=0x7ff010134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 shutdown_connections 2024-09-15T03:45:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 >> 172.21.15.26:0/4143328172 conn(0x7ff010063dc0 msgr2=0x7ff01006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 shutdown_connections 2024-09-15T03:45:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.083+0000 7ff015e6a700 1 -- 172.21.15.26:0/4143328172 wait complete. 2024-09-15T03:45:01.107 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"flags":0,"active_gid":4124,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6812","nonce":20895}]},"active_addr":"172.21.15.122:6812/20895","active_change":"2024-09-15T03:44:51.444965+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3713420102}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2508635318}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":274625072}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2673584113}]}]} 2024-09-15T03:45:01.110 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T03:45:01.110 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T03:45:01.110 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient: monmap: 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.217129+0000 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.217129+0000 2024-09-15T03:45:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.219 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 1 Processor -- start 2024-09-15T03:45:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 1 -- start start 2024-09-15T03:45:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient: init 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient: _add_conns ranks=[4,1,3,2,7,5,6,8,0] 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient(hunting): picked mon.noname-c con 0x7f190c070de0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient(hunting): picked mon.noname-f con 0x7f190c06dea0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 10 monclient(hunting): picked mon.noname-g con 0x7f190c1360f0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.211+0000 7f1911936700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c04ca10 con 0x7f190c06dea0 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c0a1a80 con 0x7f190c1360f0 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c06d4e0 con 0x7f190c070de0 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient(hunting): authenticate will time out at 845.648202s 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f190affd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f190c070de0 0x7f190c069870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35782/0 (socket says 172.21.15.26:35782) 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f190affd700 1 -- 172.21.15.26:0/212624041 learned_addr learned my addr 172.21.15.26:0/212624041 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840458707 0 0) 0x7f190c06d4e0 con 0x7f190c070de0 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): my global_id is 4225 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f4003570 con 0x7f190c070de0 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361112725 0 0) 0x7f190c04ca10 con 0x7f190c06dea0 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): my global_id is 4294 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c06d4e0 con 0x7f190c06dea0 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351283040 0 0) 0x7f190c0a1a80 con 0x7f190c1360f0 2024-09-15T03:45:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient(hunting): my global_id is 4269 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c04ca10 con 0x7f190c1360f0 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620458724 0 0) 0x7f18f4003570 con 0x7f190c070de0 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 >> v1:172.21.15.122:6790/0 conn(0x7f190c1360f0 legacy=0x7f190c1384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 >> v1:172.21.15.122:6789/0 conn(0x7f190c06dea0 legacy=0x7f190c069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 monclient: found mon.noname-c 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c06b2c0 con 0x7f190c070de0 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f18fc0017b0 con 0x7f190c070de0 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 5 monclient: authenticate success, global_id 4225 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: dump: 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f18fc001a90 con 0x7f190c070de0 2024-09-15T03:45:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 4 set_mon_vals no callback set 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 1 -- 172.21.15.26:0/212624041 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f18fc004c30 con 0x7f190c070de0 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1909ffb700 10 monclient: dump: 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient: shutdown 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- 172.21.15.26:0/212624041 >> v1:172.21.15.26:6791/0 conn(0x7f190c070de0 legacy=0x7f190c069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- 172.21.15.26:0/212624041 shutdown_connections 2024-09-15T03:45:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- 172.21.15.26:0/212624041 >> 172.21.15.26:0/212624041 conn(0x7f190c0a3360 msgr2=0x7f190c09f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- 172.21.15.26:0/212624041 shutdown_connections 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- 172.21.15.26:0/212624041 wait complete. 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient: monmap: 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.232 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 Processor -- start 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 1 -- start start 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.215+0000 7f1911936700 10 monclient: init 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient: _add_conns ranks=[5,8,4,3,7,0,1,6,2] 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient(hunting): picked mon.noname-h con 0x7f190c06dea0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient(hunting): picked mon.noname-e con 0x7f190c070de0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:01.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient(hunting): picked mon.noname-c con 0x7f190c1360f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c09cc70 con 0x7f190c1360f0 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c06aad0 con 0x7f190c06dea0 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c11fe70 con 0x7f190c070de0 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient(hunting): authenticate will time out at 845.652495s 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f190a7fc700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f190c070de0 0x7f190c119880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48744/0 (socket says 172.21.15.26:48744) 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f190a7fc700 1 -- 172.21.15.26:0/3724549219 learned_addr learned my addr 172.21.15.26:0/3724549219 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f190b7fe700 1 --1- 172.21.15.26:0/3724549219 >> v1:172.21.15.26:6791/0 conn(0x7f190c1360f0 0x7f190c14a5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35794/0 (socket says 172.21.15.26:35794) 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009821313 0 0) 0x7f190c11fe70 con 0x7f190c070de0 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): my global_id is 4256 2024-09-15T03:45:01.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f8003570 con 0x7f190c070de0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231255009 0 0) 0x7f190c09cc70 con 0x7f190c1360f0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): my global_id is 4234 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c11fe70 con 0x7f190c1360f0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105829444 0 0) 0x7f190c06aad0 con 0x7f190c06dea0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): my global_id is 4235 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c09cc70 con 0x7f190c06dea0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4072126605 0 0) 0x7f18f8003570 con 0x7f190c070de0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f190c06aad0 con 0x7f190c070de0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4197188706 0 0) 0x7f190c11fe70 con 0x7f190c1360f0 2024-09-15T03:45:01.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f18f8003570 con 0x7f190c1360f0 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1900001b20 con 0x7f190c070de0 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.239 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1904001d40 con 0x7f190c1360f0 2024-09-15T03:45:01.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.241 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1298665276 0 0) 0x7f190c09cc70 con 0x7f190c06dea0 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f190c11fe70 con 0x7f190c06dea0 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f18fc004b70 con 0x7f190c06dea0 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3359138700 0 0) 0x7f18f8003570 con 0x7f190c1360f0 2024-09-15T03:45:01.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 >> v1:172.21.15.26:6793/0 conn(0x7f190c070de0 legacy=0x7f190c119880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 >> v1:172.21.15.122:6791/0 conn(0x7f190c06dea0 legacy=0x7f190c119170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 monclient: found mon.c 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c1203e0 con 0x7f190c1360f0 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 5 monclient: authenticate success, global_id 4234 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient: _renew_subs 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0a12c0 con 0x7f190c1360f0 2024-09-15T03:45:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient: _renew_subs 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c11a7d0 con 0x7f190c1360f0 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1904003010 con 0x7f190c1360f0 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1904002550 con 0x7f190c1360f0 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f19097fa700 4 set_mon_vals no callback set 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f18f2ffd700 10 monclient: dump: 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.219+0000 7f1911936700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.223+0000 7f1911936700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.223+0000 7f1911936700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.223+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec0050b0 con 0x7f190c1360f0 2024-09-15T03:45:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.223+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1904002730 con 0x7f190c1360f0 2024-09-15T03:45:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.223+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f1904043e90 con 0x7f190c1360f0 2024-09-15T03:45:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.223+0000 7f190affd700 10 monclient: get_auth_request con 0x7f18f8038030 auth_method 0 2024-09-15T03:45:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.231+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f1904014e20 con 0x7f190c1360f0 2024-09-15T03:45:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.231+0000 7f18f2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.231+0000 7f18f2ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f1911936700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:01.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f1911936700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f1911936700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18ec005360 con 0x7f190c1360f0 2024-09-15T03:45:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f18f2ffd700 1 -- 172.21.15.26:0/3724549219 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+10084 (unknown 1054498076 0 2475284439) 0x7f1904014820 con 0x7f190c1360f0 2024-09-15T03:45:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f18f2ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:01.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.391+0000 7f18f2ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:01.400 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:01.400 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T03:45:00.725199+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T03:44:56.976968+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-15T03:44:57.527097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:45:01.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 >> v1:172.21.15.122:6812/20895 conn(0x7f18f8038030 legacy=0x7f18f803a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 10 monclient: shutdown 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 >> v1:172.21.15.26:6791/0 conn(0x7f190c1360f0 legacy=0x7f190c14a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 shutdown_connections 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 >> 172.21.15.26:0/3724549219 conn(0x7f190c0a3360 msgr2=0x7f190c1353a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 shutdown_connections 2024-09-15T03:45:01.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.395+0000 7f1911936700 1 -- 172.21.15.26:0/3724549219 wait complete. 2024-09-15T03:45:01.420 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T03:45:01.420 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:45:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient: monmap: 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.521318+0000 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.521318+0000 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.523 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.524 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.525 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.525 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 1 Processor -- start 2024-09-15T03:45:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 1 -- start start 2024-09-15T03:45:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient: init 2024-09-15T03:45:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient: _add_conns ranks=[5,1,8,6,7,4,0,2,3] 2024-09-15T03:45:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient(hunting): picked mon.noname-h con 0x7fd4c80a38e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient(hunting): picked mon.noname-f con 0x7fd4c809cfb0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient(hunting): picked mon.noname-e con 0x7fd4c806de10 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c804ca10 con 0x7fd4c809cfb0 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c809e010 con 0x7fd4c80a38e0 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c80a3cb0 con 0x7fd4c806de10 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c7fff700 10 monclient(hunting): authenticate will time out at 845.950914s 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c77fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fd4c806de10 0x7fd4c8139660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48752/0 (socket says 172.21.15.26:48752) 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c77fe700 1 -- 172.21.15.26:0/2594328212 learned_addr learned my addr 172.21.15.26:0/2594328212 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126272722 0 0) 0x7fd4c80a3cb0 con 0x7fd4c806de10 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): my global_id is 4265 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4a8003570 con 0x7fd4c806de10 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056961096 0 0) 0x7fd4c809e010 con 0x7fd4c80a38e0 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): my global_id is 4244 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c80a3cb0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506600420 0 0) 0x7fd4c804ca10 con 0x7fd4c809cfb0 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 10 monclient(hunting): my global_id is 4303 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.515+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c809e010 con 0x7fd4c809cfb0 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 34231137 0 0) 0x7fd4a8003570 con 0x7fd4c806de10 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 >> v1:172.21.15.122:6791/0 conn(0x7fd4c80a38e0 legacy=0x7fd4c8063340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 >> v1:172.21.15.122:6789/0 conn(0x7fd4c809cfb0 legacy=0x7fd4c8063a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 monclient: found mon.noname-e 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c8064160 con 0x7fd4c806de10 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd4c00017b0 con 0x7fd4c806de10 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 5 monclient: authenticate success, global_id 4265 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: dump: 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd4c0001a20 con 0x7fd4c806de10 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 1 -- 172.21.15.26:0/2594328212 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd4c0004c30 con 0x7fd4c806de10 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c5ffb700 10 monclient: dump: 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 4 set_mon_vals no callback set 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 10 monclient: shutdown 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2594328212 >> v1:172.21.15.26:6793/0 conn(0x7fd4c806de10 legacy=0x7fd4c8139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2594328212 shutdown_connections 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2594328212 >> 172.21.15.26:0/2594328212 conn(0x7fd4c8071860 msgr2=0x7fd4c80a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2594328212 shutdown_connections 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2594328212 wait complete. 2024-09-15T03:45:01.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 10 monclient: monmap: 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.542 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 Processor -- start 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 1 -- start start 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.519+0000 7fd4c7fff700 10 monclient: init 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _add_conns ranks=[4,7,6,5,3,0,1,8,2] 2024-09-15T03:45:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient(hunting): picked mon.noname-c con 0x7fd4c806de10 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient(hunting): picked mon.noname-i con 0x7fd4c809cfb0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient(hunting): picked mon.noname-d con 0x7fd4c80a38e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c809eaf0 con 0x7fd4c806de10 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c809e720 con 0x7fd4c80a38e0 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c6ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd4c806de10 0x7fd4c813da50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35796/0 (socket says 172.21.15.26:35796) 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c6ffd700 1 -- 172.21.15.26:0/2568961510 learned_addr learned my addr 172.21.15.26:0/2568961510 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c813e160 con 0x7fd4c809cfb0 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient(hunting): authenticate will time out at 845.956396s 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393829847 0 0) 0x7fd4c809eaf0 con 0x7fd4c806de10 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): my global_id is 4243 2024-09-15T03:45:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4ac003570 con 0x7fd4c806de10 2024-09-15T03:45:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998915613 0 0) 0x7fd4c809e720 con 0x7fd4c80a38e0 2024-09-15T03:45:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): my global_id is 4254 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c809eaf0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689088659 0 0) 0x7fd4c813e160 con 0x7fd4c809cfb0 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): my global_id is 4345 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c809e720 con 0x7fd4c809cfb0 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2696632484 0 0) 0x7fd4c809eaf0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd4c813e160 con 0x7fd4c80a38e0 2024-09-15T03:45:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd4c0001770 con 0x7fd4c80a38e0 2024-09-15T03:45:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): dump: 2024-09-15T03:45:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 627653549 0 0) 0x7fd4ac003570 con 0x7fd4c806de10 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd4c809eaf0 con 0x7fd4c806de10 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd4b8002d50 con 0x7fd4c806de10 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient(hunting): dump: 2024-09-15T03:45:01.549 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.550 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 401793306 0 0) 0x7fd4c813e160 con 0x7fd4c80a38e0 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 >> v1:172.21.15.122:6792/0 conn(0x7fd4c809cfb0 legacy=0x7fd4c8146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 >> v1:172.21.15.26:6791/0 conn(0x7fd4c806de10 legacy=0x7fd4c813da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 monclient: found mon.d 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c813ef70 con 0x7fd4c80a38e0 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 5 monclient: authenticate success, global_id 4254 2024-09-15T03:45:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _renew_subs 2024-09-15T03:45:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c8122af0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _renew_subs 2024-09-15T03:45:01.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c8098d50 con 0x7fd4c80a38e0 2024-09-15T03:45:01.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd4c0004b70 con 0x7fd4c80a38e0 2024-09-15T03:45:01.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd4c0005240 con 0x7fd4c80a38e0 2024-09-15T03:45:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c57fa700 4 set_mon_vals no callback set 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 10 monclient: dump: 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a00050b0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.523+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd4c000fc00 con 0x7fd4c80a38e0 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.527+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7fd4c0043db0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.527+0000 7fd4c6ffd700 10 monclient: get_auth_request con 0x7fd4ac037e20 auth_method 0 2024-09-15T03:45:01.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.535+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fd4c00149e0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.535+0000 7fd4beffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:01.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.535+0000 7fd4beffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.695+0000 7fd4c7fff700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:01.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.695+0000 7fd4c7fff700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:01.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.695+0000 7fd4c7fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.695+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4a0005360 con 0x7fd4c80a38e0 2024-09-15T03:45:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4beffd700 1 -- 172.21.15.26:0/2568961510 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+10084 (unknown 1054498076 0 2475284439) 0x7fd4c00149e0 con 0x7fd4c80a38e0 2024-09-15T03:45:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4beffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4beffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:01.705 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:01.705 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T03:45:00.725199+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T03:44:56.976968+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-15T03:44:57.527097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:45:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 >> v1:172.21.15.122:6812/20895 conn(0x7fd4ac037e20 legacy=0x7fd4ac03a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 10 monclient: shutdown 2024-09-15T03:45:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 >> v1:172.21.15.26:6792/0 conn(0x7fd4c80a38e0 legacy=0x7fd4c814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 shutdown_connections 2024-09-15T03:45:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 >> 172.21.15.26:0/2568961510 conn(0x7fd4c8071860 msgr2=0x7fd4c80a10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 shutdown_connections 2024-09-15T03:45:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.699+0000 7fd4c7fff700 1 -- 172.21.15.26:0/2568961510 wait complete. 2024-09-15T03:45:01.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T03:45:01.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T03:45:01.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T03:45:01.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T03:45:01.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T03:45:01.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T03:45:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.827+0000 7fd030f76700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.827+0000 7fd030f76700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.827+0000 7fd030f76700 10 monclient: monmap: 2024-09-15T03:45:01.837 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.835176+0000 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.835176+0000 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.838 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.839 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 Processor -- start 2024-09-15T03:45:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- start start 2024-09-15T03:45:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient: init 2024-09-15T03:45:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient: _add_conns ranks=[4,0,5,6,2,3,7,8,1] 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient(hunting): picked mon.noname-c con 0x7fd02c097e00 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient(hunting): picked mon.noname-a con 0x7fd02c12f5f0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient(hunting): picked mon.noname-h con 0x7fd02c130110 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c04ca10 con 0x7fd02c12f5f0 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c06e380 con 0x7fd02c097e00 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c09e5f0 con 0x7fd02c130110 2024-09-15T03:45:01.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient(hunting): authenticate will time out at 846.265163s 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02b7fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd02c097e00 0x7fd02c09dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35798/0 (socket says 172.21.15.26:35798) 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02b7fe700 1 -- 172.21.15.26:0/3704577652 learned_addr learned my addr 172.21.15.26:0/3704577652 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02affd700 1 --1- 172.21.15.26:0/3704577652 >> v1:172.21.15.26:6789/0 conn(0x7fd02c12f5f0 0x7fd02c12fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53000/0 (socket says 172.21.15.26:53000) 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203999279 0 0) 0x7fd02c04ca10 con 0x7fd02c12f5f0 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): my global_id is 4203 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd014003570 con 0x7fd02c12f5f0 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868497202 0 0) 0x7fd02c06e380 con 0x7fd02c097e00 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): my global_id is 4252 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c04ca10 con 0x7fd02c097e00 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762938703 0 0) 0x7fd02c09e5f0 con 0x7fd02c130110 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient(hunting): my global_id is 4253 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c06e380 con 0x7fd02c130110 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226120465 0 0) 0x7fd014003570 con 0x7fd02c12f5f0 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 >> v1:172.21.15.122:6791/0 conn(0x7fd02c130110 legacy=0x7fd02c13aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 >> v1:172.21.15.26:6791/0 conn(0x7fd02c097e00 legacy=0x7fd02c09dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 monclient: found mon.noname-a 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c09cb80 con 0x7fd02c12f5f0 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd020001730 con 0x7fd02c12f5f0 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: dump: 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd020001950 con 0x7fd02c12f5f0 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 1 -- 172.21.15.26:0/3704577652 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd020004830 con 0x7fd02c12f5f0 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd02a7fc700 10 monclient: dump: 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 5 monclient: authenticate success, global_id 4203 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 4 set_mon_vals no callback set 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 10 monclient: shutdown 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- 172.21.15.26:0/3704577652 >> v1:172.21.15.26:6789/0 conn(0x7fd02c12f5f0 legacy=0x7fd02c12fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- 172.21.15.26:0/3704577652 shutdown_connections 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- 172.21.15.26:0/3704577652 >> 172.21.15.26:0/3704577652 conn(0x7fd02c063dc0 msgr2=0x7fd02c0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- 172.21.15.26:0/3704577652 shutdown_connections 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.831+0000 7fd030f76700 1 -- 172.21.15.26:0/3704577652 wait complete. 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient: monmap: 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.849 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 1 Processor -- start 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 1 -- start start 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient: init 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient: _add_conns ranks=[8,0,5,6,4,1,3,2,7] 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient(hunting): picked mon.noname-e con 0x7fd02c097e00 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient(hunting): picked mon.noname-a con 0x7fd02c12f5f0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient(hunting): picked mon.noname-h con 0x7fd02c130110 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c098b90 con 0x7fd02c12f5f0 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c0987a0 con 0x7fd02c130110 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c11e320 con 0x7fd02c097e00 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd030f76700 10 monclient(hunting): authenticate will time out at 846.270160s 2024-09-15T03:45:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd02affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd02c12f5f0 0x7fd02c11b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53014/0 (socket says 172.21.15.26:53014) 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd02affd700 1 -- 172.21.15.26:0/2151563545 learned_addr learned my addr 172.21.15.26:0/2151563545 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382361965 0 0) 0x7fd02c11e320 con 0x7fd02c097e00 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): my global_id is 4274 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0140037b0 con 0x7fd02c097e00 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334197313 0 0) 0x7fd02c098b90 con 0x7fd02c12f5f0 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): my global_id is 4212 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c11e320 con 0x7fd02c12f5f0 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141246362 0 0) 0x7fd02c0987a0 con 0x7fd02c130110 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): my global_id is 4262 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c098b90 con 0x7fd02c130110 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2640327253 0 0) 0x7fd0140037b0 con 0x7fd02c097e00 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd02c0987a0 con 0x7fd02c097e00 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3194633786 0 0) 0x7fd02c11e320 con 0x7fd02c12f5f0 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd0140037b0 con 0x7fd02c12f5f0 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd01c001990 con 0x7fd02c097e00 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): dump: 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.857 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd0200016b0 con 0x7fd02c12f5f0 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 10 monclient(hunting): dump: 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 732513690 0 0) 0x7fd02c098b90 con 0x7fd02c130110 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd02c11e320 con 0x7fd02c130110 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.835+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 703364405 0 0) 0x7fd02c0987a0 con 0x7fd02c097e00 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 >> v1:172.21.15.122:6791/0 conn(0x7fd02c130110 legacy=0x7fd02c149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 >> v1:172.21.15.26:6789/0 conn(0x7fd02c12f5f0 legacy=0x7fd02c11b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 monclient: found mon.e 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c11e8e0 con 0x7fd02c097e00 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 5 monclient: authenticate success, global_id 4274 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 10 monclient: _renew_subs 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c12f3c0 con 0x7fd02c097e00 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 10 monclient: _renew_subs 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c11d430 con 0x7fd02c097e00 2024-09-15T03:45:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd01c0048e0 con 0x7fd02c097e00 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0297fa700 4 set_mon_vals no callback set 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd01c004d90 con 0x7fd02c097e00 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: dump: 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.26:6793/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd018000f20 con 0x7fd02c097e00 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7f0e0359e700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7f0e0359e700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7f0e0359e700 10 monclient: monmap: 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.846573+0000 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.846573+0000 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.863 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd01c004ef0 con 0x7fd02c097e00 2024-09-15T03:45:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7fd01c043a70 con 0x7fd02c097e00 2024-09-15T03:45:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.122:6800/21255 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd014042fc0 con 0x7fd01403f320 2024-09-15T03:45:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7fd01c043ea0 con 0x7fd02c097e00 2024-09-15T03:45:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd0137fe700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-15T03:45:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd02affd700 10 monclient: get_auth_request con 0x7fd0140388b0 auth_method 0 2024-09-15T03:45:01.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.839+0000 7fd02bfff700 10 monclient: get_auth_request con 0x7fd01403f320 auth_method 0 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 Processor -- start 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- start start 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: init 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: _add_conns ranks=[2,5,1,0,4,7,6,3,8] 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient(hunting): picked mon.noname-b con 0x7f0e040a2aa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient(hunting): picked mon.noname-h con 0x7f0e0406e160 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient(hunting): picked mon.noname-f con 0x7f0e0409b310 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0404ca10 con 0x7f0e0409b310 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e04099950 con 0x7f0e040a2aa0 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e040995e0 con 0x7f0e0406e160 2024-09-15T03:45:01.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient(hunting): authenticate will time out at 846.276948s 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0259c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0e040a2aa0 0x7f0e0409c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54234/0 (socket says 172.21.15.26:54234) 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0259c700 1 -- 172.21.15.26:0/1387722506 learned_addr learned my addr 172.21.15.26:0/1387722506 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270391794 0 0) 0x7f0e04099950 con 0x7f0e040a2aa0 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient(hunting): my global_id is 4268 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df0004680 con 0x7f0e040a2aa0 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852631824 0 0) 0x7f0e040995e0 con 0x7f0e0406e160 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient(hunting): my global_id is 4271 2024-09-15T03:45:01.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e04099950 con 0x7f0e0406e160 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2449986861 0 0) 0x7f0df0004680 con 0x7f0e040a2aa0 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 >> v1:172.21.15.122:6791/0 conn(0x7f0e0406e160 legacy=0x7f0e0409cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 >> v1:172.21.15.122:6789/0 conn(0x7f0e0409b310 legacy=0x7f0e04137640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 monclient: found mon.noname-b 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0409d630 con 0x7f0e040a2aa0 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0df40017b0 con 0x7f0e040a2aa0 2024-09-15T03:45:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 5 monclient: authenticate success, global_id 4268 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: dump: 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0df40019d0 con 0x7f0e040a2aa0 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 4 set_mon_vals no callback set 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 1 -- 172.21.15.26:0/1387722506 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0df4004c30 con 0x7f0e040a2aa0 2024-09-15T03:45:01.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0159a700 10 monclient: dump: 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== osd.3 v1:172.21.15.122:6800/21255 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fd014042fc0 con 0x7fd01403f320 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: shutdown 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- 172.21.15.26:0/1387722506 >> v1:172.21.15.26:6790/0 conn(0x7f0e040a2aa0 legacy=0x7f0e0409c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- 172.21.15.26:0/1387722506 shutdown_connections 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- 172.21.15.26:0/1387722506 >> 172.21.15.26:0/1387722506 conn(0x7f0e04063cd0 msgr2=0x7f0e040a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- 172.21.15.26:0/1387722506 shutdown_connections 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 -- 172.21.15.26:0/1387722506 wait complete. 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 10 monclient: monmap: 2024-09-15T03:45:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.843+0000 7f0e0359e700 1 Processor -- start 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 1 -- start start 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient: init 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient: _add_conns ranks=[2,5,3,4,7,1,6,8,0] 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient(hunting): picked mon.noname-b con 0x7f0e0406e160 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient(hunting): picked mon.noname-h con 0x7f0e0409b310 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient(hunting): picked mon.noname-g con 0x7f0e0411d300 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0409be80 con 0x7f0e0406e160 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0409b9b0 con 0x7f0e0411d300 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0411fb10 con 0x7f0e0409b310 2024-09-15T03:45:01.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0359e700 10 monclient(hunting): authenticate will time out at 846.280653s 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0259c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0e0406e160 0x7f0e04132710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54248/0 (socket says 172.21.15.26:54248) 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.847+0000 7f0e0259c700 1 -- 172.21.15.26:0/3666561426 learned_addr learned my addr 172.21.15.26:0/3666561426 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567299862 0 0) 0x7f0e0409be80 con 0x7f0e0406e160 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): my global_id is 4277 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de4003570 con 0x7f0e0406e160 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802200595 0 0) 0x7f0e0411fb10 con 0x7f0e0409b310 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): my global_id is 4280 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0409be80 con 0x7f0e0409b310 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658652548 0 0) 0x7f0e0409b9b0 con 0x7f0e0411d300 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): my global_id is 4278 2024-09-15T03:45:01.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0411fb10 con 0x7f0e0411d300 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f200759e700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f200759e700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f200759e700 10 monclient: monmap: 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.857636+0000 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.857636+0000 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.881 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.882 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3664181019 0 0) 0x7f0de4003570 con 0x7f0e0406e160 2024-09-15T03:45:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e0409b9b0 con 0x7f0e0406e160 2024-09-15T03:45:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0df40016f0 con 0x7f0e0406e160 2024-09-15T03:45:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:01.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2598076579 0 0) 0x7f0e0411fb10 con 0x7f0e0411d300 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0de4003570 con 0x7f0e0411d300 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0dfc001ca0 con 0x7f0e0411d300 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.886 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2107487279 0 0) 0x7f0e0409b9b0 con 0x7f0e0406e160 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 >> v1:172.21.15.122:6791/0 conn(0x7f0e0409b310 legacy=0x7f0e04146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 >> v1:172.21.15.122:6790/0 conn(0x7f0e0411d300 legacy=0x7f0e0414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 monclient: found mon.b 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0411f830 con 0x7f0e0406e160 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 5 monclient: authenticate success, global_id 4277 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 10 monclient: _renew_subs 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0411dd00 con 0x7f0e0406e160 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 10 monclient: _renew_subs 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0411e8d0 con 0x7f0e0406e160 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0df4001210 con 0x7f0e0406e160 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0df40051a0 con 0x7f0e0406e160 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0e00d99700 4 set_mon_vals no callback set 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 10 monclient: dump: 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.851+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0df400fb60 con 0x7f0e0406e160 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0e0359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0de8000f20 con 0x7f0e0406e160 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f0df4043f80 con 0x7f0e0406e160 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6808/21148 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0de40424a0 con 0x7f0de403e800 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0e01d9b700 10 monclient: get_auth_request con 0x7f0de4037c20 auth_method 0 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7f0df4014bf0 con 0x7f0e0406e160 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0e02d9d700 10 monclient: get_auth_request con 0x7f0de403e800 auth_method 0 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0dfa7fc700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== osd.2 v1:172.21.15.26:6808/21148 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f0de40424a0 con 0x7f0de403e800 2024-09-15T03:45:01.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 1 Processor -- start 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7fd5c0c32700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7fd5c0c32700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7fd5c0c32700 10 monclient: monmap: 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.862937+0000 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.862937+0000 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.892 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.893 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 1 -- start start 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 10 monclient: init 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 10 monclient: _add_conns ranks=[8,3,6,7,0,2,1,4,5] 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7efc0b33d700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7efc0b33d700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7efc0b33d700 10 monclient: monmap: 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.894 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.863275+0000 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.863275+0000 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.895 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.896 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 10 monclient(hunting): picked mon.noname-e con 0x7f20080a6880 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:01.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f200759e700 10 monclient(hunting): picked mon.noname-g con 0x7f200809faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient(hunting): picked mon.noname-d con 0x7f2008138d30 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:01.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200804ca10 con 0x7f200809faa0 2024-09-15T03:45:01.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 Processor -- start 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080a0580 con 0x7f2008138d30 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200659c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f20080a6880 0x7f2008133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48776/0 (socket says 172.21.15.26:48776) 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- 172.21.15.26:0/1170364792 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2008133780 con 0x7f20080a6880 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f92b277a700 10 monclient: get_monmap_and_config 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f92b277a700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.855+0000 7f92b277a700 10 monclient: monmap: 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.897 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:01.863479+0000 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:01.863479+0000 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.898 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200659c700 1 -- 172.21.15.26:0/1170364792 learned_addr learned my addr 172.21.15.26:0/1170364792 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient(hunting): authenticate will time out at 846.292203s 2024-09-15T03:45:01.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- start start 2024-09-15T03:45:01.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: init 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183147705 0 0) 0x7f2008133780 con 0x7f20080a6880 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient(hunting): my global_id is 4283 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff0003570 con 0x7f20080a6880 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222730226 0 0) 0x7f200804ca10 con 0x7f200809faa0 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient(hunting): my global_id is 4287 2024-09-15T03:45:01.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2008133780 con 0x7f200809faa0 2024-09-15T03:45:01.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: _add_conns ranks=[7,6,5,2,8,3,0,1,4] 2024-09-15T03:45:01.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient(hunting): picked mon.noname-i con 0x7fd5bc0a2a70 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:01.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient(hunting): picked mon.noname-d con 0x7fd5bc09e380 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:01.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient(hunting): picked mon.noname-h con 0x7fd5bc09bfc0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc04ca10 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1293167787 0 0) 0x7f1ff0003570 con 0x7f20080a6880 2024-09-15T03:45:01.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 >> v1:172.21.15.26:6792/0 conn(0x7f2008138d30 legacy=0x7f200813b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:01.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc09d3e0 con 0x7fd5bc09e380 2024-09-15T03:45:01.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc098880 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient(hunting): authenticate will time out at 846.293094s 2024-09-15T03:45:01.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5baffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd5bc09e380 0x7fd5bc098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54988/0 (socket says 172.21.15.26:54988) 2024-09-15T03:45:01.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5baffd700 1 -- 172.21.15.26:0/2757624224 learned_addr learned my addr 172.21.15.26:0/2757624224 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 >> v1:172.21.15.122:6790/0 conn(0x7f200809faa0 legacy=0x7f200809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 monclient: found mon.noname-e 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080a6c50 con 0x7f20080a6880 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1ff80017b0 con 0x7f20080a6880 2024-09-15T03:45:01.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 5 monclient: authenticate success, global_id 4283 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: dump: 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.905 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028418584 0 0) 0x7fd5bc09d3e0 con 0x7fd5bc09e380 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient(hunting): my global_id is 4263 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a4003570 con 0x7fd5bc09e380 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ff8001a90 con 0x7f20080a6880 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 1 -- 172.21.15.26:0/1170364792 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1ff8004c30 con 0x7f20080a6880 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:01.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200559a700 10 monclient: dump: 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.908 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.909 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.909 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 4 set_mon_vals no callback set 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3162433890 0 0) 0x7fd5a4003570 con 0x7fd5bc09e380 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 >> v1:172.21.15.122:6792/0 conn(0x7fd5bc0a2a70 legacy=0x7fd5bc097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 >> v1:172.21.15.122:6791/0 conn(0x7fd5bc09bfc0 legacy=0x7fd5bc138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 monclient: found mon.noname-d 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc09d100 con 0x7fd5bc09e380 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 5 monclient: authenticate success, global_id 4263 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5b00017d0 con 0x7fd5bc09e380 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: dump: 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.911 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd5b0001af0 con 0x7fd5bc09e380 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 1 -- 172.21.15.26:0/2757624224 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5b0004c30 con 0x7fd5bc09e380 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 4 set_mon_vals no callback set 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5ba7fc700 10 monclient: dump: 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.913 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.914 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient: shutdown 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- 172.21.15.26:0/1170364792 >> v1:172.21.15.26:6793/0 conn(0x7f20080a6880 legacy=0x7f2008133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- 172.21.15.26:0/1170364792 shutdown_connections 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- 172.21.15.26:0/1170364792 >> 172.21.15.26:0/1170364792 conn(0x7f200806e2a0 msgr2=0x7f20080a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- 172.21.15.26:0/1170364792 shutdown_connections 2024-09-15T03:45:01.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- 172.21.15.26:0/1170364792 wait complete. 2024-09-15T03:45:01.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: shutdown 2024-09-15T03:45:01.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- 172.21.15.26:0/2757624224 >> v1:172.21.15.26:6792/0 conn(0x7fd5bc09e380 legacy=0x7fd5bc098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- 172.21.15.26:0/2757624224 shutdown_connections 2024-09-15T03:45:01.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- 172.21.15.26:0/2757624224 >> 172.21.15.26:0/2757624224 conn(0x7fd5bc063cd0 msgr2=0x7fd5bc0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- 172.21.15.26:0/2757624224 shutdown_connections 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 1 -- 172.21.15.26:0/2757624224 wait complete. 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient: monmap: 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.917 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 Processor -- start 2024-09-15T03:45:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 1 -- start start 2024-09-15T03:45:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7f200759e700 10 monclient: init 2024-09-15T03:45:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.859+0000 7fd5c0c32700 10 monclient: monmap: 2024-09-15T03:45:01.919 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.920 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.921 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 1 Processor -- start 2024-09-15T03:45:01.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 1 -- start start 2024-09-15T03:45:01.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient: init 2024-09-15T03:45:01.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 1 Processor -- start 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _add_conns ranks=[6,3,4,7,8,2,5,0,1] 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient(hunting): picked mon.noname-d con 0x7f200809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient(hunting): picked mon.noname-g con 0x7f20080a6880 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient(hunting): picked mon.noname-c con 0x7f2008138d30 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080a18c0 con 0x7f20080a6880 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200809cf10 con 0x7f2008138d30 2024-09-15T03:45:01.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2008131e70 con 0x7f200809faa0 2024-09-15T03:45:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient(hunting): authenticate will time out at 846.296585s 2024-09-15T03:45:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 1 Processor -- start 2024-09-15T03:45:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 1 -- start start 2024-09-15T03:45:01.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient: init 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient: _add_conns ranks=[7,3,1,4,5,8,2,0,6] 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient(hunting): picked mon.noname-i con 0x7efc040a2ab0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient(hunting): picked mon.noname-g con 0x7efc0412f7d0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient(hunting): picked mon.noname-f con 0x7efc041302f0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0404ca10 con 0x7efc041302f0 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc04098b90 con 0x7efc0412f7d0 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0409d3b0 con 0x7efc040a2ab0 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc0b33d700 10 monclient(hunting): authenticate will time out at 846.296862s 2024-09-15T03:45:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200659c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f200809faa0 0x7f2008130450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54998/0 (socket says 172.21.15.26:54998) 2024-09-15T03:45:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200659c700 1 -- 172.21.15.26:0/501807672 learned_addr learned my addr 172.21.15.26:0/501807672 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 1 -- start start 2024-09-15T03:45:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient: init 2024-09-15T03:45:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc090d9700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7efc040a2ab0 0x7efc0409e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:55886/0 (socket says 172.21.15.26:55886) 2024-09-15T03:45:01.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc090d9700 1 -- 172.21.15.26:0/503757326 learned_addr learned my addr 172.21.15.26:0/503757326 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7efc098da700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7efc041302f0 0x7efc0413b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:36022/0 (socket says 172.21.15.26:36022) 2024-09-15T03:45:01.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient: _add_conns ranks=[3,8,6,0,7,4,1,5,2] 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient(hunting): picked mon.noname-g con 0x7f92ac0a28b0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient(hunting): picked mon.noname-e con 0x7f92ac137a60 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient(hunting): picked mon.noname-d con 0x7f92ac137e60 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac04ca10 con 0x7f92ac0a28b0 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac09c4d0 con 0x7f92ac137e60 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac09cb10 con 0x7f92ac137a60 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92b277a700 10 monclient(hunting): authenticate will time out at 846.297429s 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92ab7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f92ac137a60 0x7f92ac13c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48790/0 (socket says 172.21.15.26:48790) 2024-09-15T03:45:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f92ab7fe700 1 -- 172.21.15.26:0/2823741604 learned_addr learned my addr 172.21.15.26:0/2823741604 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268400139 0 0) 0x7f2008131e70 con 0x7f200809faa0 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): my global_id is 4272 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8003570 con 0x7f200809faa0 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient: _add_conns ranks=[7,3,2,5,0,6,4,8,1] 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient(hunting): picked mon.noname-i con 0x7fd5bc09bfc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient(hunting): picked mon.noname-g con 0x7fd5bc09e380 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient(hunting): picked mon.noname-b con 0x7fd5bc0a2a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc11a5f0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc09cad0 con 0x7fd5bc09e380 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5bc11a310 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5c0c32700 10 monclient(hunting): authenticate will time out at 846.297881s 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267597528 0 0) 0x7f20080a18c0 con 0x7f20080a6880 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5baffd700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fd5bc09e380 0x7fd5bc14ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:33992/0 (socket says 172.21.15.26:33992) 2024-09-15T03:45:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7fd5baffd700 1 -- 172.21.15.26:0/3159662440 learned_addr learned my addr 172.21.15.26:0/3159662440 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): my global_id is 4296 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2008131e70 con 0x7f20080a6880 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3193731391 0 0) 0x7f1fe8003570 con 0x7f200809faa0 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20080a18c0 con 0x7f200809faa0 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1ff8004b70 con 0x7f200809faa0 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): dump: 2024-09-15T03:45:01.933 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.934 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202019813 0 0) 0x7f200809cf10 con 0x7f2008138d30 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient(hunting): my global_id is 4261 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8003570 con 0x7f2008138d30 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1887333785 0 0) 0x7f2008131e70 con 0x7f20080a6880 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f200809cf10 con 0x7f20080a6880 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4202160247 0 0) 0x7f20080a18c0 con 0x7f200809faa0 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 >> v1:172.21.15.26:6791/0 conn(0x7f2008138d30 legacy=0x7f200814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 >> v1:172.21.15.122:6790/0 conn(0x7f20080a6880 legacy=0x7f2008130b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 monclient: found mon.d 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008131270 con 0x7f200809faa0 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 5 monclient: authenticate success, global_id 4272 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _renew_subs 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20081346b0 con 0x7f200809faa0 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _renew_subs 2024-09-15T03:45:01.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2008133910 con 0x7f200809faa0 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ff8001860 con 0x7f200809faa0 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1ff8005230 con 0x7f200809faa0 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff7fff700 4 set_mon_vals no callback set 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 10 monclient: dump: 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.937 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.26:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1fec000f20 con 0x7f200809faa0 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.863+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1ff800fbf0 con 0x7f200809faa0 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f1ff8043fc0 con 0x7f200809faa0 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.122:6804/21256 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1fe8042100 con 0x7f1fe803e460 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7f1ff80443f0 con 0x7f200809faa0 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f1ff67fc700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f2005d9b700 10 monclient: get_auth_request con 0x7f1fe8037880 auth_method 0 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f2006d9d700 10 monclient: get_auth_request con 0x7f1fe803e460 auth_method 0 2024-09-15T03:45:01.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 --> v1:172.21.15.122:6800/21255 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd018002ba0 con 0x7fd01403f320 2024-09-15T03:45:01.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205174540 0 0) 0x7efc0404ca10 con 0x7efc041302f0 2024-09-15T03:45:01.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): my global_id is 4321 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf0003570 con 0x7efc041302f0 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366808984 0 0) 0x7efc0409d3b0 con 0x7efc040a2ab0 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): my global_id is 4363 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc0404ca10 con 0x7efc040a2ab0 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958188259 0 0) 0x7efc04098b90 con 0x7efc0412f7d0 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient(hunting): my global_id is 4314 2024-09-15T03:45:01.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc0409d3b0 con 0x7efc0412f7d0 2024-09-15T03:45:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984197709 0 0) 0x7f92ac04ca10 con 0x7f92ac0a28b0 2024-09-15T03:45:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): my global_id is 4305 2024-09-15T03:45:01.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9294003570 con 0x7f92ac0a28b0 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573929950 0 0) 0x7f92ac09cb10 con 0x7f92ac137a60 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): my global_id is 4292 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac04ca10 con 0x7f92ac137a60 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177593266 0 0) 0x7f92ac09c4d0 con 0x7f92ac137e60 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient(hunting): my global_id is 4281 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac09cb10 con 0x7f92ac137e60 2024-09-15T03:45:01.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571728938 0 0) 0x7fd5bc11a5f0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): my global_id is 4286 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a8003570 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183193700 0 0) 0x7fd5bc11a310 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): my global_id is 4372 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5bc11a5f0 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962497971 0 0) 0x7fd5bc09cad0 con 0x7fd5bc09e380 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 10 monclient(hunting): my global_id is 4323 2024-09-15T03:45:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5bc11a310 con 0x7fd5bc09e380 2024-09-15T03:45:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== osd.5 v1:172.21.15.122:6804/21256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f1fe8042100 con 0x7f1fe803e460 2024-09-15T03:45:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4041661672 0 0) 0x7f92ac09cb10 con 0x7f92ac137e60 2024-09-15T03:45:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 >> v1:172.21.15.26:6793/0 conn(0x7f92ac137a60 legacy=0x7f92ac13c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 >> v1:172.21.15.122:6790/0 conn(0x7f92ac0a28b0 legacy=0x7f92ac137350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 monclient: found mon.noname-d 2024-09-15T03:45:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0a2c80 con 0x7f92ac137e60 2024-09-15T03:45:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd0137fe700 1 -- 172.21.15.26:0/2151563545 <== osd.3 v1:172.21.15.122:6800/21255 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3580782990) 0x7fd018002ba0 con 0x7fd01403f320 2024-09-15T03:45:01.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4167376540 0 0) 0x7efc0409d3b0 con 0x7efc0412f7d0 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 >> v1:172.21.15.122:6792/0 conn(0x7efc040a2ab0 legacy=0x7efc0409e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 >> v1:172.21.15.122:6789/0 conn(0x7efc041302f0 legacy=0x7efc0413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 monclient: found mon.noname-g 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0409cd60 con 0x7efc0412f7d0 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92b277a700 5 monclient: authenticate success, global_id 4281 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92b277a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f929c001600 con 0x7f92ac137e60 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efbf4001600 con 0x7efc0412f7d0 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efc0b33d700 5 monclient: authenticate success, global_id 4314 2024-09-15T03:45:01.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efc0b33d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: dump: 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.949 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92b277a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 1 -- 172.21.15.26:0/2823741604 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f929c001870 con 0x7f92ac137e60 2024-09-15T03:45:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92aaffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92b277a700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7f92b277a700 4 set_mon_vals no callback set 2024-09-15T03:45:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: dump: 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.952 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efc0b33d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:01.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 >> v1:172.21.15.122:6800/21255 conn(0x7fd01403f320 legacy=0x7fd014041750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efbf4001870 con 0x7efc0412f7d0 2024-09-15T03:45:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.867+0000 7efbfbfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efbfbfff700 1 -- 172.21.15.26:0/503757326 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efbf4001f00 con 0x7efc0412f7d0 2024-09-15T03:45:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efc0b33d700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efc0b33d700 4 set_mon_vals no callback set 2024-09-15T03:45:01.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efbfbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efbfbfff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efbfbfff700 10 monclient: dump: 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.955 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient: shutdown 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- 172.21.15.26:0/2823741604 >> v1:172.21.15.26:6792/0 conn(0x7f92ac137e60 legacy=0x7f92ac13fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2015197980 0 0) 0x7fd5a8003570 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5bc09cad0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- 172.21.15.26:0/2823741604 shutdown_connections 2024-09-15T03:45:01.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- 172.21.15.26:0/2823741604 >> 172.21.15.26:0/2823741604 conn(0x7f92ac063ad0 msgr2=0x7f92ac0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1557076105 0 0) 0x7fd5bc11a5f0 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5a8003570 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2060262545 0 0) 0x7fd5bc11a310 con 0x7fd5bc09e380 2024-09-15T03:45:01.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5bc11a5f0 con 0x7fd5bc09e380 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5b4001bd0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): dump: 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.959 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5ac001820 con 0x7fd5bc09bfc0 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): dump: 2024-09-15T03:45:01.960 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.961 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5b0001790 con 0x7fd5bc09e380 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient(hunting): dump: 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.962 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2112941345 0 0) 0x7fd5bc09cad0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 >> v1:172.21.15.122:6792/0 conn(0x7fd5bc09bfc0 legacy=0x7fd5bc12cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 >> v1:172.21.15.122:6790/0 conn(0x7fd5bc09e380 legacy=0x7fd5bc14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 monclient: found mon.b 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc11a030 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 >> v1:172.21.15.122:6812/20895 conn(0x7fd0140388b0 legacy=0x7fd01403ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 10 monclient: shutdown 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 >> v1:172.21.15.26:6793/0 conn(0x7fd02c097e00 legacy=0x7fd02c11aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd02bfff700 1 -- 172.21.15.26:0/2151563545 reap_dead start 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 shutdown_connections 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 >> 172.21.15.26:0/2151563545 conn(0x7fd02c063dc0 msgr2=0x7fd02c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 shutdown_connections 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd030f76700 1 -- 172.21.15.26:0/2151563545 wait complete. 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 5 monclient: authenticate success, global_id 4286 2024-09-15T03:45:01.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 10 monclient: _renew_subs 2024-09-15T03:45:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc09ac00 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- 172.21.15.26:0/2823741604 shutdown_connections 2024-09-15T03:45:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efc0b33d700 10 monclient: shutdown 2024-09-15T03:45:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efc0b33d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:01.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7efc0b33d700 1 -- 172.21.15.26:0/503757326 >> v1:172.21.15.122:6790/0 conn(0x7efc0412f7d0 legacy=0x7efc0412fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 10 monclient: _renew_subs 2024-09-15T03:45:01.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc12d310 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd5b4001df0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5b9ffb700 4 set_mon_vals no callback set 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5b40022a0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.26:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd59c000f20 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 10 monclient: dump: 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.969 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- 172.21.15.26:0/2823741604 wait complete. 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient: monmap: 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.971 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 Processor -- start 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- start start 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient: init 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient: _add_conns ranks=[5,8,4,2,7,1,3,0,6] 2024-09-15T03:45:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient(hunting): picked mon.noname-h con 0x7f92ac0a28b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient(hunting): picked mon.noname-e con 0x7f92ac137a60 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient(hunting): picked mon.noname-c con 0x7f92ac137e60 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac09d000 con 0x7f92ac137e60 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac09d4e0 con 0x7f92ac0a28b0 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac1356c0 con 0x7f92ac137a60 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92b277a700 10 monclient(hunting): authenticate will time out at 846.307442s 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92ab7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f92ac137a60 0x7f92ac11fce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48806/0 (socket says 172.21.15.26:48806) 2024-09-15T03:45:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7f92ab7fe700 1 -- 172.21.15.26:0/1242558309 learned_addr learned my addr 172.21.15.26:0/1242558309 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd5b40024c0 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7fd5b4041570 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.871+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.122:6808/21254 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5a8042a50 con 0x7fd5a803edb0 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (unknown 3847591702 0 0) 0x7fd5b4041930 con 0x7fd5bc0a2a70 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7fd5a37fe700 10 monclient: handle_get_version_reply finishing 1 version 16 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7fd5bb7fe700 10 monclient: get_auth_request con 0x7fd5a8038340 auth_method 0 2024-09-15T03:45:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- 172.21.15.26:0/503757326 shutdown_connections 2024-09-15T03:45:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- 172.21.15.26:0/503757326 >> 172.21.15.26:0/503757326 conn(0x7efc04063ad0 msgr2=0x7efc040a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344336217 0 0) 0x7f92ac1356c0 con 0x7f92ac137a60 2024-09-15T03:45:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): my global_id is 4301 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92940031b0 con 0x7f92ac137a60 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950326552 0 0) 0x7f92ac09d000 con 0x7f92ac137e60 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): my global_id is 4270 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac1356c0 con 0x7f92ac137e60 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543496248 0 0) 0x7f92ac09d4e0 con 0x7f92ac0a28b0 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): my global_id is 4298 2024-09-15T03:45:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- 172.21.15.26:0/503757326 shutdown_connections 2024-09-15T03:45:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ac09d000 con 0x7f92ac0a28b0 2024-09-15T03:45:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7fd5baffd700 10 monclient: get_auth_request con 0x7fd5a803edb0 auth_method 0 2024-09-15T03:45:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4009661610 0 0) 0x7f92940031b0 con 0x7f92ac137a60 2024-09-15T03:45:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92ac09d4e0 con 0x7f92ac137a60 2024-09-15T03:45:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- 172.21.15.26:0/503757326 wait complete. 2024-09-15T03:45:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1504472980 0 0) 0x7f92ac1356c0 con 0x7f92ac137e60 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92940031b0 con 0x7f92ac137e60 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92a00019e0 con 0x7f92ac137a60 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): dump: 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7f9293fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient: build_initial_monmap 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient: monmap: 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 Processor -- start 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- start start 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient: init 2024-09-15T03:45:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== osd.4 v1:172.21.15.122:6808/21254 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fd5a8042a50 con 0x7fd5a803edb0 2024-09-15T03:45:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient: _add_conns ranks=[2,5,4,8,7,1,3,0,6] 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient(hunting): picked mon.noname-b con 0x7efc040a2ab0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient(hunting): picked mon.noname-h con 0x7efc0412f7d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient(hunting): picked mon.noname-c con 0x7efc041302f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0406e360 con 0x7efc040a2ab0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0406dfa0 con 0x7efc041302f0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc04146f80 con 0x7efc0412f7d0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc0b33d700 10 monclient(hunting): authenticate will time out at 846.310882s 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc090d9700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7efc040a2ab0 0x7efc04146160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54260/0 (socket says 172.21.15.26:54260) 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efc090d9700 1 -- 172.21.15.26:0/186200709 learned_addr learned my addr 172.21.15.26:0/186200709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897871905 0 0) 0x7efc0406e360 con 0x7efc040a2ab0 2024-09-15T03:45:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): my global_id is 4295 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe8003570 con 0x7efc040a2ab0 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216351952 0 0) 0x7efc0406dfa0 con 0x7efc041302f0 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): my global_id is 4279 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc0406e360 con 0x7efc041302f0 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519733021 0 0) 0x7efc04146f80 con 0x7efc0412f7d0 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): my global_id is 4307 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc0406dfa0 con 0x7efc0412f7d0 2024-09-15T03:45:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3844215115 0 0) 0x7efbe8003570 con 0x7efc040a2ab0 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efc04146f80 con 0x7efc040a2ab0 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efbfc001d80 con 0x7efc040a2ab0 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): dump: 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1042294325 0 0) 0x7efc0406e360 con 0x7efc041302f0 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efbe8003570 con 0x7efc041302f0 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efc00001830 con 0x7efc041302f0 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.875+0000 7efbf97fa700 10 monclient(hunting): dump: 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.990 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.990 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.990 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.990 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1549820480 0 0) 0x7f92ac09d4e0 con 0x7f92ac137a60 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 >> v1:172.21.15.122:6791/0 conn(0x7f92ac0a28b0 legacy=0x7f92ac14e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 >> v1:172.21.15.26:6791/0 conn(0x7f92ac137e60 legacy=0x7f92ac156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 monclient: found mon.e 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac136a30 con 0x7f92ac137a60 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 5 monclient: authenticate success, global_id 4301 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 10 monclient: _renew_subs 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac14ef80 con 0x7f92ac137a60 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 10 monclient: _renew_subs 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac135b40 con 0x7f92ac137a60 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f92a0001d20 con 0x7f92ac137a60 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92a0004e20 con 0x7f92ac137a60 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 10 monclient: dump: 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92aa7fc700 4 set_mon_vals no callback set 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f92a0004fc0 con 0x7f92ac137a60 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f92a0043c80 con 0x7f92ac137a60 2024-09-15T03:45:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.879+0000 7f92abfff700 10 monclient: get_auth_request con 0x7f92940380e0 auth_method 0 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3413022702 0 0) 0x7efc0406dfa0 con 0x7efc0412f7d0 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efc0406e360 con 0x7efc0412f7d0 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 917048224 0 0) 0x7efbe8003570 con 0x7efc041302f0 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 >> v1:172.21.15.122:6791/0 conn(0x7efc0412f7d0 legacy=0x7efc04146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 >> v1:172.21.15.26:6790/0 conn(0x7efc040a2ab0 legacy=0x7efc04146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: _finish_hunting 0 2024-09-15T03:45:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 monclient: found mon.c 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0406b810 con 0x7efc041302f0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: _finish_auth 0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 5 monclient: authenticate success, global_id 4279 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 10 monclient: _renew_subs 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0406b300 con 0x7efc041302f0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 10 monclient: _renew_subs 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc0b33d700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0406bed0 con 0x7efc041302f0 2024-09-15T03:45:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efc00004830 con 0x7efc041302f0 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efc00001990 con 0x7efc041302f0 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbfb7fe700 4 set_mon_vals no callback set 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 10 monclient: dump: 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:01.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 --> v1:172.21.15.26:6808/21148 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0de8002ba0 con 0x7f0de403e800 2024-09-15T03:45:01.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7efc0000f540 con 0x7efc041302f0 2024-09-15T03:45:01.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0dfa7fc700 1 -- 172.21.15.26:0/3666561426 <== osd.2 v1:172.21.15.26:6808/21148 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 654699149) 0x7f0de8002ba0 con 0x7f0de403e800 2024-09-15T03:45:01.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7efc00043a30 con 0x7efc041302f0 2024-09-15T03:45:01.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7efc090d9700 10 monclient: get_auth_request con 0x7efbe8037ff0 auth_method 0 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 >> v1:172.21.15.26:6808/21148 conn(0x7f0de403e800 legacy=0x7f0de4040c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 >> v1:172.21.15.122:6812/20895 conn(0x7f0de4037c20 legacy=0x7f0de403a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 10 monclient: shutdown 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 >> v1:172.21.15.26:6790/0 conn(0x7f0e0406e160 legacy=0x7f0e04132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e02d9d700 1 -- 172.21.15.26:0/3666561426 reap_dead start 2024-09-15T03:45:02.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 shutdown_connections 2024-09-15T03:45:02.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 >> 172.21.15.26:0/3666561426 conn(0x7f0e04063cd0 msgr2=0x7f0e040a1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 shutdown_connections 2024-09-15T03:45:02.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.883+0000 7f0e0359e700 1 -- 172.21.15.26:0/3666561426 wait complete. 2024-09-15T03:45:02.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.887+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6800/21130 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbe0005330 con 0x7efbe00015d0 2024-09-15T03:45:02.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.887+0000 7efc088d8700 10 monclient: get_auth_request con 0x7efbe00015d0 auth_method 0 2024-09-15T03:45:02.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.887+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== osd.1 v1:172.21.15.26:6800/21130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7efbe0005330 con 0x7efbe00015d0 2024-09-15T03:45:02.002 INFO:teuthology.orchestra.run.smithi026.stdout:47244640261 2024-09-15T03:45:02.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T03:45:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.887+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 --> v1:172.21.15.122:6804/21256 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1fec002ba0 con 0x7f1fe803e460 2024-09-15T03:45:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.887+0000 7f1ff67fc700 1 -- 172.21.15.26:0/501807672 <== osd.5 v1:172.21.15.122:6804/21256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 654699149) 0x7f1fec002ba0 con 0x7f1fe803e460 2024-09-15T03:45:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.891+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6804/21131 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f928c005330 con 0x7f928c0015d0 2024-09-15T03:45:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.891+0000 7f92b0d17700 10 monclient: get_auth_request con 0x7f928c0015d0 auth_method 0 2024-09-15T03:45:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.891+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== osd.0 v1:172.21.15.26:6804/21131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f928c005330 con 0x7f928c0015d0 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 >> v1:172.21.15.122:6804/21256 conn(0x7f1fe803e460 legacy=0x7f1fe8040890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 >> v1:172.21.15.122:6812/20895 conn(0x7f1fe8037880 legacy=0x7f1fe8039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 10 monclient: shutdown 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 >> v1:172.21.15.26:6792/0 conn(0x7f200809faa0 legacy=0x7f2008130450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f2006d9d700 1 -- 172.21.15.26:0/501807672 reap_dead start 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 shutdown_connections 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 >> 172.21.15.26:0/501807672 conn(0x7f200806e2a0 msgr2=0x7f200812ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 shutdown_connections 2024-09-15T03:45:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7f200759e700 1 -- 172.21.15.26:0/501807672 wait complete. 2024-09-15T03:45:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 --> v1:172.21.15.122:6808/21254 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd59c002ba0 con 0x7fd5a803edb0 2024-09-15T03:45:02.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5a37fe700 1 -- 172.21.15.26:0/3159662440 <== osd.4 v1:172.21.15.122:6808/21254 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1800444376) 0x7fd59c002ba0 con 0x7fd5a803edb0 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 >> v1:172.21.15.122:6808/21254 conn(0x7fd5a803edb0 legacy=0x7fd5a80411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 >> v1:172.21.15.122:6812/20895 conn(0x7fd5a8038340 legacy=0x7fd5a803a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5c0c32700 10 monclient: shutdown 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5c0c32700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.895+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 >> v1:172.21.15.26:6790/0 conn(0x7fd5bc0a2a70 legacy=0x7fd5bc14e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.899+0000 7fd5bbfff700 1 -- 172.21.15.26:0/3159662440 reap_dead start 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.899+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 shutdown_connections 2024-09-15T03:45:02.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.899+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 >> 172.21.15.26:0/3159662440 conn(0x7fd5bc063cd0 msgr2=0x7fd5bc12f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.899+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 shutdown_connections 2024-09-15T03:45:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.899+0000 7fd5c0c32700 1 -- 172.21.15.26:0/3159662440 wait complete. 2024-09-15T03:45:02.008 INFO:teuthology.orchestra.run.smithi026.stdout:47244640260 2024-09-15T03:45:02.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T03:45:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 --> v1:172.21.15.26:6800/21130 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efbe0006f80 con 0x7efbe00015d0 2024-09-15T03:45:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbf97fa700 1 -- 172.21.15.26:0/186200709 <== osd.1 v1:172.21.15.26:6800/21130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3580782990) 0x7efbe0006f80 con 0x7efbe00015d0 2024-09-15T03:45:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 >> v1:172.21.15.26:6800/21130 conn(0x7efbe00015d0 legacy=0x7efbe0003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 >> v1:172.21.15.122:6812/20895 conn(0x7efbe8037ff0 legacy=0x7efbe803a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 10 monclient: shutdown 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 >> v1:172.21.15.26:6791/0 conn(0x7efc041302f0 legacy=0x7efc0414e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efc098da700 1 -- 172.21.15.26:0/186200709 reap_dead start 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 shutdown_connections 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 >> 172.21.15.26:0/186200709 conn(0x7efc04063ad0 msgr2=0x7efc0406f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 shutdown_connections 2024-09-15T03:45:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.907+0000 7efbeeffd700 1 -- 172.21.15.26:0/186200709 wait complete. 2024-09-15T03:45:02.012 INFO:teuthology.orchestra.run.smithi026.stdout:47244640260 2024-09-15T03:45:02.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T03:45:02.013 INFO:teuthology.orchestra.run.smithi026.stdout:47244640259 2024-09-15T03:45:02.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T03:45:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 --> v1:172.21.15.26:6804/21131 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f928c006f80 con 0x7f928c0015d0 2024-09-15T03:45:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f9293fff700 1 -- 172.21.15.26:0/1242558309 <== osd.0 v1:172.21.15.26:6804/21131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 654699149) 0x7f928c006f80 con 0x7f928c0015d0 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 >> v1:172.21.15.26:6804/21131 conn(0x7f928c0015d0 legacy=0x7f928c003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 >> v1:172.21.15.122:6812/20895 conn(0x7f92940380e0 legacy=0x7f929403a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 10 monclient: shutdown 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 >> v1:172.21.15.26:6793/0 conn(0x7f92ac137a60 legacy=0x7f92ac11fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b0d17700 1 -- 172.21.15.26:0/1242558309 reap_dead start 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 shutdown_connections 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 >> 172.21.15.26:0/1242558309 conn(0x7f92ac063ad0 msgr2=0x7f92ac0a14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 shutdown_connections 2024-09-15T03:45:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:01.911+0000 7f92b277a700 1 -- 172.21.15.26:0/1242558309 wait complete. 2024-09-15T03:45:02.016 INFO:teuthology.orchestra.run.smithi026.stdout:47244640261 2024-09-15T03:45:02.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T03:45:02.017 INFO:teuthology.orchestra.run.smithi026.stdout:47244640260 2024-09-15T03:45:02.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient: get_monmap_and_config 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient: monmap: 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:02.158103+0000 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:02.158103+0000 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7ff7b08c2700 10 monclient: get_monmap_and_config 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7ff7b08c2700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7ff7b08c2700 10 monclient: monmap: 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:02.158863+0000 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:02.158863+0000 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.163 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 1 Processor -- start 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 1 -- start start 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient: init 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient: _add_conns ranks=[8,7,4,2,6,3,5,1,0] 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient(hunting): picked mon.noname-e con 0x7f744406d250 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient(hunting): picked mon.noname-i con 0x7f744406d620 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient(hunting): picked mon.noname-c con 0x7f7444139050 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744404ca10 con 0x7f7444139050 2024-09-15T03:45:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744409cf40 con 0x7f744406d620 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7444098330 con 0x7f744406d250 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.151+0000 7f74489be700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 10 monclient(hunting): authenticate will time out at 846.588011s 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74437fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f744406d250 0x7f744409def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48820/0 (socket says 172.21.15.26:48820) 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74437fe700 1 -- 172.21.15.26:0/4135349260 learned_addr learned my addr 172.21.15.26:0/4135349260 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 1 Processor -- start 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 1 -- start start 2024-09-15T03:45:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient: init 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449667644 0 0) 0x7f744404ca10 con 0x7f7444139050 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): my global_id is 4288 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f742c003570 con 0x7f7444139050 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005315059 0 0) 0x7f7444098330 con 0x7f744406d250 2024-09-15T03:45:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): my global_id is 4310 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f744404ca10 con 0x7f744406d250 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450253199 0 0) 0x7f744409cf40 con 0x7f744406d620 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient(hunting): my global_id is 4381 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7444098330 con 0x7f744406d620 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400314325 0 0) 0x7f744404ca10 con 0x7f744406d250 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient: _add_conns ranks=[5,1,7,4,0,3,8,2,6] 2024-09-15T03:45:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 >> v1:172.21.15.122:6792/0 conn(0x7f744406d620 legacy=0x7f744406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient(hunting): picked mon.noname-h con 0x7ff7ac0a4ae0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient(hunting): picked mon.noname-f con 0x7ff7ac09de50 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 >> v1:172.21.15.26:6791/0 conn(0x7f7444139050 legacy=0x7f744413b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 monclient: found mon.noname-e 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744409cc60 con 0x7f744406d250 2024-09-15T03:45:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient(hunting): picked mon.noname-i con 0x7ff7ac09e260 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac04ca10 con 0x7ff7ac09de50 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0a4eb0 con 0x7ff7ac0a4ae0 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0a0910 con 0x7ff7ac09e260 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient(hunting): authenticate will time out at 846.589690s 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7434001750 con 0x7f744406d250 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: dump: 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7aad9d700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7ff7ac09e260 0x7ff7ac13fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:55912/0 (socket says 172.21.15.26:55912) 2024-09-15T03:45:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7aad9d700 1 -- 172.21.15.26:0/2245445278 learned_addr learned my addr 172.21.15.26:0/2245445278 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 5 monclient: authenticate success, global_id 4310 2024-09-15T03:45:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f74340019e0 con 0x7f744406d250 2024-09-15T03:45:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 1 -- 172.21.15.26:0/4135349260 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7434004c30 con 0x7f744406d250 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 4 set_mon_vals no callback set 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74427fc700 10 monclient: dump: 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.175 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300164762 0 0) 0x7ff7ac0a0910 con 0x7ff7ac09e260 2024-09-15T03:45:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient(hunting): my global_id is 4390 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff794003570 con 0x7ff7ac09e260 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286557094 0 0) 0x7ff7ac0a4eb0 con 0x7ff7ac0a4ae0 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient(hunting): my global_id is 4316 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ac0a0910 con 0x7ff7ac0a4ae0 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311279589 0 0) 0x7ff794003570 con 0x7ff7ac09e260 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 >> v1:172.21.15.122:6791/0 conn(0x7ff7ac0a4ae0 legacy=0x7ff7ac09b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 >> v1:172.21.15.122:6789/0 conn(0x7ff7ac09de50 legacy=0x7ff7ac13c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 monclient: found mon.noname-i 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac068730 con 0x7ff7ac09e260 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7a4001b30 con 0x7ff7ac09e260 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: dump: 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.180 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 10 monclient: shutdown 2024-09-15T03:45:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 1 -- 172.21.15.26:0/4135349260 >> v1:172.21.15.26:6793/0 conn(0x7f744406d250 legacy=0x7f744409def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 1 -- 172.21.15.26:0/4135349260 shutdown_connections 2024-09-15T03:45:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 1 -- 172.21.15.26:0/4135349260 >> 172.21.15.26:0/4135349260 conn(0x7f7444063dc0 msgr2=0x7f74440a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 5 monclient: authenticate success, global_id 4390 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7a4001e10 con 0x7ff7ac09e260 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 1 -- 172.21.15.26:0/2245445278 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7a4002000 con 0x7ff7ac09e260 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7b08c2700 4 set_mon_vals no callback set 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7ff7a959a700 10 monclient: dump: 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.183 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 1 -- 172.21.15.26:0/4135349260 shutdown_connections 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.155+0000 7f74489be700 1 -- 172.21.15.26:0/4135349260 wait complete. 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient: monmap: 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.185 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient: shutdown 2024-09-15T03:45:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/2245445278 >> v1:172.21.15.122:6792/0 conn(0x7ff7ac09e260 legacy=0x7ff7ac13fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 1 Processor -- start 2024-09-15T03:45:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/2245445278 shutdown_connections 2024-09-15T03:45:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/2245445278 >> 172.21.15.26:0/2245445278 conn(0x7ff7ac063a10 msgr2=0x7ff7ac0a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 1 -- start start 2024-09-15T03:45:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient: init 2024-09-15T03:45:02.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/2245445278 shutdown_connections 2024-09-15T03:45:02.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/2245445278 wait complete. 2024-09-15T03:45:02.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient: monmap: 2024-09-15T03:45:02.188 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.190 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.190 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.190 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient: _add_conns ranks=[0,5,3,4,2,7,6,8,1] 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient(hunting): picked mon.noname-a con 0x7f744406d250 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient(hunting): picked mon.noname-h con 0x7f744406d620 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient(hunting): picked mon.noname-g con 0x7f7444139050 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7444098ca0 con 0x7f744406d250 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7444098840 con 0x7f7444139050 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7444118680 con 0x7f744406d620 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74489be700 10 monclient(hunting): authenticate will time out at 846.593775s 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74437fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f744406d250 0x7f7444136800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53030/0 (socket says 172.21.15.26:53030) 2024-09-15T03:45:02.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7f74437fe700 1 -- 172.21.15.26:0/1507651588 learned_addr learned my addr 172.21.15.26:0/1507651588 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 Processor -- start 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- start start 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient: init 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient: _add_conns ranks=[1,0,3,7,2,4,8,5,6] 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient(hunting): picked mon.noname-f con 0x7ff7ac09de50 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient(hunting): picked mon.noname-a con 0x7ff7ac09e260 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient(hunting): picked mon.noname-g con 0x7ff7ac0a4ae0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac06e370 con 0x7ff7ac09e260 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7a9d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff7ac09e260 0x7ff7ac11ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53044/0 (socket says 172.21.15.26:53044) 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7a9d9b700 1 -- 172.21.15.26:0/902858224 learned_addr learned my addr 172.21.15.26:0/902858224 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac09aef0 con 0x7ff7ac09de50 2024-09-15T03:45:02.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac11d8d0 con 0x7ff7ac0a4ae0 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff7b08c2700 10 monclient(hunting): authenticate will time out at 846.595079s 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237189820 0 0) 0x7ff7ac06e370 con 0x7ff7ac09e260 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): my global_id is 4230 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff794004aa0 con 0x7ff7ac09e260 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258050750 0 0) 0x7ff7ac09aef0 con 0x7ff7ac09de50 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): my global_id is 4339 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ac06e370 con 0x7ff7ac09de50 2024-09-15T03:45:02.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741469746 0 0) 0x7ff7ac11d8d0 con 0x7ff7ac0a4ae0 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): my global_id is 4341 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ac09aef0 con 0x7ff7ac0a4ae0 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1558448971 0 0) 0x7ff794004aa0 con 0x7ff7ac09e260 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7ac11d8d0 con 0x7ff7ac09e260 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7a0001a70 con 0x7ff7ac09e260 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.197 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.159+0000 7ff79a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:02.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 533344072 0 0) 0x7ff7ac06e370 con 0x7ff7ac09de50 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff794004aa0 con 0x7ff7ac09de50 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff79c0018f0 con 0x7ff7ac09de50 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.199 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 102867808 0 0) 0x7ff7ac11d8d0 con 0x7ff7ac09e260 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 >> v1:172.21.15.122:6790/0 conn(0x7ff7ac0a4ae0 legacy=0x7ff7ac11c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 >> v1:172.21.15.122:6789/0 conn(0x7ff7ac09de50 legacy=0x7ff7ac11b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 monclient: found mon.a 2024-09-15T03:45:02.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac11d210 con 0x7ff7ac09e260 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 5 monclient: authenticate success, global_id 4230 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 monclient: _renew_subs 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac1276c0 con 0x7ff7ac09e260 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 monclient: _renew_subs 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac139120 con 0x7ff7ac09e260 2024-09-15T03:45:02.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7a0002270 con 0x7ff7ac09e260 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7a00027a0 con 0x7ff7ac09e260 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 10 monclient: dump: 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.202 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7a8d99700 4 set_mon_vals no callback set 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634691110 0 0) 0x7f7444098ca0 con 0x7f744406d250 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): my global_id is 4221 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7430003570 con 0x7f744406d250 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169173600 0 0) 0x7f7444118680 con 0x7f744406d620 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): my global_id is 4325 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7444098ca0 con 0x7f744406d620 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147576567 0 0) 0x7f7444098840 con 0x7f7444139050 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): my global_id is 4332 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7444118680 con 0x7f7444139050 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2246883367 0 0) 0x7f7430003570 con 0x7f744406d250 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7444098840 con 0x7f744406d250 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7434001810 con 0x7f744406d250 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): dump: 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.205 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff7a0002900 con 0x7ff7ac09e260 2024-09-15T03:45:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7ff7a0043d60 con 0x7ff7ac09e260 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 492632155 0 0) 0x7f7444098ca0 con 0x7f744406d620 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7430003570 con 0x7f744406d620 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1117131584 0 0) 0x7f7444118680 con 0x7f7444139050 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7444098ca0 con 0x7f7444139050 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 255875425 0 0) 0x7f7444098840 con 0x7f744406d250 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 >> v1:172.21.15.122:6791/0 conn(0x7f744406d620 legacy=0x7f7444136f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 >> v1:172.21.15.122:6790/0 conn(0x7f7444139050 legacy=0x7f74441499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 monclient: found mon.a 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7444137620 con 0x7f744406d250 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 5 monclient: authenticate success, global_id 4221 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 10 monclient: _renew_subs 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7444119980 con 0x7f744406d250 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 10 monclient: _renew_subs 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f744412fec0 con 0x7f744406d250 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7434004ef0 con 0x7f744406d250 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f74340011b0 con 0x7f744406d250 2024-09-15T03:45:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f74417fa700 4 set_mon_vals no callback set 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 10 monclient: dump: 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f743400fc00 con 0x7f744406d250 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7aa59c700 10 monclient: get_auth_request con 0x7ff794037f90 auth_method 0 2024-09-15T03:45:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.163+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7900050b0 con 0x7ff7ac09e260 2024-09-15T03:45:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.167+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f7434043e60 con 0x7f744406d250 2024-09-15T03:45:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.167+0000 7f7442ffd700 10 monclient: get_auth_request con 0x7f7430037f60 auth_method 0 2024-09-15T03:45:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.167+0000 7f74489be700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.167+0000 7f74489be700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.167+0000 7f74489be700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.167+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74240050b0 con 0x7f744406d250 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.171+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ff7a0014ca0 con 0x7ff7ac09e260 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.171+0000 7ff79a7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.171+0000 7ff79a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.175+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f7434014b50 con 0x7f744406d250 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.175+0000 7f742b7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.175+0000 7f742b7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7f5ceed72700 10 monclient: get_monmap_and_config 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7f5ceed72700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7f5ceed72700 10 monclient: monmap: 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:02.197921+0000 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:02.197921+0000 2024-09-15T03:45:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.215 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient: get_monmap_and_config 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient: monmap: 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:02.197948+0000 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:02.197948+0000 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 1 Processor -- start 2024-09-15T03:45:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7f5ceed72700 1 Processor -- start 2024-09-15T03:45:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 1 -- start start 2024-09-15T03:45:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient: init 2024-09-15T03:45:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient: _add_conns ranks=[1,5,2,3,8,7,6,0,4] 2024-09-15T03:45:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient(hunting): picked mon.noname-f con 0x7ffa780a2710 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient(hunting): picked mon.noname-h con 0x7ffa7809d900 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient(hunting): picked mon.noname-b con 0x7ffa7809e3e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7f5ceed72700 1 -- start start 2024-09-15T03:45:02.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7f5ceed72700 10 monclient: init 2024-09-15T03:45:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7804ca10 con 0x7ffa780a2710 2024-09-15T03:45:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa78098780 con 0x7ffa7809e3e0 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780a2ae0 con 0x7ffa7809d900 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7f0a1700 10 monclient(hunting): authenticate will time out at 846.627673s 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7d63e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ffa7809e3e0 0x7ffa78138f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54268/0 (socket says 172.21.15.26:54268) 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.191+0000 7ffa7d63e700 1 -- 172.21.15.26:0/1455546566 learned_addr learned my addr 172.21.15.26:0/1455546566 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507661985 0 0) 0x7ffa78098780 con 0x7ffa7809e3e0 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient(hunting): my global_id is 4304 2024-09-15T03:45:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa60003570 con 0x7ffa7809e3e0 2024-09-15T03:45:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient: _add_conns ranks=[0,6,5,2,4,8,1,3,7] 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient(hunting): picked mon.noname-a con 0x7f5ce80a2bd0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient(hunting): picked mon.noname-d con 0x7f5ce809b9e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient(hunting): picked mon.noname-h con 0x7f5ce8134f90 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce804ca10 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce809e240 con 0x7f5ce8134f90 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce812f8c0 con 0x7f5ce809b9e0 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient(hunting): authenticate will time out at 846.628684s 2024-09-15T03:45:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce7fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5ce809b9e0 0x7f5ce809bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55030/0 (socket says 172.21.15.26:55030) 2024-09-15T03:45:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce7fff700 1 -- 172.21.15.26:0/209466028 learned_addr learned my addr 172.21.15.26:0/209466028 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5cecb0e700 1 --1- 172.21.15.26:0/209466028 >> v1:172.21.15.26:6789/0 conn(0x7f5ce80a2bd0 0x7f5ce812f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53046/0 (socket says 172.21.15.26:53046) 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111350756 0 0) 0x7ffa780a2ae0 con 0x7ffa7809d900 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient(hunting): my global_id is 4334 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa78098780 con 0x7ffa7809d900 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726570525 0 0) 0x7ffa60003570 con 0x7ffa7809e3e0 2024-09-15T03:45:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 >> v1:172.21.15.122:6791/0 conn(0x7ffa7809d900 legacy=0x7ffa7809dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122798340 0 0) 0x7f5ce804ca10 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient(hunting): my global_id is 4239 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd0003570 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878766340 0 0) 0x7f5ce812f8c0 con 0x7f5ce809b9e0 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient(hunting): my global_id is 4290 2024-09-15T03:45:02.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce804ca10 con 0x7f5ce809b9e0 2024-09-15T03:45:02.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 >> v1:172.21.15.122:6789/0 conn(0x7ffa780a2710 legacy=0x7ffa78098070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 monclient: found mon.noname-b 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7809ceb0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 5 monclient: authenticate success, global_id 4304 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa6c001730 con 0x7ffa7809e3e0 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: dump: 2024-09-15T03:45:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa6c001950 con 0x7ffa7809e3e0 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 1 -- 172.21.15.26:0/1455546566 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa6c0048f0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa777fe700 10 monclient: dump: 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.232 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:02.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 4 set_mon_vals no callback set 2024-09-15T03:45:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 10 monclient: shutdown 2024-09-15T03:45:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/1455546566 >> v1:172.21.15.26:6790/0 conn(0x7ffa7809e3e0 legacy=0x7ffa78138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515004897 0 0) 0x7f5cd0003570 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 >> v1:172.21.15.26:6792/0 conn(0x7f5ce809b9e0 legacy=0x7f5ce809bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 >> v1:172.21.15.122:6791/0 conn(0x7f5ce8134f90 legacy=0x7f5ce8137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 monclient: found mon.noname-a 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce809c4c0 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ce0001b30 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: dump: 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.236 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.237 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 5 monclient: authenticate success, global_id 4239 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ce0001d50 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 1 -- 172.21.15.26:0/209466028 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ce0004bb0 con 0x7f5ce80a2bd0 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 4 set_mon_vals no callback set 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:02.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ce77fe700 10 monclient: dump: 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient: shutdown 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- 172.21.15.26:0/209466028 >> v1:172.21.15.26:6789/0 conn(0x7f5ce80a2bd0 legacy=0x7f5ce812f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/1455546566 shutdown_connections 2024-09-15T03:45:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/1455546566 >> 172.21.15.26:0/1455546566 conn(0x7ffa78063ad0 msgr2=0x7ffa780a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/1455546566 shutdown_connections 2024-09-15T03:45:02.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/1455546566 wait complete. 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- 172.21.15.26:0/209466028 shutdown_connections 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- 172.21.15.26:0/209466028 >> 172.21.15.26:0/209466028 conn(0x7f5ce8063ad0 msgr2=0x7f5ce80a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- 172.21.15.26:0/209466028 shutdown_connections 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 1 -- 172.21.15.26:0/209466028 wait complete. 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fe23361e700 10 monclient: get_monmap_and_config 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fe23361e700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fe23361e700 10 monclient: monmap: 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:02.202599+0000 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:02.202599+0000 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fe23361e700 1 Processor -- start 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fe23361e700 1 -- start start 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fe23361e700 10 monclient: init 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fd9914f6700 10 monclient: get_monmap_and_config 2024-09-15T03:45:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fd9914f6700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fd9914f6700 10 monclient: monmap: 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:02.202490+0000 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:02.202490+0000 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fd9914f6700 1 Processor -- start 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fd9914f6700 1 -- start start 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7fd9914f6700 10 monclient: init 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7f5ceed72700 10 monclient: monmap: 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.195+0000 7ffa7f0a1700 10 monclient: monmap: 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 1 Processor -- start 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 1 -- start start 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient: init 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient: _add_conns ranks=[2,3,8,4,7,0,5,1,6] 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient(hunting): picked mon.noname-b con 0x7fe22c0a2ab0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient(hunting): picked mon.noname-g con 0x7fe22c12f7d0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient(hunting): picked mon.noname-e con 0x7fe22c1302f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c04ca10 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c098b90 con 0x7fe22c12f7d0 2024-09-15T03:45:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c09d3b0 con 0x7fe22c1302f0 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient(hunting): authenticate will time out at 846.632681s 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe231bbb700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fe22c1302f0 0x7fe22c13b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48842/0 (socket says 172.21.15.26:48842) 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe231bbb700 1 -- 172.21.15.26:0/3136318276 learned_addr learned my addr 172.21.15.26:0/3136318276 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe2313ba700 1 --1- 172.21.15.26:0/3136318276 >> v1:172.21.15.26:6790/0 conn(0x7fe22c0a2ab0 0x7fe22c09e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54270/0 (socket says 172.21.15.26:54270) 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient: _add_conns ranks=[7,8,6,1,5,2,0,3,4] 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient(hunting): picked mon.noname-i con 0x7fd98c0a2910 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient(hunting): picked mon.noname-e con 0x7fd98c09e0a0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient(hunting): picked mon.noname-d con 0x7fd98c139230 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c04ca10 con 0x7fd98c139230 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c09caf0 con 0x7fd98c0a2910 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c098880 con 0x7fd98c09e0a0 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient(hunting): authenticate will time out at 846.632650s 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd98b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd98c139230 0x7fd98c13b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55042/0 (socket says 172.21.15.26:55042) 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd98b7fe700 1 -- 172.21.15.26:0/3759088277 learned_addr learned my addr 172.21.15.26:0/3759088277 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 1 Processor -- start 2024-09-15T03:45:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 1 -- start start 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient: init 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient: _add_conns ranks=[1,7,0,8,5,6,3,2,4] 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient(hunting): picked mon.noname-f con 0x7f5ce809b9e0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient(hunting): picked mon.noname-i con 0x7f5ce8134f90 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient(hunting): picked mon.noname-a con 0x7f5ce8134730 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce809d8c0 con 0x7f5ce8134730 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce8134b40 con 0x7f5ce809b9e0 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce806bca0 con 0x7f5ce8134f90 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ceed72700 10 monclient(hunting): authenticate will time out at 846.633314s 2024-09-15T03:45:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient: _add_conns ranks=[7,2,0,3,4,6,8,5,1] 2024-09-15T03:45:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ced30f700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ce8134730 0x7f5ce8146440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53056/0 (socket says 172.21.15.26:53056) 2024-09-15T03:45:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7f5ced30f700 1 -- 172.21.15.26:0/3764505900 learned_addr learned my addr 172.21.15.26:0/3764505900 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient(hunting): picked mon.noname-i con 0x7ffa7809d900 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient(hunting): picked mon.noname-b con 0x7ffa7809e3e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient(hunting): picked mon.noname-a con 0x7ffa780a2710 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7809bbb0 con 0x7ffa780a2710 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa78142930 con 0x7ffa7809e3e0 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7811c040 con 0x7ffa7809d900 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132543150 0 0) 0x7fe22c09d3b0 con 0x7fe22c1302f0 2024-09-15T03:45:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7f0a1700 10 monclient(hunting): authenticate will time out at 846.633680s 2024-09-15T03:45:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): my global_id is 4319 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842465012 0 0) 0x7fd98c09caf0 con 0x7fd98c0a2910 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient(hunting): my global_id is 4399 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd974003570 con 0x7fd98c0a2910 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe218003570 con 0x7fe22c1302f0 2024-09-15T03:45:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa77fff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ffa7809e3e0 0x7ffa7814b430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54278/0 (socket says 172.21.15.26:54278) 2024-09-15T03:45:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa77fff700 1 -- 172.21.15.26:0/2516542239 learned_addr learned my addr 172.21.15.26:0/2516542239 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820491730 0 0) 0x7fd98c098880 con 0x7fd98c09e0a0 2024-09-15T03:45:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient(hunting): my global_id is 4328 2024-09-15T03:45:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd98c09caf0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa7d63e700 1 --1- 172.21.15.26:0/2516542239 >> v1:172.21.15.26:6789/0 conn(0x7ffa780a2710 0x7ffa7814eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53058/0 (socket says 172.21.15.26:53058) 2024-09-15T03:45:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111132014 0 0) 0x7ffa78142930 con 0x7ffa7809e3e0 2024-09-15T03:45:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468893718 0 0) 0x7fd974003570 con 0x7fd98c0a2910 2024-09-15T03:45:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 >> v1:172.21.15.26:6793/0 conn(0x7fd98c09e0a0 legacy=0x7fd98c098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 >> v1:172.21.15.26:6792/0 conn(0x7fd98c139230 legacy=0x7fd98c13b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 monclient: found mon.noname-i 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c098c00 con 0x7fd98c0a2910 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd9800017b0 con 0x7fd98c0a2910 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: dump: 2024-09-15T03:45:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 5 monclient: authenticate success, global_id 4399 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): my global_id is 4322 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa64003570 con 0x7ffa7809e3e0 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724936145 0 0) 0x7ffa7811c040 con 0x7ffa7809d900 2024-09-15T03:45:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): my global_id is 4408 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa78142930 con 0x7ffa7809d900 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609758656 0 0) 0x7ffa7809bbb0 con 0x7ffa780a2710 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): my global_id is 4257 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa7811c040 con 0x7ffa780a2710 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142415405 0 0) 0x7fe22c04ca10 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): my global_id is 4313 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c09d3b0 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd9800019d0 con 0x7fd98c0a2910 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 1 -- 172.21.15.26:0/3759088277 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd980001f00 con 0x7fd98c0a2910 2024-09-15T03:45:02.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 4 set_mon_vals no callback set 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd989ffb700 10 monclient: dump: 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.265 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1594103439 0 0) 0x7ffa64003570 con 0x7ffa7809e3e0 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa7809bbb0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3750750257 0 0) 0x7ffa7811c040 con 0x7ffa780a2710 2024-09-15T03:45:02.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa64003570 con 0x7ffa780a2710 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400456801 0 0) 0x7fe22c098b90 con 0x7fe22c12f7d0 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient(hunting): my global_id is 4350 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c04ca10 con 0x7fe22c12f7d0 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2422474127 0 0) 0x7fe218003570 con 0x7fe22c1302f0 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 >> v1:172.21.15.122:6790/0 conn(0x7fe22c12f7d0 legacy=0x7fe22c12fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 >> v1:172.21.15.26:6790/0 conn(0x7fe22c0a2ab0 legacy=0x7fe22c09e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 646329701 0 0) 0x7ffa78142930 con 0x7ffa7809d900 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa7811c040 con 0x7ffa7809d900 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa68002b70 con 0x7ffa7809e3e0 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.268 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:02.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa6c0016b0 con 0x7ffa780a2710 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.270 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa70001c20 con 0x7ffa7809d900 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.271 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2641701761 0 0) 0x7ffa7809bbb0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 >> v1:172.21.15.122:6792/0 conn(0x7ffa7809d900 legacy=0x7ffa78142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 >> v1:172.21.15.26:6789/0 conn(0x7ffa780a2710 legacy=0x7ffa7814eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 monclient: found mon.noname-e 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c09cd60 con 0x7fe22c1302f0 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe224001ab0 con 0x7fe22c1302f0 2024-09-15T03:45:02.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 monclient: found mon.b 2024-09-15T03:45:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa781435f0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: dump: 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.275 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe224001d40 con 0x7fe22c1302f0 2024-09-15T03:45:02.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 1 -- 172.21.15.26:0/3136318276 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe224004fb0 con 0x7fe22c1302f0 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe223fff700 10 monclient: dump: 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 5 monclient: authenticate success, global_id 4319 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fe23361e700 4 set_mon_vals no callback set 2024-09-15T03:45:02.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 10 monclient: shutdown 2024-09-15T03:45:02.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7fd9914f6700 1 -- 172.21.15.26:0/3759088277 >> v1:172.21.15.122:6792/0 conn(0x7fd98c0a2910 legacy=0x7fd98c097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.199+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa680016a0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa68001b50 con 0x7ffa7809e3e0 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 10 monclient: dump: 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.280 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa76ffd700 4 set_mon_vals no callback set 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 5 monclient: authenticate success, global_id 4322 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 monclient: _renew_subs 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7811bd60 con 0x7ffa7809e3e0 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 monclient: _renew_subs 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:02.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7811ae70 con 0x7ffa7809e3e0 2024-09-15T03:45:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fe23361e700 10 monclient: shutdown 2024-09-15T03:45:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fe23361e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fe23361e700 1 -- 172.21.15.26:0/3136318276 >> v1:172.21.15.26:6793/0 conn(0x7fe22c1302f0 legacy=0x7fe22c13b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffa68001d30 con 0x7ffa7809e3e0 2024-09-15T03:45:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:02.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7804bbf0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- 172.21.15.26:0/3759088277 shutdown_connections 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- 172.21.15.26:0/3759088277 >> 172.21.15.26:0/3759088277 conn(0x7fd98c063ad0 msgr2=0x7fd98c0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- 172.21.15.26:0/3759088277 shutdown_connections 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- 172.21.15.26:0/3759088277 wait complete. 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient: monmap: 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.285 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 Processor -- start 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- start start 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient: init 2024-09-15T03:45:02.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient: _add_conns ranks=[4,1,3,2,8,0,7,6,5] 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient(hunting): picked mon.noname-c con 0x7fd98c09e0a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient(hunting): picked mon.noname-f con 0x7fd98c0a2910 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient(hunting): picked mon.noname-g con 0x7fd98c139230 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c06e470 con 0x7fd98c0a2910 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c06e010 con 0x7fd98c139230 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c132ee0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7fd9914f6700 10 monclient(hunting): authenticate will time out at 846.639995s 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7ffa68043d00 con 0x7ffa7809e3e0 2024-09-15T03:45:02.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.203+0000 7ffa7ce3d700 10 monclient: get_auth_request con 0x7ffa640382c0 auth_method 0 2024-09-15T03:45:02.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd98affd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd98c09e0a0 0x7fd98c1316b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35840/0 (socket says 172.21.15.26:35840) 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd98affd700 1 -- 172.21.15.26:0/3462757599 learned_addr learned my addr 172.21.15.26:0/3462757599 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641047062 0 0) 0x7fd98c132ee0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): my global_id is 4297 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003570 con 0x7fd98c09e0a0 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1111519543 0 0) 0x7fd978003570 con 0x7fd98c09e0a0 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd98c132ee0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd9800016b0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.291 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593307275 0 0) 0x7fd98c06e470 con 0x7fd98c0a2910 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): my global_id is 4366 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003570 con 0x7fd98c0a2910 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112649480 0 0) 0x7fd98c06e010 con 0x7fd98c139230 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient(hunting): my global_id is 4359 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd98c06e470 con 0x7fd98c139230 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1283326767 0 0) 0x7fd98c132ee0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 >> v1:172.21.15.122:6790/0 conn(0x7fd98c139230 legacy=0x7fd98c14a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 >> v1:172.21.15.122:6789/0 conn(0x7fd98c0a2910 legacy=0x7fd98c131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 monclient: found mon.c 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c134a10 con 0x7fd98c09e0a0 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 5 monclient: authenticate success, global_id 4297 2024-09-15T03:45:02.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 monclient: _renew_subs 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c138970 con 0x7fd98c09e0a0 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 monclient: _renew_subs 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c133c50 con 0x7fd98c09e0a0 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd980002580 con 0x7fd98c09e0a0 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd980002ab0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 10 monclient: dump: 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c04bbf0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd980002c10 con 0x7fd98c09e0a0 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7fd980043f80 con 0x7fd98c09e0a0 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ffa68011380 con 0x7ffa7809e3e0 2024-09-15T03:45:02.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7ffa74ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7ffa74ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:02.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.207+0000 7fd98a7fc700 10 monclient: get_auth_request con 0x7fd9780376e0 auth_method 0 2024-09-15T03:45:02.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7fd9897fa700 4 set_mon_vals no callback set 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977252732 0 0) 0x7f5ce809c4c0 con 0x7f5ce809b9e0 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): my global_id is 4357 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd4003570 con 0x7f5ce809b9e0 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812421448 0 0) 0x7f5ce806bca0 con 0x7f5ce8134f90 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): my global_id is 4417 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce809c4c0 con 0x7f5ce8134f90 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133400917 0 0) 0x7f5ce809d8c0 con 0x7f5ce8134730 2024-09-15T03:45:02.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): my global_id is 4248 2024-09-15T03:45:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7fe23361e700 1 -- 172.21.15.26:0/3136318276 shutdown_connections 2024-09-15T03:45:02.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7fe23361e700 1 -- 172.21.15.26:0/3136318276 >> 172.21.15.26:0/3136318276 conn(0x7fe22c063ad0 msgr2=0x7fe22c0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce806bca0 con 0x7f5ce8134730 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2337903887 0 0) 0x7f5ce806bca0 con 0x7f5ce8134730 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ce809d8c0 con 0x7f5ce8134730 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cd8001910 con 0x7f5ce8134730 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.301 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1583790156 0 0) 0x7f5ce809d8c0 con 0x7f5ce8134730 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 >> v1:172.21.15.122:6792/0 conn(0x7f5ce8134f90 legacy=0x7f5ce806b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 >> v1:172.21.15.122:6789/0 conn(0x7f5ce809b9e0 legacy=0x7f5ce806ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 monclient: found mon.a 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce812fe50 con 0x7f5ce8134730 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 5 monclient: authenticate success, global_id 4248 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 10 monclient: _renew_subs 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce806c930 con 0x7f5ce8134730 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 10 monclient: _renew_subs 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce8130510 con 0x7f5ce8134730 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5cd8001bc0 con 0x7f5ce8134730 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cd8002070 con 0x7f5ce8134730 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce67fc700 4 set_mon_vals no callback set 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 10 monclient: dump: 2024-09-15T03:45:02.303 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.304 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5cd8002250 con 0x7f5ce8134730 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.211+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f5cd8043c10 con 0x7f5ce8134730 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 -- 172.21.15.26:0/3136318276 shutdown_connections 2024-09-15T03:45:02.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 -- 172.21.15.26:0/3136318276 wait complete. 2024-09-15T03:45:02.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fd9800150c0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fd972ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fd972ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:02.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7f5cecb0e700 10 monclient: get_auth_request con 0x7f5cd40377e0 auth_method 0 2024-09-15T03:45:02.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient: build_initial_monmap 2024-09-15T03:45:02.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient: monmap: 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 Processor -- start 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 -- start start 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient: init 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient: _add_conns ranks=[2,4,7,6,3,0,5,1,8] 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient(hunting): picked mon.noname-b con 0x7fe22c0a2ab0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient(hunting): picked mon.noname-c con 0x7fe22c12f7d0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient(hunting): picked mon.noname-i con 0x7fe22c1302f0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c06e360 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c06dfa0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c11d4f0 con 0x7fe22c1302f0 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe23361e700 10 monclient(hunting): authenticate will time out at 846.650516s 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe2313ba700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe22c0a2ab0 0x7fe22c119f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54288/0 (socket says 172.21.15.26:54288) 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.215+0000 7fe2313ba700 1 -- 172.21.15.26:0/807211798 learned_addr learned my addr 172.21.15.26:0/807211798 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.219+0000 7f5ceed72700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.219+0000 7f5ceed72700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.219+0000 7f5ceed72700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.219+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc80050b0 con 0x7f5ce8134730 2024-09-15T03:45:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599322780 0 0) 0x7fe22c06e360 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): my global_id is 4331 2024-09-15T03:45:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe210003570 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761632297 0 0) 0x7fe22c06dfa0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): my global_id is 4306 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c06e360 con 0x7fe22c12f7d0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000956470 0 0) 0x7fe22c11d4f0 con 0x7fe22c1302f0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): my global_id is 4426 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe22c06dfa0 con 0x7fe22c1302f0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3143287697 0 0) 0x7fe22c06e360 con 0x7fe22c12f7d0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe22c11d4f0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe228001960 con 0x7fe22c12f7d0 2024-09-15T03:45:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): dump: 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4103713356 0 0) 0x7fe210003570 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe22c06e360 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 104163843 0 0) 0x7fe22c06dfa0 con 0x7fe22c1302f0 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe210003570 con 0x7fe22c1302f0 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe21c0013a0 con 0x7fe22c0a2ab0 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient(hunting): dump: 2024-09-15T03:45:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2677386432 0 0) 0x7fe22c11d4f0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 >> v1:172.21.15.122:6792/0 conn(0x7fe22c1302f0 legacy=0x7fe22c11ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 >> v1:172.21.15.26:6790/0 conn(0x7fe22c0a2ab0 legacy=0x7fe22c119f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: _finish_hunting 0 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 monclient: found mon.c 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c11d110 con 0x7fe22c12f7d0 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: _finish_auth 0 2024-09-15T03:45:02.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 5 monclient: authenticate success, global_id 4306 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe228002c70 con 0x7fe22c12f7d0 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe2280021b0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 10 monclient: _renew_subs 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c11b9c0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 10 monclient: _renew_subs 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c14b080 con 0x7fe22c12f7d0 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2237fe700 4 set_mon_vals no callback set 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 10 monclient: dump: 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe228002310 con 0x7fe22c12f7d0 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7fe228041210 con 0x7fe22c12f7d0 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.223+0000 7fe2313ba700 10 monclient: get_auth_request con 0x7fe2100382c0 auth_method 0 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7fe23361e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7fe23361e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7fe23361e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2080050b0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f5cd8014ba0 con 0x7f5ce8134730 2024-09-15T03:45:02.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7f5ce4ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.227+0000 7f5ce4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:02.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.235+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fe228011d80 con 0x7fe22c12f7d0 2024-09-15T03:45:02.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.235+0000 7fe2217fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:02.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.235+0000 7fe2217fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.347+0000 7f74489be700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.347+0000 7f74489be700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.347+0000 7f74489be700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.347+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7424005360 con 0x7f744406d250 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f742b7fe700 1 -- 172.21.15.26:0/1507651588 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 555833778) 0x7f7434014970 con 0x7f744406d250 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f742b7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T03:45:02.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f742b7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:02.358 INFO:teuthology.orchestra.run.smithi026.stdout:47244640258 2024-09-15T03:45:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 >> v1:172.21.15.122:6812/20895 conn(0x7f7430037f60 legacy=0x7f743003a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 10 monclient: shutdown 2024-09-15T03:45:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 >> v1:172.21.15.26:6789/0 conn(0x7f744406d250 legacy=0x7f7444136800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 shutdown_connections 2024-09-15T03:45:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 >> 172.21.15.26:0/1507651588 conn(0x7f7444063dc0 msgr2=0x7f744409fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 shutdown_connections 2024-09-15T03:45:02.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.351+0000 7f74489be700 1 -- 172.21.15.26:0/1507651588 wait complete. 2024-09-15T03:45:02.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T03:45:02.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T03:45:02.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff790005360 con 0x7ff7ac09e260 2024-09-15T03:45:02.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff79a7fc700 1 -- 172.21.15.26:0/902858224 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 3327677263) 0x7ff7a0014ca0 con 0x7ff7ac09e260 2024-09-15T03:45:02.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff79a7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T03:45:02.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff79a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:02.363 INFO:teuthology.orchestra.run.smithi026.stdout:47244640257 2024-09-15T03:45:02.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 >> v1:172.21.15.122:6812/20895 conn(0x7ff794037f90 legacy=0x7ff79403a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 10 monclient: shutdown 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 >> v1:172.21.15.26:6789/0 conn(0x7ff7ac09e260 legacy=0x7ff7ac11ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 shutdown_connections 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.355+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 >> 172.21.15.26:0/902858224 conn(0x7ff7ac063a10 msgr2=0x7ff7ac09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.359+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 shutdown_connections 2024-09-15T03:45:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.359+0000 7ff7b08c2700 1 -- 172.21.15.26:0/902858224 wait complete. 2024-09-15T03:45:02.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.2 2024-09-15T03:45:02.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640257 for osd.4 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd98c06f230 con 0x7fd98c09e0a0 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd972ffd700 1 -- 172.21.15.26:0/3462757599 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 847495621) 0x7fd980014ac0 con 0x7fd98c09e0a0 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd972ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T03:45:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd972ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:02.467 INFO:teuthology.orchestra.run.smithi026.stdout:47244640259 2024-09-15T03:45:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa7f0a1700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T03:45:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa7f0a1700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T03:45:02.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa7f0a1700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:02.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa7806c9c0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa74ff9700 1 -- 172.21.15.26:0/2516542239 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 847495621) 0x7ffa680146f0 con 0x7ffa7809e3e0 2024-09-15T03:45:02.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa74ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T03:45:02.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7ffa74ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:02.468 INFO:teuthology.orchestra.run.smithi026.stdout:47244640259 2024-09-15T03:45:02.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 >> v1:172.21.15.122:6812/20895 conn(0x7fd9780376e0 legacy=0x7fd978039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 10 monclient: shutdown 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 >> v1:172.21.15.26:6791/0 conn(0x7fd98c09e0a0 legacy=0x7fd98c1316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 shutdown_connections 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 >> 172.21.15.26:0/3462757599 conn(0x7fd98c063ad0 msgr2=0x7fd98c12cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 shutdown_connections 2024-09-15T03:45:02.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.459+0000 7fd9914f6700 1 -- 172.21.15.26:0/3462757599 wait complete. 2024-09-15T03:45:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 >> v1:172.21.15.122:6812/20895 conn(0x7ffa640382c0 legacy=0x7ffa6403a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 10 monclient: shutdown 2024-09-15T03:45:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 >> v1:172.21.15.26:6790/0 conn(0x7ffa7809e3e0 legacy=0x7ffa7814b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 shutdown_connections 2024-09-15T03:45:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 >> 172.21.15.26:0/2516542239 conn(0x7ffa78063ad0 msgr2=0x7ffa780a0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 shutdown_connections 2024-09-15T03:45:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.463+0000 7ffa7f0a1700 1 -- 172.21.15.26:0/2516542239 wait complete. 2024-09-15T03:45:02.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.1 2024-09-15T03:45:02.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.3 2024-09-15T03:45:02.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ceed72700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T03:45:02.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ceed72700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T03:45:02.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ceed72700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:02.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5cc8005360 con 0x7f5ce8134730 2024-09-15T03:45:02.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ce4ff9700 1 -- 172.21.15.26:0/3764505900 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 555833778) 0x7f5cd80145a0 con 0x7f5ce8134730 2024-09-15T03:45:02.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ce4ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T03:45:02.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.479+0000 7f5ce4ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:02.488 INFO:teuthology.orchestra.run.smithi026.stdout:47244640258 2024-09-15T03:45:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 >> v1:172.21.15.122:6812/20895 conn(0x7f5cd40377e0 legacy=0x7f5cd4039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 10 monclient: shutdown 2024-09-15T03:45:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 >> v1:172.21.15.26:6789/0 conn(0x7f5ce8134730 legacy=0x7f5ce8146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 shutdown_connections 2024-09-15T03:45:02.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 >> 172.21.15.26:0/3764505900 conn(0x7f5ce8063ad0 msgr2=0x7f5ce8097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 shutdown_connections 2024-09-15T03:45:02.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.483+0000 7f5ceed72700 1 -- 172.21.15.26:0/3764505900 wait complete. 2024-09-15T03:45:02.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T03:45:02.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T03:45:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe208005360 con 0x7fe22c12f7d0 2024-09-15T03:45:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe2217fa700 1 -- 172.21.15.26:0/807211798 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 555833778) 0x7fe228011ba0 con 0x7fe22c12f7d0 2024-09-15T03:45:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe2217fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T03:45:02.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe2217fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:02.499 INFO:teuthology.orchestra.run.smithi026.stdout:47244640258 2024-09-15T03:45:02.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.0 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 >> v1:172.21.15.122:6812/20895 conn(0x7fe2100382c0 legacy=0x7fe21003a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 10 monclient: shutdown 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 >> v1:172.21.15.26:6791/0 conn(0x7fe22c12f7d0 legacy=0x7fe22c11a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 shutdown_connections 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.491+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 >> 172.21.15.26:0/807211798 conn(0x7fe22c063ad0 msgr2=0x7fe22c0a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.495+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 shutdown_connections 2024-09-15T03:45:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:02.495+0000 7fe23361e700 1 -- 172.21.15.26:0/807211798 wait complete. 2024-09-15T03:45:02.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.5 2024-09-15T03:45:03.372 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T03:45:03.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T03:45:03.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T03:45:03.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.487+0000 7f3aa6046700 10 monclient: get_monmap_and_config 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.487+0000 7f3aa6046700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.487+0000 7f3aa6046700 10 monclient: monmap: 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:03.495070+0000 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:03.495070+0000 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.497 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.487+0000 7f3aa6046700 1 Processor -- start 2024-09-15T03:45:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- start start 2024-09-15T03:45:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient: init 2024-09-15T03:45:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient: _add_conns ranks=[4,2,3,7,1,6,8,0,5] 2024-09-15T03:45:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient(hunting): picked mon.noname-c con 0x7f3aa00a3860 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:03.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient(hunting): picked mon.noname-b con 0x7f3aa009e0b0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:03.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient(hunting): picked mon.noname-g con 0x7f3aa009e480 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:03.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa004ca10 con 0x7f3aa009e0b0 2024-09-15T03:45:03.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa0098c50 con 0x7f3aa009e480 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa0098970 con 0x7f3aa00a3860 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9f7fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f3aa00a3860 0x7f3aa009d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35852/0 (socket says 172.21.15.26:35852) 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient(hunting): authenticate will time out at 847.924961s 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9f7fe700 1 -- 172.21.15.26:0/1659571320 learned_addr learned my addr 172.21.15.26:0/1659571320 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544551462 0 0) 0x7f3aa0098970 con 0x7f3aa00a3860 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): my global_id is 4315 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a8c003570 con 0x7f3aa00a3860 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830124075 0 0) 0x7f3aa004ca10 con 0x7f3aa009e0b0 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): my global_id is 4340 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa0098970 con 0x7f3aa009e0b0 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906301194 0 0) 0x7f3aa0098c50 con 0x7f3aa009e480 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient(hunting): my global_id is 4368 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa004ca10 con 0x7f3aa009e480 2024-09-15T03:45:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626546574 0 0) 0x7f3aa0098970 con 0x7f3aa009e0b0 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 >> v1:172.21.15.26:6791/0 conn(0x7f3aa00a3860 legacy=0x7f3aa009d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 >> v1:172.21.15.122:6790/0 conn(0x7f3aa009e480 legacy=0x7f3aa01382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 monclient: found mon.noname-b 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0098f30 con 0x7f3aa009e0b0 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 5 monclient: authenticate success, global_id 4340 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a88001790 con 0x7f3aa009e0b0 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: dump: 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.506 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.507 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.507 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:03.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3a880019b0 con 0x7f3aa009e0b0 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 1 -- 172.21.15.26:0/1659571320 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a88001ea0 con 0x7f3aa009e0b0 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3a9effd700 10 monclient: dump: 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.508 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 4 set_mon_vals no callback set 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 10 monclient: shutdown 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- 172.21.15.26:0/1659571320 >> v1:172.21.15.26:6790/0 conn(0x7f3aa009e0b0 legacy=0x7f3aa0134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- 172.21.15.26:0/1659571320 shutdown_connections 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- 172.21.15.26:0/1659571320 >> 172.21.15.26:0/1659571320 conn(0x7f3aa0071350 msgr2=0x7f3aa00a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- 172.21.15.26:0/1659571320 shutdown_connections 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.491+0000 7f3aa6046700 1 -- 172.21.15.26:0/1659571320 wait complete. 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient: monmap: 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.510 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.511 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.511 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.511 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.511 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.511 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.511 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T03:45:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 1 Processor -- start 2024-09-15T03:45:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 1 -- start start 2024-09-15T03:45:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient: init 2024-09-15T03:45:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient: _add_conns ranks=[0,6,4,8,2,1,5,7,3] 2024-09-15T03:45:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient(hunting): picked mon.noname-a con 0x7f3aa009e480 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient(hunting): picked mon.noname-d con 0x7f3aa00a3860 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient(hunting): picked mon.noname-c con 0x7f3aa012f330 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa006e590 con 0x7f3aa009e480 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa006e1d0 con 0x7f3aa012f330 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa011ade0 con 0x7f3aa00a3860 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3aa6046700 10 monclient(hunting): authenticate will time out at 847.930047s 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3a97fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f3aa00a3860 0x7f3aa0143370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55048/0 (socket says 172.21.15.26:55048) 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.495+0000 7f3a97fff700 1 -- 172.21.15.26:0/2954426579 learned_addr learned my addr 172.21.15.26:0/2954426579 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886754169 0 0) 0x7f3aa011ade0 con 0x7f3aa00a3860 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): my global_id is 4308 2024-09-15T03:45:03.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a80003570 con 0x7f3aa00a3860 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39261071 0 0) 0x7f3aa006e1d0 con 0x7f3aa012f330 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): my global_id is 4324 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa011ade0 con 0x7f3aa012f330 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155452519 0 0) 0x7f3aa006e590 con 0x7f3aa009e480 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): my global_id is 4266 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa006e1d0 con 0x7f3aa009e480 2024-09-15T03:45:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3559725646 0 0) 0x7f3a80003570 con 0x7f3aa00a3860 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3aa006e590 con 0x7f3aa00a3860 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a88001690 con 0x7f3aa00a3860 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): dump: 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 243069320 0 0) 0x7f3aa006e1d0 con 0x7f3aa009e480 2024-09-15T03:45:03.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3a80003570 con 0x7f3aa009e480 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3394484801 0 0) 0x7f3aa011ade0 con 0x7f3aa012f330 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3aa006e1d0 con 0x7f3aa012f330 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a90001960 con 0x7f3aa009e480 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): dump: 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a98001d60 con 0x7f3aa012f330 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:03.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient(hunting): dump: 2024-09-15T03:45:03.524 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.524 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.524 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.525 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 855137953 0 0) 0x7f3aa006e590 con 0x7f3aa00a3860 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 >> v1:172.21.15.26:6791/0 conn(0x7f3aa012f330 legacy=0x7f3aa0146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 >> v1:172.21.15.26:6789/0 conn(0x7f3aa009e480 legacy=0x7f3aa012ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 monclient: found mon.d 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa011bd40 con 0x7f3aa00a3860 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3a977fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 5 monclient: authenticate success, global_id 4308 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 10 monclient: _renew_subs 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa006db40 con 0x7f3aa00a3860 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 10 monclient: _renew_subs 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.499+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa013e6b0 con 0x7f3aa00a3860 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3a880017f0 con 0x7f3aa00a3860 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a9e7fc700 4 set_mon_vals no callback set 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a88002820 con 0x7f3aa00a3860 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 10 monclient: dump: 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3a88002a00 con 0x7f3aa00a3860 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f3a88043e40 con 0x7f3aa00a3860 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.503+0000 7f3a9f7fe700 10 monclient: get_auth_request con 0x7f3a80038240 auth_method 0 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.507+0000 7f3aa6046700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.507+0000 7f3aa6046700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.507+0000 7f3aa6046700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.507+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a840050b0 con 0x7f3aa00a3860 2024-09-15T03:45:03.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T03:45:03.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.511+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f3a88014f50 con 0x7f3aa00a3860 2024-09-15T03:45:03.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.511+0000 7f3a977fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.511+0000 7f3a977fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.515+0000 7f87d327f700 10 monclient: get_monmap_and_config 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.515+0000 7f87d327f700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.515+0000 7f87d327f700 10 monclient: monmap: 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:03.520638+0000 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:03.520638+0000 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.515+0000 7f87d327f700 1 Processor -- start 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.515+0000 7f87d327f700 1 -- start start 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.515+0000 7f87d327f700 10 monclient: init 2024-09-15T03:45:03.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient: _add_conns ranks=[0,6,3,5,2,7,8,4,1] 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient(hunting): picked mon.noname-a con 0x7f87cc0a2ab0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient(hunting): picked mon.noname-d con 0x7f87cc12e7c0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient(hunting): picked mon.noname-g con 0x7f87cc12f2c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc04ca10 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc099c70 con 0x7f87cc12f2c0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc098dd0 con 0x7f87cc12e7c0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient(hunting): authenticate will time out at 847.952490s 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d101b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87cc0a2ab0 0x7f87cc130000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53088/0 (socket says 172.21.15.26:53088) 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d101b700 1 -- 172.21.15.26:0/245294871 learned_addr learned my addr 172.21.15.26:0/245294871 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953196719 0 0) 0x7f87cc098dd0 con 0x7f87cc12e7c0 2024-09-15T03:45:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): my global_id is 4317 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b8003570 con 0x7f87cc12e7c0 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795479573 0 0) 0x7f87cc099c70 con 0x7f87cc12f2c0 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): my global_id is 4377 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87cc098dd0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337446023 0 0) 0x7f87cc04ca10 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient(hunting): my global_id is 4275 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87cc099c70 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847250114 0 0) 0x7f87b8003570 con 0x7f87cc12e7c0 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 >> v1:172.21.15.122:6790/0 conn(0x7f87cc12f2c0 legacy=0x7f87cc137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 >> v1:172.21.15.26:6789/0 conn(0x7f87cc0a2ab0 legacy=0x7f87cc130000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 monclient: found mon.noname-d 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc130710 con 0x7f87cc12e7c0 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87bc0017f0 con 0x7f87cc12e7c0 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: dump: 2024-09-15T03:45:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f87bc001a80 con 0x7f87cc12e7c0 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 5 monclient: authenticate success, global_id 4317 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 1 -- 172.21.15.26:0/245294871 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87bc002240 con 0x7f87cc12e7c0 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 4 set_mon_vals no callback set 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87c3fff700 10 monclient: dump: 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 10 monclient: shutdown 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- 172.21.15.26:0/245294871 >> v1:172.21.15.26:6792/0 conn(0x7f87cc12e7c0 legacy=0x7f87cc12ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- 172.21.15.26:0/245294871 shutdown_connections 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- 172.21.15.26:0/245294871 >> 172.21.15.26:0/245294871 conn(0x7f87cc063ad0 msgr2=0x7f87cc0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- 172.21.15.26:0/245294871 shutdown_connections 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.519+0000 7f87d327f700 1 -- 172.21.15.26:0/245294871 wait complete. 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient: monmap: 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 1 Processor -- start 2024-09-15T03:45:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 1 -- start start 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient: init 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient: _add_conns ranks=[5,7,2,8,4,0,6,1,3] 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient(hunting): picked mon.noname-h con 0x7f87cc0a2ab0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient(hunting): picked mon.noname-i con 0x7f87cc12e7c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient(hunting): picked mon.noname-b con 0x7f87cc12f2c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc09e110 con 0x7f87cc12f2c0 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc09c440 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc11eb00 con 0x7f87cc12e7c0 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d327f700 10 monclient(hunting): authenticate will time out at 847.957245s 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d181c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f87cc12f2c0 0x7f87cc1485e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54298/0 (socket says 172.21.15.26:54298) 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.523+0000 7f87d181c700 1 -- 172.21.15.26:0/1124279571 learned_addr learned my addr 172.21.15.26:0/1124279571 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804155140 0 0) 0x7f87cc09e110 con 0x7f87cc12f2c0 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): my global_id is 4349 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b00035a0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585801090 0 0) 0x7f87cc09c440 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): my global_id is 4352 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87cc09e110 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553185593 0 0) 0x7f87cc11eb00 con 0x7f87cc12e7c0 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): my global_id is 4435 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87cc09c440 con 0x7f87cc12e7c0 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1276663160 0 0) 0x7f87b00035a0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87cc11eb00 con 0x7f87cc12f2c0 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87c4002db0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): dump: 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 791364927 0 0) 0x7f87cc09e110 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87b00035a0 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87c8001960 con 0x7f87cc0a2ab0 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient(hunting): dump: 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1064773422 0 0) 0x7f87cc11eb00 con 0x7f87cc12f2c0 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 >> v1:172.21.15.122:6792/0 conn(0x7f87cc12e7c0 legacy=0x7f87cc144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 >> v1:172.21.15.122:6791/0 conn(0x7f87cc0a2ab0 legacy=0x7f87cc13fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 monclient: found mon.b 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc11e480 con 0x7f87cc12f2c0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 5 monclient: authenticate success, global_id 4349 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 10 monclient: _renew_subs 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc11e790 con 0x7f87cc12f2c0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 10 monclient: _renew_subs 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc11d8a0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f87c4001880 con 0x7f87cc12f2c0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87c4001db0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c37fe700 4 set_mon_vals no callback set 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 10 monclient: dump: 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f87c4001f10 con 0x7f87cc12f2c0 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f87c4043fc0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.527+0000 7f87d101b700 10 monclient: get_auth_request con 0x7f87b0037e30 auth_method 0 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.535+0000 7f87d327f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.535+0000 7f87d327f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.535+0000 7f87d327f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.535+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac0050b0 con 0x7f87cc12f2c0 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.543+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f87c4015060 con 0x7f87cc12f2c0 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.543+0000 7f87c17fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.543+0000 7f87c17fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient: get_monmap_and_config 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient: monmap: 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:03.584671+0000 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:03.584671+0000 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.586 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 1 Processor -- start 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 1 -- start start 2024-09-15T03:45:03.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient: init 2024-09-15T03:45:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient: _add_conns ranks=[4,7,8,5,6,2,0,3,1] 2024-09-15T03:45:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient(hunting): picked mon.noname-c con 0x7f5b6c0a2ab0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient(hunting): picked mon.noname-i con 0x7f5b6c12f7d0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient(hunting): picked mon.noname-e con 0x7f5b6c1302f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c04ca10 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c098b90 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c09d3b0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 10 monclient(hunting): authenticate will time out at 848.014574s 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b6affd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f5b6c0a2ab0 0x7f5b6c09e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35868/0 (socket says 172.21.15.26:35868) 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b6affd700 1 -- 172.21.15.26:0/2378642544 learned_addr learned my addr 172.21.15.26:0/2378642544 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756419437 0 0) 0x7f5b6c098b90 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): my global_id is 4444 2024-09-15T03:45:03.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b54003570 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50150039 0 0) 0x7f5b6c09d3b0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): my global_id is 4337 2024-09-15T03:45:03.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6c098b90 con 0x7f5b6c1302f0 2024-09-15T03:45:03.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415648172 0 0) 0x7f5b6c04ca10 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient(hunting): my global_id is 4333 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6c09d3b0 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 648447474 0 0) 0x7f5b54003570 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 >> v1:172.21.15.26:6793/0 conn(0x7f5b6c1302f0 legacy=0x7f5b6c13b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 >> v1:172.21.15.26:6791/0 conn(0x7f5b6c0a2ab0 legacy=0x7f5b6c09e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 monclient: found mon.noname-i 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c09cd60 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b60001730 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b69ffb700 10 monclient: dump: 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 5 monclient: authenticate success, global_id 4444 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.579+0000 7f5b71745700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5b60001950 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b69ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b69ffb700 1 -- 172.21.15.26:0/2378642544 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b60001f00 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 4 set_mon_vals no callback set 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b69ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b69ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b69ffb700 10 monclient: dump: 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.595 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: shutdown 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- 172.21.15.26:0/2378642544 >> v1:172.21.15.122:6792/0 conn(0x7f5b6c12f7d0 legacy=0x7f5b6c12fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- 172.21.15.26:0/2378642544 shutdown_connections 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- 172.21.15.26:0/2378642544 >> 172.21.15.26:0/2378642544 conn(0x7f5b6c063ad0 msgr2=0x7f5b6c0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- 172.21.15.26:0/2378642544 shutdown_connections 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- 172.21.15.26:0/2378642544 wait complete. 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: monmap: 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.597 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 Processor -- start 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- start start 2024-09-15T03:45:03.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: init 2024-09-15T03:45:03.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient: _add_conns ranks=[2,3,0,5,4,1,6,8,7] 2024-09-15T03:45:03.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient(hunting): picked mon.noname-b con 0x7f5b6c0a2ab0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:03.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient(hunting): picked mon.noname-g con 0x7f5b6c12f7d0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient(hunting): picked mon.noname-a con 0x7f5b6c1302f0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c11a300 con 0x7f5b6c1302f0 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c06e360 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6c06dfa0 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b71745700 10 monclient(hunting): authenticate will time out at 848.019713s 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b6b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5b6c1302f0 0x7f5b6c14a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53104/0 (socket says 172.21.15.26:53104) 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.583+0000 7f5b6b7fe700 1 -- 172.21.15.26:0/1243645442 learned_addr learned my addr 172.21.15.26:0/1243645442 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894415585 0 0) 0x7f5b6c11a300 con 0x7f5b6c1302f0 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): my global_id is 4284 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b58003570 con 0x7f5b6c1302f0 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925903964 0 0) 0x7f5b6c06e360 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): my global_id is 4358 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6c11a300 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566083008 0 0) 0x7f5b6c06dfa0 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): my global_id is 4386 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b6c06e360 con 0x7f5b6c12f7d0 2024-09-15T03:45:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1228813568 0 0) 0x7f5b58003570 con 0x7f5b6c1302f0 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b6c06dfa0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b64002dc0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1463361418 0 0) 0x7f5b6c11a300 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b58003570 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b5c001a60 con 0x7f5b6c0a2ab0 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 395257850 0 0) 0x7f5b6c06dfa0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 >> v1:172.21.15.122:6790/0 conn(0x7f5b6c12f7d0 legacy=0x7f5b6c1471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 >> v1:172.21.15.26:6790/0 conn(0x7f5b6c0a2ab0 legacy=0x7f5b6c12e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 monclient: found mon.a 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c11a950 con 0x7f5b6c1302f0 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 5 monclient: authenticate success, global_id 4284 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 10 monclient: _renew_subs 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6c11b8b0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 10 monclient: _renew_subs 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c119d80 con 0x7f5b6c1302f0 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5b64003080 con 0x7f5b6c1302f0 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b52ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.587+0000 7f5b697fa700 4 set_mon_vals no callback set 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b64001c70 con 0x7f5b6c1302f0 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b52ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b52ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b52ffd700 10 monclient: dump: 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5b64001dd0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f5b640440c0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b6affd700 10 monclient: get_auth_request con 0x7f5b58037d90 auth_method 0 2024-09-15T03:45:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b71745700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b71745700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b71745700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.591+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c0050b0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f5b640150d0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5b52ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5b52ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5d0b2d0700 10 monclient: get_monmap_and_config 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5d0b2d0700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5d0b2d0700 10 monclient: monmap: 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:03.606186+0000 2024-09-15T03:45:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:03.606186+0000 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5d0b2d0700 1 Processor -- start 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5d0b2d0700 1 -- start start 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.599+0000 7f5d0b2d0700 10 monclient: init 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient: _add_conns ranks=[6,2,3,8,5,4,0,7,1] 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient(hunting): picked mon.noname-d con 0x7f5d04097e00 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient(hunting): picked mon.noname-b con 0x7f5d0412f5f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient(hunting): picked mon.noname-g con 0x7f5d04130110 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0404ca10 con 0x7f5d0412f5f0 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0406e380 con 0x7f5d04130110 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0409e5f0 con 0x7f5d04097e00 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient(hunting): authenticate will time out at 848.036199s 2024-09-15T03:45:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d09acd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f5d0412f5f0 0x7f5d0412fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54310/0 (socket says 172.21.15.26:54310) 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d09acd700 1 -- 172.21.15.26:0/2901674908 learned_addr learned my addr 172.21.15.26:0/2901674908 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684677703 0 0) 0x7f5d0404ca10 con 0x7f5d0412f5f0 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient(hunting): my global_id is 4367 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf0003570 con 0x7f5d0412f5f0 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427636365 0 0) 0x7f5d0406e380 con 0x7f5d04130110 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient(hunting): my global_id is 4395 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d0404ca10 con 0x7f5d04130110 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965474913 0 0) 0x7f5cf0003570 con 0x7f5d0412f5f0 2024-09-15T03:45:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 >> v1:172.21.15.26:6792/0 conn(0x7f5d04097e00 legacy=0x7f5d0409dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 >> v1:172.21.15.122:6790/0 conn(0x7f5d04130110 legacy=0x7f5d0413aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 monclient: found mon.noname-b 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0409cb80 con 0x7f5d0412f5f0 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cfc001b30 con 0x7f5d0412f5f0 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: dump: 2024-09-15T03:45:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 5 monclient: authenticate success, global_id 4367 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5cfc001da0 con 0x7f5d0412f5f0 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 1 -- 172.21.15.26:0/2901674908 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cfc004fb0 con 0x7f5d0412f5f0 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 4 set_mon_vals no callback set 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d092cc700 10 monclient: dump: 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient: shutdown 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/2901674908 >> v1:172.21.15.26:6790/0 conn(0x7f5d0412f5f0 legacy=0x7f5d0412fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/2901674908 shutdown_connections 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/2901674908 >> 172.21.15.26:0/2901674908 conn(0x7f5d04063dc0 msgr2=0x7f5d040a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/2901674908 shutdown_connections 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/2901674908 wait complete. 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.603+0000 7f5d0b2d0700 10 monclient: monmap: 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 Processor -- start 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 -- start start 2024-09-15T03:45:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: init 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: _add_conns ranks=[8,2,6,1,7,4,3,5,0] 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient(hunting): picked mon.noname-e con 0x7f5d04097e00 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient(hunting): picked mon.noname-b con 0x7f5d0412f5f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient(hunting): picked mon.noname-d con 0x7f5d04130110 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d04098b90 con 0x7f5d0412f5f0 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d040987a0 con 0x7f5d04130110 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0412d550 con 0x7f5d04097e00 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient(hunting): authenticate will time out at 848.040819s 2024-09-15T03:45:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0a2ce700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5d04097e00 0x7f5d04119940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48858/0 (socket says 172.21.15.26:48858) 2024-09-15T03:45:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0a2ce700 1 -- 172.21.15.26:0/972648717 learned_addr learned my addr 172.21.15.26:0/972648717 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175741779 0 0) 0x7f5d0412d550 con 0x7f5d04097e00 2024-09-15T03:45:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): my global_id is 4346 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce8003570 con 0x7f5d04097e00 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656186297 0 0) 0x7f5d04098b90 con 0x7f5d0412f5f0 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): my global_id is 4376 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d0412d550 con 0x7f5d0412f5f0 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860684449 0 0) 0x7f5d040987a0 con 0x7f5d04130110 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): my global_id is 4326 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d04098b90 con 0x7f5d04130110 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2249683248 0 0) 0x7f5ce8003570 con 0x7f5d04097e00 2024-09-15T03:45:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d040987a0 con 0x7f5d04097e00 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cf40019a0 con 0x7f5d04097e00 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3211495372 0 0) 0x7f5d0412d550 con 0x7f5d0412f5f0 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ce8003570 con 0x7f5d0412f5f0 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 288188357 0 0) 0x7f5d04098b90 con 0x7f5d04130110 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d0412d550 con 0x7f5d04130110 2024-09-15T03:45:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cfc001af0 con 0x7f5d0412f5f0 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d00001900 con 0x7f5d04130110 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.631 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2092840550 0 0) 0x7f5d040987a0 con 0x7f5d04097e00 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 >> v1:172.21.15.26:6792/0 conn(0x7f5d04130110 legacy=0x7f5d0411a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 >> v1:172.21.15.26:6790/0 conn(0x7f5d0412f5f0 legacy=0x7f5d0411a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 monclient: found mon.e 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0411d560 con 0x7f5d04097e00 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 5 monclient: authenticate success, global_id 4346 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: _renew_subs 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0411b4e0 con 0x7f5d04097e00 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: _renew_subs 2024-09-15T03:45:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5d0b2d0700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0411c0b0 con 0x7f5d04097e00 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5cf4004950 con 0x7f5d04097e00 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5d08acb700 4 set_mon_vals no callback set 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.607+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5cf4004e00 con 0x7f5d04097e00 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5cfa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5cfa7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5cfa7fc700 10 monclient: dump: 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5cf4004f60 con 0x7f5d04097e00 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f5cf4043cf0 con 0x7f5d04097e00 2024-09-15T03:45:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5ce7fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5d09acd700 10 monclient: get_auth_request con 0x7f5ce80381b0 auth_method 0 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5ce7fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5ce7fff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.611+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0404bbf0 con 0x7f5d04097e00 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.619+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f5cf4006550 con 0x7f5d04097e00 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.619+0000 7f5cfa7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.619+0000 7f5cfa7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.639+0000 7f812357f700 10 monclient: get_monmap_and_config 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.639+0000 7f812357f700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.639+0000 7f812357f700 10 monclient: monmap: 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:03.647033+0000 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:03.647033+0000 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.649 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 Processor -- start 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- start start 2024-09-15T03:45:03.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: init 2024-09-15T03:45:03.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: _add_conns ranks=[3,1,0,4,7,5,2,8,6] 2024-09-15T03:45:03.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): picked mon.noname-g con 0x7f811c09d920 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:03.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): picked mon.noname-f con 0x7f811c09e4b0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:03.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): picked mon.noname-a con 0x7f811c098280 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c04ca10 con 0x7f811c098280 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c098c80 con 0x7f811c09e4b0 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c099bf0 con 0x7f811c09d920 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): authenticate will time out at 848.077470s 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f8122d7e700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f811c098280 0x7f811c138130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53116/0 (socket says 172.21.15.26:53116) 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f8121d7c700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f811c09e4b0 0x7f811c134a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:36086/0 (socket says 172.21.15.26:36086) 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f8121d7c700 1 -- 172.21.15.26:0/2668457784 learned_addr learned my addr 172.21.15.26:0/2668457784 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812257d700 1 --1- 172.21.15.26:0/2668457784 >> v1:172.21.15.122:6790/0 conn(0x7f811c09d920 0x7f811c09b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:34074/0 (socket says 172.21.15.26:34074) 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337630456 0 0) 0x7f811c098c80 con 0x7f811c09e4b0 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): my global_id is 4375 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8108003570 con 0x7f811c09e4b0 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592692057 0 0) 0x7f811c04ca10 con 0x7f811c098280 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): my global_id is 4293 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c098c80 con 0x7f811c098280 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523685586 0 0) 0x7f811c099bf0 con 0x7f811c09d920 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient(hunting): my global_id is 4404 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c04ca10 con 0x7f811c09d920 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1057452864 0 0) 0x7f811c098c80 con 0x7f811c098280 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 >> v1:172.21.15.122:6790/0 conn(0x7f811c09d920 legacy=0x7f811c09b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 >> v1:172.21.15.122:6789/0 conn(0x7f811c09e4b0 legacy=0x7f811c134a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 monclient: found mon.noname-a 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c099ed0 con 0x7f811c098280 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8118001710 con 0x7f811c098280 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: dump: 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.657 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 5 monclient: authenticate success, global_id 4293 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8118001a30 con 0x7f811c098280 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 1 -- 172.21.15.26:0/2668457784 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8118001c20 con 0x7f811c098280 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 4 set_mon_vals no callback set 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812157b700 10 monclient: dump: 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.659 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.660 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.660 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.660 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: shutdown 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- 172.21.15.26:0/2668457784 >> v1:172.21.15.26:6789/0 conn(0x7f811c098280 legacy=0x7f811c138130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- 172.21.15.26:0/2668457784 shutdown_connections 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- 172.21.15.26:0/2668457784 >> 172.21.15.26:0/2668457784 conn(0x7f811c063dc0 msgr2=0x7f811c0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- 172.21.15.26:0/2668457784 shutdown_connections 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- 172.21.15.26:0/2668457784 wait complete. 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: monmap: 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.663 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.664 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 Processor -- start 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- start start 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: init 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient: _add_conns ranks=[4,1,0,5,7,2,8,6,3] 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): picked mon.noname-c con 0x7f811c098280 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): picked mon.noname-f con 0x7f811c09d920 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): picked mon.noname-a con 0x7f811c09e4b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c06de20 con 0x7f811c09e4b0 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c11da20 con 0x7f811c09d920 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c142830 con 0x7f811c098280 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812357f700 10 monclient(hunting): authenticate will time out at 848.083351s 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812257d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f811c098280 0x7f811c141a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35874/0 (socket says 172.21.15.26:35874) 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.647+0000 7f812257d700 1 -- 172.21.15.26:0/1333035649 learned_addr learned my addr 172.21.15.26:0/1333035649 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517624812 0 0) 0x7f811c142830 con 0x7f811c098280 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): my global_id is 4342 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8108003700 con 0x7f811c098280 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222126266 0 0) 0x7f811c11da20 con 0x7f811c09d920 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): my global_id is 4384 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c142830 con 0x7f811c09d920 2024-09-15T03:45:03.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250729455 0 0) 0x7f811c06de20 con 0x7f811c09e4b0 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): my global_id is 4302 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c11da20 con 0x7f811c09e4b0 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3034804713 0 0) 0x7f8108003700 con 0x7f811c098280 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f811c06de20 con 0x7f811c098280 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2511843835 0 0) 0x7f811c11da20 con 0x7f811c09e4b0 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8108003700 con 0x7f811c09e4b0 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8114001de0 con 0x7f811c098280 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): dump: 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.667 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8118001740 con 0x7f811c09e4b0 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): dump: 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.669 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 528246577 0 0) 0x7f811c142830 con 0x7f811c09d920 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f811c11da20 con 0x7f811c09d920 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f810c001960 con 0x7f811c09d920 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient(hunting): dump: 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.670 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.671 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2362152189 0 0) 0x7f8108003700 con 0x7f811c09e4b0 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 >> v1:172.21.15.26:6791/0 conn(0x7f811c098280 legacy=0x7f811c141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 >> v1:172.21.15.122:6789/0 conn(0x7f811c09d920 legacy=0x7f811c142120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 monclient: found mon.a 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c11d740 con 0x7f811c09e4b0 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 5 monclient: authenticate success, global_id 4302 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 10 monclient: _renew_subs 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c06d250 con 0x7f811c09e4b0 2024-09-15T03:45:03.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 10 monclient: _renew_subs 2024-09-15T03:45:03.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c06c690 con 0x7f811c09e4b0 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8118001ba0 con 0x7f811c09e4b0 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8118001060 con 0x7f811c09e4b0 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f81127fc700 10 monclient: dump: 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.674 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.655+0000 7f8120d7a700 4 set_mon_vals no callback set 2024-09-15T03:45:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.659+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f811800fa60 con 0x7f811c09e4b0 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.659+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f8118043d00 con 0x7f811c09e4b0 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.659+0000 7f812257d700 10 monclient: get_auth_request con 0x7f8108038680 auth_method 0 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.663+0000 7f812357f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.663+0000 7f812357f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.663+0000 7f812357f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.663+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81040050b0 con 0x7f811c09e4b0 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.667+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f8118010e90 con 0x7f811c09e4b0 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.667+0000 7f81127fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.667+0000 7f81127fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.667+0000 7f943a2a0700 10 monclient: get_monmap_and_config 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.667+0000 7f943a2a0700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.667+0000 7f943a2a0700 10 monclient: monmap: 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:03.675789+0000 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:03.675789+0000 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 1 Processor -- start 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 1 -- start start 2024-09-15T03:45:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient: init 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient: _add_conns ranks=[3,6,4,1,0,7,2,8,5] 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient(hunting): picked mon.noname-g con 0x7f94340a2910 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient(hunting): picked mon.noname-d con 0x7f943412d7b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient(hunting): picked mon.noname-c con 0x7f943412e2d0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943404ca10 con 0x7f94340a2910 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9434098b90 con 0x7f943412e2d0 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94340a2ce0 con 0x7f943412d7b0 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient(hunting): authenticate will time out at 848.105922s 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f94337fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f943412d7b0 0x7f943412dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55074/0 (socket says 172.21.15.26:55074) 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943883d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f943412e2d0 0x7f9434138fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35886/0 (socket says 172.21.15.26:35886) 2024-09-15T03:45:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f94337fe700 1 -- 172.21.15.26:0/2573394433 learned_addr learned my addr 172.21.15.26:0/2573394433 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812539969 0 0) 0x7f9434098b90 con 0x7f943412e2d0 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient(hunting): my global_id is 4351 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f941c003570 con 0x7f943412e2d0 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986568595 0 0) 0x7f94340a2ce0 con 0x7f943412d7b0 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient(hunting): my global_id is 4335 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9434098b90 con 0x7f943412d7b0 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4125348731 0 0) 0x7f941c003570 con 0x7f943412e2d0 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 >> v1:172.21.15.26:6792/0 conn(0x7f943412d7b0 legacy=0x7f943412dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 >> v1:172.21.15.122:6790/0 conn(0x7f94340a2910 legacy=0x7f9434130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 monclient: found mon.noname-c 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94341316f0 con 0x7f943412e2d0 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f942c001b30 con 0x7f943412e2d0 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: dump: 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.683 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.684 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 5 monclient: authenticate success, global_id 4351 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f942c001d00 con 0x7f943412e2d0 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 1 -- 172.21.15.26:0/2573394433 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f942c004bb0 con 0x7f943412e2d0 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f9432ffd700 10 monclient: dump: 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 4 set_mon_vals no callback set 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 10 monclient: shutdown 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.671+0000 7f943a2a0700 1 -- 172.21.15.26:0/2573394433 >> v1:172.21.15.26:6791/0 conn(0x7f943412e2d0 legacy=0x7f9434138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.675+0000 7f943a2a0700 1 -- 172.21.15.26:0/2573394433 shutdown_connections 2024-09-15T03:45:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.675+0000 7f943a2a0700 1 -- 172.21.15.26:0/2573394433 >> 172.21.15.26:0/2573394433 conn(0x7f9434063ad0 msgr2=0x7f94340a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 -- 172.21.15.26:0/2573394433 shutdown_connections 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 -- 172.21.15.26:0/2573394433 wait complete. 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient: build_initial_monmap 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient: monmap: 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 Processor -- start 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 -- start start 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient: init 2024-09-15T03:45:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient: _add_conns ranks=[2,7,1,3,4,8,5,0,6] 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient(hunting): picked mon.noname-b con 0x7f94340a2910 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient(hunting): picked mon.noname-i con 0x7f943412d7b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient(hunting): picked mon.noname-f con 0x7f943412e2d0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943409bad0 con 0x7f943412e2d0 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943406e060 con 0x7f94340a2910 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9434119950 con 0x7f943412d7b0 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f943a2a0700 10 monclient(hunting): authenticate will time out at 848.114331s 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f9433fff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f94340a2910 0x7f943412c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54328/0 (socket says 172.21.15.26:54328) 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f9433fff700 1 -- 172.21.15.26:0/2942506381 learned_addr learned my addr 172.21.15.26:0/2942506381 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349711235 0 0) 0x7f943406e060 con 0x7f94340a2910 2024-09-15T03:45:03.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): my global_id is 4385 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9420003570 con 0x7f94340a2910 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204540102 0 0) 0x7f9434119950 con 0x7f943412d7b0 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): my global_id is 4453 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f943406e060 con 0x7f943412d7b0 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466766565 0 0) 0x7f943409bad0 con 0x7f943412e2d0 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): my global_id is 4393 2024-09-15T03:45:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9434119950 con 0x7f943412e2d0 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3754348377 0 0) 0x7f9420003570 con 0x7f94340a2910 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f943409bad0 con 0x7f94340a2910 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9424001a60 con 0x7f94340a2910 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): dump: 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.694 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 392080747 0 0) 0x7f943406e060 con 0x7f943412d7b0 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9420003570 con 0x7f943412d7b0 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f94280019b0 con 0x7f943412d7b0 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.679+0000 7f941bfff700 10 monclient(hunting): dump: 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2321643146 0 0) 0x7f943409bad0 con 0x7f94340a2910 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 >> v1:172.21.15.122:6792/0 conn(0x7f943412d7b0 legacy=0x7f9434144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 >> v1:172.21.15.122:6789/0 conn(0x7f943412e2d0 legacy=0x7f94341486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 monclient: found mon.b 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9434135190 con 0x7f94340a2910 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: _finish_auth 0 2024-09-15T03:45:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 5 monclient: authenticate success, global_id 4385 2024-09-15T03:45:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 monclient: _renew_subs 2024-09-15T03:45:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9434140a70 con 0x7f94340a2910 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 monclient: _renew_subs 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f943412cf40 con 0x7f94340a2910 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943404bbf0 con 0x7f94340a2910 2024-09-15T03:45:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9424001d40 con 0x7f94340a2910 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f94327fc700 4 set_mon_vals no callback set 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f94240025b0 con 0x7f94340a2910 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 10 monclient: dump: 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.683+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9424002710 con 0x7f94340a2910 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.691+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f9424043cf0 con 0x7f94340a2910 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.691+0000 7f94337fe700 10 monclient: get_auth_request con 0x7f9420037f00 auth_method 0 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.695+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f9424014d00 con 0x7f94340a2910 2024-09-15T03:45:03.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.695+0000 7f941bfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:03.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.695+0000 7f941bfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3aa6046700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T03:45:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3aa6046700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T03:45:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3aa6046700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:03.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a84005360 con 0x7f3aa00a3860 2024-09-15T03:45:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3a977fe700 1 -- 172.21.15.26:0/2954426579 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 1444633209) 0x7f3a88014950 con 0x7f3aa00a3860 2024-09-15T03:45:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3a977fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-09-15T03:45:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.795+0000 7f3a977fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:03.799 INFO:teuthology.orchestra.run.smithi026.stdout:47244640260 2024-09-15T03:45:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 >> v1:172.21.15.122:6812/20895 conn(0x7f3a80038240 legacy=0x7f3a8003a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 10 monclient: shutdown 2024-09-15T03:45:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 >> v1:172.21.15.26:6792/0 conn(0x7f3aa00a3860 legacy=0x7f3aa0143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 shutdown_connections 2024-09-15T03:45:03.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 >> 172.21.15.26:0/2954426579 conn(0x7f3aa0071350 msgr2=0x7f3aa0131900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 shutdown_connections 2024-09-15T03:45:03.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.799+0000 7f3aa6046700 1 -- 172.21.15.26:0/2954426579 wait complete. 2024-09-15T03:45:03.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.2 2024-09-15T03:45:03.816 DEBUG:teuthology.parallel:result is None 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87d327f700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87d327f700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87d327f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f87ac005360 con 0x7f87cc12f2c0 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87c17fa700 1 -- 172.21.15.26:0/1124279571 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (unknown 1823589948 0 847495621) 0x7f87c4014a60 con 0x7f87cc12f2c0 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87c17fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f87c17fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:03.871 INFO:teuthology.orchestra.run.smithi026.stdout:47244640259 2024-09-15T03:45:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b71745700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T03:45:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b71745700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T03:45:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b71745700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b4c005360 con 0x7f5b6c1302f0 2024-09-15T03:45:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b52ffd700 1 -- 172.21.15.26:0/1243645442 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (unknown 383520633 0 1169801742) 0x7f5b64014ad0 con 0x7f5b6c1302f0 2024-09-15T03:45:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b52ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-09-15T03:45:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.867+0000 7f5b52ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:03.873 INFO:teuthology.orchestra.run.smithi026.stdout:47244640261 2024-09-15T03:45:03.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 >> v1:172.21.15.122:6812/20895 conn(0x7f87b0037e30 legacy=0x7f87b003a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 10 monclient: shutdown 2024-09-15T03:45:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 >> v1:172.21.15.26:6790/0 conn(0x7f87cc12f2c0 legacy=0x7f87cc1485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 shutdown_connections 2024-09-15T03:45:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 >> 172.21.15.26:0/1124279571 conn(0x7f87cc063ad0 msgr2=0x7f87cc06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 shutdown_connections 2024-09-15T03:45:03.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.871+0000 7f87d327f700 1 -- 172.21.15.26:0/1124279571 wait complete. 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 >> v1:172.21.15.122:6812/20895 conn(0x7f5b58037d90 legacy=0x7f5b5803a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 10 monclient: shutdown 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 >> v1:172.21.15.26:6789/0 conn(0x7f5b6c1302f0 legacy=0x7f5b6c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 shutdown_connections 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 >> 172.21.15.26:0/1243645442 conn(0x7f5b6c063ad0 msgr2=0x7f5b6c0a00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 shutdown_connections 2024-09-15T03:45:03.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.875+0000 7f5b71745700 1 -- 172.21.15.26:0/1243645442 wait complete. 2024-09-15T03:45:03.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640259 got 47244640259 for osd.4 2024-09-15T03:45:03.890 DEBUG:teuthology.parallel:result is None 2024-09-15T03:45:03.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.3 2024-09-15T03:45:03.891 DEBUG:teuthology.parallel:result is None 2024-09-15T03:45:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5ce7fff700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T03:45:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5ce7fff700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T03:45:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5ce7fff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5d04066de0 con 0x7f5d04097e00 2024-09-15T03:45:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5cfa7fc700 1 -- 172.21.15.26:0/972648717 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 1169801742) 0x7f5cf4014860 con 0x7f5d04097e00 2024-09-15T03:45:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5cfa7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-09-15T03:45:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.927+0000 7f5cfa7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:03.933 INFO:teuthology.orchestra.run.smithi026.stdout:47244640261 2024-09-15T03:45:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 >> v1:172.21.15.122:6812/20895 conn(0x7f5ce80381b0 legacy=0x7f5ce803a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 10 monclient: shutdown 2024-09-15T03:45:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 >> v1:172.21.15.26:6793/0 conn(0x7f5d04097e00 legacy=0x7f5d04119940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 shutdown_connections 2024-09-15T03:45:03.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 >> 172.21.15.26:0/972648717 conn(0x7f5d04063dc0 msgr2=0x7f5d0412ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 shutdown_connections 2024-09-15T03:45:03.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.931+0000 7f5ce7fff700 1 -- 172.21.15.26:0/972648717 wait complete. 2024-09-15T03:45:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.943+0000 7f943a2a0700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T03:45:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.943+0000 7f943a2a0700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T03:45:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.943+0000 7f943a2a0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.943+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f943406f230 con 0x7f94340a2910 2024-09-15T03:45:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f941bfff700 1 -- 172.21.15.26:0/2942506381 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (unknown 2131975755 0 1444633209) 0x7f9424014700 con 0x7f94340a2910 2024-09-15T03:45:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f941bfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-09-15T03:45:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f941bfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:03.951 INFO:teuthology.orchestra.run.smithi026.stdout:47244640260 2024-09-15T03:45:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 >> v1:172.21.15.122:6812/20895 conn(0x7f9420037f00 legacy=0x7f942003a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 10 monclient: shutdown 2024-09-15T03:45:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 >> v1:172.21.15.26:6790/0 conn(0x7f94340a2910 legacy=0x7f943412c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 shutdown_connections 2024-09-15T03:45:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 >> 172.21.15.26:0/2942506381 conn(0x7f9434063ad0 msgr2=0x7f943412e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 shutdown_connections 2024-09-15T03:45:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.947+0000 7f943a2a0700 1 -- 172.21.15.26:0/2942506381 wait complete. 2024-09-15T03:45:03.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.1 2024-09-15T03:45:03.955 DEBUG:teuthology.parallel:result is None 2024-09-15T03:45:03.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.5 2024-09-15T03:45:03.962 DEBUG:teuthology.parallel:result is None 2024-09-15T03:45:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f812357f700 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T03:45:03.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f812357f700 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T03:45:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f812357f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8104005360 con 0x7f811c09e4b0 2024-09-15T03:45:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f81127fc700 1 -- 172.21.15.26:0/1333035649 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 1444633209) 0x7f8118014750 con 0x7f811c09e4b0 2024-09-15T03:45:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f81127fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-09-15T03:45:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.963+0000 7f81127fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:03.971 INFO:teuthology.orchestra.run.smithi026.stdout:47244640260 2024-09-15T03:45:03.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 >> v1:172.21.15.122:6812/20895 conn(0x7f8108038680 legacy=0x7f810803ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 10 monclient: shutdown 2024-09-15T03:45:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 >> v1:172.21.15.26:6789/0 conn(0x7f811c09e4b0 legacy=0x7f811c149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 shutdown_connections 2024-09-15T03:45:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 >> 172.21.15.26:0/1333035649 conn(0x7f811c063dc0 msgr2=0x7f811c12ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:03.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 shutdown_connections 2024-09-15T03:45:03.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:03.967+0000 7f812357f700 1 -- 172.21.15.26:0/1333035649 wait complete. 2024-09-15T03:45:03.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.0 2024-09-15T03:45:03.982 DEBUG:teuthology.parallel:result is None 2024-09-15T03:45:03.983 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T03:45:03.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient: get_monmap_and_config 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient: build_initial_monmap 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient: monmap: 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:04.082767+0000 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:04.082767+0000 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:04.085 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 1 Processor -- start 2024-09-15T03:45:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 1 -- start start 2024-09-15T03:45:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient: init 2024-09-15T03:45:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient: _add_conns ranks=[5,8,3,6,2,0,7,1,4] 2024-09-15T03:45:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient(hunting): picked mon.noname-h con 0x7f2e9009ba90 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.079+0000 7f2e9794e700 10 monclient(hunting): picked mon.noname-e con 0x7f2e90098a70 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient(hunting): picked mon.noname-g con 0x7f2e9013d2a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9004ca10 con 0x7f2e9013d2a0 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9009d080 con 0x7f2e9009ba90 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9009e490 con 0x7f2e90098a70 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient(hunting): authenticate will time out at 848.512098s 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e94ee9700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f2e90098a70 0x7f2e90099f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48868/0 (socket says 172.21.15.26:48868) 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e94ee9700 1 -- 172.21.15.26:0/2122514251 learned_addr learned my addr 172.21.15.26:0/2122514251 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645302529 0 0) 0x7f2e9009e490 con 0x7f2e90098a70 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): my global_id is 4355 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c003570 con 0x7f2e90098a70 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17052481 0 0) 0x7f2e9004ca10 con 0x7f2e9013d2a0 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): my global_id is 4422 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9009e490 con 0x7f2e9013d2a0 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258051439 0 0) 0x7f2e9009d080 con 0x7f2e9009ba90 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient(hunting): my global_id is 4361 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9004ca10 con 0x7f2e9009ba90 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 674313895 0 0) 0x7f2e7c003570 con 0x7f2e90098a70 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 >> v1:172.21.15.122:6791/0 conn(0x7f2e9009ba90 legacy=0x7f2e9009be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 >> v1:172.21.15.122:6790/0 conn(0x7f2e9013d2a0 legacy=0x7f2e9013f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 monclient: found mon.noname-e 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9009a680 con 0x7f2e90098a70 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: _finish_auth 0 2024-09-15T03:45:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 5 monclient: authenticate success, global_id 4355 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e8c001b30 con 0x7f2e90098a70 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: dump: 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.094 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:04.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2e8c001d80 con 0x7f2e90098a70 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 1 -- 172.21.15.26:0/2122514251 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e8c004fb0 con 0x7f2e90098a70 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 4 set_mon_vals no callback set 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e87fff700 10 monclient: dump: 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient: shutdown 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:04.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- 172.21.15.26:0/2122514251 >> v1:172.21.15.26:6793/0 conn(0x7f2e90098a70 legacy=0x7f2e90099f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- 172.21.15.26:0/2122514251 shutdown_connections 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- 172.21.15.26:0/2122514251 >> 172.21.15.26:0/2122514251 conn(0x7f2e90063f30 msgr2=0x7f2e900a2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- 172.21.15.26:0/2122514251 shutdown_connections 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 1 -- 172.21.15.26:0/2122514251 wait complete. 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient: build_initial_monmap 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.083+0000 7f2e9794e700 10 monclient: monmap: 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:04.098 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 Processor -- start 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 -- start start 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: init 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: _add_conns ranks=[2,4,6,7,5,0,3,8,1] 2024-09-15T03:45:04.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient(hunting): picked mon.noname-b con 0x7f2e90098a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient(hunting): picked mon.noname-c con 0x7f2e9009ba90 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient(hunting): picked mon.noname-d con 0x7f2e9013d2a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9009d5a0 con 0x7f2e90098a70 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9006e5a0 con 0x7f2e9009ba90 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9006d250 con 0x7f2e9013d2a0 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient(hunting): authenticate will time out at 848.517263s 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e94ee9700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f2e9009ba90 0x7f2e9013c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35898/0 (socket says 172.21.15.26:35898) 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e956ea700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f2e90098a70 0x7f2e9013b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54332/0 (socket says 172.21.15.26:54332) 2024-09-15T03:45:04.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e94ee9700 1 -- 172.21.15.26:0/2006504368 learned_addr learned my addr 172.21.15.26:0/2006504368 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535792 0 0) 0x7f2e9009d5a0 con 0x7f2e90098a70 2024-09-15T03:45:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): my global_id is 4394 2024-09-15T03:45:04.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e74003570 con 0x7f2e90098a70 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173216309 0 0) 0x7f2e9006d250 con 0x7f2e9013d2a0 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): my global_id is 4344 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9009d5a0 con 0x7f2e9013d2a0 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277873778 0 0) 0x7f2e9006e5a0 con 0x7f2e9009ba90 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): my global_id is 4360 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e9006d250 con 0x7f2e9009ba90 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3243723926 0 0) 0x7f2e9006d250 con 0x7f2e9009ba90 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e9006e5a0 con 0x7f2e9009ba90 2024-09-15T03:45:04.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 392576022 0 0) 0x7f2e9009d5a0 con 0x7f2e9013d2a0 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e9006d250 con 0x7f2e9013d2a0 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1242102023 0 0) 0x7f2e74003570 con 0x7f2e90098a70 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e9009d5a0 con 0x7f2e90098a70 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e8c001af0 con 0x7f2e9009ba90 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): dump: 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.106 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e88001900 con 0x7f2e9013d2a0 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): dump: 2024-09-15T03:45:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e800019e0 con 0x7f2e90098a70 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient(hunting): dump: 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4268616496 0 0) 0x7f2e9006e5a0 con 0x7f2e9009ba90 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 >> v1:172.21.15.26:6792/0 conn(0x7f2e9013d2a0 legacy=0x7f2e9013c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 >> v1:172.21.15.26:6790/0 conn(0x7f2e90098a70 legacy=0x7f2e9013b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: _finish_hunting 0 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 monclient: found mon.c 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9006ad50 con 0x7f2e9009ba90 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: _finish_auth 0 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 5 monclient: authenticate success, global_id 4360 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: _renew_subs 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e900692b0 con 0x7f2e9009ba90 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: _renew_subs 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9006a000 con 0x7f2e9009ba90 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2e8c001550 con 0x7f2e9009ba90 2024-09-15T03:45:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e8c004f00 con 0x7f2e9009ba90 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e877fe700 4 set_mon_vals no callback set 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.087+0000 7f2e857fa700 10 monclient: dump: 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e9794e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e9794e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e9794e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e700050b0 con 0x7f2e9009ba90 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2e8c00fe70 con 0x7f2e9009ba90 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f2e8c0441a0 con 0x7f2e9009ba90 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.091+0000 7f2e956ea700 10 monclient: get_auth_request con 0x7f2e74038170 auth_method 0 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.095+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f2e8c0151f0 con 0x7f2e9009ba90 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.095+0000 7f2e857fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.095+0000 7f2e857fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:04.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.259+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e70000b90 con 0x7f2e74038170 2024-09-15T03:45:04.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e857fa700 1 -- 172.21.15.26:0/2006504368 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41010 (unknown 2965378022 0 1150296108) 0x7f2e70000b90 con 0x7f2e74038170 2024-09-15T03:45:04.269 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 >> v1:172.21.15.122:6812/20895 conn(0x7f2e74038170 legacy=0x7f2e7403a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 10 monclient: shutdown 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 >> v1:172.21.15.26:6791/0 conn(0x7f2e9009ba90 legacy=0x7f2e9013c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 shutdown_connections 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 >> 172.21.15.26:0/2006504368 conn(0x7f2e90063f30 msgr2=0x7f2e900a1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:04.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.263+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 shutdown_connections 2024-09-15T03:45:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.267+0000 7f2e9794e700 1 -- 172.21.15.26:0/2006504368 wait complete. 2024-09-15T03:45:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T03:45:04.280 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-15T03:45:03.460602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786272,"kb_used_data":1864,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444768,"statfs":{"total":579820584960,"available":577991442432,"internally_reserved":0,"allocated":1908736,"data_stored":1309276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900929+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T03:45:00.900929+0000","last_peered":"2024-09-15T03:45:00.900929+0000","last_clean":"2024-09-15T03:45:00.900929+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T03:45:00.900929+0000","last_undegraded":"2024-09-15T03:45:00.900929+0000","last_fullsized":"2024-09-15T03:45:00.900929+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747766+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T03:45:00.747766+0000","last_peered":"2024-09-15T03:45:00.747766+0000","last_clean":"2024-09-15T03:45:00.747766+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T03:45:00.747766+0000","last_undegraded":"2024-09-15T03:45:00.747766+0000","last_fullsized":"2024-09-15T03:45:00.747766+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747588+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T03:45:00.747588+0000","last_peered":"2024-09-15T03:45:00.747588+0000","last_clean":"2024-09-15T03:45:00.747588+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T03:45:00.747588+0000","last_undegraded":"2024-09-15T03:45:00.747588+0000","last_fullsized":"2024-09-15T03:45:00.747588+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.746955+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T03:45:00.746955+0000","last_peered":"2024-09-15T03:45:00.746955+0000","last_clean":"2024-09-15T03:45:00.746955+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T03:45:00.746955+0000","last_undegraded":"2024-09-15T03:45:00.746955+0000","last_fullsized":"2024-09-15T03:45:00.746955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.745376+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T03:45:00.745376+0000","last_peered":"2024-09-15T03:45:00.745376+0000","last_clean":"2024-09-15T03:45:00.745376+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T03:45:00.745376+0000","last_undegraded":"2024-09-15T03:45:00.745376+0000","last_fullsized":"2024-09-15T03:45:00.745376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747049+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T03:45:00.747049+0000","last_peered":"2024-09-15T03:45:00.747049+0000","last_clean":"2024-09-15T03:45:00.747049+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T03:45:00.747049+0000","last_undegraded":"2024-09-15T03:45:00.747049+0000","last_fullsized":"2024-09-15T03:45:00.747049+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.748385+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T03:45:00.748385+0000","last_peered":"2024-09-15T03:45:00.748385+0000","last_clean":"2024-09-15T03:45:00.748385+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T03:45:00.748385+0000","last_undegraded":"2024-09-15T03:45:00.748385+0000","last_fullsized":"2024-09-15T03:45:00.748385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900829+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T03:45:00.900829+0000","last_peered":"2024-09-15T03:45:00.900829+0000","last_clean":"2024-09-15T03:45:00.900829+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T03:45:00.900829+0000","last_undegraded":"2024-09-15T03:45:00.900829+0000","last_fullsized":"2024-09-15T03:45:00.900829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747827+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T03:45:00.747827+0000","last_peered":"2024-09-15T03:45:00.747827+0000","last_clean":"2024-09-15T03:45:00.747827+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T03:45:00.747827+0000","last_undegraded":"2024-09-15T03:45:00.747827+0000","last_fullsized":"2024-09-15T03:45:00.747827+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:45:04.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient: get_monmap_and_config 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient: build_initial_monmap 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient: monmap: 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:04.384059+0000 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:04.384059+0000 2024-09-15T03:45:04.385 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:04.386 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 1 Processor -- start 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 1 -- start start 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient: init 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient: _add_conns ranks=[6,7,5,1,0,2,4,3,8] 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient(hunting): picked mon.noname-d con 0x7f8e440715a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient(hunting): picked mon.noname-i con 0x7f8e4409f640 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient(hunting): picked mon.noname-h con 0x7f8e44099c90 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4404ca10 con 0x7f8e44099c90 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44098ca0 con 0x7f8e440715a0 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44071970 con 0x7f8e4409f640 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e499ea700 10 monclient(hunting): authenticate will time out at 848.814986s 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e42ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8e440715a0 0x7f8e4409da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55100/0 (socket says 172.21.15.26:55100) 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e42ffd700 1 -- 172.21.15.26:0/1826377489 learned_addr learned my addr 172.21.15.26:0/1826377489 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558372125 0 0) 0x7f8e44098ca0 con 0x7f8e440715a0 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): my global_id is 4353 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003570 con 0x7f8e440715a0 2024-09-15T03:45:04.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044550074 0 0) 0x7f8e44071970 con 0x7f8e4409f640 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): my global_id is 4462 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e44098ca0 con 0x7f8e4409f640 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329636883 0 0) 0x7f8e4404ca10 con 0x7f8e44099c90 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 10 monclient(hunting): my global_id is 4370 2024-09-15T03:45:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.383+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e44071970 con 0x7f8e44099c90 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740017079 0 0) 0x7f8e2c003570 con 0x7f8e440715a0 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 >> v1:172.21.15.122:6792/0 conn(0x7f8e4409f640 legacy=0x7f8e4409e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 >> v1:172.21.15.122:6791/0 conn(0x7f8e44099c90 legacy=0x7f8e44137d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 monclient: found mon.noname-d 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4409e8a0 con 0x7f8e440715a0 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 5 monclient: authenticate success, global_id 4353 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e340017d0 con 0x7f8e440715a0 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: dump: 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8e34001ab0 con 0x7f8e440715a0 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 4 set_mon_vals no callback set 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 1 -- 172.21.15.26:0/1826377489 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e34004c30 con 0x7f8e440715a0 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e41ffb700 10 monclient: dump: 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: shutdown 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- 172.21.15.26:0/1826377489 >> v1:172.21.15.26:6792/0 conn(0x7f8e440715a0 legacy=0x7f8e4409da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- 172.21.15.26:0/1826377489 shutdown_connections 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- 172.21.15.26:0/1826377489 >> 172.21.15.26:0/1826377489 conn(0x7f8e440a3740 msgr2=0x7f8e44070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- 172.21.15.26:0/1826377489 shutdown_connections 2024-09-15T03:45:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- 172.21.15.26:0/1826377489 wait complete. 2024-09-15T03:45:04.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: build_initial_monmap 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: monmap: 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:04.402 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 Processor -- start 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- start start 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: init 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient: _add_conns ranks=[6,3,8,7,2,4,1,5,0] 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient(hunting): picked mon.noname-d con 0x7f8e440715a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient(hunting): picked mon.noname-g con 0x7f8e44099c90 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:04.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient(hunting): picked mon.noname-e con 0x7f8e4409f640 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44120d20 con 0x7f8e44099c90 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44120a60 con 0x7f8e440715a0 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44121000 con 0x7f8e4409f640 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.387+0000 7f8e499ea700 10 monclient(hunting): authenticate will time out at 848.819860s 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e42ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8e440715a0 0x7f8e44138970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55112/0 (socket says 172.21.15.26:55112) 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e42ffd700 1 -- 172.21.15.26:0/1104644354 learned_addr learned my addr 172.21.15.26:0/1104644354 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396636010 0 0) 0x7f8e44121000 con 0x7f8e4409f640 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): my global_id is 4364 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e30003570 con 0x7f8e4409f640 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205115778 0 0) 0x7f8e44120a60 con 0x7f8e440715a0 2024-09-15T03:45:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): my global_id is 4362 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e44121000 con 0x7f8e440715a0 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535200411 0 0) 0x7f8e44120d20 con 0x7f8e44099c90 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): my global_id is 4431 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e44120a60 con 0x7f8e44099c90 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 817897147 0 0) 0x7f8e30003570 con 0x7f8e4409f640 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e44120d20 con 0x7f8e4409f640 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e3c001de0 con 0x7f8e4409f640 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): dump: 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.406 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3942216131 0 0) 0x7f8e44121000 con 0x7f8e440715a0 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e30003570 con 0x7f8e440715a0 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e34004b70 con 0x7f8e440715a0 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): dump: 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.407 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.408 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2561791756 0 0) 0x7f8e44120a60 con 0x7f8e44099c90 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e44121000 con 0x7f8e44099c90 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e38001930 con 0x7f8e44099c90 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient(hunting): dump: 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.409 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 251085433 0 0) 0x7f8e30003570 con 0x7f8e440715a0 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 >> v1:172.21.15.26:6793/0 conn(0x7f8e4409f640 legacy=0x7f8e44148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 >> v1:172.21.15.122:6790/0 conn(0x7f8e44099c90 legacy=0x7f8e44144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 monclient: found mon.d 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4411f620 con 0x7f8e440715a0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: _finish_auth 0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 5 monclient: authenticate success, global_id 4362 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 monclient: _renew_subs 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e44139080 con 0x7f8e440715a0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 monclient: _renew_subs 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4411ebe0 con 0x7f8e440715a0 2024-09-15T03:45:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8e340017c0 con 0x7f8e440715a0 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e34005240 con 0x7f8e440715a0 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e417fa700 4 set_mon_vals no callback set 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 10 monclient: dump: 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.412 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:04.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e240050b0 con 0x7f8e440715a0 2024-09-15T03:45:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8e3400fc00 con 0x7f8e440715a0 2024-09-15T03:45:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.391+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f8e34043f20 con 0x7f8e440715a0 2024-09-15T03:45:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.395+0000 7f8e427fc700 10 monclient: get_auth_request con 0x7f8e30037f20 auth_method 0 2024-09-15T03:45:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.399+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f8e34014f70 con 0x7f8e440715a0 2024-09-15T03:45:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.399+0000 7f8e2affd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.399+0000 7f8e2affd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:04.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.559+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e24000b90 con 0x7f8e30037f20 2024-09-15T03:45:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e2affd700 1 -- 172.21.15.26:0/1104644354 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41010 (unknown 2965378022 0 1150296108) 0x7f8e24000b90 con 0x7f8e30037f20 2024-09-15T03:45:04.566 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 >> v1:172.21.15.122:6812/20895 conn(0x7f8e30037f20 legacy=0x7f8e3003a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 10 monclient: shutdown 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 >> v1:172.21.15.26:6792/0 conn(0x7f8e440715a0 legacy=0x7f8e44138970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 shutdown_connections 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 >> 172.21.15.26:0/1104644354 conn(0x7f8e440a3740 msgr2=0x7f8e4409b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 shutdown_connections 2024-09-15T03:45:04.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.563+0000 7f8e499ea700 1 -- 172.21.15.26:0/1104644354 wait complete. 2024-09-15T03:45:04.568 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T03:45:04.579 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-15T03:45:03.460602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786272,"kb_used_data":1864,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444768,"statfs":{"total":579820584960,"available":577991442432,"internally_reserved":0,"allocated":1908736,"data_stored":1309276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900929+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T03:45:00.900929+0000","last_peered":"2024-09-15T03:45:00.900929+0000","last_clean":"2024-09-15T03:45:00.900929+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T03:45:00.900929+0000","last_undegraded":"2024-09-15T03:45:00.900929+0000","last_fullsized":"2024-09-15T03:45:00.900929+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747766+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T03:45:00.747766+0000","last_peered":"2024-09-15T03:45:00.747766+0000","last_clean":"2024-09-15T03:45:00.747766+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T03:45:00.747766+0000","last_undegraded":"2024-09-15T03:45:00.747766+0000","last_fullsized":"2024-09-15T03:45:00.747766+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747588+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T03:45:00.747588+0000","last_peered":"2024-09-15T03:45:00.747588+0000","last_clean":"2024-09-15T03:45:00.747588+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T03:45:00.747588+0000","last_undegraded":"2024-09-15T03:45:00.747588+0000","last_fullsized":"2024-09-15T03:45:00.747588+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.746955+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T03:45:00.746955+0000","last_peered":"2024-09-15T03:45:00.746955+0000","last_clean":"2024-09-15T03:45:00.746955+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T03:45:00.746955+0000","last_undegraded":"2024-09-15T03:45:00.746955+0000","last_fullsized":"2024-09-15T03:45:00.746955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.745376+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T03:45:00.745376+0000","last_peered":"2024-09-15T03:45:00.745376+0000","last_clean":"2024-09-15T03:45:00.745376+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T03:45:00.745376+0000","last_undegraded":"2024-09-15T03:45:00.745376+0000","last_fullsized":"2024-09-15T03:45:00.745376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747049+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T03:45:00.747049+0000","last_peered":"2024-09-15T03:45:00.747049+0000","last_clean":"2024-09-15T03:45:00.747049+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T03:45:00.747049+0000","last_undegraded":"2024-09-15T03:45:00.747049+0000","last_fullsized":"2024-09-15T03:45:00.747049+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.748385+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T03:45:00.748385+0000","last_peered":"2024-09-15T03:45:00.748385+0000","last_clean":"2024-09-15T03:45:00.748385+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T03:45:00.748385+0000","last_undegraded":"2024-09-15T03:45:00.748385+0000","last_fullsized":"2024-09-15T03:45:00.748385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900829+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T03:45:00.900829+0000","last_peered":"2024-09-15T03:45:00.900829+0000","last_clean":"2024-09-15T03:45:00.900829+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T03:45:00.900829+0000","last_undegraded":"2024-09-15T03:45:00.900829+0000","last_fullsized":"2024-09-15T03:45:00.900829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747827+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T03:45:00.747827+0000","last_peered":"2024-09-15T03:45:00.747827+0000","last_clean":"2024-09-15T03:45:00.747827+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T03:45:00.747827+0000","last_undegraded":"2024-09-15T03:45:00.747827+0000","last_fullsized":"2024-09-15T03:45:00.747827+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:45:04.580 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T03:45:04.580 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T03:45:04.580 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T03:45:04.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T03:45:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient: get_monmap_and_config 2024-09-15T03:45:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient: build_initial_monmap 2024-09-15T03:45:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient: monmap: 2024-09-15T03:45:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:04.677151+0000 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:04.677151+0000 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:04.680 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 1 Processor -- start 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 1 -- start start 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient: init 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient: _add_conns ranks=[0,5,8,2,4,3,7,1,6] 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient(hunting): picked mon.noname-a con 0x7f643c09f590 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient(hunting): picked mon.noname-h con 0x7f643c1331b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient(hunting): picked mon.noname-e con 0x7f643c139730 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c04ca10 con 0x7f643c09f590 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c096720 con 0x7f643c1331b0 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c09f960 con 0x7f643c139730 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 10 monclient(hunting): authenticate will time out at 849.106788s 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f643c09f590 0x7f643c098bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53140/0 (socket says 172.21.15.26:53140) 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643b7fe700 1 -- 172.21.15.26:0/2746546643 learned_addr learned my addr 172.21.15.26:0/2746546643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620784349 0 0) 0x7f643c09f960 con 0x7f643c139730 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): my global_id is 4373 2024-09-15T03:45:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6424003570 con 0x7f643c139730 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080796575 0 0) 0x7f643c04ca10 con 0x7f643c09f590 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): my global_id is 4311 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c09f960 con 0x7f643c09f590 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278356868 0 0) 0x7f643c096720 con 0x7f643c1331b0 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient(hunting): my global_id is 4379 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c04ca10 con 0x7f643c1331b0 2024-09-15T03:45:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852520751 0 0) 0x7f6424003570 con 0x7f643c139730 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 >> v1:172.21.15.122:6791/0 conn(0x7f643c1331b0 legacy=0x7f643c138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 >> v1:172.21.15.26:6789/0 conn(0x7f643c09f590 legacy=0x7f643c098bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 monclient: found mon.noname-e 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c06e070 con 0x7f643c139730 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 5 monclient: authenticate success, global_id 4373 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6434001b70 con 0x7f643c139730 2024-09-15T03:45:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f643a7fc700 10 monclient: dump: 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.675+0000 7f6441df6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6434001d90 con 0x7f643c139730 2024-09-15T03:45:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f643a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f643a7fc700 1 -- 172.21.15.26:0/2746546643 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6434001f80 con 0x7f643c139730 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f643a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f643a7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f643a7fc700 10 monclient: dump: 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 4 set_mon_vals no callback set 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 10 monclient: shutdown 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 -- 172.21.15.26:0/2746546643 >> v1:172.21.15.26:6793/0 conn(0x7f643c139730 legacy=0x7f643c13bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 -- 172.21.15.26:0/2746546643 shutdown_connections 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 -- 172.21.15.26:0/2746546643 >> 172.21.15.26:0/2746546643 conn(0x7f643c0a2e20 msgr2=0x7f643c09ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 -- 172.21.15.26:0/2746546643 shutdown_connections 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 -- 172.21.15.26:0/2746546643 wait complete. 2024-09-15T03:45:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 10 monclient: build_initial_monmap 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 10 monclient: monmap: 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:04.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 Processor -- start 2024-09-15T03:45:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 1 -- start start 2024-09-15T03:45:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.679+0000 7f6441df6700 10 monclient: init 2024-09-15T03:45:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient: _add_conns ranks=[3,7,0,6,8,4,5,1,2] 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient(hunting): picked mon.noname-g con 0x7f643c09f590 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient(hunting): picked mon.noname-i con 0x7f643c1331b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient(hunting): picked mon.noname-a con 0x7f643c139730 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c06e2a0 con 0x7f643c139730 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c135730 con 0x7f643c09f590 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c09bfd0 con 0x7f643c1331b0 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient(hunting): authenticate will time out at 849.113153s 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f643bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f643c139730 0x7f643c14a0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53142/0 (socket says 172.21.15.26:53142) 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f643bfff700 1 -- 172.21.15.26:0/1709490103 learned_addr learned my addr 172.21.15.26:0/1709490103 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f643affd700 1 --1- 172.21.15.26:0/1709490103 >> v1:172.21.15.122:6792/0 conn(0x7f643c1331b0 0x7f643c141cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:55982/0 (socket says 172.21.15.26:55982) 2024-09-15T03:45:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224437606 0 0) 0x7f643c09bfd0 con 0x7f643c1331b0 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): my global_id is 4471 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6428003570 con 0x7f643c1331b0 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401748989 0 0) 0x7f643c135730 con 0x7f643c09f590 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): my global_id is 4440 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c09bfd0 con 0x7f643c09f590 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692704797 0 0) 0x7f643c06e2a0 con 0x7f643c139730 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): my global_id is 4320 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c135730 con 0x7f643c139730 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 681324805 0 0) 0x7f6428003570 con 0x7f643c1331b0 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f643c06e2a0 con 0x7f643c1331b0 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2591853749 0 0) 0x7f643c09bfd0 con 0x7f643c09f590 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6428003570 con 0x7f643c09f590 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6430002a10 con 0x7f643c1331b0 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): dump: 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1999419350 0 0) 0x7f643c135730 con 0x7f643c139730 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f643c09bfd0 con 0x7f643c139730 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f642c001a60 con 0x7f643c09f590 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): dump: 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.703 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64340025a0 con 0x7f643c139730 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient(hunting): dump: 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.704 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.705 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2971969544 0 0) 0x7f643c06e2a0 con 0x7f643c1331b0 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 >> v1:172.21.15.122:6790/0 conn(0x7f643c09f590 legacy=0x7f643c138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 >> v1:172.21.15.26:6789/0 conn(0x7f643c139730 legacy=0x7f643c14a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 monclient: found mon.i 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c1344f0 con 0x7f643c1331b0 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 5 monclient: authenticate success, global_id 4471 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient: _renew_subs 2024-09-15T03:45:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c135450 con 0x7f643c1331b0 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient: _renew_subs 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c138920 con 0x7f643c1331b0 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6430002da0 con 0x7f643c1331b0 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6430001880 con 0x7f643c1331b0 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64397fa700 4 set_mon_vals no callback set 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.683+0000 7f64237fe700 10 monclient: dump: 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:04.707 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:04.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f6441df6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f6441df6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f6441df6700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f64300019e0 con 0x7f643c1331b0 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c0050b0 con 0x7f643c1331b0 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f6430043c20 con 0x7f643c1331b0 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.687+0000 7f643b7fe700 10 monclient: get_auth_request con 0x7f6428038050 auth_method 0 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.695+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f6430014bb0 con 0x7f643c1331b0 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.699+0000 7f64237fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.699+0000 7f64237fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-09-15T03:45:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-09-15T03:45:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:45:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f641c0058e0 con 0x7f643c1331b0 2024-09-15T03:45:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f64237fe700 1 -- 172.21.15.26:0/1709490103 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f64300145b0 con 0x7f643c1331b0 2024-09-15T03:45:04.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f64237fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-09-15T03:45:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f64237fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:04.899 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:04.899 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T03:45:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 >> v1:172.21.15.122:6812/20895 conn(0x7f6428038050 legacy=0x7f642803a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 10 monclient: shutdown 2024-09-15T03:45:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 >> v1:172.21.15.122:6792/0 conn(0x7f643c1331b0 legacy=0x7f643c141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 shutdown_connections 2024-09-15T03:45:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.895+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 >> 172.21.15.26:0/1709490103 conn(0x7f643c0a2e20 msgr2=0x7f643c097ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.899+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 shutdown_connections 2024-09-15T03:45:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:04.899+0000 7f6441df6700 1 -- 172.21.15.26:0/1709490103 wait complete. 2024-09-15T03:45:04.912 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T03:45:04.912 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-15T03:45:04.927 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-15T03:45:04.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T03:45:05.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.027+0000 7f4051979700 10 monclient: get_monmap_and_config 2024-09-15T03:45:05.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.027+0000 7f4051979700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.027+0000 7f4051979700 10 monclient: monmap: 2024-09-15T03:45:05.032 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:05.031370+0000 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:05.031370+0000 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.034 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 Processor -- start 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- start start 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: init 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: _add_conns ranks=[0,6,2,5,3,1,8,7,4] 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient(hunting): picked mon.noname-a con 0x7f404c099310 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient(hunting): picked mon.noname-d con 0x7f404c132fa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient(hunting): picked mon.noname-b con 0x7f404c1333f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c04ca10 con 0x7f404c099310 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c097e50 con 0x7f404c1333f0 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c0a1410 con 0x7f404c132fa0 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient(hunting): authenticate will time out at 849.460739s 2024-09-15T03:45:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f404b7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f404c1333f0 0x7f404c13b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54346/0 (socket says 172.21.15.26:54346) 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f404a7fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f404c132fa0 0x7f404c1380a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55120/0 (socket says 172.21.15.26:55120) 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f404b7fe700 1 -- 172.21.15.26:0/4219744983 learned_addr learned my addr 172.21.15.26:0/4219744983 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296447235 0 0) 0x7f404c097e50 con 0x7f404c1333f0 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): my global_id is 4403 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4034003570 con 0x7f404c1333f0 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664298684 0 0) 0x7f404c0a1410 con 0x7f404c132fa0 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): my global_id is 4371 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f404c097e50 con 0x7f404c132fa0 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331046256 0 0) 0x7f404c04ca10 con 0x7f404c099310 2024-09-15T03:45:05.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient(hunting): my global_id is 4329 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f404c0a1410 con 0x7f404c099310 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061507888 0 0) 0x7f404c097e50 con 0x7f404c132fa0 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 >> v1:172.21.15.26:6790/0 conn(0x7f404c1333f0 legacy=0x7f404c13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 >> v1:172.21.15.26:6789/0 conn(0x7f404c099310 legacy=0x7f404c0996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 monclient: found mon.noname-d 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c0a2710 con 0x7f404c132fa0 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f403c0017d0 con 0x7f404c132fa0 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: dump: 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 5 monclient: authenticate success, global_id 4371 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f403c0019f0 con 0x7f404c132fa0 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 1 -- 172.21.15.26:0/4219744983 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f403c001be0 con 0x7f404c132fa0 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 4 set_mon_vals no callback set 2024-09-15T03:45:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4049ffb700 10 monclient: dump: 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: shutdown 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- 172.21.15.26:0/4219744983 >> v1:172.21.15.26:6792/0 conn(0x7f404c132fa0 legacy=0x7f404c1380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- 172.21.15.26:0/4219744983 shutdown_connections 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- 172.21.15.26:0/4219744983 >> 172.21.15.26:0/4219744983 conn(0x7f404c0a6c30 msgr2=0x7f404c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- 172.21.15.26:0/4219744983 shutdown_connections 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 1 -- 172.21.15.26:0/4219744983 wait complete. 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.031+0000 7f4051979700 10 monclient: monmap: 2024-09-15T03:45:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.044 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 Processor -- start 2024-09-15T03:45:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 -- start start 2024-09-15T03:45:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: init 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: _add_conns ranks=[0,6,2,7,5,4,3,8,1] 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient(hunting): picked mon.noname-a con 0x7f404c099310 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient(hunting): picked mon.noname-d con 0x7f404c132fa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient(hunting): picked mon.noname-b con 0x7f404c1333f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c11e8b0 con 0x7f404c099310 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c11fee0 con 0x7f404c1333f0 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c11eca0 con 0x7f404c132fa0 2024-09-15T03:45:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient(hunting): authenticate will time out at 849.464831s 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f404b7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f404c1333f0 0x7f404c14a820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54358/0 (socket says 172.21.15.26:54358) 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f404b7fe700 1 -- 172.21.15.26:0/1942477963 learned_addr learned my addr 172.21.15.26:0/1942477963 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297358743 0 0) 0x7f404c11fee0 con 0x7f404c1333f0 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): my global_id is 4412 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4038003570 con 0x7f404c1333f0 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082760754 0 0) 0x7f404c11eca0 con 0x7f404c132fa0 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): my global_id is 4380 2024-09-15T03:45:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f404c11fee0 con 0x7f404c132fa0 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528136880 0 0) 0x7f404c11e8b0 con 0x7f404c099310 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): my global_id is 4338 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f404c11eca0 con 0x7f404c099310 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1924166246 0 0) 0x7f4038003570 con 0x7f404c1333f0 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f404c11e8b0 con 0x7f404c1333f0 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4044001d60 con 0x7f404c1333f0 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2483958820 0 0) 0x7f404c11fee0 con 0x7f404c132fa0 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4038003570 con 0x7f404c132fa0 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3922695015 0 0) 0x7f404c11eca0 con 0x7f404c099310 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f404c11fee0 con 0x7f404c099310 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f403c001840 con 0x7f404c132fa0 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f40400019e0 con 0x7f404c099310 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient(hunting): dump: 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 306776351 0 0) 0x7f404c11fee0 con 0x7f404c099310 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 >> v1:172.21.15.26:6792/0 conn(0x7f404c132fa0 legacy=0x7f404c1470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 >> v1:172.21.15.26:6790/0 conn(0x7f404c1333f0 legacy=0x7f404c14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 monclient: found mon.a 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c11fc00 con 0x7f404c099310 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 5 monclient: authenticate success, global_id 4338 2024-09-15T03:45:05.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: _renew_subs 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c1201c0 con 0x7f404c099310 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: _renew_subs 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c130360 con 0x7f404c099310 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4040002cd0 con 0x7f404c099310 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4040002270 con 0x7f404c099310 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f40497fa700 4 set_mon_vals no callback set 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 10 monclient: dump: 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f40400023d0 con 0x7f404c099310 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.035+0000 7f4051979700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.039+0000 7f4051979700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.039+0000 7f4051979700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.039+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c0050b0 con 0x7f404c099310 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.039+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f4040041290 con 0x7f404c099310 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.039+0000 7f404a7fc700 10 monclient: get_auth_request con 0x7f4038038070 auth_method 0 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.043+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f4040011e00 con 0x7f404c099310 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.043+0000 7f4032ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.043+0000 7f4032ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.211+0000 7f4051979700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.211+0000 7f4051979700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.211+0000 7f4051979700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.211+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f402c005360 con 0x7f404c099310 2024-09-15T03:45:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4032ffd700 1 -- 172.21.15.26:0/1942477963 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+10084 (unknown 1054498076 0 2475284439) 0x7f4040011c20 con 0x7f404c099310 2024-09-15T03:45:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4032ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T03:45:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4032ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:05.217 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:05.218 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T03:45:00.725199+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T03:44:56.976968+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-15T03:44:57.527097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 >> v1:172.21.15.122:6812/20895 conn(0x7f4038038070 legacy=0x7f403803a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 10 monclient: shutdown 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 >> v1:172.21.15.26:6789/0 conn(0x7f404c099310 legacy=0x7f404c12fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 shutdown_connections 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 >> 172.21.15.26:0/1942477963 conn(0x7f404c0a6c30 msgr2=0x7f404c1325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 shutdown_connections 2024-09-15T03:45:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.215+0000 7f4051979700 1 -- 172.21.15.26:0/1942477963 wait complete. 2024-09-15T03:45:05.232 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-15T03:44:56.976968+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 16, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-15T03:44:57.527097+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-15T03:45:05.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.343+0000 7f6427fff700 10 monclient: get_monmap_and_config 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.343+0000 7f6427fff700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.343+0000 7f6427fff700 10 monclient: monmap: 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:05.347217+0000 2024-09-15T03:45:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:05.347217+0000 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.349 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 1 Processor -- start 2024-09-15T03:45:05.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 1 -- start start 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient: init 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient: _add_conns ranks=[6,8,0,2,4,5,7,1,3] 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient(hunting): picked mon.noname-d con 0x7f64280a6880 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient(hunting): picked mon.noname-e con 0x7f642809faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient(hunting): picked mon.noname-a con 0x7f6428138d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642804ca10 con 0x7f6428138d30 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280a0580 con 0x7f64280a6880 2024-09-15T03:45:05.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6428133780 con 0x7f642809faa0 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient(hunting): authenticate will time out at 849.777688s 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f64277fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6428138d30 0x7f642813b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53172/0 (socket says 172.21.15.26:53172) 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f64277fe700 1 -- 172.21.15.26:0/1833599087 learned_addr learned my addr 172.21.15.26:0/1833599087 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497640541 0 0) 0x7f642804ca10 con 0x7f6428138d30 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): my global_id is 4347 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6410003570 con 0x7f6428138d30 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839421884 0 0) 0x7f6428133780 con 0x7f642809faa0 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): my global_id is 4382 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f642804ca10 con 0x7f642809faa0 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52161382 0 0) 0x7f6418001110 con 0x7f64280a6880 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient(hunting): my global_id is 4389 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6428133780 con 0x7f64280a6880 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3621655670 0 0) 0x7f6410003570 con 0x7f6428138d30 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 >> v1:172.21.15.26:6793/0 conn(0x7f642809faa0 legacy=0x7f642809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 >> v1:172.21.15.26:6792/0 conn(0x7f64280a6880 legacy=0x7f6428133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 monclient: found mon.noname-a 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280a6c50 con 0x7f6428138d30 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 5 monclient: authenticate success, global_id 4347 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6420001b30 con 0x7f6428138d30 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: dump: 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.366 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.367 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6420001d50 con 0x7f6428138d30 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6427fff700 4 set_mon_vals no callback set 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 1 -- 172.21.15.26:0/1833599087 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6420004c70 con 0x7f6428138d30 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.347+0000 7f6425ffb700 10 monclient: dump: 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.368 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.369 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient: shutdown 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- 172.21.15.26:0/1833599087 >> v1:172.21.15.26:6789/0 conn(0x7f6428138d30 legacy=0x7f642813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- 172.21.15.26:0/1833599087 shutdown_connections 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- 172.21.15.26:0/1833599087 >> 172.21.15.26:0/1833599087 conn(0x7f642806e2a0 msgr2=0x7f64280a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- 172.21.15.26:0/1833599087 shutdown_connections 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- 172.21.15.26:0/1833599087 wait complete. 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient: monmap: 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:05.370 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 Processor -- start 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- start start 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient: init 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient: _add_conns ranks=[4,5,2,0,6,8,1,7,3] 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient(hunting): picked mon.noname-c con 0x7f642809faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient(hunting): picked mon.noname-h con 0x7f64280a6880 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient(hunting): picked mon.noname-b con 0x7f6428138d30 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642811d310 con 0x7f6428138d30 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64280a18c0 con 0x7f642809faa0 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642809cf10 con 0x7f64280a6880 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6427fff700 10 monclient(hunting): authenticate will time out at 849.783656s 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f64277fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6428138d30 0x7f642814a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54370/0 (socket says 172.21.15.26:54370) 2024-09-15T03:45:05.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f6426ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f642809faa0 0x7f6428135350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35910/0 (socket says 172.21.15.26:35910) 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.351+0000 7f64277fe700 1 -- 172.21.15.26:0/2221021809 learned_addr learned my addr 172.21.15.26:0/2221021809 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144021014 0 0) 0x7f642811d310 con 0x7f6428138d30 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): my global_id is 4421 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6414003570 con 0x7f6428138d30 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305320224 0 0) 0x7f64280a18c0 con 0x7f642809faa0 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): my global_id is 4369 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f642811d310 con 0x7f642809faa0 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449276026 0 0) 0x7f642809cf10 con 0x7f64280a6880 2024-09-15T03:45:05.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): my global_id is 4388 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64280a18c0 con 0x7f64280a6880 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3131955153 0 0) 0x7f64280a18c0 con 0x7f64280a6880 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f642809cf10 con 0x7f64280a6880 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f641c001640 con 0x7f64280a6880 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): dump: 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:05.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1504014636 0 0) 0x7f6414003570 con 0x7f6428138d30 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64280a18c0 con 0x7f6428138d30 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6420001a30 con 0x7f6428138d30 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): dump: 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.376 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.377 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4285070228 0 0) 0x7f642811d310 con 0x7f642809faa0 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6414003570 con 0x7f642809faa0 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64180017c0 con 0x7f642809faa0 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient(hunting): dump: 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.378 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 485480868 0 0) 0x7f642809cf10 con 0x7f64280a6880 2024-09-15T03:45:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 >> v1:172.21.15.26:6791/0 conn(0x7f642809faa0 legacy=0x7f6428135350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 >> v1:172.21.15.26:6790/0 conn(0x7f6428138d30 legacy=0x7f642814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 monclient: found mon.h 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6428136170 con 0x7f64280a6880 2024-09-15T03:45:05.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 5 monclient: authenticate success, global_id 4388 2024-09-15T03:45:05.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 monclient: _renew_subs 2024-09-15T03:45:05.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:45:05.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6428137310 con 0x7f64280a6880 2024-09-15T03:45:05.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 monclient: _renew_subs 2024-09-15T03:45:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:45:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6428137550 con 0x7f64280a6880 2024-09-15T03:45:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f641c002080 con 0x7f64280a6880 2024-09-15T03:45:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f64257fa700 4 set_mon_vals no callback set 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f641c0025b0 con 0x7f64280a6880 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 10 monclient: dump: 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:45:05.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64080050b0 con 0x7f64280a6880 2024-09-15T03:45:05.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f641c002790 con 0x7f64280a6880 2024-09-15T03:45:05.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.355+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f641c041510 con 0x7f64280a6880 2024-09-15T03:45:05.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.359+0000 7f6426ffd700 10 monclient: get_auth_request con 0x7f64140380a0 auth_method 0 2024-09-15T03:45:05.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.367+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f641c0124a0 con 0x7f64280a6880 2024-09-15T03:45:05.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.367+0000 7f640effd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.367+0000 7f640effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:05.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T03:45:05.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T03:45:05.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:45:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f64080058e0 con 0x7f64280a6880 2024-09-15T03:45:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f640effd700 1 -- 172.21.15.26:0/2221021809 <== mon.5 v1:172.21.15.122:6791/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (unknown 116879340 0 265718887) 0x7f641c011ea0 con 0x7f64280a6880 2024-09-15T03:45:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f640effd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-09-15T03:45:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f640effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:05.535 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 8 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 >> v1:172.21.15.122:6812/20895 conn(0x7f64140380a0 legacy=0x7f641403a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 10 monclient: shutdown 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.531+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 >> v1:172.21.15.122:6791/0 conn(0x7f64280a6880 legacy=0x7f6428135a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.535+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 shutdown_connections 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.535+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 >> 172.21.15.26:0/2221021809 conn(0x7f642806e2a0 msgr2=0x7f642806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.535+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 shutdown_connections 2024-09-15T03:45:05.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.535+0000 7f6427fff700 1 -- 172.21.15.26:0/2221021809 wait complete. 2024-09-15T03:45:05.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient: get_monmap_and_config 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient: monmap: 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:05.657050+0000 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:05.657050+0000 2024-09-15T03:45:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.659 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 1 Processor -- start 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 1 -- start start 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient: init 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient: _add_conns ranks=[3,0,5,1,6,4,2,7,8] 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient(hunting): picked mon.noname-g con 0x7f8144132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient(hunting): picked mon.noname-a con 0x7f814409faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient(hunting): picked mon.noname-h con 0x7f8144138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814404ca10 con 0x7f814409faa0 2024-09-15T03:45:05.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814409d5d0 con 0x7f8144132b40 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8144133620 con 0x7f8144138d40 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8149e92700 10 monclient(hunting): authenticate will time out at 850.086996s 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8143fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f814409faa0 0x7f814409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53188/0 (socket says 172.21.15.26:53188) 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f8143fff700 1 -- 172.21.15.26:0/1932260570 learned_addr learned my addr 172.21.15.26:0/1932260570 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516909475 0 0) 0x7f814404ca10 con 0x7f814409faa0 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): my global_id is 4356 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8130003570 con 0x7f814409faa0 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742573803 0 0) 0x7f8144133620 con 0x7f8144138d40 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): my global_id is 4397 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f814404ca10 con 0x7f8144138d40 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772746504 0 0) 0x7f814409d5d0 con 0x7f8144132b40 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 10 monclient(hunting): my global_id is 4449 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.655+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8144133620 con 0x7f8144132b40 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1120833143 0 0) 0x7f8130003570 con 0x7f814409faa0 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 >> v1:172.21.15.122:6791/0 conn(0x7f8144138d40 legacy=0x7f814413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 >> v1:172.21.15.122:6790/0 conn(0x7f8144132b40 legacy=0x7f8144132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 monclient: found mon.noname-a 2024-09-15T03:45:05.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440a0cc0 con 0x7f814409faa0 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81340017b0 con 0x7f814409faa0 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: dump: 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.666 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 5 monclient: authenticate success, global_id 4356 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8134001a20 con 0x7f814409faa0 2024-09-15T03:45:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 1 -- 172.21.15.26:0/1932260570 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8134001c10 con 0x7f814409faa0 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f81437fe700 10 monclient: dump: 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.668 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 4 set_mon_vals no callback set 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 10 monclient: shutdown 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 -- 172.21.15.26:0/1932260570 >> v1:172.21.15.26:6789/0 conn(0x7f814409faa0 legacy=0x7f814409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 -- 172.21.15.26:0/1932260570 shutdown_connections 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 -- 172.21.15.26:0/1932260570 >> 172.21.15.26:0/1932260570 conn(0x7f814406e370 msgr2=0x7f81440a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 -- 172.21.15.26:0/1932260570 shutdown_connections 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 -- 172.21.15.26:0/1932260570 wait complete. 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 10 monclient: monmap: 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.671 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 Processor -- start 2024-09-15T03:45:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 1 -- start start 2024-09-15T03:45:05.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.659+0000 7f8149e92700 10 monclient: init 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _add_conns ranks=[8,4,7,0,6,1,5,3,2] 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient(hunting): picked mon.noname-e con 0x7f814409faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient(hunting): picked mon.noname-c con 0x7f8144132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient(hunting): picked mon.noname-i con 0x7f8144138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814409cfa0 con 0x7f8144132b40 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814409ccc0 con 0x7f8144138d40 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8148e90700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f814409faa0 0x7f8144135890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:48896/0 (socket says 172.21.15.26:48896) 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8148e90700 1 -- 172.21.15.26:0/1630285102 learned_addr learned my addr 172.21.15.26:0/1630285102 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8144137130 con 0x7f814409faa0 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient(hunting): authenticate will time out at 850.092441s 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993730575 0 0) 0x7f8144133620 con 0x7f814409faa0 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): my global_id is 4391 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8124003570 con 0x7f814409faa0 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978715762 0 0) 0x7f814409cfa0 con 0x7f8144132b40 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): my global_id is 4378 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8144133620 con 0x7f8144132b40 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391064387 0 0) 0x7f814409ccc0 con 0x7f8144138d40 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): my global_id is 4480 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f814409cfa0 con 0x7f8144138d40 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1129489751 0 0) 0x7f8124003570 con 0x7f814409faa0 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f814409ccc0 con 0x7f814409faa0 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f812c001b20 con 0x7f814409faa0 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 424295333 0 0) 0x7f814409ccc0 con 0x7f814409faa0 2024-09-15T03:45:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 >> v1:172.21.15.122:6792/0 conn(0x7f8144138d40 legacy=0x7f814414a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 >> v1:172.21.15.26:6791/0 conn(0x7f8144132b40 legacy=0x7f8144135fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 monclient: found mon.e 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81441375d0 con 0x7f814409faa0 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 5 monclient: authenticate success, global_id 4391 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _renew_subs 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814411c950 con 0x7f814409faa0 2024-09-15T03:45:05.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _renew_subs 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8144137c90 con 0x7f814409faa0 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f812c0020b0 con 0x7f814409faa0 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f812c002dd0 con 0x7f814409faa0 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f81427fc700 4 set_mon_vals no callback set 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 10 monclient: dump: 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f812c0025e0 con 0x7f814409faa0 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81280050b0 con 0x7f814409faa0 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.663+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f812c043bd0 con 0x7f814409faa0 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.667+0000 7f8143fff700 10 monclient: get_auth_request con 0x7f81240377c0 auth_method 0 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.671+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f812c014b60 con 0x7f814409faa0 2024-09-15T03:45:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.671+0000 7f8140ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.671+0000 7f8140ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:05.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8149e92700 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T03:45:05.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8149e92700 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T03:45:05.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8149e92700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:05.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81280058e0 con 0x7f814409faa0 2024-09-15T03:45:05.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8140ff9700 1 -- 172.21.15.26:0/1630285102 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (unknown 3870207488 0 2170607528) 0x7f812c014560 con 0x7f814409faa0 2024-09-15T03:45:05.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8140ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-09-15T03:45:05.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.843+0000 7f8140ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:05.847 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-09-15T03:45:05.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 >> v1:172.21.15.122:6812/20895 conn(0x7f81240377c0 legacy=0x7f8124039c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 10 monclient: shutdown 2024-09-15T03:45:05.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 >> v1:172.21.15.26:6793/0 conn(0x7f814409faa0 legacy=0x7f8144135890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 shutdown_connections 2024-09-15T03:45:05.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 >> 172.21.15.26:0/1630285102 conn(0x7f814406e370 msgr2=0x7f814409e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 shutdown_connections 2024-09-15T03:45:05.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.847+0000 7f8149e92700 1 -- 172.21.15.26:0/1630285102 wait complete. 2024-09-15T03:45:05.861 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-15T03:45:05.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.959+0000 7ffa58212700 10 monclient: get_monmap_and_config 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.959+0000 7ffa58212700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.959+0000 7ffa58212700 10 monclient: monmap: 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:05.963670+0000 2024-09-15T03:45:05.965 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:05.963670+0000 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.966 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.967 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 Processor -- start 2024-09-15T03:45:05.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- start start 2024-09-15T03:45:05.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient: init 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient: _add_conns ranks=[1,4,2,8,7,0,5,3,6] 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient(hunting): picked mon.noname-f con 0x7ffa50099310 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient(hunting): picked mon.noname-c con 0x7ffa50132fa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient(hunting): picked mon.noname-b con 0x7ffa501333f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5004ca10 con 0x7ffa50099310 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa50097e50 con 0x7ffa501333f0 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa500a1410 con 0x7ffa50132fa0 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient(hunting): authenticate will time out at 850.392990s 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa567af700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ffa501333f0 0x7ffa5013b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54382/0 (socket says 172.21.15.26:54382) 2024-09-15T03:45:05.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa567af700 1 -- 172.21.15.26:0/3190887884 learned_addr learned my addr 172.21.15.26:0/3190887884 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa557ad700 1 --1- 172.21.15.26:0/3190887884 >> v1:172.21.15.26:6791/0 conn(0x7ffa50132fa0 0x7ffa501380a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35936/0 (socket says 172.21.15.26:35936) 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465216105 0 0) 0x7ffa5004ca10 con 0x7ffa50099310 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): my global_id is 4402 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa3c003570 con 0x7ffa50099310 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401110929 0 0) 0x7ffa50097e50 con 0x7ffa501333f0 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): my global_id is 4430 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5004ca10 con 0x7ffa501333f0 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324183218 0 0) 0x7ffa500a1410 con 0x7ffa50132fa0 2024-09-15T03:45:05.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient(hunting): my global_id is 4387 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa50097e50 con 0x7ffa50132fa0 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926611870 0 0) 0x7ffa50097e50 con 0x7ffa50132fa0 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 >> v1:172.21.15.26:6790/0 conn(0x7ffa501333f0 legacy=0x7ffa5013b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 >> v1:172.21.15.122:6789/0 conn(0x7ffa50099310 legacy=0x7ffa500996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 monclient: found mon.noname-c 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500a2710 con 0x7ffa50132fa0 2024-09-15T03:45:05.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa400016d0 con 0x7ffa50132fa0 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: dump: 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.976 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 5 monclient: authenticate success, global_id 4387 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa40001870 con 0x7ffa50132fa0 2024-09-15T03:45:05.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 1 -- 172.21.15.26:0/3190887884 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa40001da0 con 0x7ffa50132fa0 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 4 set_mon_vals no callback set 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa54fac700 10 monclient: dump: 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.978 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 10 monclient: shutdown 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- 172.21.15.26:0/3190887884 >> v1:172.21.15.26:6791/0 conn(0x7ffa50132fa0 legacy=0x7ffa501380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- 172.21.15.26:0/3190887884 shutdown_connections 2024-09-15T03:45:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- 172.21.15.26:0/3190887884 >> 172.21.15.26:0/3190887884 conn(0x7ffa500a6c30 msgr2=0x7ffa5006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- 172.21.15.26:0/3190887884 shutdown_connections 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.963+0000 7ffa58212700 1 -- 172.21.15.26:0/3190887884 wait complete. 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: build_initial_monmap 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: monmap: 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 Processor -- start 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 -- start start 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: init 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: _add_conns ranks=[5,6,7,1,8,2,3,4,0] 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient(hunting): picked mon.noname-h con 0x7ffa50099310 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:05.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient(hunting): picked mon.noname-d con 0x7ffa50132fa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient(hunting): picked mon.noname-i con 0x7ffa501333f0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5006cfb0 con 0x7ffa50099310 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5006ccd0 con 0x7ffa50132fa0 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5006d290 con 0x7ffa501333f0 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient(hunting): authenticate will time out at 850.398008s 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa557ad700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ffa50132fa0 0x7ffa5014b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55138/0 (socket says 172.21.15.26:55138) 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa557ad700 1 -- 172.21.15.26:0/541897253 learned_addr learned my addr 172.21.15.26:0/541897253 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621510970 0 0) 0x7ffa5006ccd0 con 0x7ffa50132fa0 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): my global_id is 4398 2024-09-15T03:45:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa34003570 con 0x7ffa50132fa0 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892200048 0 0) 0x7ffa5006cfb0 con 0x7ffa50099310 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): my global_id is 4406 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5006ccd0 con 0x7ffa50099310 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672178067 0 0) 0x7ffa5006d290 con 0x7ffa501333f0 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): my global_id is 4489 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5006cfb0 con 0x7ffa501333f0 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4061559161 0 0) 0x7ffa34003570 con 0x7ffa50132fa0 2024-09-15T03:45:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa5006d290 con 0x7ffa50132fa0 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa40001650 con 0x7ffa50132fa0 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): dump: 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.984 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.985 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3805494942 0 0) 0x7ffa5006d290 con 0x7ffa50132fa0 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 >> v1:172.21.15.122:6792/0 conn(0x7ffa501333f0 legacy=0x7ffa5014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 >> v1:172.21.15.122:6791/0 conn(0x7ffa50099310 legacy=0x7ffa50146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 monclient: found mon.d 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa501255b0 con 0x7ffa50132fa0 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 5 monclient: authenticate success, global_id 4398 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: _renew_subs 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa500a1d70 con 0x7ffa50132fa0 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: _renew_subs 2024-09-15T03:45:05.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5006aa00 con 0x7ffa50132fa0 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa40004c50 con 0x7ffa50132fa0 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa40005180 con 0x7ffa50132fa0 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa47fff700 4 set_mon_vals no callback set 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.967+0000 7ffa45ffb700 10 monclient: dump: 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:05.990 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:05.991 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:05.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa58212700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa58212700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa58212700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:05.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa300050b0 con 0x7ffa50132fa0 2024-09-15T03:45:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffa40005360 con 0x7ffa50132fa0 2024-09-15T03:45:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7ffa40043fa0 con 0x7ffa50132fa0 2024-09-15T03:45:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.971+0000 7ffa55fae700 10 monclient: get_auth_request con 0x7ffa34037500 auth_method 0 2024-09-15T03:45:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.979+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ffa40014b70 con 0x7ffa50132fa0 2024-09-15T03:45:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.979+0000 7ffa45ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:05.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:05.979+0000 7ffa45ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.143+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa30000b90 con 0x7ffa34037500 2024-09-15T03:45:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.147+0000 7ffa45ffb700 1 -- 172.21.15.26:0/541897253 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41010 (unknown 2965378022 0 2191580444) 0x7ffa30000b90 con 0x7ffa34037500 2024-09-15T03:45:06.152 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 >> v1:172.21.15.122:6812/20895 conn(0x7ffa34037500 legacy=0x7ffa340399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 10 monclient: shutdown 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 >> v1:172.21.15.26:6792/0 conn(0x7ffa50132fa0 legacy=0x7ffa5014b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 shutdown_connections 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 >> 172.21.15.26:0/541897253 conn(0x7ffa500a6c30 msgr2=0x7ffa50067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 shutdown_connections 2024-09-15T03:45:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.151+0000 7ffa58212700 1 -- 172.21.15.26:0/541897253 wait complete. 2024-09-15T03:45:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T03:45:06.166 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-15T03:45:05.461255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786272,"kb_used_data":1864,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444768,"statfs":{"total":579820584960,"available":577991442432,"internally_reserved":0,"allocated":1908736,"data_stored":1309276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000652"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900929+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T03:45:00.900929+0000","last_peered":"2024-09-15T03:45:00.900929+0000","last_clean":"2024-09-15T03:45:00.900929+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T03:45:00.900929+0000","last_undegraded":"2024-09-15T03:45:00.900929+0000","last_fullsized":"2024-09-15T03:45:00.900929+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747766+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T03:45:00.747766+0000","last_peered":"2024-09-15T03:45:00.747766+0000","last_clean":"2024-09-15T03:45:00.747766+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T03:45:00.747766+0000","last_undegraded":"2024-09-15T03:45:00.747766+0000","last_fullsized":"2024-09-15T03:45:00.747766+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747588+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T03:45:00.747588+0000","last_peered":"2024-09-15T03:45:00.747588+0000","last_clean":"2024-09-15T03:45:00.747588+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T03:45:00.747588+0000","last_undegraded":"2024-09-15T03:45:00.747588+0000","last_fullsized":"2024-09-15T03:45:00.747588+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.746955+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T03:45:00.746955+0000","last_peered":"2024-09-15T03:45:00.746955+0000","last_clean":"2024-09-15T03:45:00.746955+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T03:45:00.746955+0000","last_undegraded":"2024-09-15T03:45:00.746955+0000","last_fullsized":"2024-09-15T03:45:00.746955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.745376+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T03:45:00.745376+0000","last_peered":"2024-09-15T03:45:00.745376+0000","last_clean":"2024-09-15T03:45:00.745376+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T03:45:00.745376+0000","last_undegraded":"2024-09-15T03:45:00.745376+0000","last_fullsized":"2024-09-15T03:45:00.745376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747049+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T03:45:00.747049+0000","last_peered":"2024-09-15T03:45:00.747049+0000","last_clean":"2024-09-15T03:45:00.747049+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T03:45:00.747049+0000","last_undegraded":"2024-09-15T03:45:00.747049+0000","last_fullsized":"2024-09-15T03:45:00.747049+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.748385+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T03:45:00.748385+0000","last_peered":"2024-09-15T03:45:00.748385+0000","last_clean":"2024-09-15T03:45:00.748385+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T03:45:00.748385+0000","last_undegraded":"2024-09-15T03:45:00.748385+0000","last_fullsized":"2024-09-15T03:45:00.748385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900829+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T03:45:00.900829+0000","last_peered":"2024-09-15T03:45:00.900829+0000","last_clean":"2024-09-15T03:45:00.900829+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T03:45:00.900829+0000","last_undegraded":"2024-09-15T03:45:00.900829+0000","last_fullsized":"2024-09-15T03:45:00.900829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747827+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T03:45:00.747827+0000","last_peered":"2024-09-15T03:45:00.747827+0000","last_clean":"2024-09-15T03:45:00.747827+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T03:45:00.747827+0000","last_undegraded":"2024-09-15T03:45:00.747827+0000","last_fullsized":"2024-09-15T03:45:00.747827+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:45:06.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T03:45:06.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.267+0000 7f2c8d721700 10 monclient: get_monmap_and_config 2024-09-15T03:45:06.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.267+0000 7f2c8d721700 10 monclient: build_initial_monmap 2024-09-15T03:45:06.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.267+0000 7f2c8d721700 10 monclient: monmap: 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:06.271397+0000 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:06.271397+0000 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:06.273 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:06.274 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:06.274 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:06.274 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 Processor -- start 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- start start 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient: init 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient: _add_conns ranks=[6,4,1,0,2,5,8,3,7] 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient(hunting): picked mon.noname-d con 0x7f2c88068a30 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient(hunting): picked mon.noname-c con 0x7f2c88132b80 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:06.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient(hunting): picked mon.noname-f con 0x7f2c88138ec0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:06.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8804ca10 con 0x7f2c88138ec0 2024-09-15T03:45:06.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880a2920 con 0x7f2c88132b80 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880a2640 con 0x7f2c88068a30 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient(hunting): authenticate will time out at 850.701259s 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c87fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f2c88068a30 0x7f2c8809fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55154/0 (socket says 172.21.15.26:55154) 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c87fff700 1 -- 172.21.15.26:0/2573277271 learned_addr learned my addr 172.21.15.26:0/2573277271 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c877fe700 1 --1- 172.21.15.26:0/2573277271 >> v1:172.21.15.26:6791/0 conn(0x7f2c88132b80 0x7f2c88132f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35938/0 (socket says 172.21.15.26:35938) 2024-09-15T03:45:06.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261061508 0 0) 0x7f2c880a2920 con 0x7f2c88132b80 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): my global_id is 4396 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c70003570 con 0x7f2c88132b80 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138060581 0 0) 0x7f2c880a2640 con 0x7f2c88068a30 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): my global_id is 4407 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c880a2920 con 0x7f2c88068a30 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564761650 0 0) 0x7f2c8804ca10 con 0x7f2c88138ec0 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient(hunting): my global_id is 4411 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c880a2640 con 0x7f2c88138ec0 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355027335 0 0) 0x7f2c70003570 con 0x7f2c88132b80 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 >> v1:172.21.15.26:6792/0 conn(0x7f2c88068a30 legacy=0x7f2c8809fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 >> v1:172.21.15.122:6789/0 conn(0x7f2c88138ec0 legacy=0x7f2c8813b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 monclient: found mon.noname-c 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880a0320 con 0x7f2c88132b80 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c780017b0 con 0x7f2c88132b80 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 5 monclient: authenticate success, global_id 4396 2024-09-15T03:45:06.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: dump: 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.283 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c78001a90 con 0x7f2c88132b80 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 1 -- 172.21.15.26:0/2573277271 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c78004b70 con 0x7f2c88132b80 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c86ffd700 10 monclient: dump: 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 4 set_mon_vals no callback set 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 10 monclient: shutdown 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- 172.21.15.26:0/2573277271 >> v1:172.21.15.26:6791/0 conn(0x7f2c88132b80 legacy=0x7f2c88132f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- 172.21.15.26:0/2573277271 shutdown_connections 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- 172.21.15.26:0/2573277271 >> 172.21.15.26:0/2573277271 conn(0x7f2c8806e430 msgr2=0x7f2c880a6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- 172.21.15.26:0/2573277271 shutdown_connections 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.271+0000 7f2c8d721700 1 -- 172.21.15.26:0/2573277271 wait complete. 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: build_initial_monmap 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: monmap: 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:06.286 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:06.287 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 Processor -- start 2024-09-15T03:45:06.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 -- start start 2024-09-15T03:45:06.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: init 2024-09-15T03:45:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: _add_conns ranks=[1,4,0,2,7,5,8,6,3] 2024-09-15T03:45:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient(hunting): picked mon.noname-f con 0x7f2c88068a30 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient(hunting): picked mon.noname-c con 0x7f2c88132b80 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient(hunting): picked mon.noname-a con 0x7f2c88138ec0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880a1720 con 0x7f2c88138ec0 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c88099910 con 0x7f2c88068a30 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c880a1c10 con 0x7f2c88132b80 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient(hunting): authenticate will time out at 850.706255s 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c877fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f2c88132b80 0x7f2c88131930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:35946/0 (socket says 172.21.15.26:35946) 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c877fe700 1 -- 172.21.15.26:0/4069605237 learned_addr learned my addr 172.21.15.26:0/4069605237 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8cf20700 1 --1- 172.21.15.26:0/4069605237 >> v1:172.21.15.26:6789/0 conn(0x7f2c88138ec0 0x7f2c8814a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53198/0 (socket says 172.21.15.26:53198) 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068806089 0 0) 0x7f2c880a1c10 con 0x7f2c88132b80 2024-09-15T03:45:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): my global_id is 4405 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c74003570 con 0x7f2c88132b80 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909577456 0 0) 0x7f2c880a1720 con 0x7f2c88138ec0 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): my global_id is 4365 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c880a1c10 con 0x7f2c88138ec0 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070807239 0 0) 0x7f2c88099910 con 0x7f2c88068a30 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): my global_id is 4420 2024-09-15T03:45:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c880a1720 con 0x7f2c88068a30 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3734830818 0 0) 0x7f2c880a1c10 con 0x7f2c88138ec0 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c88099910 con 0x7f2c88138ec0 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c80001d40 con 0x7f2c88138ec0 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): dump: 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1809375051 0 0) 0x7f2c74003570 con 0x7f2c88132b80 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c880a1c10 con 0x7f2c88132b80 2024-09-15T03:45:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c78001770 con 0x7f2c88132b80 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient(hunting): dump: 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2977288025 0 0) 0x7f2c88099910 con 0x7f2c88138ec0 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 >> v1:172.21.15.26:6791/0 conn(0x7f2c88132b80 legacy=0x7f2c88131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 >> v1:172.21.15.122:6789/0 conn(0x7f2c88068a30 legacy=0x7f2c88131220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 monclient: found mon.a 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8809a2f0 con 0x7f2c88138ec0 2024-09-15T03:45:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: _finish_auth 0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 5 monclient: authenticate success, global_id 4365 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: _renew_subs 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c88132040 con 0x7f2c88138ec0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: _renew_subs 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c880986d0 con 0x7f2c88138ec0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c80002020 con 0x7f2c88138ec0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c80002550 con 0x7f2c88138ec0 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c867fc700 4 set_mon_vals no callback set 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.275+0000 7f2c6ffff700 10 monclient: dump: 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c8d721700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c8d721700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c8d721700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8804bbf0 con 0x7f2c88138ec0 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2c80002730 con 0x7f2c88138ec0 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f2c80043eb0 con 0x7f2c88138ec0 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.279+0000 7f2c87fff700 10 monclient: get_auth_request con 0x7f2c74037dd0 auth_method 0 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.287+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f2c80014a80 con 0x7f2c88138ec0 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.287+0000 7f2c6ffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.287+0000 7f2c6ffff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:06.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.451+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c88099710 con 0x7f2c74037dd0 2024-09-15T03:45:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c6ffff700 1 -- 172.21.15.26:0/4069605237 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+41010 (unknown 2965378022 0 2191580444) 0x7f2c88099710 con 0x7f2c74037dd0 2024-09-15T03:45:06.457 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T03:45:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 >> v1:172.21.15.122:6812/20895 conn(0x7f2c74037dd0 legacy=0x7f2c7403a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 10 monclient: shutdown 2024-09-15T03:45:06.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 >> v1:172.21.15.26:6789/0 conn(0x7f2c88138ec0 legacy=0x7f2c8814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 shutdown_connections 2024-09-15T03:45:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 >> 172.21.15.26:0/4069605237 conn(0x7f2c8806e430 msgr2=0x7f2c8812c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 shutdown_connections 2024-09-15T03:45:06.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.455+0000 7f2c8d721700 1 -- 172.21.15.26:0/4069605237 wait complete. 2024-09-15T03:45:06.461 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T03:45:06.472 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-15T03:45:05.461255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1786272,"kb_used_data":1864,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":564444768,"statfs":{"total":579820584960,"available":577991442432,"internally_reserved":0,"allocated":1908736,"data_stored":1309276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000652"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900929+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T03:45:00.900929+0000","last_peered":"2024-09-15T03:45:00.900929+0000","last_clean":"2024-09-15T03:45:00.900929+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T03:45:00.900929+0000","last_undegraded":"2024-09-15T03:45:00.900929+0000","last_fullsized":"2024-09-15T03:45:00.900929+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747766+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T03:45:00.747766+0000","last_peered":"2024-09-15T03:45:00.747766+0000","last_clean":"2024-09-15T03:45:00.747766+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T03:45:00.747766+0000","last_undegraded":"2024-09-15T03:45:00.747766+0000","last_fullsized":"2024-09-15T03:45:00.747766+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747588+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T03:45:00.747588+0000","last_peered":"2024-09-15T03:45:00.747588+0000","last_clean":"2024-09-15T03:45:00.747588+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T03:45:00.747588+0000","last_undegraded":"2024-09-15T03:45:00.747588+0000","last_fullsized":"2024-09-15T03:45:00.747588+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.746955+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T03:45:00.746955+0000","last_peered":"2024-09-15T03:45:00.746955+0000","last_clean":"2024-09-15T03:45:00.746955+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T03:45:00.746955+0000","last_undegraded":"2024-09-15T03:45:00.746955+0000","last_fullsized":"2024-09-15T03:45:00.746955+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"15'72","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.745376+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T03:45:00.745376+0000","last_peered":"2024-09-15T03:45:00.745376+0000","last_clean":"2024-09-15T03:45:00.745376+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T03:45:00.745376+0000","last_undegraded":"2024-09-15T03:45:00.745376+0000","last_fullsized":"2024-09-15T03:45:00.745376+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747049+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T03:45:00.747049+0000","last_peered":"2024-09-15T03:45:00.747049+0000","last_clean":"2024-09-15T03:45:00.747049+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T03:45:00.747049+0000","last_undegraded":"2024-09-15T03:45:00.747049+0000","last_fullsized":"2024-09-15T03:45:00.747049+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.748385+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T03:45:00.748385+0000","last_peered":"2024-09-15T03:45:00.748385+0000","last_clean":"2024-09-15T03:45:00.748385+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T03:45:00.748385+0000","last_undegraded":"2024-09-15T03:45:00.748385+0000","last_fullsized":"2024-09-15T03:45:00.748385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.900829+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T03:45:00.900829+0000","last_peered":"2024-09-15T03:45:00.900829+0000","last_clean":"2024-09-15T03:45:00.900829+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T03:45:00.900829+0000","last_undegraded":"2024-09-15T03:45:00.900829+0000","last_fullsized":"2024-09-15T03:45:00.900829+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-15T03:45:00.747827+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T03:45:00.747827+0000","last_peered":"2024-09-15T03:45:00.747827+0000","last_clean":"2024-09-15T03:45:00.747827+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T03:45:00.747827+0000","last_undegraded":"2024-09-15T03:45:00.747827+0000","last_fullsized":"2024-09-15T03:45:00.747827+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.349}]}]},{"osd":4,"up_from":11,"seq":47244640259,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":3,"up_from":11,"seq":47244640261,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6020000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]}]},{"osd":2,"up_from":11,"seq":47244640260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":139264,"data_stored":41897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298140,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073700,"statfs":{"total":96636764160,"available":96331468800,"internally_reserved":0,"allocated":671744,"data_stored":570825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":143360,"data_stored":41916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T03:45:06.473 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-15T03:45:06.474 DEBUG:tasks.mon_thrash:Yielding 2024-09-15T03:45:06.474 INFO:teuthology.run_tasks:Running task rados... 2024-09-15T03:45:06.486 INFO:tasks.rados:Beginning rados... 2024-09-15T03:45:06.486 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2024-09-15T03:45:06.486 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-15T03:45:06.487 INFO:tasks.rados:config is {'clients': ['client.0'], 'objects': 50, 'op_weights': {'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000} 2024-09-15T03:45:06.487 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-15T03:45:06.495 INFO:tasks.rados:joining rados 2024-09-15T03:45:06.495 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-15T03:45:06.495 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726371906, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-15T03:45:06.495 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:45:06.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:45:06.496 INFO:tasks.rados:clients are ['client.0'] 2024-09-15T03:45:06.496 INFO:tasks.rados:starting run 0 out of 1 2024-09-15T03:45:06.496 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-15T03:45:06.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-15T03:45:06.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.595+0000 7fa5c2b43700 10 monclient: get_monmap_and_config 2024-09-15T03:45:06.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.595+0000 7fa5c2b43700 10 monclient: build_initial_monmap 2024-09-15T03:45:06.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.595+0000 7fa5c2b43700 10 monclient: monmap: 2024-09-15T03:45:06.599 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:06.599 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.599 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:06.598135+0000 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:06.598135+0000 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:06.601 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:06.601 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:06.601 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:06.601 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.595+0000 7fa5c2b43700 1 Processor -- start 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.595+0000 7fa5c2b43700 1 -- start start 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.595+0000 7fa5c2b43700 10 monclient: init 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: _add_conns ranks=[5,2,1,7,4,3,0,6,8] 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient(hunting): picked mon.noname-h con 0x7fa5bc070fb0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient(hunting): picked mon.noname-b con 0x7fa5bc0950b0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient(hunting): picked mon.noname-f con 0x7fa5bc095b90 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc04ca10 con 0x7fa5bc095b90 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc0678d0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc09b4a0 con 0x7fa5bc070fb0 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient(hunting): authenticate will time out at 851.028363s 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c1340700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fa5bc0950b0 0x7fa5bc095480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39698/0 (socket says 172.21.15.26:39698) 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c1340700 1 -- 172.21.15.26:0/749205765 learned_addr learned my addr 172.21.15.26:0/749205765 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475971111 0 0) 0x7fa5ac001110 con 0x7fa5bc0950b0 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient(hunting): my global_id is 4439 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a8003570 con 0x7fa5bc0950b0 2024-09-15T03:45:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602435987 0 0) 0x7fa5a8003570 con 0x7fa5bc0950b0 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 >> v1:172.21.15.122:6791/0 conn(0x7fa5bc070fb0 legacy=0x7fa5bc09bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 >> v1:172.21.15.122:6789/0 conn(0x7fa5bc095b90 legacy=0x7fa5bc137f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: _finish_hunting 0 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 monclient: found mon.noname-b 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc09b1c0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: _finish_auth 0 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 5 monclient: authenticate success, global_id 4439 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5ac001940 con 0x7fa5bc0950b0 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: dump: 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5ac001c60 con 0x7fa5bc0950b0 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 4 set_mon_vals no callback set 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 1 -- 172.21.15.26:0/749205765 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5ac004c30 con 0x7fa5bc0950b0 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c0b3f700 10 monclient: dump: 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: shutdown 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- 172.21.15.26:0/749205765 >> v1:172.21.15.26:6790/0 conn(0x7fa5bc0950b0 legacy=0x7fa5bc095480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- 172.21.15.26:0/749205765 shutdown_connections 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- 172.21.15.26:0/749205765 >> 172.21.15.26:0/749205765 conn(0x7fa5bc0a24e0 msgr2=0x7fa5bc0a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- 172.21.15.26:0/749205765 shutdown_connections 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- 172.21.15.26:0/749205765 wait complete. 2024-09-15T03:45:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: build_initial_monmap 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: monmap: 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:06.612 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 Processor -- start 2024-09-15T03:45:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 1 -- start start 2024-09-15T03:45:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: init 2024-09-15T03:45:06.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient: _add_conns ranks=[4,6,7,8,3,0,5,2,1] 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.599+0000 7fa5c2b43700 10 monclient(hunting): picked mon.noname-c con 0x7fa5bc0950b0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient(hunting): picked mon.noname-d con 0x7fa5bc095b90 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient(hunting): picked mon.noname-i con 0x7fa5bc06e3e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc12a5e0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc09c7b0 con 0x7fa5bc095b90 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc06c060 con 0x7fa5bc06e3e0 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient(hunting): authenticate will time out at 851.032156s 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c1b41700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fa5bc0950b0 0x7fa5bc13e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49954/0 (socket says 172.21.15.26:49954) 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c1b41700 1 -- 172.21.15.26:0/3597410365 learned_addr learned my addr 172.21.15.26:0/3597410365 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276764519 0 0) 0x7fa5bc09c7b0 con 0x7fa5bc095b90 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): my global_id is 4416 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a0003570 con 0x7fa5bc095b90 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681768009 0 0) 0x7fa5bc12a5e0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): my global_id is 4414 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc09c7b0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284016161 0 0) 0x7fa5bc06c060 con 0x7fa5bc06e3e0 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): my global_id is 4498 2024-09-15T03:45:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc12a5e0 con 0x7fa5bc06e3e0 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient: get_monmap_and_config 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient: build_initial_monmap 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient: monmap: 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:06.604980+0000 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:06.604980+0000 2024-09-15T03:45:06.616 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:06.617 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3745391539 0 0) 0x7fa5bc09c7b0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5bc06c060 con 0x7fa5bc0950b0 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5b8001a60 con 0x7fa5bc0950b0 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): dump: 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3076219557 0 0) 0x7fa5a0003570 con 0x7fa5bc095b90 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5bc09c7b0 con 0x7fa5bc095b90 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5ac001900 con 0x7fa5bc095b90 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient(hunting): dump: 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 21568658 0 0) 0x7fa5bc12a5e0 con 0x7fa5bc06e3e0 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5a0003570 con 0x7fa5bc06e3e0 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2793042441 0 0) 0x7fa5bc06c060 con 0x7fa5bc0950b0 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 >> v1:172.21.15.122:6792/0 conn(0x7fa5bc06e3e0 legacy=0x7fa5bc146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 >> v1:172.21.15.26:6792/0 conn(0x7fa5bc095b90 legacy=0x7fa5bc13f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 monclient: found mon.c 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc06d690 con 0x7fa5bc0950b0 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 5 monclient: authenticate success, global_id 4414 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient: _renew_subs 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc13f870 con 0x7fa5bc0950b0 2024-09-15T03:45:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient: _renew_subs 2024-09-15T03:45:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc06d390 con 0x7fa5bc0950b0 2024-09-15T03:45:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5b8001d60 con 0x7fa5bc0950b0 2024-09-15T03:45:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5b8002570 con 0x7fa5bc0950b0 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b3fff700 4 set_mon_vals no callback set 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 10 monclient: dump: 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.627 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 1 Processor -- start 2024-09-15T03:45:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 1 -- start start 2024-09-15T03:45:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient: init 2024-09-15T03:45:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c0050b0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa5b8002750 con 0x7fa5bc0950b0 2024-09-15T03:45:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7fa5b8043ce0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient: _add_conns ranks=[4,2,8,1,7,5,3,0,6] 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient(hunting): picked mon.noname-c con 0x7f3b20063ac0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient(hunting): picked mon.noname-b con 0x7f3b2009dcf0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient(hunting): picked mon.noname-e con 0x7f3b2009e0c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2004ca10 con 0x7f3b2009dcf0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2009cc00 con 0x7f3b20063ac0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b20063e90 con 0x7f3b2009e0c0 2024-09-15T03:45:06.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b270d1700 10 monclient(hunting): authenticate will time out at 851.035872s 2024-09-15T03:45:06.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7fa5c1340700 10 monclient: get_auth_request con 0x7fa5a00381c0 auth_method 0 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b260cf700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f3b20063ac0 0x7f3b2009d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49960/0 (socket says 172.21.15.26:49960) 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.603+0000 7f3b258ce700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f3b2009dcf0 0x7f3b201356c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39704/0 (socket says 172.21.15.26:39704) 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b260cf700 1 -- 172.21.15.26:0/2817836006 learned_addr learned my addr 172.21.15.26:0/2817836006 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195446049 0 0) 0x7f3b20063e90 con 0x7f3b2009e0c0 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient(hunting): my global_id is 4400 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b0c003570 con 0x7f3b2009e0c0 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183266493 0 0) 0x7f3b2004ca10 con 0x7f3b2009dcf0 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient(hunting): my global_id is 4448 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b20063e90 con 0x7f3b2009dcf0 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3543323198 0 0) 0x7f3b0c003570 con 0x7f3b2009e0c0 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 >> v1:172.21.15.26:6791/0 conn(0x7f3b20063ac0 legacy=0x7f3b2009d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 >> v1:172.21.15.26:6790/0 conn(0x7f3b2009dcf0 legacy=0x7f3b201356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 monclient: found mon.noname-e 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2009c2f0 con 0x7f3b2009e0c0 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: _finish_auth 0 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:06.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 5 monclient: authenticate success, global_id 4400 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3b100017f0 con 0x7f3b2009e0c0 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: dump: 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.639 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3b10001a10 con 0x7f3b2009e0c0 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 4 set_mon_vals no callback set 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 1 -- 172.21.15.26:0/2817836006 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3b10001c00 con 0x7f3b2009e0c0 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b250cd700 10 monclient: dump: 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.641 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: shutdown 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- 172.21.15.26:0/2817836006 >> v1:172.21.15.26:6793/0 conn(0x7f3b2009e0c0 legacy=0x7f3b20138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- 172.21.15.26:0/2817836006 shutdown_connections 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- 172.21.15.26:0/2817836006 >> 172.21.15.26:0/2817836006 conn(0x7f3b200a4410 msgr2=0x7f3b200a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- 172.21.15.26:0/2817836006 shutdown_connections 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- 172.21.15.26:0/2817836006 wait complete. 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: build_initial_monmap 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: monmap: 2024-09-15T03:45:06.642 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:06.643 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 Processor -- start 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- start start 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: init 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient: _add_conns ranks=[6,3,4,8,5,0,2,1,7] 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient(hunting): picked mon.noname-d con 0x7f3b20063ac0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient(hunting): picked mon.noname-g con 0x7f3b2009dcf0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient(hunting): picked mon.noname-c con 0x7f3b2009e0c0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b20123410 con 0x7f3b2009dcf0 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b20123130 con 0x7f3b2009e0c0 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b201205d0 con 0x7f3b20063ac0 2024-09-15T03:45:06.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b270d1700 10 monclient(hunting): authenticate will time out at 851.039451s 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b260cf700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f3b20063ac0 0x7f3b2011f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41536/0 (socket says 172.21.15.26:41536) 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b260cf700 1 -- 172.21.15.26:0/4020767457 learned_addr learned my addr 172.21.15.26:0/4020767457 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697213521 0 0) 0x7f3b201205d0 con 0x7f3b20063ac0 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b167fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b167fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b167fc700 10 monclient(hunting): my global_id is 4425 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.607+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b0c004aa0 con 0x7f3b20063ac0 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2610417656 0 0) 0x7f3b0c004aa0 con 0x7f3b20063ac0 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b201205d0 con 0x7f3b20063ac0 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778691785 0 0) 0x7f3b20123410 con 0x7f3b2009dcf0 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:06.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): my global_id is 4458 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b0c004aa0 con 0x7f3b2009dcf0 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3b1c001950 con 0x7f3b20063ac0 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): dump: 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.646 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 208524106 0 0) 0x7f3b201205d0 con 0x7f3b20063ac0 2024-09-15T03:45:06.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 >> v1:172.21.15.26:6791/0 conn(0x7f3b2009e0c0 legacy=0x7f3b2014a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 >> v1:172.21.15.122:6790/0 conn(0x7f3b2009dcf0 legacy=0x7f3b2011fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 monclient: found mon.d 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2009ae50 con 0x7f3b20063ac0 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 5 monclient: authenticate success, global_id 4425 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 monclient: _renew_subs 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b20122e50 con 0x7f3b20063ac0 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 monclient: _renew_subs 2024-09-15T03:45:06.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b20123b10 con 0x7f3b20063ac0 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3b1c001d20 con 0x7f3b20063ac0 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3b1c004ea0 con 0x7f3b20063ac0 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b248cc700 4 set_mon_vals no callback set 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 10 monclient: dump: 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:06.649 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b040050b0 con 0x7f3b20063ac0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3b1c005080 con 0x7f3b20063ac0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 3558+0+0 (unknown 835503630 0 0) 0x7f3b1c043d90 con 0x7f3b20063ac0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7f3b258ce700 10 monclient: get_auth_request con 0x7f3b0c037a60 auth_method 0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fa5b8014cd0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7fa5b1ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.611+0000 7fa5b1ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:06.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.619+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f3b1c014d80 con 0x7f3b20063ac0 2024-09-15T03:45:06.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.619+0000 7f3b167fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:06.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.619+0000 7f3b167fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:06.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.815+0000 7f3b270d1700 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-15T03:45:06.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.815+0000 7f3b270d1700 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-15T03:45:06.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.815+0000 7f3b270d1700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:06.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.815+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} v 0) v1 -- 0x7f3b040058e0 con 0x7f3b20063ac0 2024-09-15T03:45:06.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.879+0000 7fa5c2b43700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:45:06.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.879+0000 7fa5c2b43700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:45:06.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.879+0000 7fa5c2b43700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:06.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.879+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa59c0058e0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.883+0000 7fa5b1ffb700 1 -- 172.21.15.26:0/3597410365 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 1829575245) 0x7fa5b80146d0 con 0x7fa5bc0950b0 2024-09-15T03:45:06.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.883+0000 7fa5b1ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:45:06.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.883+0000 7fa5b1ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:06.886 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":20,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:45:06.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 >> v1:172.21.15.122:6812/20895 conn(0x7fa5a00381c0 legacy=0x7fa5a003a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 10 monclient: shutdown 2024-09-15T03:45:06.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:06.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 >> v1:172.21.15.26:6791/0 conn(0x7fa5bc0950b0 legacy=0x7fa5bc13e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:06.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 shutdown_connections 2024-09-15T03:45:06.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 >> 172.21.15.26:0/3597410365 conn(0x7fa5bc0a24e0 msgr2=0x7fa5bc1295c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:06.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 shutdown_connections 2024-09-15T03:45:06.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:06.887+0000 7fa5c2b43700 1 -- 172.21.15.26:0/3597410365 wait complete. 2024-09-15T03:45:06.904 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:45:06.904 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:45:06.904 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:45:06.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:45:07.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient: monmap: 2024-09-15T03:45:07.011 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.011 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.011 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.010034+0000 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.010034+0000 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.013 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 1 Processor -- start 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 1 -- start start 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient: init 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient: _add_conns ranks=[0,7,3,2,6,4,5,1,8] 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient(hunting): picked mon.noname-a con 0x7f91b40990c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient(hunting): picked mon.noname-i con 0x7f91b40a1310 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient(hunting): picked mon.noname-g con 0x7f91b41335a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b404ca10 con 0x7f91b40990c0 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b40a1b20 con 0x7f91b41335a0 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b40a10b0 con 0x7f91b40a1310 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.007+0000 7f91bb249700 10 monclient(hunting): authenticate will time out at 851.439970s 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b8fe5700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91b40990c0 0x7f91b4132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41360/0 (socket says 172.21.15.26:41360) 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b8fe5700 1 -- 172.21.15.26:0/1656465077 learned_addr learned my addr 172.21.15.26:0/1656465077 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596593520 0 0) 0x7f91b404ca10 con 0x7f91b40990c0 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): my global_id is 4374 2024-09-15T03:45:07.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a0003570 con 0x7f91b40990c0 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951436705 0 0) 0x7f91b40a1b20 con 0x7f91b41335a0 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): my global_id is 4467 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b404ca10 con 0x7f91b41335a0 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974726331 0 0) 0x7f91b40a10b0 con 0x7f91b40a1310 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient(hunting): my global_id is 4507 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b40a1b20 con 0x7f91b40a1310 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670078157 0 0) 0x7f91a0003570 con 0x7f91b40990c0 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 >> v1:172.21.15.122:6792/0 conn(0x7f91b40a1310 legacy=0x7f91b4099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 >> v1:172.21.15.122:6790/0 conn(0x7f91b41335a0 legacy=0x7f91b413afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 monclient: found mon.noname-a 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4099ba0 con 0x7f91b40990c0 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 5 monclient: authenticate success, global_id 4374 2024-09-15T03:45:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f919c0017b0 con 0x7f91b40990c0 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: dump: 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f919c001a90 con 0x7f91b40990c0 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 4 set_mon_vals no callback set 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 1 -- 172.21.15.26:0/1656465077 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f919c001c80 con 0x7f91b40990c0 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91b37fe700 10 monclient: dump: 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 10 monclient: shutdown 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 1 -- 172.21.15.26:0/1656465077 >> v1:172.21.15.26:6789/0 conn(0x7f91b40990c0 legacy=0x7f91b4132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 1 -- 172.21.15.26:0/1656465077 shutdown_connections 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 1 -- 172.21.15.26:0/1656465077 >> 172.21.15.26:0/1656465077 conn(0x7f91b406e370 msgr2=0x7f91b40a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 1 -- 172.21.15.26:0/1656465077 shutdown_connections 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 1 -- 172.21.15.26:0/1656465077 wait complete. 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 10 monclient: monmap: 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.025 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.011+0000 7f91bb249700 1 Processor -- start 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- start start 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: init 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: _add_conns ranks=[1,6,7,2,3,8,0,4,5] 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient(hunting): picked mon.noname-f con 0x7f91b40990c0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient(hunting): picked mon.noname-d con 0x7f91b40a1310 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient(hunting): picked mon.noname-i con 0x7f91b41335a0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b40a1ce0 con 0x7f91b40990c0 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b409a010 con 0x7f91b40a1310 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b411ff40 con 0x7f91b41335a0 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient(hunting): authenticate will time out at 851.444976s 2024-09-15T03:45:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b3fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f91b40a1310 0x7f91b4147030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41550/0 (socket says 172.21.15.26:41550) 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b3fff700 1 -- 172.21.15.26:0/3602991353 learned_addr learned my addr 172.21.15.26:0/3602991353 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198619066 0 0) 0x7f91b409a010 con 0x7f91b40a1310 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): my global_id is 4434 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9194003570 con 0x7f91b40a1310 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577901373 0 0) 0x7f91b411ff40 con 0x7f91b41335a0 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): my global_id is 4516 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b409a010 con 0x7f91b41335a0 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166021303 0 0) 0x7f91b40a1ce0 con 0x7f91b40990c0 2024-09-15T03:45:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): my global_id is 4438 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b411ff40 con 0x7f91b40990c0 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 308269344 0 0) 0x7f9194003570 con 0x7f91b40a1310 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91b40a1ce0 con 0x7f91b40a1310 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91a4001b20 con 0x7f91b40a1310 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 387249257 0 0) 0x7f91b409a010 con 0x7f91b41335a0 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9194003570 con 0x7f91b41335a0 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91ac001cc0 con 0x7f91b41335a0 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 166135637 0 0) 0x7f91b411ff40 con 0x7f91b40990c0 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91b409a010 con 0x7f91b40990c0 2024-09-15T03:45:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f919c001770 con 0x7f91b40990c0 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient(hunting): dump: 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 747979192 0 0) 0x7f91b40a1ce0 con 0x7f91b40a1310 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.122:6792/0 conn(0x7f91b41335a0 legacy=0x7f91b414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.122:6789/0 conn(0x7f91b40990c0 legacy=0x7f91b4142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 monclient: found mon.d 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4120640 con 0x7f91b40a1310 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 5 monclient: authenticate success, global_id 4434 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: _renew_subs 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:07.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b4120b10 con 0x7f91b40a1310 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: _renew_subs 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b411f410 con 0x7f91b40a1310 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f91a40020b0 con 0x7f91b40a1310 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91a40025e0 con 0x7f91b40a1310 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b2ffd700 4 set_mon_vals no callback set 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:07.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91b0ff9700 10 monclient: dump: 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.015+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9198006590 con 0x7f9198002840 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882431852 0 0) 0x7f9198006590 con 0x7f9198002840 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient(hunting): my global_id is 4383 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b40a1ce0 con 0x7f9198002840 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1490068507 0 0) 0x7f91b40a1ce0 con 0x7f9198002840 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9198006590 con 0x7f9198002840 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f919c0010e0 con 0x7f9198002840 2024-09-15T03:45:07.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient: dump: 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.043 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f91a40027c0 con 0x7f91b40a1310 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3530+0+0 (unknown 378337516 0 0) 0x7f91a4043dc0 con 0x7f91b40a1310 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1913604400 0 0) 0x7f9198006590 con 0x7f9198002840 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f919800a130 con 0x7f9198002840 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b97e6700 10 monclient: get_auth_request con 0x7f91940381f0 auth_method 0 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f919c0070b0 con 0x7f9198002840 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.26:6789/0 conn(0x7f9198002840 legacy=0x7f9198004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.019+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.26:6789/0 conn(0x7f9198002840 legacy=0x7f9198004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91bb249700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9198007550 con 0x7f9198006900 2024-09-15T03:45:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547423306 0 0) 0x7f9198007550 con 0x7f9198006900 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 10 monclient(hunting): my global_id is 4392 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9198006590 con 0x7f9198006900 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4121091703 0 0) 0x7f9198006590 con 0x7f9198006900 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9198007550 con 0x7f9198006900 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f919c0057f0 con 0x7f9198006900 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.027+0000 7f91b0ff9700 10 monclient: dump: 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 535019337 0 0) 0x7f9198007550 con 0x7f9198006900 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f91980087a0 con 0x7f9198006900 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7401 (unknown 0 0 136412485) 0x7f919c006d00 con 0x7f9198006900 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.26:6789/0 conn(0x7f9198006900 legacy=0x7f9198006d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b0ff9700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.26:6789/0 conn(0x7f9198006900 legacy=0x7f9198006d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.049 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:07.050 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 19, 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.051 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.052 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:07.053 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.054 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.057 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.058 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.061 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.062 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.064 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.065 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.066 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.067 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.068 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.069 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:07.069 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.122:6812/20895 conn(0x7f91940381f0 legacy=0x7f919403a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 10 monclient: shutdown 2024-09-15T03:45:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 >> v1:172.21.15.26:6792/0 conn(0x7f91b40a1310 legacy=0x7f91b4147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b97e6700 1 -- 172.21.15.26:0/3602991353 reap_dead start 2024-09-15T03:45:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91b97e6700 1 -- 172.21.15.26:0/3602991353 reap_dead start 2024-09-15T03:45:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 shutdown_connections 2024-09-15T03:45:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 >> 172.21.15.26:0/3602991353 conn(0x7f91b406e370 msgr2=0x7f91b40a3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 shutdown_connections 2024-09-15T03:45:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.031+0000 7f91bb249700 1 -- 172.21.15.26:0/3602991353 wait complete. 2024-09-15T03:45:07.071 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:07.071 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:45:07.071 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:45:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient: monmap: 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.186041+0000 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.186041+0000 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 1 Processor -- start 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 1 -- start start 2024-09-15T03:45:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient: init 2024-09-15T03:45:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient: _add_conns ranks=[5,3,1,0,8,6,7,4,2] 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient(hunting): picked mon.noname-h con 0x7f9234064390 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient(hunting): picked mon.noname-g con 0x7f923406c580 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient(hunting): picked mon.noname-f con 0x7f9234138cb0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923404ca10 con 0x7f9234138cb0 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923406bc80 con 0x7f923406c580 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923406d2b0 con 0x7f9234064390 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923b20d700 10 monclient(hunting): authenticate will time out at 851.615421s 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f9239a0a700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f923406c580 0x7f923406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:37388/0 (socket says 172.21.15.26:37388) 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f9239a0a700 1 -- 172.21.15.26:0/2693844815 learned_addr learned my addr 172.21.15.26:0/2693844815 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.183+0000 7f923a20b700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f9234064390 0x7f923406a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:60868/0 (socket says 172.21.15.26:60868) 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208047182 0 0) 0x7f923406bc80 con 0x7f923406c580 2024-09-15T03:45:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): my global_id is 4476 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f921c003570 con 0x7f923406c580 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137958995 0 0) 0x7f923406d2b0 con 0x7f9234064390 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): my global_id is 4424 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923406bc80 con 0x7f9234064390 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361431407 0 0) 0x7f923404ca10 con 0x7f9234138cb0 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient(hunting): my global_id is 4447 2024-09-15T03:45:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923406d2b0 con 0x7f9234138cb0 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 265343526 0 0) 0x7f921c003570 con 0x7f923406c580 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 >> v1:172.21.15.122:6791/0 conn(0x7f9234064390 legacy=0x7f923406a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 >> v1:172.21.15.122:6789/0 conn(0x7f9234138cb0 legacy=0x7f923413b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 monclient: found mon.noname-g 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923406b630 con 0x7f923406c580 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 5 monclient: authenticate success, global_id 4476 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9224001b30 con 0x7f923406c580 2024-09-15T03:45:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: dump: 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9224001da0 con 0x7f923406c580 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 4 set_mon_vals no callback set 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 1 -- 172.21.15.26:0/2693844815 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9224004fb0 con 0x7f923406c580 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f9239209700 10 monclient: dump: 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.199 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 10 monclient: shutdown 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 -- 172.21.15.26:0/2693844815 >> v1:172.21.15.122:6790/0 conn(0x7f923406c580 legacy=0x7f923406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 -- 172.21.15.26:0/2693844815 shutdown_connections 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 -- 172.21.15.26:0/2693844815 >> 172.21.15.26:0/2693844815 conn(0x7f9234063340 msgr2=0x7f9234063740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 -- 172.21.15.26:0/2693844815 shutdown_connections 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 -- 172.21.15.26:0/2693844815 wait complete. 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 10 monclient: monmap: 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.201 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 Processor -- start 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 1 -- start start 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.187+0000 7f923b20d700 10 monclient: init 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient: _add_conns ranks=[2,5,0,8,3,7,4,1,6] 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient(hunting): picked mon.noname-b con 0x7f9234064390 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient(hunting): picked mon.noname-h con 0x7f923406c580 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient(hunting): picked mon.noname-a con 0x7f9234138cb0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923412cf50 con 0x7f9234138cb0 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923406cad0 con 0x7f9234064390 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923409d630 con 0x7f923406c580 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient(hunting): authenticate will time out at 851.620616s 2024-09-15T03:45:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923aa0c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9234138cb0 0x7f9234131e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41386/0 (socket says 172.21.15.26:41386) 2024-09-15T03:45:07.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923a20b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f9234064390 0x7f9234131020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39710/0 (socket says 172.21.15.26:39710) 2024-09-15T03:45:07.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923aa0c700 1 -- 172.21.15.26:0/1820771390 learned_addr learned my addr 172.21.15.26:0/1820771390 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586528183 0 0) 0x7f923412cf50 con 0x7f9234138cb0 2024-09-15T03:45:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): my global_id is 4401 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9220003570 con 0x7f9234138cb0 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167028870 0 0) 0x7f923406cad0 con 0x7f9234064390 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): my global_id is 4457 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923412cf50 con 0x7f9234064390 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317647088 0 0) 0x7f923409d630 con 0x7f923406c580 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): my global_id is 4433 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923406cad0 con 0x7f923406c580 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 828618956 0 0) 0x7f923412cf50 con 0x7f9234064390 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f923409d630 con 0x7f9234064390 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9228001a60 con 0x7f9234064390 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): dump: 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1029754108 0 0) 0x7f923406cad0 con 0x7f923406c580 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f923412cf50 con 0x7f923406c580 2024-09-15T03:45:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1567327807 0 0) 0x7f9220003570 con 0x7f9234138cb0 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f923406cad0 con 0x7f9234138cb0 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9224001af0 con 0x7f923406c580 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): dump: 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f922c002990 con 0x7f9234138cb0 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient(hunting): dump: 2024-09-15T03:45:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2435093280 0 0) 0x7f923409d630 con 0x7f9234064390 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.122:6791/0 conn(0x7f923406c580 legacy=0x7f9234131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.26:6789/0 conn(0x7f9234138cb0 legacy=0x7f9234131e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 monclient: found mon.b 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923409d2e0 con 0x7f9234064390 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 5 monclient: authenticate success, global_id 4457 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient: _renew_subs 2024-09-15T03:45:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923409cd20 con 0x7f9234064390 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient: _renew_subs 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f923409f9d0 con 0x7f9234064390 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9228004ae0 con 0x7f9234064390 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9228005010 con 0x7f9234064390 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f9238a08700 4 set_mon_vals no callback set 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.191+0000 7f92327fc700 10 monclient: dump: 2024-09-15T03:45:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9228005170 con 0x7f9234064390 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f923b20d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9214006590 con 0x7f9214002840 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3530+0+0 (unknown 378337516 0 0) 0x7f9228043c70 con 0x7f9234064390 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980563272 0 0) 0x7f9214006590 con 0x7f9214002840 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient(hunting): my global_id is 4466 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f923409d630 con 0x7f9214002840 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f923aa0c700 10 monclient: get_auth_request con 0x7f92200381f0 auth_method 0 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1531145495 0 0) 0x7f923409d630 con 0x7f9214002840 2024-09-15T03:45:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9214006590 con 0x7f9214002840 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92240015b0 con 0x7f9214002840 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient: dump: 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 641366846 0 0) 0x7f9214006590 con 0x7f9214002840 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f921400a130 con 0x7f9214002840 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f92240073b0 con 0x7f9214002840 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.26:6790/0 conn(0x7f9214002840 legacy=0x7f9214004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.195+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.26:6790/0 conn(0x7f9214002840 legacy=0x7f9214004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f923b20d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9214009640 con 0x7f92140090b0 2024-09-15T03:45:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812512776 0 0) 0x7f9214009640 con 0x7f92140090b0 2024-09-15T03:45:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient(hunting): my global_id is 4475 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9214006590 con 0x7f92140090b0 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3078443015 0 0) 0x7f9214006590 con 0x7f92140090b0 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9214009640 con 0x7f92140090b0 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9224005b30 con 0x7f92140090b0 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient: dump: 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 994971097 0 0) 0x7f9214009640 con 0x7f92140090b0 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f92140098f0 con 0x7f92140090b0 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3080743114) 0x7f9224007000 con 0x7f92140090b0 2024-09-15T03:45:07.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.26:6790/0 conn(0x7f92140090b0 legacy=0x7f9214008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.207+0000 7f92327fc700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.26:6790/0 conn(0x7f92140090b0 legacy=0x7f9214008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:07.224 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 20, 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.225 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:07.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:07.227 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.228 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:07.229 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.230 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.231 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.232 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.233 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.236 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.237 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.240 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.241 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:07.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:07.243 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:07.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.122:6812/20895 conn(0x7f92200381f0 legacy=0x7f922003a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 10 monclient: shutdown 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 >> v1:172.21.15.26:6790/0 conn(0x7f9234064390 legacy=0x7f9234131020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923aa0c700 1 -- 172.21.15.26:0/1820771390 reap_dead start 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923aa0c700 1 -- 172.21.15.26:0/1820771390 reap_dead start 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 shutdown_connections 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 >> 172.21.15.26:0/1820771390 conn(0x7f9234063340 msgr2=0x7f923406f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 shutdown_connections 2024-09-15T03:45:07.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.211+0000 7f923b20d700 1 -- 172.21.15.26:0/1820771390 wait complete. 2024-09-15T03:45:07.245 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:07.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:45:07.246 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:45:07.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient: monmap: 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.361738+0000 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.361738+0000 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.364 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 1 Processor -- start 2024-09-15T03:45:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 1 -- start start 2024-09-15T03:45:07.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient: init 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient: _add_conns ranks=[5,3,2,1,8,7,0,6,4] 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient(hunting): picked mon.noname-h con 0x7fb1640a4840 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient(hunting): picked mon.noname-g con 0x7fb16409aa90 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.359+0000 7fb16bd9c700 10 monclient(hunting): picked mon.noname-b con 0x7fb1641392c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16404ca10 con 0x7fb1641392c0 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb164068c10 con 0x7fb16409aa90 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1640a4c10 con 0x7fb1640a4840 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 10 monclient(hunting): authenticate will time out at 851.792260s 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16a339700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb1641392c0 0x7fb16413b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39744/0 (socket says 172.21.15.26:39744) 2024-09-15T03:45:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16a339700 1 -- 172.21.15.26:0/802616295 learned_addr learned my addr 172.21.15.26:0/802616295 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139990217 0 0) 0x7fb16404ca10 con 0x7fb1641392c0 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): my global_id is 4484 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb150003570 con 0x7fb1641392c0 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235170239 0 0) 0x7fb164068c10 con 0x7fb16409aa90 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): my global_id is 4485 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb16404ca10 con 0x7fb16409aa90 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472070290 0 0) 0x7fb1640a4c10 con 0x7fb1640a4840 2024-09-15T03:45:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient(hunting): my global_id is 4442 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb164068c10 con 0x7fb1640a4840 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839636025 0 0) 0x7fb150003570 con 0x7fb1641392c0 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 >> v1:172.21.15.122:6791/0 conn(0x7fb1640a4840 legacy=0x7fb16409db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 >> v1:172.21.15.122:6790/0 conn(0x7fb16409aa90 legacy=0x7fb164137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 monclient: found mon.noname-b 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164068930 con 0x7fb1641392c0 2024-09-15T03:45:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb15c001b30 con 0x7fb1641392c0 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 5 monclient: authenticate success, global_id 4484 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: dump: 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb15c001e10 con 0x7fb1641392c0 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 4 set_mon_vals no callback set 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 1 -- 172.21.15.26:0/802616295 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb15c004fb0 con 0x7fb1641392c0 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb168b36700 10 monclient: dump: 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 10 monclient: shutdown 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- 172.21.15.26:0/802616295 >> v1:172.21.15.26:6790/0 conn(0x7fb1641392c0 legacy=0x7fb16413b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- 172.21.15.26:0/802616295 shutdown_connections 2024-09-15T03:45:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- 172.21.15.26:0/802616295 >> 172.21.15.26:0/802616295 conn(0x7fb1640639b0 msgr2=0x7fb1640a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- 172.21.15.26:0/802616295 shutdown_connections 2024-09-15T03:45:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 1 -- 172.21.15.26:0/802616295 wait complete. 2024-09-15T03:45:07.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.363+0000 7fb16bd9c700 10 monclient: monmap: 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 1 Processor -- start 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 1 -- start start 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient: init 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient: _add_conns ranks=[6,1,5,0,7,2,8,4,3] 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient(hunting): picked mon.noname-d con 0x7fb16409aa90 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient(hunting): picked mon.noname-f con 0x7fb1640a4840 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient(hunting): picked mon.noname-h con 0x7fb1641392c0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16406e2f0 con 0x7fb1640a4840 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16406ddc0 con 0x7fb1641392c0 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb164067a10 con 0x7fb16409aa90 2024-09-15T03:45:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb16bd9c700 10 monclient(hunting): authenticate will time out at 851.798195s 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb169b38700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fb16409aa90 0x7fb164138300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41554/0 (socket says 172.21.15.26:41554) 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb169b38700 1 -- 172.21.15.26:0/1821880038 learned_addr learned my addr 172.21.15.26:0/1821880038 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406286107 0 0) 0x7fb164067a10 con 0x7fb16409aa90 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): my global_id is 4443 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb148003570 con 0x7fb16409aa90 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262858002 0 0) 0x7fb16406ddc0 con 0x7fb1641392c0 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): my global_id is 4451 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb164067a10 con 0x7fb1641392c0 2024-09-15T03:45:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597469657 0 0) 0x7fb16406e2f0 con 0x7fb1640a4840 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): my global_id is 4456 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb16406ddc0 con 0x7fb1640a4840 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2653325766 0 0) 0x7fb148003570 con 0x7fb16409aa90 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb16406e2f0 con 0x7fb16409aa90 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb154002bb0 con 0x7fb16409aa90 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): dump: 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.384 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2683866613 0 0) 0x7fb164067a10 con 0x7fb1641392c0 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb148003570 con 0x7fb1641392c0 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb15c004ef0 con 0x7fb1641392c0 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient(hunting): dump: 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2836252363 0 0) 0x7fb16406e2f0 con 0x7fb16409aa90 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.122:6791/0 conn(0x7fb1641392c0 legacy=0x7fb16414a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.122:6789/0 conn(0x7fb1640a4840 legacy=0x7fb164146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.367+0000 7fb159ffb700 1 monclient: found mon.d 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16409a810 con 0x7fb16409aa90 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 5 monclient: authenticate success, global_id 4443 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 10 monclient: _renew_subs 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16406da50 con 0x7fb16409aa90 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 10 monclient: _renew_subs 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb164138a10 con 0x7fb16409aa90 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb154001660 con 0x7fb16409aa90 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb154001b90 con 0x7fb16409aa90 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb15bfff700 4 set_mon_vals no callback set 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: dump: 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb164138cc0 con 0x7fb164141f60 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb154001d70 con 0x7fb16409aa90 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3530+0+0 (unknown 378337516 0 0) 0x7fb154043dd0 con 0x7fb16409aa90 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149142035 0 0) 0x7fb164138cc0 con 0x7fb164141f60 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient(hunting): my global_id is 4423 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb16406e2f0 con 0x7fb164141f60 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb16a339700 10 monclient: get_auth_request con 0x7fb148037cf0 auth_method 0 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 14789431 0 0) 0x7fb16406e2f0 con 0x7fb164141f60 2024-09-15T03:45:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb164138cc0 con 0x7fb164141f60 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb1600020b0 con 0x7fb164141f60 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 10 monclient: dump: 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 826131512 0 0) 0x7fb164138cc0 con 0x7fb164141f60 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.371+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb16404bbf0 con 0x7fb164141f60 2024-09-15T03:45:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.375+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb1600044a0 con 0x7fb164141f60 2024-09-15T03:45:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.375+0000 7fb159ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.375+0000 7fb159ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.375+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.26:6791/0 conn(0x7fb164141f60 legacy=0x7fb1640a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.375+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.26:6791/0 conn(0x7fb164141f60 legacy=0x7fb1640a28f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb16bd9c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:07.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb16409a100 con 0x7fb164099d10 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982566937 0 0) 0x7fb16409a100 con 0x7fb164099d10 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient(hunting): my global_id is 4432 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb164138cc0 con 0x7fb164099d10 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3843495956 0 0) 0x7fb164138cc0 con 0x7fb164099d10 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb16409a100 con 0x7fb164099d10 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb160002b50 con 0x7fb164099d10 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:07.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient: dump: 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3630077579 0 0) 0x7fb16409a100 con 0x7fb164099d10 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb16406f230 con 0x7fb164099d10 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 326710857) 0x7fb160003fd0 con 0x7fb164099d10 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.26:6791/0 conn(0x7fb164099d10 legacy=0x7fb16402ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.383+0000 7fb159ffb700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.26:6791/0 conn(0x7fb164099d10 legacy=0x7fb16402ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:07.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:07.399 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 20, 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.400 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.401 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:07.402 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.403 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.405 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.406 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.407 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:07.408 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.409 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.410 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:07.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.412 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:07.419 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.122:6812/20895 conn(0x7fb148037cf0 legacy=0x7fb14803a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 10 monclient: shutdown 2024-09-15T03:45:07.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 >> v1:172.21.15.26:6792/0 conn(0x7fb16409aa90 legacy=0x7fb164138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16a339700 1 -- 172.21.15.26:0/1821880038 reap_dead start 2024-09-15T03:45:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 shutdown_connections 2024-09-15T03:45:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 >> 172.21.15.26:0/1821880038 conn(0x7fb1640639b0 msgr2=0x7fb164133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 shutdown_connections 2024-09-15T03:45:07.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.387+0000 7fb16bd9c700 1 -- 172.21.15.26:0/1821880038 wait complete. 2024-09-15T03:45:07.421 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:07.421 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:45:07.421 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:45:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.527+0000 7fbd9281d700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.527+0000 7fbd9281d700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.527+0000 7fbd9281d700 10 monclient: monmap: 2024-09-15T03:45:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.531237+0000 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.531237+0000 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.534 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 Processor -- start 2024-09-15T03:45:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- start start 2024-09-15T03:45:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient: init 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient: _add_conns ranks=[1,5,8,2,4,6,7,3,0] 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient(hunting): picked mon.noname-f con 0x7fbd8c0990c0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient(hunting): picked mon.noname-h con 0x7fbd8c0a1310 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient(hunting): picked mon.noname-e con 0x7fbd8c1335a0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c04ca10 con 0x7fbd8c0990c0 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c0a1b20 con 0x7fbd8c0a1310 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c0a10b0 con 0x7fbd8c1335a0 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient(hunting): authenticate will time out at 851.961122s 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd90dba700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fbd8c1335a0 0x7fbd8c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51172/0 (socket says 172.21.15.26:51172) 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd90dba700 1 -- 172.21.15.26:0/1433555168 learned_addr learned my addr 172.21.15.26:0/1433555168 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591676087 0 0) 0x7fbd8c0a10b0 con 0x7fbd8c1335a0 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): my global_id is 4409 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd74003570 con 0x7fbd8c1335a0 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990790678 0 0) 0x7fbd8c04ca10 con 0x7fbd8c0990c0 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): my global_id is 4465 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c0a10b0 con 0x7fbd8c0990c0 2024-09-15T03:45:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451481405 0 0) 0x7fbd8c0a1b20 con 0x7fbd8c0a1310 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient(hunting): my global_id is 4460 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c04ca10 con 0x7fbd8c0a1310 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4068695994 0 0) 0x7fbd74003570 con 0x7fbd8c1335a0 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 >> v1:172.21.15.122:6791/0 conn(0x7fbd8c0a1310 legacy=0x7fbd8c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 >> v1:172.21.15.122:6789/0 conn(0x7fbd8c0990c0 legacy=0x7fbd8c132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 monclient: found mon.noname-e 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c099ba0 con 0x7fbd8c1335a0 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd840017b0 con 0x7fbd8c1335a0 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 5 monclient: authenticate success, global_id 4409 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: dump: 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd84001a90 con 0x7fbd8c1335a0 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 4 set_mon_vals no callback set 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 1 -- 172.21.15.26:0/1433555168 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd84004c30 con 0x7fbd8c1335a0 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd8affd700 10 monclient: dump: 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.552 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 10 monclient: shutdown 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- 172.21.15.26:0/1433555168 >> v1:172.21.15.26:6793/0 conn(0x7fbd8c1335a0 legacy=0x7fbd8c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- 172.21.15.26:0/1433555168 shutdown_connections 2024-09-15T03:45:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- 172.21.15.26:0/1433555168 >> 172.21.15.26:0/1433555168 conn(0x7fbd8c06e370 msgr2=0x7fbd8c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- 172.21.15.26:0/1433555168 shutdown_connections 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.531+0000 7fbd9281d700 1 -- 172.21.15.26:0/1433555168 wait complete. 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: monmap: 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 1 Processor -- start 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 1 -- start start 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: init 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: _add_conns ranks=[1,2,5,3,6,8,4,7,0] 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient(hunting): picked mon.noname-f con 0x7fbd8c0990c0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient(hunting): picked mon.noname-b con 0x7fbd8c0a1310 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient(hunting): picked mon.noname-h con 0x7fbd8c1335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c0a1ce0 con 0x7fbd8c0990c0 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c09a010 con 0x7fbd8c0a1310 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8c122260 con 0x7fbd8c1335a0 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient(hunting): authenticate will time out at 851.966077s 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd8b7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fbd8c0a1310 0x7fbd8c147030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39760/0 (socket says 172.21.15.26:39760) 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd8b7fe700 1 -- 172.21.15.26:0/192810421 learned_addr learned my addr 172.21.15.26:0/192810421 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047034079 0 0) 0x7fbd8c0a1ce0 con 0x7fbd8c0990c0 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): my global_id is 4474 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd78003570 con 0x7fbd8c0990c0 2024-09-15T03:45:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333801412 0 0) 0x7fbd8c09a010 con 0x7fbd8c0a1310 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): my global_id is 4493 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c0a1ce0 con 0x7fbd8c0a1310 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122983034 0 0) 0x7fbd8c122260 con 0x7fbd8c1335a0 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): my global_id is 4469 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c09a010 con 0x7fbd8c1335a0 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 634688308 0 0) 0x7fbd8c0a1ce0 con 0x7fbd8c0a1310 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd8c122260 con 0x7fbd8c0a1310 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3174312072 0 0) 0x7fbd78003570 con 0x7fbd8c0990c0 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd8c0a1ce0 con 0x7fbd8c0990c0 2024-09-15T03:45:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd80001b20 con 0x7fbd8c0a1310 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): dump: 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd7c001de0 con 0x7fbd8c0990c0 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): dump: 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3733922526 0 0) 0x7fbd8c09a010 con 0x7fbd8c1335a0 2024-09-15T03:45:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd78003570 con 0x7fbd8c1335a0 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd84004b70 con 0x7fbd8c1335a0 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient(hunting): dump: 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3437525948 0 0) 0x7fbd8c122260 con 0x7fbd8c0a1310 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.122:6791/0 conn(0x7fbd8c1335a0 legacy=0x7fbd8c14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.122:6789/0 conn(0x7fbd8c0990c0 legacy=0x7fbd8c131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 monclient: found mon.b 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c121ca0 con 0x7fbd8c0a1310 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd73fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 5 monclient: authenticate success, global_id 4493 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: _renew_subs 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.535+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8c142ba0 con 0x7fbd8c0a1310 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd9281d700 10 monclient: _renew_subs 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd9281d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8c120d40 con 0x7fbd8c0a1310 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd80002080 con 0x7fbd8c0a1310 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd8a7fc700 4 set_mon_vals no callback set 2024-09-15T03:45:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd800025b0 con 0x7fbd8c0a1310 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: dump: 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd9281d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbd6c006590 con 0x7fbd6c002840 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbd80002790 con 0x7fbd8c0a1310 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3530+0+0 (unknown 378337516 0 0) 0x7fbd80043dd0 con 0x7fbd8c0a1310 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466752450 0 0) 0x7fbd6c006590 con 0x7fbd6c002840 2024-09-15T03:45:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient(hunting): my global_id is 4452 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c122260 con 0x7fbd6c002840 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd90dba700 10 monclient: get_auth_request con 0x7fbd78038140 auth_method 0 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 472515796 0 0) 0x7fbd8c122260 con 0x7fbd6c002840 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd6c006590 con 0x7fbd6c002840 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd7c0024f0 con 0x7fbd6c002840 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 10 monclient: dump: 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1601013760 0 0) 0x7fbd6c006590 con 0x7fbd6c002840 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.539+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd6c00a130 con 0x7fbd6c002840 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.543+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbd7c002920 con 0x7fbd6c002840 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.543+0000 7fbd73fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.543+0000 7fbd73fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.543+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.26:6792/0 conn(0x7fbd6c002840 legacy=0x7fbd6c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.543+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.26:6792/0 conn(0x7fbd6c002840 legacy=0x7fbd6c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbd6c009640 con 0x7fbd6c0090b0 2024-09-15T03:45:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774295477 0 0) 0x7fbd6c009640 con 0x7fbd6c0090b0 2024-09-15T03:45:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient(hunting): my global_id is 4461 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c006590 con 0x7fbd6c0090b0 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2861040083 0 0) 0x7fbd6c006590 con 0x7fbd6c0090b0 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd6c009640 con 0x7fbd6c0090b0 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd7c0032b0 con 0x7fbd6c0090b0 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient: dump: 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.571 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1333912055 0 0) 0x7fbd6c009640 con 0x7fbd6c0090b0 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbd6c0098f0 con 0x7fbd6c0090b0 2024-09-15T03:45:07.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 394537757) 0x7fbd7c0057d0 con 0x7fbd6c0090b0 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.26:6792/0 conn(0x7fbd6c0090b0 legacy=0x7fbd6c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd73fff700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.26:6792/0 conn(0x7fbd6c0090b0 legacy=0x7fbd6c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:07.573 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 20, 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.574 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:07.575 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:07.576 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:07.577 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.578 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.579 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.580 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.581 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.583 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.585 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.586 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:07.588 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.590 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.591 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:07.592 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:07.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.122:6812/20895 conn(0x7fbd78038140 legacy=0x7fbd7803a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 10 monclient: shutdown 2024-09-15T03:45:07.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 >> v1:172.21.15.26:6790/0 conn(0x7fbd8c0a1310 legacy=0x7fbd8c147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd90dba700 1 -- 172.21.15.26:0/192810421 reap_dead start 2024-09-15T03:45:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 shutdown_connections 2024-09-15T03:45:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 >> 172.21.15.26:0/192810421 conn(0x7fbd8c06e370 msgr2=0x7fbd8c097f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 shutdown_connections 2024-09-15T03:45:07.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.551+0000 7fbd9281d700 1 -- 172.21.15.26:0/192810421 wait complete. 2024-09-15T03:45:07.594 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:07.594 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:45:07.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:45:07.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient: monmap: 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.713138+0000 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.713138+0000 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.715 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.716 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.716 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.716 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.716 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.716 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 1 Processor -- start 2024-09-15T03:45:07.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 1 -- start start 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient: init 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient: _add_conns ranks=[0,1,5,7,6,8,3,2,4] 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient(hunting): picked mon.noname-a con 0x7fdec409bba0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient(hunting): picked mon.noname-f con 0x7fdec409bf70 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient(hunting): picked mon.noname-h con 0x7fdec409c340 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec404ca10 con 0x7fdec409bba0 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec4099340 con 0x7fdec409bf70 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec409c780 con 0x7fdec409c340 2024-09-15T03:45:07.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec359e700 10 monclient(hunting): authenticate will time out at 852.143112s 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec259c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdec409bba0 0x7fdec4130c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41400/0 (socket says 172.21.15.26:41400) 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec259c700 1 -- 172.21.15.26:0/1899445480 learned_addr learned my addr 172.21.15.26:0/1899445480 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588788634 0 0) 0x7fdec404ca10 con 0x7fdec409bba0 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.711+0000 7fdec159a700 10 monclient(hunting): my global_id is 4410 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeac003570 con 0x7fdec409bba0 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517171152 0 0) 0x7fdec4099340 con 0x7fdec409bf70 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient(hunting): my global_id is 4483 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec404ca10 con 0x7fdec409bf70 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183068379 0 0) 0x7fdec409c780 con 0x7fdec409c340 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient(hunting): my global_id is 4478 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec4099340 con 0x7fdec409c340 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483615409 0 0) 0x7fdeac003570 con 0x7fdec409bba0 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 >> v1:172.21.15.122:6791/0 conn(0x7fdec409c340 legacy=0x7fdec4139240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 >> v1:172.21.15.122:6789/0 conn(0x7fdec409bf70 legacy=0x7fdec4135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 monclient: found mon.noname-a 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4098d10 con 0x7fdec409bba0 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdeb80017b0 con 0x7fdec409bba0 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: dump: 2024-09-15T03:45:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 5 monclient: authenticate success, global_id 4410 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdeb8001a90 con 0x7fdec409bba0 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 1 -- 172.21.15.26:0/1899445480 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdeb8004c30 con 0x7fdec409bba0 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 4 set_mon_vals no callback set 2024-09-15T03:45:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec159a700 10 monclient: dump: 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 10 monclient: shutdown 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 -- 172.21.15.26:0/1899445480 >> v1:172.21.15.26:6789/0 conn(0x7fdec409bba0 legacy=0x7fdec4130c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 -- 172.21.15.26:0/1899445480 shutdown_connections 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 -- 172.21.15.26:0/1899445480 >> 172.21.15.26:0/1899445480 conn(0x7fdec40a2e80 msgr2=0x7fdec409eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 -- 172.21.15.26:0/1899445480 shutdown_connections 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 -- 172.21.15.26:0/1899445480 wait complete. 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 10 monclient: monmap: 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 Processor -- start 2024-09-15T03:45:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 1 -- start start 2024-09-15T03:45:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.715+0000 7fdec359e700 10 monclient: init 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient: _add_conns ranks=[0,7,1,4,3,8,6,5,2] 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient(hunting): picked mon.noname-a con 0x7fdec409bf70 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient(hunting): picked mon.noname-i con 0x7fdec409c340 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient(hunting): picked mon.noname-f con 0x7fdec406e740 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec409f910 con 0x7fdec409bf70 2024-09-15T03:45:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec41282b0 con 0x7fdec406e740 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec4140750 con 0x7fdec409c340 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient(hunting): authenticate will time out at 852.149117s 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec259c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdec409bf70 0x7fdec413f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41410/0 (socket says 172.21.15.26:41410) 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec259c700 1 -- 172.21.15.26:0/754295731 learned_addr learned my addr 172.21.15.26:0/754295731 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878538669 0 0) 0x7fdec409f910 con 0x7fdec409bf70 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): my global_id is 4419 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdea4003570 con 0x7fdec409bf70 2024-09-15T03:45:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322754891 0 0) 0x7fdec41282b0 con 0x7fdec406e740 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): my global_id is 4492 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec409f910 con 0x7fdec406e740 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733988972 0 0) 0x7fdec4140750 con 0x7fdec409c340 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): my global_id is 4525 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec41282b0 con 0x7fdec409c340 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1556967446 0 0) 0x7fdea4003570 con 0x7fdec409bf70 2024-09-15T03:45:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdec4140750 con 0x7fdec409bf70 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdeb8004b70 con 0x7fdec409bf70 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): dump: 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2173248956 0 0) 0x7fdec409f910 con 0x7fdec406e740 2024-09-15T03:45:07.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdea4003570 con 0x7fdec406e740 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdebc001d40 con 0x7fdec406e740 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient(hunting): dump: 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2280505136 0 0) 0x7fdec41282b0 con 0x7fdec409c340 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdec409f910 con 0x7fdec409c340 2024-09-15T03:45:07.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3187102327 0 0) 0x7fdec4140750 con 0x7fdec409bf70 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.122:6792/0 conn(0x7fdec409c340 legacy=0x7fdec412da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.122:6789/0 conn(0x7fdec406e740 legacy=0x7fdec4147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 monclient: found mon.a 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4127f40 con 0x7fdec409bf70 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 5 monclient: authenticate success, global_id 4419 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient: _renew_subs 2024-09-15T03:45:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec4127c60 con 0x7fdec409bf70 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient: _renew_subs 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec4140150 con 0x7fdec409bf70 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdeb8004f00 con 0x7fdec409bf70 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdeb80053b0 con 0x7fdec409bf70 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdec0d99700 4 set_mon_vals no callback set 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.719+0000 7fdeb27fc700 10 monclient: dump: 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdec359e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdea8006590 con 0x7fdea8002840 2024-09-15T03:45:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdeb800fd70 con 0x7fdec409bf70 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(17..17 src has 1..17) v4 ==== 3530+0+0 (unknown 378337516 0 0) 0x7fdeb8043e40 con 0x7fdec409bf70 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806001014 0 0) 0x7fdea8006590 con 0x7fdea8002840 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient(hunting): my global_id is 4418 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec4140750 con 0x7fdea8002840 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdec2d9d700 10 monclient: get_auth_request con 0x7fdea4037de0 auth_method 0 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1918661171 0 0) 0x7fdec4140750 con 0x7fdea8002840 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdea8006590 con 0x7fdea8002840 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdeb4002170 con 0x7fdea8002840 2024-09-15T03:45:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient: dump: 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 418123106 0 0) 0x7fdea8006590 con 0x7fdea8002840 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdea800a130 con 0x7fdea8002840 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdeb40025b0 con 0x7fdea8002840 2024-09-15T03:45:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.26:6793/0 conn(0x7fdea8002840 legacy=0x7fdea8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.723+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.26:6793/0 conn(0x7fdea8002840 legacy=0x7fdea8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdec359e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdea8009640 con 0x7fdea80090b0 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917852070 0 0) 0x7fdea8009640 con 0x7fdea80090b0 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 10 monclient(hunting): my global_id is 4427 2024-09-15T03:45:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdea8006590 con 0x7fdea80090b0 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 397376573 0 0) 0x7fdea8006590 con 0x7fdea80090b0 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdea8009640 con 0x7fdea80090b0 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdeb4002d70 con 0x7fdea80090b0 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 10 monclient: dump: 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.754 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.755 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3168990804 0 0) 0x7fdea8009640 con 0x7fdea80090b0 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.731+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdea80098f0 con 0x7fdea80090b0 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 2105224890) 0x7fdeb4004ff0 con 0x7fdea80090b0 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdeb27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdeb27fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.26:6793/0 conn(0x7fdea80090b0 legacy=0x7fdea8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdeb27fc700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.26:6793/0 conn(0x7fdea80090b0 legacy=0x7fdea8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.756 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:07.757 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 20, 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.758 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.759 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:07.760 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:07.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.762 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.765 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.767 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.771 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:07.774 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.774 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:07.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.778 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.779 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.780 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.781 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.781 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.781 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:07.781 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.122:6812/20895 conn(0x7fdea4037de0 legacy=0x7fdea403a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 10 monclient: shutdown 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 >> v1:172.21.15.26:6789/0 conn(0x7fdec409bf70 legacy=0x7fdec413f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec2d9d700 1 -- 172.21.15.26:0/754295731 reap_dead start 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec2d9d700 1 -- 172.21.15.26:0/754295731 reap_dead start 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 shutdown_connections 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 >> 172.21.15.26:0/754295731 conn(0x7fdec40a2e80 msgr2=0x7fdec40a1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 shutdown_connections 2024-09-15T03:45:07.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.735+0000 7fdec359e700 1 -- 172.21.15.26:0/754295731 wait complete. 2024-09-15T03:45:07.784 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:07.784 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:45:07.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:45:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.795+0000 7f3b167fc700 1 -- 172.21.15.26:0/4020767457 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]=0 pool 'unique_pool_0' created v18) v1 ==== 130+0+0 (unknown 425855328 0 0) 0x7f3b1c014780 con 0x7f3b20063ac0 2024-09-15T03:45:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.795+0000 7f3b167fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}] 2024-09-15T03:45:07.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.795+0000 7f3b167fc700 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' created 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 >> v1:172.21.15.122:6812/20895 conn(0x7f3b0c037a60 legacy=0x7f3b0c039f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 10 monclient: shutdown 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 >> v1:172.21.15.26:6792/0 conn(0x7f3b20063ac0 legacy=0x7f3b2011f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 shutdown_connections 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 >> 172.21.15.26:0/4020767457 conn(0x7f3b200a4410 msgr2=0x7f3b200669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 shutdown_connections 2024-09-15T03:45:07.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.799+0000 7f3b270d1700 1 -- 172.21.15.26:0/4020767457 wait complete. 2024-09-15T03:45:07.803 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'unique_pool_0' created 2024-09-15T03:45:07.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-15T03:45:07.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient: monmap: 2024-09-15T03:45:07.891 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.891 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.889834+0000 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.889834+0000 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 1 Processor -- start 2024-09-15T03:45:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 1 -- start start 2024-09-15T03:45:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient: init 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient: _add_conns ranks=[7,5,0,6,2,1,8,4,3] 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient(hunting): picked mon.noname-i con 0x7fc20c070de0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient(hunting): picked mon.noname-h con 0x7fc20c09c580 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient(hunting): picked mon.noname-a con 0x7fc20c09c9b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c04ca10 con 0x7fc20c09c9b0 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c0711b0 con 0x7fc20c09c580 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c09ea10 con 0x7fc20c070de0 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc214022700 10 monclient(hunting): authenticate will time out at 852.319337s 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc2125bf700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc20c09c9b0 0x7fc20c13b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41418/0 (socket says 172.21.15.26:41418) 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc2125bf700 1 -- 172.21.15.26:0/2582105317 learned_addr learned my addr 172.21.15.26:0/2582105317 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856558213 0 0) 0x7fc20c04ca10 con 0x7fc20c09c9b0 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc210dbc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc210dbc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc210dbc700 10 monclient(hunting): my global_id is 4428 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.887+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f8003570 con 0x7fc20c09c9b0 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007168734 0 0) 0x7fc20c09ea10 con 0x7fc20c070de0 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient(hunting): my global_id is 4534 2024-09-15T03:45:07.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c04ca10 con 0x7fc20c070de0 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690051409 0 0) 0x7fc20c0711b0 con 0x7fc20c09c580 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient(hunting): my global_id is 4487 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c09ea10 con 0x7fc20c09c580 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4125922340 0 0) 0x7fc1f8003570 con 0x7fc20c09c9b0 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 >> v1:172.21.15.122:6792/0 conn(0x7fc20c070de0 legacy=0x7fc20c09be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 >> v1:172.21.15.122:6791/0 conn(0x7fc20c09c580 legacy=0x7fc20c137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 monclient: found mon.noname-a 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c09ecf0 con 0x7fc20c09c9b0 2024-09-15T03:45:07.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 5 monclient: authenticate success, global_id 4428 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1f40017b0 con 0x7fc20c09c9b0 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: dump: 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.899 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc1f40019d0 con 0x7fc20c09c9b0 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 4 set_mon_vals no callback set 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 1 -- 172.21.15.26:0/2582105317 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1f40048f0 con 0x7fc20c09c9b0 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc210dbc700 10 monclient: dump: 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.901 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: shutdown 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 -- 172.21.15.26:0/2582105317 >> v1:172.21.15.26:6789/0 conn(0x7fc20c09c9b0 legacy=0x7fc20c13b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 -- 172.21.15.26:0/2582105317 shutdown_connections 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 -- 172.21.15.26:0/2582105317 >> 172.21.15.26:0/2582105317 conn(0x7fc20c063e10 msgr2=0x7fc20c0a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 -- 172.21.15.26:0/2582105317 shutdown_connections 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 -- 172.21.15.26:0/2582105317 wait complete. 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: monmap: 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.903 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.904 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 Processor -- start 2024-09-15T03:45:07.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 1 -- start start 2024-09-15T03:45:07.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: init 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient: _add_conns ranks=[8,5,1,4,3,6,7,0,2] 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.891+0000 7fc214022700 10 monclient(hunting): picked mon.noname-e con 0x7fc20c070de0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient(hunting): picked mon.noname-h con 0x7fc20c09c580 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient(hunting): picked mon.noname-f con 0x7fc20c09c9b0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c137050 con 0x7fc20c09c9b0 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c142a70 con 0x7fc20c09c580 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c135220 con 0x7fc20c070de0 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient(hunting): authenticate will time out at 852.324148s 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc211dbe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fc20c070de0 0x7fc20c142360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51200/0 (socket says 172.21.15.26:51200) 2024-09-15T03:45:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc211dbe700 1 -- 172.21.15.26:0/3038300449 learned_addr learned my addr 172.21.15.26:0/3038300449 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043878124 0 0) 0x7fc20c135220 con 0x7fc20c070de0 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): my global_id is 4436 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ec003570 con 0x7fc20c070de0 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784468213 0 0) 0x7fc20c142a70 con 0x7fc20c09c580 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): my global_id is 4496 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c135220 con 0x7fc20c09c580 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342581706 0 0) 0x7fc20c137050 con 0x7fc20c09c9b0 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): my global_id is 4501 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c142a70 con 0x7fc20c09c9b0 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2262569463 0 0) 0x7fc1ec003570 con 0x7fc20c070de0 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc20c137050 con 0x7fc20c070de0 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1fc001d10 con 0x7fc20c070de0 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): dump: 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.911 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 427359906 0 0) 0x7fc20c137050 con 0x7fc20c070de0 2024-09-15T03:45:07.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6791/0 conn(0x7fc20c09c580 legacy=0x7fc20c146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6789/0 conn(0x7fc20c09c9b0 legacy=0x7fc20c14a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 monclient: found mon.e 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c119210 con 0x7fc20c070de0 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 5 monclient: authenticate success, global_id 4436 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient: _renew_subs 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c118ea0 con 0x7fc20c070de0 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient: _renew_subs 2024-09-15T03:45:07.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c12c660 con 0x7fc20c070de0 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc1fc0017a0 con 0x7fc20c070de0 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc20b7fe700 4 set_mon_vals no callback set 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1fc002520 con 0x7fc20c070de0 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: dump: 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.914 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc1f0006590 con 0x7fc1f0002840 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc1fc00fb20 con 0x7fc20c070de0 2024-09-15T03:45:07.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3854+0+0 (unknown 1213324056 0 0) 0x7fc1fc043ea0 con 0x7fc20c070de0 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc2125bf700 10 monclient: get_auth_request con 0x7fc1ec0376e0 auth_method 0 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992433941 0 0) 0x7fc1f0006590 con 0x7fc1f0002840 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient(hunting): my global_id is 4510 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c137050 con 0x7fc1f0002840 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1083182150 0 0) 0x7fc20c137050 con 0x7fc1f0002840 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1f0006590 con 0x7fc1f0002840 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc204001db0 con 0x7fc1f0002840 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.895+0000 7fc209ffb700 10 monclient: dump: 2024-09-15T03:45:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1941356367 0 0) 0x7fc1f0006590 con 0x7fc1f0002840 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1f000a130 con 0x7fc1f0002840 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc204004160 con 0x7fc1f0002840 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6789/0 conn(0x7fc1f0002840 legacy=0x7fc1f0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.899+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6789/0 conn(0x7fc1f0002840 legacy=0x7fc1f0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc214022700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc1f0007520 con 0x7fc1f00068d0 2024-09-15T03:45:07.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280094209 0 0) 0x7fc1f0007520 con 0x7fc1f00068d0 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 10 monclient(hunting): my global_id is 4519 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f0006590 con 0x7fc1f00068d0 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1933149921 0 0) 0x7fc1f0006590 con 0x7fc1f00068d0 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1f0007520 con 0x7fc1f00068d0 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc204002880 con 0x7fc1f00068d0 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 10 monclient: dump: 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 79666607 0 0) 0x7fc1f0007520 con 0x7fc1f00068d0 2024-09-15T03:45:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.907+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc1f0008760 con 0x7fc1f00068d0 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: get_monmap_and_config 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: monmap: 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:07.912350+0000 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:07.912350+0000 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.923 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7739 (unknown 0 0 594794151) 0x7fc204003dc0 con 0x7fc1f00068d0 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc209ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc209ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6789/0 conn(0x7fc1f00068d0 legacy=0x7fc1f0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc209ffb700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6789/0 conn(0x7fc1f00068d0 legacy=0x7fc1f0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:07.924 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.925 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 20, 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.926 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:07.928 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:07.929 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.932 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.936 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:07.937 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:07.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.939 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.940 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.941 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.941 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:07.941 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:07.942 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:07.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:07.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:07.945 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:07.946 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.122:6812/20895 conn(0x7fc1ec0376e0 legacy=0x7fc1ec039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 10 monclient: shutdown 2024-09-15T03:45:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 >> v1:172.21.15.26:6793/0 conn(0x7fc20c070de0 legacy=0x7fc20c142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc2125bf700 1 -- 172.21.15.26:0/3038300449 reap_dead start 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc2125bf700 1 -- 172.21.15.26:0/3038300449 reap_dead start 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 shutdown_connections 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 >> 172.21.15.26:0/3038300449 conn(0x7fc20c063e10 msgr2=0x7fc20c12ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 1 Processor -- start 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 shutdown_connections 2024-09-15T03:45:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fc214022700 1 -- 172.21.15.26:0/3038300449 wait complete. 2024-09-15T03:45:07.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 1 -- start start 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: init 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: _add_conns ranks=[6,8,1,3,7,2,0,5,4] 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient(hunting): picked mon.noname-d con 0x7fcdf4063340 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient(hunting): picked mon.noname-e con 0x7fcdf4063710 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient(hunting): picked mon.noname-f con 0x7fcdf4138eb0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf4070420 con 0x7fcdf4138eb0 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf409f400 con 0x7fcdf4063340 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf4133720 con 0x7fcdf4063710 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient(hunting): authenticate will time out at 852.342640s 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf27fc700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fcdf4063710 0x7fcdf4063ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51206/0 (socket says 172.21.15.26:51206) 2024-09-15T03:45:07.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf27fc700 1 -- 172.21.15.26:0/2664173435 learned_addr learned my addr 172.21.15.26:0/2664173435 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474383982 0 0) 0x7fcdf4133720 con 0x7fcdf4063710 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): my global_id is 4445 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcddc003570 con 0x7fcdf4063710 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671249494 0 0) 0x7fcdf409f400 con 0x7fcdf4063340 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): my global_id is 4470 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf4133720 con 0x7fcdf4063340 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187075151 0 0) 0x7fcdf4070420 con 0x7fcdf4138eb0 2024-09-15T03:45:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient(hunting): my global_id is 4528 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf409f400 con 0x7fcdf4138eb0 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977970748 0 0) 0x7fcddc003570 con 0x7fcdf4063710 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 >> v1:172.21.15.26:6792/0 conn(0x7fcdf4063340 legacy=0x7fcdf4133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 >> v1:172.21.15.122:6789/0 conn(0x7fcdf4138eb0 legacy=0x7fcdf413b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 monclient: found mon.noname-e 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4097b00 con 0x7fcdf4063710 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcde80017b0 con 0x7fcdf4063710 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: dump: 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 5 monclient: authenticate success, global_id 4445 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcde8001a90 con 0x7fcdf4063710 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 1 -- 172.21.15.26:0/2664173435 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcde8001c80 con 0x7fcdf4063710 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf1ffb700 10 monclient: dump: 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.911+0000 7fcdf3fff700 4 set_mon_vals no callback set 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient: shutdown 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2664173435 >> v1:172.21.15.26:6793/0 conn(0x7fcdf4063710 legacy=0x7fcdf4063ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2664173435 shutdown_connections 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2664173435 >> 172.21.15.26:0/2664173435 conn(0x7fcdf40a39b0 msgr2=0x7fcdf4070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2664173435 shutdown_connections 2024-09-15T03:45:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2664173435 wait complete. 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient: build_initial_monmap 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient: monmap: 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 Processor -- start 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- start start 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient: init 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient: _add_conns ranks=[1,4,5,0,8,3,2,7,6] 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient(hunting): picked mon.noname-f con 0x7fcdf4063340 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient(hunting): picked mon.noname-c con 0x7fcdf4063710 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient(hunting): picked mon.noname-h con 0x7fcdf4138eb0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf409e840 con 0x7fcdf4063340 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf411c4f0 con 0x7fcdf4063710 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf409e470 con 0x7fcdf4138eb0 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf3fff700 10 monclient(hunting): authenticate will time out at 852.347110s 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf27fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fcdf4063710 0x7fcdf4119ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:50004/0 (socket says 172.21.15.26:50004) 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcdf27fc700 1 -- 172.21.15.26:0/2583404537 learned_addr learned my addr 172.21.15.26:0/2583404537 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203820589 0 0) 0x7fcdf411c4f0 con 0x7fcdf4063710 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): my global_id is 4441 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde0003570 con 0x7fcdf4063710 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 730169018 0 0) 0x7fcde0003570 con 0x7fcdf4063710 2024-09-15T03:45:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdf411c4f0 con 0x7fcdf4063710 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcde8001770 con 0x7fcdf4063710 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): dump: 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048330392 0 0) 0x7fcdf409e840 con 0x7fcdf4063340 2024-09-15T03:45:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.915+0000 7fcddaffd700 10 monclient(hunting): my global_id is 4537 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde0003570 con 0x7fcdf4063340 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2840450389 0 0) 0x7fcdf411c4f0 con 0x7fcdf4063710 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6791/0 conn(0x7fcdf4138eb0 legacy=0x7fcdf414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6789/0 conn(0x7fcdf4063340 legacy=0x7fcdf4119590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 monclient: found mon.c 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf411a3b0 con 0x7fcdf4063710 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 5 monclient: authenticate success, global_id 4441 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 monclient: _renew_subs 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf411cab0 con 0x7fcdf4063710 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 monclient: _renew_subs 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf411b600 con 0x7fcdf4063710 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcde8004f80 con 0x7fcdf4063710 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf0ff9700 4 set_mon_vals no callback set 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcde8005430 con 0x7fcdf4063710 2024-09-15T03:45:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 10 monclient: dump: 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcde800fdf0 con 0x7fcdf4063710 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3854+0+0 (unknown 1213324056 0 0) 0x7fcde8044270 con 0x7fcdf4063710 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd40050b0 con 0x7fcdf4063710 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.919+0000 7fcdf2ffd700 10 monclient: get_auth_request con 0x7fcde0037480 auth_method 0 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.923+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fcde80151f0 con 0x7fcdf4063710 2024-09-15T03:45:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.923+0000 7fcddaffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:07.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:07.923+0000 7fcddaffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:07.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:07.970 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:45:07.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:45:08.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient: get_monmap_and_config 2024-09-15T03:45:08.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient: build_initial_monmap 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient: monmap: 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:08.082053+0000 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:08.082053+0000 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 1 Processor -- start 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 1 -- start start 2024-09-15T03:45:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient: init 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient: _add_conns ranks=[2,1,6,0,4,8,3,5,7] 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient(hunting): picked mon.noname-b con 0x7f917c0990c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient(hunting): picked mon.noname-f con 0x7f917c0a1310 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient(hunting): picked mon.noname-d con 0x7f917c1335a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c04ca10 con 0x7f917c0a1310 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0a1b20 con 0x7f917c0990c0 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0a10b0 con 0x7f917c1335a0 2024-09-15T03:45:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9183780700 10 monclient(hunting): authenticate will time out at 852.511392s 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f918151c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f917c0990c0 0x7f917c132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39772/0 (socket says 172.21.15.26:39772) 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f9181d1d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f917c1335a0 0x7f917c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41594/0 (socket says 172.21.15.26:41594) 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.079+0000 7f918151c700 1 -- 172.21.15.26:0/3986255543 learned_addr learned my addr 172.21.15.26:0/3986255543 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276971189 0 0) 0x7f917c0a10b0 con 0x7f917c1335a0 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): my global_id is 4479 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9168003570 con 0x7f917c1335a0 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027679776 0 0) 0x7f917c0a1b20 con 0x7f917c0990c0 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): my global_id is 4502 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c0a10b0 con 0x7f917c0990c0 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504846552 0 0) 0x7f917c04ca10 con 0x7f917c0a1310 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient(hunting): my global_id is 4546 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c0a1b20 con 0x7f917c0a1310 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559255483 0 0) 0x7f917c0a10b0 con 0x7f917c0990c0 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 >> v1:172.21.15.26:6792/0 conn(0x7f917c1335a0 legacy=0x7f917c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 >> v1:172.21.15.122:6789/0 conn(0x7f917c0a1310 legacy=0x7f917c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 monclient: found mon.noname-b 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c099ba0 con 0x7f917c0990c0 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: _finish_auth 0 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9178001790 con 0x7f917c0990c0 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: dump: 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.096 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 5 monclient: authenticate success, global_id 4502 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9178001ab0 con 0x7f917c0990c0 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 1 -- 172.21.15.26:0/3986255543 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9178004c30 con 0x7f917c0990c0 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 4 set_mon_vals no callback set 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f916ffff700 10 monclient: dump: 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.098 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.099 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 10 monclient: shutdown 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 -- 172.21.15.26:0/3986255543 >> v1:172.21.15.26:6790/0 conn(0x7f917c0990c0 legacy=0x7f917c132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 -- 172.21.15.26:0/3986255543 shutdown_connections 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 -- 172.21.15.26:0/3986255543 >> 172.21.15.26:0/3986255543 conn(0x7f917c06e370 msgr2=0x7f917c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 -- 172.21.15.26:0/3986255543 shutdown_connections 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 -- 172.21.15.26:0/3986255543 wait complete. 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 10 monclient: build_initial_monmap 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 10 monclient: monmap: 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 Processor -- start 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 1 -- start start 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.083+0000 7f9183780700 10 monclient: init 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient: _add_conns ranks=[8,2,6,1,0,3,5,4,7] 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient(hunting): picked mon.noname-e con 0x7f917c0990c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient(hunting): picked mon.noname-b con 0x7f917c0a1310 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient(hunting): picked mon.noname-d con 0x7f917c1335a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0a1ce0 con 0x7f917c0a1310 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c09a010 con 0x7f917c1335a0 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c121da0 con 0x7f917c0990c0 2024-09-15T03:45:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient(hunting): authenticate will time out at 852.516295s 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9181d1d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f917c1335a0 0x7f917c14a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41596/0 (socket says 172.21.15.26:41596) 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9181d1d700 1 -- 172.21.15.26:0/3979262003 learned_addr learned my addr 172.21.15.26:0/3979262003 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9180d1b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f917c0a1310 0x7f917c147030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39780/0 (socket says 172.21.15.26:39780) 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f918151c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f917c0990c0 0x7f917c142080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51208/0 (socket says 172.21.15.26:51208) 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198354373 0 0) 0x7f917c0a1ce0 con 0x7f917c0a1310 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): my global_id is 4511 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9160003570 con 0x7f917c0a1310 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651263416 0 0) 0x7f917c09a010 con 0x7f917c1335a0 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): my global_id is 4488 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c0a1ce0 con 0x7f917c1335a0 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550901761 0 0) 0x7f917c121da0 con 0x7f917c0990c0 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): my global_id is 4454 2024-09-15T03:45:08.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c09a010 con 0x7f917c0990c0 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3889578489 0 0) 0x7f917c0a1ce0 con 0x7f917c1335a0 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f917c121da0 con 0x7f917c1335a0 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 316023662 0 0) 0x7f9160003570 con 0x7f917c0a1310 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f917c0a1ce0 con 0x7f917c0a1310 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9174001ca0 con 0x7f917c1335a0 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): dump: 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91700019e0 con 0x7f917c0a1310 2024-09-15T03:45:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): dump: 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1002457185 0 0) 0x7f917c121da0 con 0x7f917c1335a0 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.26:6793/0 conn(0x7f917c0990c0 legacy=0x7f917c142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.26:6790/0 conn(0x7f917c0a1310 legacy=0x7f917c147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: _finish_hunting 0 2024-09-15T03:45:08.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 monclient: found mon.d 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c121ac0 con 0x7f917c1335a0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: _finish_auth 0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 5 monclient: authenticate success, global_id 4488 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient: _renew_subs 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c120bd0 con 0x7f917c1335a0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient: _renew_subs 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c122f10 con 0x7f917c1335a0 2024-09-15T03:45:08.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9174001fe0 con 0x7f917c1335a0 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91740051a0 con 0x7f917c1335a0 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916f7fe700 4 set_mon_vals no callback set 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: dump: 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.110 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f915c006590 con 0x7f915c002840 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f917400fb60 con 0x7f917c1335a0 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3854+0+0 (unknown 1213324056 0 0) 0x7f9174043fd0 con 0x7f917c1335a0 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f9180d1b700 10 monclient: get_auth_request con 0x7f9160037d50 auth_method 0 2024-09-15T03:45:08.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503553903 0 0) 0x7f915c006590 con 0x7f915c002840 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient(hunting): my global_id is 4494 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c121da0 con 0x7f915c002840 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2354009165 0 0) 0x7f917c121da0 con 0x7f915c002840 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f915c006590 con 0x7f915c002840 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91780011d0 con 0x7f915c002840 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.087+0000 7f916d7fa700 10 monclient: dump: 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.112 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3815732373 0 0) 0x7f915c006590 con 0x7f915c002840 2024-09-15T03:45:08.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f915c00a130 con 0x7f915c002840 2024-09-15T03:45:08.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9178005380 con 0x7f915c002840 2024-09-15T03:45:08.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:08.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.122:6790/0 conn(0x7f915c002840 legacy=0x7f915c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.122:6790/0 conn(0x7f915c002840 legacy=0x7f915c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7fcdf3fff700 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-15T03:45:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7fcdf3fff700 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-15T03:45:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7fcdf3fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:45:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.091+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcdd4005360 con 0x7fcdf4063710 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f9183780700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f915c007520 con 0x7f915c0068d0 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136705114 0 0) 0x7f915c007520 con 0x7f915c0068d0 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 10 monclient(hunting): my global_id is 4503 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c006590 con 0x7f915c0068d0 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3729819862 0 0) 0x7f915c006590 con 0x7f915c0068d0 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f915c007520 con 0x7f915c0068d0 2024-09-15T03:45:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9178005b30 con 0x7f915c0068d0 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 10 monclient: dump: 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2180220804 0 0) 0x7f915c007520 con 0x7f915c0068d0 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.099+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f915c008760 con 0x7f915c0068d0 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 2741461104) 0x7f9178008370 con 0x7f915c0068d0 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f916d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f916d7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.122:6790/0 conn(0x7f915c0068d0 legacy=0x7f915c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f916d7fa700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.122:6790/0 conn(0x7f915c0068d0 legacy=0x7f915c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:08.120 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 20, 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:08.121 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.122 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:08.123 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:08.124 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.125 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.126 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:08.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:08.132 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.135 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:08.136 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.137 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.138 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:08.139 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.122:6812/20895 conn(0x7f9160037d50 legacy=0x7f916003a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 10 monclient: shutdown 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 >> v1:172.21.15.26:6792/0 conn(0x7f917c1335a0 legacy=0x7f917c14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9181d1d700 1 -- 172.21.15.26:0/3979262003 reap_dead start 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9181d1d700 1 -- 172.21.15.26:0/3979262003 reap_dead start 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 shutdown_connections 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 >> 172.21.15.26:0/3979262003 conn(0x7f917c06e370 msgr2=0x7f917c0a4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 shutdown_connections 2024-09-15T03:45:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.103+0000 7f9183780700 1 -- 172.21.15.26:0/3979262003 wait complete. 2024-09-15T03:45:08.141 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:08.142 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:45:08.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:45:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: get_monmap_and_config 2024-09-15T03:45:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: build_initial_monmap 2024-09-15T03:45:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: monmap: 2024-09-15T03:45:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:08.252054+0000 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:08.252054+0000 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:08.255 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 Processor -- start 2024-09-15T03:45:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- start start 2024-09-15T03:45:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: init 2024-09-15T03:45:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: _add_conns ranks=[6,1,7,8,0,3,2,5,4] 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient(hunting): picked mon.noname-d con 0x7f550c132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient(hunting): picked mon.noname-f con 0x7f550c09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient(hunting): picked mon.noname-i con 0x7f550c138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c04ca10 con 0x7f550c09faa0 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c09d5d0 con 0x7f550c132b40 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c133620 con 0x7f550c138d40 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient(hunting): authenticate will time out at 852.681862s 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f550c132b40 0x7f550c132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41610/0 (socket says 172.21.15.26:41610) 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550b7fe700 1 -- 172.21.15.26:0/2450677267 learned_addr learned my addr 172.21.15.26:0/2450677267 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092851515 0 0) 0x7f550c09d5d0 con 0x7f550c132b40 2024-09-15T03:45:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): my global_id is 4497 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ec003570 con 0x7f550c132b40 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401546648 0 0) 0x7f550c04ca10 con 0x7f550c09faa0 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): my global_id is 4555 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c09d5d0 con 0x7f550c09faa0 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646158917 0 0) 0x7f550c133620 con 0x7f550c138d40 2024-09-15T03:45:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient(hunting): my global_id is 4543 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c04ca10 con 0x7f550c138d40 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434798113 0 0) 0x7f54ec003570 con 0x7f550c132b40 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 >> v1:172.21.15.122:6792/0 conn(0x7f550c138d40 legacy=0x7f550c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 >> v1:172.21.15.122:6789/0 conn(0x7f550c09faa0 legacy=0x7f550c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 monclient: found mon.noname-d 2024-09-15T03:45:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0a0cc0 con 0x7f550c132b40 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 5 monclient: authenticate success, global_id 4497 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f54f4001730 con 0x7f550c132b40 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: dump: 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.270 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f54f4001950 con 0x7f550c132b40 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 1 -- 172.21.15.26:0/2450677267 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f54f40048f0 con 0x7f550c132b40 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 4 set_mon_vals no callback set 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f550a7fc700 10 monclient: dump: 2024-09-15T03:45:08.271 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 10 monclient: shutdown 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- 172.21.15.26:0/2450677267 >> v1:172.21.15.26:6792/0 conn(0x7f550c132b40 legacy=0x7f550c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- 172.21.15.26:0/2450677267 shutdown_connections 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.251+0000 7f5510945700 1 -- 172.21.15.26:0/2450677267 >> 172.21.15.26:0/2450677267 conn(0x7f550c06e370 msgr2=0x7f550c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- 172.21.15.26:0/2450677267 shutdown_connections 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- 172.21.15.26:0/2450677267 wait complete. 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: build_initial_monmap 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: monmap: 2024-09-15T03:45:08.273 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:08.274 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 Processor -- start 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- start start 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: init 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: _add_conns ranks=[3,0,2,7,6,4,5,1,8] 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient(hunting): picked mon.noname-g con 0x7f550c09faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient(hunting): picked mon.noname-a con 0x7f550c132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient(hunting): picked mon.noname-b con 0x7f550c138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:08.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c137750 con 0x7f550c132b40 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c137240 con 0x7f550c138d40 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c09cfa0 con 0x7f550c09faa0 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient(hunting): authenticate will time out at 852.686151s 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f550affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f550c132b40 0x7f550c1473e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41428/0 (socket says 172.21.15.26:41428) 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f550affd700 1 -- 172.21.15.26:0/845993760 learned_addr learned my addr 172.21.15.26:0/845993760 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41969121 0 0) 0x7f550c137750 con 0x7f550c132b40 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): my global_id is 4437 2024-09-15T03:45:08.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f0003570 con 0x7f550c132b40 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250915 0 0) 0x7f550c09cfa0 con 0x7f550c09faa0 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): my global_id is 4512 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c137750 con 0x7f550c09faa0 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221724934 0 0) 0x7f550c137240 con 0x7f550c138d40 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): my global_id is 4520 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c09cfa0 con 0x7f550c138d40 2024-09-15T03:45:08.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2320794516 0 0) 0x7f550c09cfa0 con 0x7f550c138d40 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f550c137240 con 0x7f550c138d40 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5504001b30 con 0x7f550c138d40 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): dump: 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.278 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 628140099 0 0) 0x7f550c137750 con 0x7f550c09faa0 2024-09-15T03:45:08.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f550c09cfa0 con 0x7f550c09faa0 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f54f40016b0 con 0x7f550c09faa0 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): dump: 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.280 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 783711209 0 0) 0x7f54f0003570 con 0x7f550c132b40 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f550c137750 con 0x7f550c132b40 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f54fc002b50 con 0x7f550c132b40 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient(hunting): dump: 2024-09-15T03:45:08.281 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.282 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1872446700 0 0) 0x7f550c137240 con 0x7f550c138d40 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.122:6790/0 conn(0x7f550c09faa0 legacy=0x7f550c1423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.26:6789/0 conn(0x7f550c132b40 legacy=0x7f550c1473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 monclient: found mon.b 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c09ccc0 con 0x7f550c138d40 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:08.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 5 monclient: authenticate success, global_id 4520 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: _renew_subs 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c11d900 con 0x7f550c138d40 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: _renew_subs 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c11cd30 con 0x7f550c138d40 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5504001e50 con 0x7f550c138d40 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.255+0000 7f55037fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5504002380 con 0x7f550c138d40 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f5509ffb700 4 set_mon_vals no callback set 2024-09-15T03:45:08.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: dump: 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f5510945700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5504002560 con 0x7f550c138d40 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f54e4006590 con 0x7f54e4002840 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3854+0+0 (unknown 1213324056 0 0) 0x7f55040416b0 con 0x7f550c138d40 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f550affd700 10 monclient: get_auth_request con 0x7f54f0038020 auth_method 0 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778308699 0 0) 0x7f54e4006590 con 0x7f54e4002840 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient(hunting): my global_id is 4514 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c137240 con 0x7f54e4002840 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2442678674 0 0) 0x7f550c137240 con 0x7f54e4002840 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54e4006590 con 0x7f54e4002840 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f54f4001210 con 0x7f54e4002840 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: dump: 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.287 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3208278921 0 0) 0x7f54e4006590 con 0x7f54e4002840 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54e400a130 con 0x7f54e4002840 2024-09-15T03:45:08.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f54f4007020 con 0x7f54e4002840 2024-09-15T03:45:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.122:6791/0 conn(0x7f54e4002840 legacy=0x7f54e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.259+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.122:6791/0 conn(0x7f54e4002840 legacy=0x7f54e4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f5510945700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f54e4009620 con 0x7f54e4009090 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646898395 0 0) 0x7f54e4009620 con 0x7f54e4009090 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f55037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f55037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f55037fe700 10 monclient(hunting): my global_id is 4523 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.271+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e4006590 con 0x7f54e4009090 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 542055940 0 0) 0x7f54e4006590 con 0x7f54e4009090 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54e4009620 con 0x7f54e4009090 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f54f40056b0 con 0x7f54e4009090 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 10 monclient: dump: 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.292 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1258774356 0 0) 0x7f54e4009620 con 0x7f54e4009090 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f54e4009890 con 0x7f54e4009090 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 1164591215) 0x7f54f400a010 con 0x7f54e4009090 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.122:6791/0 conn(0x7f54e4009090 legacy=0x7f54e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f55037fe700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.122:6791/0 conn(0x7f54e4009090 legacy=0x7f54e4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:08.293 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:08.294 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 21, 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.295 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.296 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:08.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.298 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:08.301 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:08.302 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:08.303 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:08.304 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:08.305 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:08.307 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.308 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.309 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.310 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.312 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.313 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:08.314 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.122:6812/20895 conn(0x7f54f0038020 legacy=0x7f54f003a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 10 monclient: shutdown 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 >> v1:172.21.15.26:6790/0 conn(0x7f550c138d40 legacy=0x7f550c14ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f550bfff700 1 -- 172.21.15.26:0/845993760 reap_dead start 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f550bfff700 1 -- 172.21.15.26:0/845993760 reap_dead start 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 shutdown_connections 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 >> 172.21.15.26:0/845993760 conn(0x7f550c06e370 msgr2=0x7f550c09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 shutdown_connections 2024-09-15T03:45:08.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.275+0000 7f5510945700 1 -- 172.21.15.26:0/845993760 wait complete. 2024-09-15T03:45:08.316 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:08.317 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:45:08.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:45:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.427+0000 7fb0a359e700 10 monclient: get_monmap_and_config 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.427+0000 7fb0a359e700 10 monclient: build_initial_monmap 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.427+0000 7fb0a359e700 10 monclient: monmap: 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:08.431666+0000 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:08.431666+0000 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:08.434 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 Processor -- start 2024-09-15T03:45:08.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- start start 2024-09-15T03:45:08.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient: init 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient: _add_conns ranks=[8,5,3,0,7,6,1,4,2] 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient(hunting): picked mon.noname-e con 0x7fb0a40a6a70 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient(hunting): picked mon.noname-h con 0x7fb0a4099020 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient(hunting): picked mon.noname-g con 0x7fb0a4138cc0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a404ca10 con 0x7fb0a4138cc0 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a4098c20 con 0x7fb0a4099020 2024-09-15T03:45:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a40a2280 con 0x7fb0a40a6a70 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient(hunting): authenticate will time out at 852.861135s 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a259c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fb0a40a6a70 0x7fb0a4098510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51222/0 (socket says 172.21.15.26:51222) 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a259c700 1 -- 172.21.15.26:0/3227324340 learned_addr learned my addr 172.21.15.26:0/3227324340 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136116778 0 0) 0x7fb0a40a2280 con 0x7fb0a40a6a70 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): my global_id is 4463 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb084003570 con 0x7fb0a40a6a70 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352906239 0 0) 0x7fb0a4098c20 con 0x7fb0a4099020 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): my global_id is 4532 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a40a2280 con 0x7fb0a4099020 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166225694 0 0) 0x7fb0a404ca10 con 0x7fb0a4138cc0 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient(hunting): my global_id is 4521 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a4098c20 con 0x7fb0a4138cc0 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 859620517 0 0) 0x7fb084003570 con 0x7fb0a40a6a70 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 >> v1:172.21.15.122:6791/0 conn(0x7fb0a4099020 legacy=0x7fb0a40993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 >> v1:172.21.15.122:6790/0 conn(0x7fb0a4138cc0 legacy=0x7fb0a413b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: _finish_hunting 0 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 monclient: found mon.noname-e 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a40a2840 con 0x7fb0a40a6a70 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: _finish_auth 0 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 5 monclient: authenticate success, global_id 4463 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb08c0017b0 con 0x7fb0a40a6a70 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:45:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: dump: 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb08c001a90 con 0x7fb0a40a6a70 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 4 set_mon_vals no callback set 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 1 -- 172.21.15.26:0/3227324340 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb08c004c30 con 0x7fb0a40a6a70 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:45:08.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a159a700 10 monclient: dump: 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 10 monclient: shutdown 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- 172.21.15.26:0/3227324340 >> v1:172.21.15.26:6793/0 conn(0x7fb0a40a6a70 legacy=0x7fb0a4098510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- 172.21.15.26:0/3227324340 shutdown_connections 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- 172.21.15.26:0/3227324340 >> 172.21.15.26:0/3227324340 conn(0x7fb0a406e1d0 msgr2=0x7fb0a40a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- 172.21.15.26:0/3227324340 shutdown_connections 2024-09-15T03:45:08.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.431+0000 7fb0a359e700 1 -- 172.21.15.26:0/3227324340 wait complete. 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient: build_initial_monmap 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient: monmap: 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:08.454 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 1 Processor -- start 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 1 -- start start 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient: init 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient: _add_conns ranks=[6,1,7,3,8,2,4,0,5] 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient(hunting): picked mon.noname-d con 0x7fb0a4099020 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient(hunting): picked mon.noname-f con 0x7fb0a40a6a70 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient(hunting): picked mon.noname-i con 0x7fb0a4138cc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a40a1770 con 0x7fb0a40a6a70 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a4099d60 con 0x7fb0a4099020 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a4121860 con 0x7fb0a4138cc0 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a359e700 10 monclient(hunting): authenticate will time out at 852.866279s 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a259c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fb0a4099020 0x7fb0a411f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41612/0 (socket says 172.21.15.26:41612) 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb0a259c700 1 -- 172.21.15.26:0/3144636992 learned_addr learned my addr 172.21.15.26:0/3144636992 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497456369 0 0) 0x7fb0a4099d60 con 0x7fb0a4099020 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): my global_id is 4506 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb088003570 con 0x7fb0a4099020 2024-09-15T03:45:08.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046361506 0 0) 0x7fb0a40a1770 con 0x7fb0a40a6a70 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): my global_id is 4564 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a4099d60 con 0x7fb0a40a6a70 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712859558 0 0) 0x7fb0a4121860 con 0x7fb0a4138cc0 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): my global_id is 4552 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a40a1770 con 0x7fb0a4138cc0 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 87662976 0 0) 0x7fb088003570 con 0x7fb0a4099020 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0a4121860 con 0x7fb0a4099020 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb08c004b70 con 0x7fb0a4099020 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 262189253 0 0) 0x7fb0a4099d60 con 0x7fb0a40a6a70 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb088003570 con 0x7fb0a40a6a70 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb094001de0 con 0x7fb0a40a6a70 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2790604856 0 0) 0x7fb0a40a1770 con 0x7fb0a4138cc0 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0a4099d60 con 0x7fb0a4138cc0 2024-09-15T03:45:08.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb09c0019c0 con 0x7fb0a4138cc0 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.435+0000 7fb09a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient(hunting): dump: 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.461 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2658323951 0 0) 0x7fb088003570 con 0x7fb0a40a6a70 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6792/0 conn(0x7fb0a4138cc0 legacy=0x7fb0a4120010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.26:6792/0 conn(0x7fb0a4099020 legacy=0x7fb0a411f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 monclient: found mon.f 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a41227c0 con 0x7fb0a40a6a70 2024-09-15T03:45:08.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 5 monclient: authenticate success, global_id 4564 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 10 monclient: _renew_subs 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a4120c90 con 0x7fb0a40a6a70 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 10 monclient: _renew_subs 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a414b180 con 0x7fb0a40a6a70 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb094003030 con 0x7fb0a40a6a70 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb0940025f0 con 0x7fb0a40a6a70 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a0d99700 4 set_mon_vals no callback set 2024-09-15T03:45:08.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: dump: 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.464 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb094002750 con 0x7fb0a40a6a70 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb07c006590 con 0x7fb07c002840 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(18..18 src has 1..18) v4 ==== 3854+0+0 (unknown 1213324056 0 0) 0x7fb094041740 con 0x7fb0a40a6a70 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb0a2d9d700 10 monclient: get_auth_request con 0x7fb088037fe0 auth_method 0 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026777938 0 0) 0x7fb07c006590 con 0x7fb07c002840 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient(hunting): my global_id is 4561 2024-09-15T03:45:08.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb088003570 con 0x7fb07c002840 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1436963780 0 0) 0x7fb088003570 con 0x7fb07c002840 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb07c006590 con 0x7fb07c002840 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb08c005420 con 0x7fb07c002840 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 10 monclient: dump: 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.466 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3053308948 0 0) 0x7fb07c006590 con 0x7fb07c002840 2024-09-15T03:45:08.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.439+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb07c00a130 con 0x7fb07c002840 2024-09-15T03:45:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.443+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb08c005730 con 0x7fb07c002840 2024-09-15T03:45:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.443+0000 7fb09a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.443+0000 7fb09a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.443+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6792/0 conn(0x7fb07c002840 legacy=0x7fb07c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.443+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6792/0 conn(0x7fb07c002840 legacy=0x7fb07c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:08.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb0a359e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb07c007520 con 0x7fb07c0068d0 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707965713 0 0) 0x7fb07c007520 con 0x7fb07c0068d0 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb09a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb09a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb09a7fc700 10 monclient(hunting): my global_id is 4570 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.451+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c006590 con 0x7fb07c0068d0 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3155153522 0 0) 0x7fb07c006590 con 0x7fb07c0068d0 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb07c007520 con 0x7fb07c0068d0 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb08c005f10 con 0x7fb07c0068d0 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 10 monclient: dump: 2024-09-15T03:45:08.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:08.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2992895025 0 0) 0x7fb07c007520 con 0x7fb07c0068d0 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb07c008760 con 0x7fb07c0068d0 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 3580067647) 0x7fb08c008150 con 0x7fb07c0068d0 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6792/0 conn(0x7fb07c0068d0 legacy=0x7fb07c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb09a7fc700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6792/0 conn(0x7fb07c0068d0 legacy=0x7fb07c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:08.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 20, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:45:08.473 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 21, 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.474 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.475 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:08.476 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:08.477 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.478 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:08.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.481 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:08.483 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:08.486 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.487 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:08.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:08.490 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:08.492 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6812/20895 conn(0x7fb088037fe0 legacy=0x7fb08803a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 10 monclient: shutdown 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 >> v1:172.21.15.122:6789/0 conn(0x7fb0a40a6a70 legacy=0x7fb0a411f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a2d9d700 1 -- 172.21.15.26:0/3144636992 reap_dead start 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 shutdown_connections 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 >> 172.21.15.26:0/3144636992 conn(0x7fb0a406e1d0 msgr2=0x7fb0a40a4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 shutdown_connections 2024-09-15T03:45:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:08.455+0000 7fb0a359e700 1 -- 172.21.15.26:0/3144636992 wait complete. 2024-09-15T03:45:08.495 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:45:08.495 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['f', 'i', 'e', 'b'] 2024-09-15T03:45:08.495 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a', 'g'] 2024-09-15T03:45:08.495 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T03:45:08.495 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T03:45:08.495 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T03:45:08.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:45:08.513 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T03:45:08.513 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-15T03:45:08.514 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T03:45:08.514 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T03:45:08.514 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:45:08.570 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T03:45:08.571 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-15T03:45:08.571 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T03:45:08.571 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-15T03:45:08.571 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:45:08.589 INFO:tasks.ceph.mon.e:Stopped 2024-09-15T03:45:08.589 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T03:45:08.589 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T03:45:08.589 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T03:45:08.589 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:45:08.604 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T03:45:08.604 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-15T03:45:08.604 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-15T03:45:08.604 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-09-15T03:45:08.604 INFO:tasks.ceph.mon.g:Sent signal 19 2024-09-15T03:45:08.604 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:45:10.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:10.915+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:10.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:10.915+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:10.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:10.915+0000 7fcdd9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:11.915+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:11.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:11.915+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:12.915+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:12.915+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:13.915+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:13.915+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:14.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:14.915+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:14.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:14.915+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:15.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:15.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:16.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:16.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:17.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:17.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:18.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:18.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:18.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:18.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:19.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:19.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:20.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:20.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:20.919+0000 7fcdd9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:21.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:21.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:22.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:22.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:23.605 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-15T03:45:23.606 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-15T03:45:23.606 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-09-15T03:45:23.606 INFO:tasks.ceph.mon.g:Sent signal 18 2024-09-15T03:45:23.606 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-15T03:45:23.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.707+0000 7fa64930b700 10 monclient: get_monmap_and_config 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.707+0000 7fa64930b700 10 monclient: build_initial_monmap 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.707+0000 7fa64930b700 10 monclient: monmap: 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:23.711823+0000 2024-09-15T03:45:23.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:23.711823+0000 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:23.714 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 1 Processor -- start 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 1 -- start start 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient: init 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient: _add_conns ranks=[0,5,2,1,4,6,7,8,3] 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient(hunting): picked mon.noname-a con 0x7fa644099910 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient(hunting): picked mon.noname-h con 0x7fa64412ff80 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient(hunting): picked mon.noname-b con 0x7fa6441303d0 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa643fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa644099910 0x7fa64412f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47418/0 (socket says 172.21.15.26:47418) 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa648b0a700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa6441303d0 legacy=0x7fa6441382d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 learned_addr learned my addr 172.21.15.26:0/1191697780 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64404ca10 con 0x7fa644099910 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644098be0 con 0x7fa6441303d0 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644099460 con 0x7fa64412ff80 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.711+0000 7fa64930b700 10 monclient(hunting): authenticate will time out at 868.143050s 2024-09-15T03:45:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.915+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa6441303d0 legacy=0x7fa6441382d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:23.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:23.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:23.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:24.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:24.319+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa6441303d0 legacy=0x7fa6441382d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:24.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:24.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:24.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:24.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:25.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:25.123+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa6441303d0 legacy=0x7fa6441382d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:25.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:25.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:25.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:25.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient(hunting): tick 2024-09-15T03:45:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:45:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6791/0 conn(0x7fa64412ff80 legacy=0x7fa644134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa6441303d0 legacy=0x7fa6441382d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6789/0 conn(0x7fa644099910 legacy=0x7fa64412f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient: _add_conns ranks=[3,7,1,6,8,4] 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-g con 0x7fa62c003810 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-i con 0x7fa62c007490 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-f con 0x7fa62c00b210 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644099110 con 0x7fa62c00b210 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644098be0 con 0x7fa62c003810 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c011020 con 0x7fa62c007490 2024-09-15T03:45:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa641ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:26.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c00b210 legacy=0x7fa62c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:26.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.711+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c007490 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:26.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.915+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c00b210 legacy=0x7fa62c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:26.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.915+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c007490 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:26.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:26.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:27.319+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c00b210 legacy=0x7fa62c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:27.319+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c007490 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:27.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:27.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:28.123+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c00b210 legacy=0x7fa62c00d6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:28.123+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c007490 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:28.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:28.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient(hunting): tick 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c007490 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6790/0 conn(0x7fa62c003810 legacy=0x7fa62c005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c00b210 legacy=0x7fa62c00d6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient: _add_conns ranks=[8,6,4] 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 reap_dead start 2024-09-15T03:45:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-e con 0x7fa62c0112f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 reap_dead start 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-d con 0x7fa62c00a110 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-c con 0x7fa62c00a4e0 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c0122d0 con 0x7fa62c00a4e0 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644099110 con 0x7fa62c00a110 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c011020 con 0x7fa62c0112f0 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa641ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.711+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.915+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:29.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:29.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:29.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:30.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:30.319+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:30.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:30.919+0000 7fcdd9ffb700 10 monclient: tick 2024-09-15T03:45:30.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:30.919+0000 7fcdd9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:30.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:30.919+0000 7fcdd9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:31.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.123+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:31.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6791/0 conn(0x7fa62c00a4e0 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-09-15T03:45:31.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6791/0 conn(0x7fa62c00a4e0 legacy=0x7fa62c009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:45:31.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fa648b0a700 1 --1- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6791/0 conn(0x7fa62c00a4e0 0x7fa62c009940 :-1 s=OPENED pgs=111 cs=1 l=1).handle_message read tag failed 2024-09-15T03:45:31.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fa648b0a700 1 --1- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6791/0 conn(0x7fa62c00a4e0 0x7fa62c009940 :-1 s=OPENED pgs=111 cs=1 l=1).fault on lossy channel, failing 2024-09-15T03:45:31.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcdf27fc700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6791/0 conn(0x7fcdf4063710 legacy=0x7fcdf4119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T03:45:31.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcdf27fc700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6791/0 conn(0x7fcdf4063710 legacy=0x7fcdf4119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:45:31.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcdf27fc700 1 --1- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6791/0 conn(0x7fcdf4063710 0x7fcdf4119ca0 :-1 s=OPENED pgs=75 cs=1 l=1).handle_message read tag failed 2024-09-15T03:45:31.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcdf27fc700 1 --1- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6791/0 conn(0x7fcdf4063710 0x7fcdf4119ca0 :-1 s=OPENED pgs=75 cs=1 l=1).fault on lossy channel, failing 2024-09-15T03:45:31.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fa642ffd700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.26:6791/0 2024-09-15T03:45:31.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient: ms_handle_reset current mon v1:172.21.15.26:6791/0 2024-09-15T03:45:31.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6791/0 conn(0x7fcdf4063710 legacy=0x7fcdf4119ca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient: _add_conns ranks=[5,4,8,0,3,2,1,7,6] 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient(hunting): picked mon.h con 0x7fcdf4138eb0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient(hunting): picked mon.c con 0x7fcde0041e60 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient(hunting): picked mon.e con 0x7fcde0045f70 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdf411c4f0 con 0x7fcde0041e60 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdf409e470 con 0x7fcdf4138eb0 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdf409e840 con 0x7fcde0045f70 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcdf2ffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6793/0 conn(0x7fcde0045f70 legacy=0x7fcde0048420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.651+0000 7fcddaffd700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.667+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6792/0 conn(0x7fa62c00a110 legacy=0x7fa62c005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T03:45:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.667+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6792/0 conn(0x7fa62c00a110 legacy=0x7fa62c005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:45:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.667+0000 7fa6437fe700 1 --1- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6792/0 conn(0x7fa62c00a110 0x7fa62c005c50 :-1 s=OPENED pgs=109 cs=1 l=1).handle_message read tag failed 2024-09-15T03:45:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.667+0000 7fa6437fe700 1 --1- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6792/0 conn(0x7fa62c00a110 0x7fa62c005c50 :-1 s=OPENED pgs=109 cs=1 l=1).fault on lossy channel, failing 2024-09-15T03:45:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.667+0000 7fa642ffd700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.26:6792/0 2024-09-15T03:45:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.855+0000 7fcdf2ffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6793/0 conn(0x7fcde0045f70 legacy=0x7fcde0048420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient(hunting): tick 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6793/0 conn(0x7fcde0045f70 legacy=0x7fcde0048420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6791/0 conn(0x7fcdf4138eb0 legacy=0x7fcde00405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6791/0 conn(0x7fcde0041e60 legacy=0x7fcde00442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:31.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient: _add_conns ranks=[3,7,0,1,6,2] 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient(hunting): picked mon.g con 0x7fcdcc003750 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient(hunting): picked mon.i con 0x7fcdcc0073b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient(hunting): picked mon.a con 0x7fcdcc00b260 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcde004d160 con 0x7fcdcc00b260 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdf409e840 con 0x7fcdcc003750 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdf37fe700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6792/0 conn(0x7fcdcc0073b0 legacy=0x7fcdcc009860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdcc0113f0 con 0x7fcdcc0073b0 2024-09-15T03:45:31.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:31.919+0000 7fcdd9ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:32.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.123+0000 7fcdf37fe700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6792/0 conn(0x7fcdcc0073b0 legacy=0x7fcdcc009860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:32.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.527+0000 7fcdf37fe700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6792/0 conn(0x7fcdcc0073b0 legacy=0x7fcdcc009860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:32.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient(hunting): tick 2024-09-15T03:45:32.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:45:32.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:45:32.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:32.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6792/0 conn(0x7fa62c00a110 legacy=0x7fa62c005c50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:32.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6791/0 conn(0x7fa62c00a4e0 legacy=0x7fa62c009940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient: _add_conns ranks=[2,7,8,5,3,6,4,0,1] 2024-09-15T03:45:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-b con 0x7fa62c00ac40 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-i con 0x7fa62c0145d0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-e con 0x7fa62c0112f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c00a9e0 con 0x7fa62c00ac40 2024-09-15T03:45:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa62c00d5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c011020 con 0x7fa62c0145d0 2024-09-15T03:45:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c01a180 con 0x7fa62c0112f0 2024-09-15T03:45:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa641ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:32.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.711+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:32.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.715+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c0145d0 legacy=0x7fa62c0169b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:32.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342688153 0 0) 0x7fcde004d160 con 0x7fcdcc00b260 2024-09-15T03:45:32.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:32.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient(hunting): _init_auth already have auth, reseting 2024-09-15T03:45:32.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6789/0 -- auth(proto 2 132 bytes epoch 0) v1 -- 0x7fcde004a0b0 con 0x7fcdcc00b260 2024-09-15T03:45:32.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 233+0+0 (unknown 3325237693 0 0) 0x7fcde004a0b0 con 0x7fcdcc00b260 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6792/0 conn(0x7fcdcc0073b0 legacy=0x7fcdcc009860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6790/0 conn(0x7fcdcc003750 legacy=0x7fcdcc005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 monclient: found mon.a 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcdf37fe700 1 -- 172.21.15.26:0/2583404537 reap_dead start 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1.5 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,mgrmap=0+,monmap=2+}) v3 -- 0x7fcdcc012640 con 0x7fcdcc00b260 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcdf37fe700 1 -- 172.21.15.26:0/2583404537 reap_dead start 2024-09-15T03:45:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcdcc0113f0 con 0x7fcdcc00b260 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcdec005bb0 con 0x7fcdcc00b260 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.855+0000 7fcddaffd700 10 monclient: dump: 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.859+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcdec005de0 con 0x7fcdcc00b260 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.859+0000 7fcddaffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.859+0000 7fcdf0ff9700 4 set_mon_vals no callback set 2024-09-15T03:45:32.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.859+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcdec005f50 con 0x7fcdcc00b260 2024-09-15T03:45:32.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.915+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:32.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.915+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa62c00d5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:32.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:32.919+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c0145d0 legacy=0x7fa62c0169b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.319+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa62c00d5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.319+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:33.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.323+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c0145d0 legacy=0x7fa62c0169b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:33.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.883+0000 7fcddaffd700 1 -- 172.21.15.26:0/2583404537 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]=0 enabled application 'rados' on pool 'unique_pool_0' v20) v1 ==== 197+0+0 (unknown 858860997 0 0) 0x7fcdec014070 con 0x7fcdcc00b260 2024-09-15T03:45:33.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.883+0000 7fcddaffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}] 2024-09-15T03:45:33.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.883+0000 7fcddaffd700 10 monclient: _finish_command 2 = system:0 enabled application 'rados' on pool 'unique_pool_0' 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.122:6812/20895 conn(0x7fcde0037480 legacy=0x7fcde0039930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 10 monclient: shutdown 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 >> v1:172.21.15.26:6789/0 conn(0x7fcdcc00b260 legacy=0x7fcdcc00d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 shutdown_connections 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 >> 172.21.15.26:0/2583404537 conn(0x7fcdf40a39b0 msgr2=0x7fcdf4098af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 shutdown_connections 2024-09-15T03:45:33.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:33.887+0000 7fcdf3fff700 1 -- 172.21.15.26:0/2583404537 wait complete. 2024-09-15T03:45:33.891 INFO:teuthology.orchestra.run.smithi026.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-15T03:45:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:34.123+0000 7fa643fff700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa62c00d5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:34.123+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:34.127+0000 7fa6437fe700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c0145d0 legacy=0x7fa62c0169b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:34.908 DEBUG:teuthology.orchestra.run.smithi122:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op write_excl 50 --pool unique_pool_0 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight read -> 100 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight write -> 50 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight delete -> 50 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight snap_create -> 50 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight snap_remove -> 50 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight rollback -> 50 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight copy_from -> 50 2024-09-15T03:45:34.928 INFO:tasks.rados.rados.0.smithi122.stdout:adding op weight write_excl -> 50 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout:ceph version 17.2.7-1643-gd75f2382 (d75f23820806da622f561ee5ee7551015285993f) quincy (stable) 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout:Configuration: 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout: Number of operations: 4000 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout: Number of objects: 50 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout: Max in flight operations: 16 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout: Object size (in bytes): 4000000 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout: Write stride min: 400000 2024-09-15T03:45:34.929 INFO:tasks.rados.rados.0.smithi122.stdout: Write stride max: 800000 2024-09-15T03:45:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient(hunting): tick 2024-09-15T03:45:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:45:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:45:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6793/0 conn(0x7fa62c0112f0 legacy=0x7fa62c0116c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6792/0 conn(0x7fa62c0145d0 legacy=0x7fa62c0169b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa62c00d5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 reap_dead start 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient: _add_conns ranks=[3,5,1,6,0,4] 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-g con 0x7fa62c00ac40 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-h con 0x7fa62c0116c0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient(hunting): picked mon.noname-f con 0x7fa62c011a90 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c01b3b0 con 0x7fa62c011a90 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c00a9e0 con 0x7fa62c00ac40 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62c011020 con 0x7fa62c0116c0 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa641ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa648b0a700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa62c005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689583705 0 0) 0x7fa62c011020 con 0x7fa62c0116c0 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient(hunting): my global_id is 4541 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa628003570 con 0x7fa62c0116c0 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402668874 0 0) 0x7fa62c00a9e0 con 0x7fa62c00ac40 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient(hunting): my global_id is 4539 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c011020 con 0x7fa62c00ac40 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2396718369 0 0) 0x7fa628003570 con 0x7fa62c0116c0 2024-09-15T03:45:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa62c00a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa62c005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 monclient: found mon.noname-h 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c01a180 con 0x7fa62c0116c0 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa630001820 con 0x7fa62c0116c0 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 5 monclient: authenticate success, global_id 4541 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: dump: 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa630001ac0 con 0x7fa62c0116c0 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 1 -- 172.21.15.26:0/1191697780 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa630001ff0 con 0x7fa62c0116c0 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 4 set_mon_vals no callback set 2024-09-15T03:45:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa642ffd700 10 monclient: dump: 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:35.727 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 10 monclient: shutdown 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.715+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 >> v1:172.21.15.122:6791/0 conn(0x7fa62c0116c0 legacy=0x7fa62c00d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 shutdown_connections 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 >> 172.21.15.26:0/1191697780 conn(0x7fa644071860 msgr2=0x7fa6440a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 shutdown_connections 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- 172.21.15.26:0/1191697780 wait complete. 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient: build_initial_monmap 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient: monmap: 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 Processor -- start 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- start start 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient: init 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient: _add_conns ranks=[2,7,1,5,6,3,8,4,0] 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient(hunting): picked mon.noname-b con 0x7fa62c00ac40 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient(hunting): picked mon.noname-i con 0x7fa62c0116c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient(hunting): picked mon.noname-f con 0x7fa62c011a90 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644097ea0 con 0x7fa62c011a90 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64411db70 con 0x7fa62c00ac40 2024-09-15T03:45:35.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644098300 con 0x7fa62c0116c0 2024-09-15T03:45:35.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:35.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa64930b700 10 monclient(hunting): authenticate will time out at 880.151959s 2024-09-15T03:45:35.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.719+0000 7fa643fff700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa64412dd10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:35.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.723+0000 7fa6437fe700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa62c0116c0 legacy=0x7fa644144eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:35.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.723+0000 7fa648b0a700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa6441485e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:35.898 INFO:tasks.rados.rados.0.smithi122.stdout:1: write initial oid 1 2024-09-15T03:45:35.898 INFO:tasks.rados.rados.0.smithi122.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2024-09-15T03:45:35.906 INFO:tasks.rados.rados.0.smithi122.stdout:1: writing smithi12223906-1 from 416807 to 892056 tid 1 2024-09-15T03:45:35.913 INFO:tasks.rados.rados.0.smithi122.stdout:1: writing smithi12223906-1 from 1542129 to 2016807 tid 2 2024-09-15T03:45:35.914 INFO:tasks.rados.rados.0.smithi122.stdout:2: write initial oid 2 2024-09-15T03:45:35.914 INFO:tasks.rados.rados.0.smithi122.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2024-09-15T03:45:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:2: writing smithi12223906-2 from 416807 to 892056 tid 1 2024-09-15T03:45:35.925 INFO:tasks.rados.rados.0.smithi122.stdout:2: writing smithi12223906-2 from 1542129 to 2016807 tid 2 2024-09-15T03:45:35.926 INFO:tasks.rados.rados.0.smithi122.stdout:3: write initial oid 3 2024-09-15T03:45:35.926 INFO:tasks.rados.rados.0.smithi122.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2024-09-15T03:45:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.927+0000 7fa643fff700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa64412dd10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.927+0000 7fa648b0a700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa6441485e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:35.927+0000 7fa6437fe700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa62c0116c0 legacy=0x7fa644144eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:35.932 INFO:tasks.rados.rados.0.smithi122.stdout:3: writing smithi12223906-3 from 433614 to 984112 tid 1 2024-09-15T03:45:35.938 INFO:tasks.rados.rados.0.smithi122.stdout:3: writing smithi12223906-3 from 1600611 to 2033614 tid 2 2024-09-15T03:45:35.938 INFO:tasks.rados.rados.0.smithi122.stdout:4: write initial oid 4 2024-09-15T03:45:35.938 INFO:tasks.rados.rados.0.smithi122.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2024-09-15T03:45:35.944 INFO:tasks.rados.rados.0.smithi122.stdout:4: writing smithi12223906-4 from 450421 to 1076168 tid 1 2024-09-15T03:45:35.949 INFO:tasks.rados.rados.0.smithi122.stdout:4: writing smithi12223906-4 from 1659093 to 2050421 tid 2 2024-09-15T03:45:35.949 INFO:tasks.rados.rados.0.smithi122.stdout:5: write initial oid 5 2024-09-15T03:45:35.950 INFO:tasks.rados.rados.0.smithi122.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2024-09-15T03:45:35.956 INFO:tasks.rados.rados.0.smithi122.stdout:5: writing smithi12223906-5 from 467228 to 1168224 tid 1 2024-09-15T03:45:35.961 INFO:tasks.rados.rados.0.smithi122.stdout:5: writing smithi12223906-5 from 1717575 to 2067228 tid 2 2024-09-15T03:45:35.961 INFO:tasks.rados.rados.0.smithi122.stdout:6: write initial oid 6 2024-09-15T03:45:35.961 INFO:tasks.rados.rados.0.smithi122.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2024-09-15T03:45:35.969 INFO:tasks.rados.rados.0.smithi122.stdout:6: writing smithi12223906-6 from 484035 to 1260280 tid 1 2024-09-15T03:45:35.973 INFO:tasks.rados.rados.0.smithi122.stdout:6: writing smithi12223906-6 from 2059704 to 2084035 tid 2 2024-09-15T03:45:35.974 INFO:tasks.rados.rados.0.smithi122.stdout:7: write initial oid 7 2024-09-15T03:45:35.974 INFO:tasks.rados.rados.0.smithi122.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2024-09-15T03:45:35.979 INFO:tasks.rados.rados.0.smithi122.stdout:7: writing smithi12223906-7 from 500842 to 952336 tid 1 2024-09-15T03:45:35.986 INFO:tasks.rados.rados.0.smithi122.stdout:7: writing smithi12223906-7 from 1718186 to 2100842 tid 2 2024-09-15T03:45:35.986 INFO:tasks.rados.rados.0.smithi122.stdout:8: write initial oid 8 2024-09-15T03:45:35.986 INFO:tasks.rados.rados.0.smithi122.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2024-09-15T03:45:35.992 INFO:tasks.rados.rados.0.smithi122.stdout:8: writing smithi12223906-8 from 517649 to 1044392 tid 1 2024-09-15T03:45:35.998 INFO:tasks.rados.rados.0.smithi122.stdout:8: writing smithi12223906-8 from 1776668 to 2117649 tid 2 2024-09-15T03:45:35.998 INFO:tasks.rados.rados.0.smithi122.stdout:9: write initial oid 9 2024-09-15T03:45:35.998 INFO:tasks.rados.rados.0.smithi122.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2024-09-15T03:45:36.005 INFO:tasks.rados.rados.0.smithi122.stdout:9: writing smithi12223906-9 from 534456 to 1252801 tid 1 2024-09-15T03:45:36.010 INFO:tasks.rados.rados.0.smithi122.stdout:9: writing smithi12223906-9 from 1951503 to 2134456 tid 2 2024-09-15T03:45:36.010 INFO:tasks.rados.rados.0.smithi122.stdout:10: write initial oid 10 2024-09-15T03:45:36.010 INFO:tasks.rados.rados.0.smithi122.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2024-09-15T03:45:36.017 INFO:tasks.rados.rados.0.smithi122.stdout:10: writing smithi12223906-10 from 551263 to 1344857 tid 1 2024-09-15T03:45:36.022 INFO:tasks.rados.rados.0.smithi122.stdout:10: writing smithi12223906-10 from 1893632 to 2151263 tid 2 2024-09-15T03:45:36.022 INFO:tasks.rados.rados.0.smithi122.stdout:11: write initial oid 11 2024-09-15T03:45:36.022 INFO:tasks.rados.rados.0.smithi122.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2024-09-15T03:45:36.028 INFO:tasks.rados.rados.0.smithi122.stdout:11: writing smithi12223906-11 from 568070 to 1036913 tid 1 2024-09-15T03:45:36.035 INFO:tasks.rados.rados.0.smithi122.stdout:11: writing smithi12223906-11 from 1552114 to 2168070 tid 2 2024-09-15T03:45:36.035 INFO:tasks.rados.rados.0.smithi122.stdout:12: write initial oid 12 2024-09-15T03:45:36.035 INFO:tasks.rados.rados.0.smithi122.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2024-09-15T03:45:36.041 INFO:tasks.rados.rados.0.smithi122.stdout:12: writing smithi12223906-12 from 584877 to 1128969 tid 1 2024-09-15T03:45:36.047 INFO:tasks.rados.rados.0.smithi122.stdout:12: writing smithi12223906-12 from 1610596 to 2172599 tid 2 2024-09-15T03:45:36.047 INFO:tasks.rados.rados.0.smithi122.stdout:13: write initial oid 13 2024-09-15T03:45:36.047 INFO:tasks.rados.rados.0.smithi122.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2024-09-15T03:45:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:13: writing smithi12223906-13 from 601684 to 1221025 tid 1 2024-09-15T03:45:36.060 INFO:tasks.rados.rados.0.smithi122.stdout:13: writing smithi12223906-13 from 1669078 to 2201684 tid 2 2024-09-15T03:45:36.060 INFO:tasks.rados.rados.0.smithi122.stdout:14: write initial oid 14 2024-09-15T03:45:36.060 INFO:tasks.rados.rados.0.smithi122.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2024-09-15T03:45:36.067 INFO:tasks.rados.rados.0.smithi122.stdout:14: writing smithi12223906-14 from 618491 to 1313081 tid 1 2024-09-15T03:45:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:14: writing smithi12223906-14 from 2011207 to 2218491 tid 2 2024-09-15T03:45:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:15: write initial oid 15 2024-09-15T03:45:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2024-09-15T03:45:36.080 INFO:tasks.rados.rados.0.smithi122.stdout:15: writing smithi12223906-15 from 635298 to 1405137 tid 1 2024-09-15T03:45:36.085 INFO:tasks.rados.rados.0.smithi122.stdout:15: writing smithi12223906-15 from 2069689 to 2235298 tid 2 2024-09-15T03:45:36.085 INFO:tasks.rados.rados.0.smithi122.stdout:16: write initial oid 16 2024-09-15T03:45:36.085 INFO:tasks.rados.rados.0.smithi122.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2024-09-15T03:45:36.091 INFO:tasks.rados.rados.0.smithi122.stdout:16: writing smithi12223906-16 from 652105 to 1097193 tid 1 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout:16: writing smithi12223906-16 from 1728171 to 2181159 tid 2 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout:1: finishing write tid 1 to smithi12223906-1 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout:1: finishing write tid 2 to smithi12223906-1 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout:1: finishing write tid 3 to smithi12223906-1 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout:1: finishing write tid 4 to smithi12223906-1 2024-09-15T03:45:36.098 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:2: finishing write tid 1 to smithi12223906-2 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:2: finishing write tid 2 to smithi12223906-2 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:2: finishing write tid 3 to smithi12223906-2 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:2: finishing write tid 4 to smithi12223906-2 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:1: done (15 left) 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:2: done (14 left) 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:17: write initial oid 17 2024-09-15T03:45:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2024-09-15T03:45:36.105 INFO:tasks.rados.rados.0.smithi122.stdout:17: writing smithi12223906-17 from 668912 to 1305602 tid 1 2024-09-15T03:45:36.110 INFO:tasks.rados.rados.0.smithi122.stdout:17: writing smithi12223906-17 from 1903006 to 2268912 tid 2 2024-09-15T03:45:36.110 INFO:tasks.rados.rados.0.smithi122.stdout:18: write initial oid 18 2024-09-15T03:45:36.111 INFO:tasks.rados.rados.0.smithi122.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2024-09-15T03:45:36.119 INFO:tasks.rados.rados.0.smithi122.stdout:18: writing smithi12223906-18 from 685719 to 1397658 tid 1 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:18: writing smithi12223906-18 from 1845135 to 2285719 tid 2 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:3: finishing write tid 1 to smithi12223906-3 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:3: finishing write tid 2 to smithi12223906-3 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:3: finishing write tid 3 to smithi12223906-3 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:3: finishing write tid 4 to smithi12223906-3 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-15T03:45:36.124 INFO:tasks.rados.rados.0.smithi122.stdout:4: finishing write tid 1 to smithi12223906-4 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:4: finishing write tid 2 to smithi12223906-4 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:4: finishing write tid 3 to smithi12223906-4 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:4: finishing write tid 4 to smithi12223906-4 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:3: done (15 left) 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:4: done (14 left) 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:19: write initial oid 19 2024-09-15T03:45:36.125 INFO:tasks.rados.rados.0.smithi122.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2024-09-15T03:45:36.133 INFO:tasks.rados.rados.0.smithi122.stdout:19: writing smithi12223906-19 from 702526 to 1489714 tid 1 2024-09-15T03:45:36.137 INFO:tasks.rados.rados.0.smithi122.stdout:19: writing smithi12223906-19 from 1903617 to 2302526 tid 2 2024-09-15T03:45:36.137 INFO:tasks.rados.rados.0.smithi122.stdout:20: write initial oid 20 2024-09-15T03:45:36.138 INFO:tasks.rados.rados.0.smithi122.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2024-09-15T03:45:36.143 INFO:tasks.rados.rados.0.smithi122.stdout:20: writing smithi12223906-20 from 719333 to 1181770 tid 1 2024-09-15T03:45:36.150 INFO:tasks.rados.rados.0.smithi122.stdout:20: writing smithi12223906-20 from 1962099 to 2319333 tid 2 2024-09-15T03:45:36.150 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:5: finishing write tid 1 to smithi12223906-5 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:5: finishing write tid 2 to smithi12223906-5 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:5: finishing write tid 3 to smithi12223906-5 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:5: finishing write tid 4 to smithi12223906-5 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:6: finishing write tid 1 to smithi12223906-6 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:6: finishing write tid 2 to smithi12223906-6 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:6: finishing write tid 3 to smithi12223906-6 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:6: finishing write tid 4 to smithi12223906-6 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 3 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:7: finishing write tid 1 to smithi12223906-7 2024-09-15T03:45:36.151 INFO:tasks.rados.rados.0.smithi122.stdout:7: finishing write tid 2 to smithi12223906-7 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:7: finishing write tid 3 to smithi12223906-7 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:7: finishing write tid 4 to smithi12223906-7 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 3 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:8: finishing write tid 1 to smithi12223906-8 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:8: finishing write tid 2 to smithi12223906-8 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:5: done (15 left) 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:6: done (14 left) 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:7: done (13 left) 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:21: write initial oid 21 2024-09-15T03:45:36.152 INFO:tasks.rados.rados.0.smithi122.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2024-09-15T03:45:36.157 INFO:tasks.rados.rados.0.smithi122.stdout:21: writing smithi12223906-21 from 736140 to 1273826 tid 1 2024-09-15T03:45:36.164 INFO:tasks.rados.rados.0.smithi122.stdout:21: writing smithi12223906-21 from 2020581 to 2336140 tid 2 2024-09-15T03:45:36.164 INFO:tasks.rados.rados.0.smithi122.stdout:22: write initial oid 22 2024-09-15T03:45:36.164 INFO:tasks.rados.rados.0.smithi122.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2024-09-15T03:45:36.171 INFO:tasks.rados.rados.0.smithi122.stdout:22: writing smithi12223906-22 from 752947 to 1365882 tid 1 2024-09-15T03:45:36.177 INFO:tasks.rados.rados.0.smithi122.stdout:22: writing smithi12223906-22 from 1962710 to 2352947 tid 2 2024-09-15T03:45:36.177 INFO:tasks.rados.rados.0.smithi122.stdout:23: write initial oid 23 2024-09-15T03:45:36.177 INFO:tasks.rados.rados.0.smithi122.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2024-09-15T03:45:36.185 INFO:tasks.rados.rados.0.smithi122.stdout:23: writing smithi12223906-23 from 769754 to 1457938 tid 1 2024-09-15T03:45:36.190 INFO:tasks.rados.rados.0.smithi122.stdout:23: writing smithi12223906-23 from 2021192 to 2369754 tid 2 2024-09-15T03:45:36.190 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.190 INFO:tasks.rados.rados.0.smithi122.stdout:8: finishing write tid 3 to smithi12223906-8 2024-09-15T03:45:36.190 INFO:tasks.rados.rados.0.smithi122.stdout:8: finishing write tid 4 to smithi12223906-8 2024-09-15T03:45:36.190 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T03:45:36.190 INFO:tasks.rados.rados.0.smithi122.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-15T03:45:36.191 INFO:tasks.rados.rados.0.smithi122.stdout:9: finishing write tid 1 to smithi12223906-9 2024-09-15T03:45:36.191 INFO:tasks.rados.rados.0.smithi122.stdout:8: done (15 left) 2024-09-15T03:45:36.191 INFO:tasks.rados.rados.0.smithi122.stdout:24: write initial oid 24 2024-09-15T03:45:36.191 INFO:tasks.rados.rados.0.smithi122.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2024-09-15T03:45:36.197 INFO:tasks.rados.rados.0.smithi122.stdout:24: writing smithi12223906-24 from 786561 to 1266347 tid 1 2024-09-15T03:45:36.202 INFO:tasks.rados.rados.0.smithi122.stdout:24: writing smithi12223906-24 from 1796027 to 2263691 tid 2 2024-09-15T03:45:36.202 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.202 INFO:tasks.rados.rados.0.smithi122.stdout:9: finishing write tid 2 to smithi12223906-9 2024-09-15T03:45:36.202 INFO:tasks.rados.rados.0.smithi122.stdout:9: finishing write tid 3 to smithi12223906-9 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:9: finishing write tid 4 to smithi12223906-9 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:10: finishing write tid 1 to smithi12223906-10 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:9: done (15 left) 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:25: write initial oid 25 2024-09-15T03:45:36.203 INFO:tasks.rados.rados.0.smithi122.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2024-09-15T03:45:36.207 INFO:tasks.rados.rados.0.smithi122.stdout:25: writing smithi12223906-25 from 403368 to 958403 tid 1 2024-09-15T03:45:36.214 INFO:tasks.rados.rados.0.smithi122.stdout:25: writing smithi12223906-25 from 1454509 to 2182184 tid 2 2024-09-15T03:45:36.214 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.214 INFO:tasks.rados.rados.0.smithi122.stdout:10: finishing write tid 2 to smithi12223906-10 2024-09-15T03:45:36.214 INFO:tasks.rados.rados.0.smithi122.stdout:10: finishing write tid 3 to smithi12223906-10 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:10: finishing write tid 4 to smithi12223906-10 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 3 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:11: finishing write tid 1 to smithi12223906-11 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:11: finishing write tid 2 to smithi12223906-11 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:11: finishing write tid 3 to smithi12223906-11 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:11: finishing write tid 4 to smithi12223906-11 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:12: finishing write tid 1 to smithi12223906-12 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:12: finishing write tid 2 to smithi12223906-12 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:12: finishing write tid 3 to smithi12223906-12 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:12: finishing write tid 4 to smithi12223906-12 2024-09-15T03:45:36.215 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:13: finishing write tid 1 to smithi12223906-13 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:13: finishing write tid 2 to smithi12223906-13 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:13: finishing write tid 3 to smithi12223906-13 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:13: finishing write tid 4 to smithi12223906-13 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:14: finishing write tid 1 to smithi12223906-14 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:14: finishing write tid 2 to smithi12223906-14 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:14: finishing write tid 3 to smithi12223906-14 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:14: finishing write tid 4 to smithi12223906-14 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-15T03:45:36.216 INFO:tasks.rados.rados.0.smithi122.stdout:15: finishing write tid 1 to smithi12223906-15 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:15: finishing write tid 2 to smithi12223906-15 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:15: finishing write tid 3 to smithi12223906-15 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:15: finishing write tid 4 to smithi12223906-15 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:16: finishing write tid 1 to smithi12223906-16 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:10: done (15 left) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:11: done (14 left) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:12: done (13 left) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:13: done (12 left) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:14: done (11 left) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:15: done (10 left) 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:26: write initial oid 26 2024-09-15T03:45:36.217 INFO:tasks.rados.rados.0.smithi122.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2024-09-15T03:45:36.220 INFO:tasks.rados.rados.0.smithi122.stdout:26: writing smithi12223906-26 from 420175 to 1050459 tid 1 2024-09-15T03:45:36.227 INFO:tasks.rados.rados.0.smithi122.stdout:26: writing smithi12223906-26 from 1796638 to 2267971 tid 2 2024-09-15T03:45:36.227 INFO:tasks.rados.rados.0.smithi122.stdout:27: write initial oid 27 2024-09-15T03:45:36.227 INFO:tasks.rados.rados.0.smithi122.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2024-09-15T03:45:36.233 INFO:tasks.rados.rados.0.smithi122.stdout:27: writing smithi12223906-27 from 436982 to 1142515 tid 1 2024-09-15T03:45:36.240 INFO:tasks.rados.rados.0.smithi122.stdout:27: writing smithi12223906-27 from 1855120 to 2436982 tid 2 2024-09-15T03:45:36.240 INFO:tasks.rados.rados.0.smithi122.stdout:28: write initial oid 28 2024-09-15T03:45:36.240 INFO:tasks.rados.rados.0.smithi122.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2024-09-15T03:45:36.247 INFO:tasks.rados.rados.0.smithi122.stdout:28: writing smithi12223906-28 from 453789 to 1234571 tid 1 2024-09-15T03:45:36.253 INFO:tasks.rados.rados.0.smithi122.stdout:28: writing smithi12223906-28 from 1913602 to 2388604 tid 2 2024-09-15T03:45:36.253 INFO:tasks.rados.rados.0.smithi122.stdout:29: write initial oid 29 2024-09-15T03:45:36.254 INFO:tasks.rados.rados.0.smithi122.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2024-09-15T03:45:36.258 INFO:tasks.rados.rados.0.smithi122.stdout:29: writing smithi12223906-29 from 470596 to 926627 tid 1 2024-09-15T03:45:36.266 INFO:tasks.rados.rados.0.smithi122.stdout:29: writing smithi12223906-29 from 1572084 to 2190744 tid 2 2024-09-15T03:45:36.266 INFO:tasks.rados.rados.0.smithi122.stdout:30: write initial oid 30 2024-09-15T03:45:36.266 INFO:tasks.rados.rados.0.smithi122.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2024-09-15T03:45:36.271 INFO:tasks.rados.rados.0.smithi122.stdout:30: writing smithi12223906-30 from 487403 to 1018683 tid 1 2024-09-15T03:45:36.277 INFO:tasks.rados.rados.0.smithi122.stdout:30: writing smithi12223906-30 from 1514213 to 1992884 tid 2 2024-09-15T03:45:36.277 INFO:tasks.rados.rados.0.smithi122.stdout:31: write initial oid 31 2024-09-15T03:45:36.277 INFO:tasks.rados.rados.0.smithi122.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2024-09-15T03:45:36.283 INFO:tasks.rados.rados.0.smithi122.stdout:31: writing smithi12223906-31 from 504210 to 1110739 tid 1 2024-09-15T03:45:36.290 INFO:tasks.rados.rados.0.smithi122.stdout:31: writing smithi12223906-31 from 1572695 to 2195024 tid 2 2024-09-15T03:45:36.290 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:16: finishing write tid 2 to smithi12223906-16 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:16: finishing write tid 3 to smithi12223906-16 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:16: finishing write tid 4 to smithi12223906-16 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 12 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:17: finishing write tid 1 to smithi12223906-17 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:17: finishing write tid 2 to smithi12223906-17 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:17: finishing write tid 3 to smithi12223906-17 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:17: finishing write tid 4 to smithi12223906-17 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 6 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:18: finishing write tid 1 to smithi12223906-18 2024-09-15T03:45:36.291 INFO:tasks.rados.rados.0.smithi122.stdout:18: finishing write tid 2 to smithi12223906-18 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:18: finishing write tid 3 to smithi12223906-18 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:18: finishing write tid 4 to smithi12223906-18 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:19: finishing write tid 1 to smithi12223906-19 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:19: finishing write tid 2 to smithi12223906-19 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:19: finishing write tid 3 to smithi12223906-19 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:19: finishing write tid 4 to smithi12223906-19 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:20: finishing write tid 1 to smithi12223906-20 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:20: finishing write tid 2 to smithi12223906-20 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:20: finishing write tid 3 to smithi12223906-20 2024-09-15T03:45:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:16: done (15 left) 2024-09-15T03:45:36.293 INFO:tasks.rados.rados.0.smithi122.stdout:17: done (14 left) 2024-09-15T03:45:36.293 INFO:tasks.rados.rados.0.smithi122.stdout:18: done (13 left) 2024-09-15T03:45:36.293 INFO:tasks.rados.rados.0.smithi122.stdout:19: done (12 left) 2024-09-15T03:45:36.293 INFO:tasks.rados.rados.0.smithi122.stdout:32: write initial oid 32 2024-09-15T03:45:36.293 INFO:tasks.rados.rados.0.smithi122.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2024-09-15T03:45:36.296 INFO:tasks.rados.rados.0.smithi122.stdout:32: writing smithi12223906-32 from 521017 to 1319148 tid 1 2024-09-15T03:45:36.301 INFO:tasks.rados.rados.0.smithi122.stdout:32: writing smithi12223906-32 from 1747530 to 2229870 tid 2 2024-09-15T03:45:36.302 INFO:tasks.rados.rados.0.smithi122.stdout:33: write initial oid 33 2024-09-15T03:45:36.302 INFO:tasks.rados.rados.0.smithi122.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2024-09-15T03:45:36.307 INFO:tasks.rados.rados.0.smithi122.stdout:33: writing smithi12223906-33 from 537824 to 1011204 tid 1 2024-09-15T03:45:36.315 INFO:tasks.rados.rados.0.smithi122.stdout:33: writing smithi12223906-33 from 1806012 to 2432010 tid 2 2024-09-15T03:45:36.315 INFO:tasks.rados.rados.0.smithi122.stdout:34: write initial oid 34 2024-09-15T03:45:36.315 INFO:tasks.rados.rados.0.smithi122.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2024-09-15T03:45:36.320 INFO:tasks.rados.rados.0.smithi122.stdout:34: writing smithi12223906-34 from 554631 to 1103260 tid 1 2024-09-15T03:45:36.327 INFO:tasks.rados.rados.0.smithi122.stdout:34: writing smithi12223906-34 from 1748141 to 2234150 tid 2 2024-09-15T03:45:36.327 INFO:tasks.rados.rados.0.smithi122.stdout:35: write initial oid 35 2024-09-15T03:45:36.327 INFO:tasks.rados.rados.0.smithi122.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2024-09-15T03:45:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:36.331+0000 7fa643fff700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa64412dd10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:36.331+0000 7fa648b0a700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa6441485e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:36.331+0000 7fa6437fe700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa62c0116c0 legacy=0x7fa644144eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:36.333 INFO:tasks.rados.rados.0.smithi122.stdout:35: writing smithi12223906-35 from 571438 to 1195316 tid 1 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:35: writing smithi12223906-35 from 1806623 to 2436290 tid 2 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:20: finishing write tid 4 to smithi12223906-20 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 3 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:20: done (15 left) 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:36: write initial oid 36 2024-09-15T03:45:36.341 INFO:tasks.rados.rados.0.smithi122.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2024-09-15T03:45:36.347 INFO:tasks.rados.rados.0.smithi122.stdout:36: writing smithi12223906-36 from 588245 to 1287372 tid 1 2024-09-15T03:45:36.353 INFO:tasks.rados.rados.0.smithi122.stdout:36: writing smithi12223906-36 from 1865105 to 2354783 tid 2 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:21: finishing write tid 1 to smithi12223906-21 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:21: finishing write tid 2 to smithi12223906-21 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:21: finishing write tid 3 to smithi12223906-21 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:21: finishing write tid 4 to smithi12223906-21 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-15T03:45:36.354 INFO:tasks.rados.rados.0.smithi122.stdout:21: done (15 left) 2024-09-15T03:45:36.355 INFO:tasks.rados.rados.0.smithi122.stdout:37: write initial oid 37 2024-09-15T03:45:36.355 INFO:tasks.rados.rados.0.smithi122.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2024-09-15T03:45:36.361 INFO:tasks.rados.rados.0.smithi122.stdout:37: writing smithi12223906-37 from 605052 to 1379428 tid 1 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:37: writing smithi12223906-37 from 1923587 to 2556923 tid 2 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:22: finishing write tid 1 to smithi12223906-22 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:22: finishing write tid 2 to smithi12223906-22 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:22: finishing write tid 3 to smithi12223906-22 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:22: finishing write tid 4 to smithi12223906-22 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 6 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:23: finishing write tid 1 to smithi12223906-23 2024-09-15T03:45:36.369 INFO:tasks.rados.rados.0.smithi122.stdout:23: finishing write tid 2 to smithi12223906-23 2024-09-15T03:45:36.370 INFO:tasks.rados.rados.0.smithi122.stdout:23: finishing write tid 3 to smithi12223906-23 2024-09-15T03:45:36.370 INFO:tasks.rados.rados.0.smithi122.stdout:22: done (15 left) 2024-09-15T03:45:36.370 INFO:tasks.rados.rados.0.smithi122.stdout:38: write initial oid 38 2024-09-15T03:45:36.370 INFO:tasks.rados.rados.0.smithi122.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2024-09-15T03:45:36.374 INFO:tasks.rados.rados.0.smithi122.stdout:38: writing smithi12223906-38 from 621859 to 1071484 tid 1 2024-09-15T03:45:36.383 INFO:tasks.rados.rados.0.smithi122.stdout:38: writing smithi12223906-38 from 1865716 to 2621859 tid 2 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout:23: finishing write tid 4 to smithi12223906-23 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 9 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout:23: done (15 left) 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout:39: write initial oid 39 2024-09-15T03:45:36.384 INFO:tasks.rados.rados.0.smithi122.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2024-09-15T03:45:36.390 INFO:tasks.rados.rados.0.smithi122.stdout:39: writing smithi12223906-39 from 638666 to 1163540 tid 1 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:39: writing smithi12223906-39 from 1924198 to 2561203 tid 2 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:24: finishing write tid 1 to smithi12223906-24 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:24: finishing write tid 2 to smithi12223906-24 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:24: finishing write tid 3 to smithi12223906-24 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:24: finishing write tid 4 to smithi12223906-24 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 9 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:25: finishing write tid 1 to smithi12223906-25 2024-09-15T03:45:36.398 INFO:tasks.rados.rados.0.smithi122.stdout:24: done (15 left) 2024-09-15T03:45:36.399 INFO:tasks.rados.rados.0.smithi122.stdout:40: write initial oid 40 2024-09-15T03:45:36.399 INFO:tasks.rados.rados.0.smithi122.stdout:25: finishing write tid 2 to smithi12223906-25 2024-09-15T03:45:36.399 INFO:tasks.rados.rados.0.smithi122.stdout:25: finishing write tid 3 to smithi12223906-25 2024-09-15T03:45:36.399 INFO:tasks.rados.rados.0.smithi122.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2024-09-15T03:45:36.404 INFO:tasks.rados.rados.0.smithi122.stdout:40: writing smithi12223906-40 from 655473 to 1371949 tid 1 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:40: writing smithi12223906-40 from 2099033 to 2655473 tid 2 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:25: finishing write tid 4 to smithi12223906-25 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 12 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:25: done (15 left) 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:41: write initial oid 41 2024-09-15T03:45:36.412 INFO:tasks.rados.rados.0.smithi122.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2024-09-15T03:45:36.419 INFO:tasks.rados.rados.0.smithi122.stdout:41: writing smithi12223906-41 from 672280 to 1464005 tid 1 2024-09-15T03:45:36.427 INFO:tasks.rados.rados.0.smithi122.stdout:41: writing smithi12223906-41 from 2157515 to 2672280 tid 2 2024-09-15T03:45:36.427 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.427 INFO:tasks.rados.rados.0.smithi122.stdout:26: finishing write tid 1 to smithi12223906-26 2024-09-15T03:45:36.427 INFO:tasks.rados.rados.0.smithi122.stdout:26: finishing write tid 2 to smithi12223906-26 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:26: finishing write tid 3 to smithi12223906-26 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:26: finishing write tid 4 to smithi12223906-26 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:27: finishing write tid 1 to smithi12223906-27 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:27: finishing write tid 2 to smithi12223906-27 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:27: finishing write tid 3 to smithi12223906-27 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:27: finishing write tid 4 to smithi12223906-27 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:28: finishing write tid 1 to smithi12223906-28 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:28: finishing write tid 2 to smithi12223906-28 2024-09-15T03:45:36.428 INFO:tasks.rados.rados.0.smithi122.stdout:28: finishing write tid 3 to smithi12223906-28 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:28: finishing write tid 4 to smithi12223906-28 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:29: finishing write tid 1 to smithi12223906-29 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:29: finishing write tid 2 to smithi12223906-29 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:29: finishing write tid 3 to smithi12223906-29 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:26: done (15 left) 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:27: done (14 left) 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:28: done (13 left) 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:42: write initial oid 42 2024-09-15T03:45:36.429 INFO:tasks.rados.rados.0.smithi122.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2024-09-15T03:45:36.432 INFO:tasks.rados.rados.0.smithi122.stdout:42: writing smithi12223906-42 from 689087 to 1156061 tid 1 2024-09-15T03:45:36.440 INFO:tasks.rados.rados.0.smithi122.stdout:42: writing smithi12223906-42 from 1699644 to 2483976 tid 2 2024-09-15T03:45:36.440 INFO:tasks.rados.rados.0.smithi122.stdout:43: write initial oid 43 2024-09-15T03:45:36.440 INFO:tasks.rados.rados.0.smithi122.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2024-09-15T03:45:36.446 INFO:tasks.rados.rados.0.smithi122.stdout:43: writing smithi12223906-43 from 705894 to 1248117 tid 1 2024-09-15T03:45:36.453 INFO:tasks.rados.rados.0.smithi122.stdout:43: writing smithi12223906-43 from 1758126 to 2402469 tid 2 2024-09-15T03:45:36.453 INFO:tasks.rados.rados.0.smithi122.stdout:44: write initial oid 44 2024-09-15T03:45:36.453 INFO:tasks.rados.rados.0.smithi122.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2024-09-15T03:45:36.459 INFO:tasks.rados.rados.0.smithi122.stdout:44: writing smithi12223906-44 from 722701 to 1340173 tid 1 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout:44: writing smithi12223906-44 from 1816608 to 2604609 tid 2 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout:29: finishing write tid 4 to smithi12223906-29 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 9 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout:29: done (15 left) 2024-09-15T03:45:36.467 INFO:tasks.rados.rados.0.smithi122.stdout:45: write initial oid 45 2024-09-15T03:45:36.468 INFO:tasks.rados.rados.0.smithi122.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2024-09-15T03:45:36.474 INFO:tasks.rados.rados.0.smithi122.stdout:45: writing smithi12223906-45 from 739508 to 1432229 tid 1 2024-09-15T03:45:36.481 INFO:tasks.rados.rados.0.smithi122.stdout:45: writing smithi12223906-45 from 1875090 to 2523102 tid 2 2024-09-15T03:45:36.481 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.481 INFO:tasks.rados.rados.0.smithi122.stdout:30: finishing write tid 1 to smithi12223906-30 2024-09-15T03:45:36.481 INFO:tasks.rados.rados.0.smithi122.stdout:30: finishing write tid 2 to smithi12223906-30 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:30: finishing write tid 3 to smithi12223906-30 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:30: finishing write tid 4 to smithi12223906-30 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 3 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:31: finishing write tid 1 to smithi12223906-31 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:31: finishing write tid 2 to smithi12223906-31 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:31: finishing write tid 3 to smithi12223906-31 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:31: finishing write tid 4 to smithi12223906-31 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:32: finishing write tid 1 to smithi12223906-32 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:32: finishing write tid 2 to smithi12223906-32 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:32: finishing write tid 3 to smithi12223906-32 2024-09-15T03:45:36.482 INFO:tasks.rados.rados.0.smithi122.stdout:32: finishing write tid 4 to smithi12223906-32 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 15 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:30: done (15 left) 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:31: done (14 left) 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:32: done (13 left) 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:46: write initial oid 46 2024-09-15T03:45:36.483 INFO:tasks.rados.rados.0.smithi122.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2024-09-15T03:45:36.488 INFO:tasks.rados.rados.0.smithi122.stdout:46: writing smithi12223906-46 from 756315 to 1524285 tid 1 2024-09-15T03:45:36.495 INFO:tasks.rados.rados.0.smithi122.stdout:46: writing smithi12223906-46 from 1933572 to 2725242 tid 2 2024-09-15T03:45:36.495 INFO:tasks.rados.rados.0.smithi122.stdout:47: write initial oid 47 2024-09-15T03:45:36.495 INFO:tasks.rados.rados.0.smithi122.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2024-09-15T03:45:36.502 INFO:tasks.rados.rados.0.smithi122.stdout:47: writing smithi12223906-47 from 773122 to 1332694 tid 1 2024-09-15T03:45:36.509 INFO:tasks.rados.rados.0.smithi122.stdout:47: writing smithi12223906-47 from 1992054 to 2643735 tid 2 2024-09-15T03:45:36.509 INFO:tasks.rados.rados.0.smithi122.stdout:48: write initial oid 48 2024-09-15T03:45:36.509 INFO:tasks.rados.rados.0.smithi122.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2024-09-15T03:45:36.516 INFO:tasks.rados.rados.0.smithi122.stdout:48: writing smithi12223906-48 from 789929 to 1424750 tid 1 2024-09-15T03:45:36.526 INFO:tasks.rados.rados.0.smithi122.stdout:48: writing smithi12223906-48 from 2050536 to 2789929 tid 2 2024-09-15T03:45:36.526 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.526 INFO:tasks.rados.rados.0.smithi122.stdout:33: finishing write tid 1 to smithi12223906-33 2024-09-15T03:45:36.526 INFO:tasks.rados.rados.0.smithi122.stdout:33: finishing write tid 2 to smithi12223906-33 2024-09-15T03:45:36.526 INFO:tasks.rados.rados.0.smithi122.stdout:33: finishing write tid 3 to smithi12223906-33 2024-09-15T03:45:36.526 INFO:tasks.rados.rados.0.smithi122.stdout:33: finishing write tid 4 to smithi12223906-33 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 18 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:34: finishing write tid 1 to smithi12223906-34 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:34: finishing write tid 2 to smithi12223906-34 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:34: finishing write tid 3 to smithi12223906-34 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:34: finishing write tid 4 to smithi12223906-34 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:35: finishing write tid 1 to smithi12223906-35 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:35: finishing write tid 2 to smithi12223906-35 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:35: finishing write tid 3 to smithi12223906-35 2024-09-15T03:45:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:35: finishing write tid 4 to smithi12223906-35 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 3 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:36: finishing write tid 1 to smithi12223906-36 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:36: finishing write tid 2 to smithi12223906-36 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:36: finishing write tid 3 to smithi12223906-36 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:36: finishing write tid 4 to smithi12223906-36 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 15 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:37: finishing write tid 1 to smithi12223906-37 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:37: finishing write tid 2 to smithi12223906-37 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:37: finishing write tid 3 to smithi12223906-37 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:37: finishing write tid 4 to smithi12223906-37 2024-09-15T03:45:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 9 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:33: done (15 left) 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:34: done (14 left) 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:35: done (13 left) 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:36: done (12 left) 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:37: done (11 left) 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:49: write initial oid 49 2024-09-15T03:45:36.529 INFO:tasks.rados.rados.0.smithi122.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2024-09-15T03:45:36.531 INFO:tasks.rados.rados.0.smithi122.stdout:49: writing smithi12223906-49 from 406736 to 1116806 tid 1 2024-09-15T03:45:36.538 INFO:tasks.rados.rados.0.smithi122.stdout:49: writing smithi12223906-49 from 1709018 to 2364368 tid 2 2024-09-15T03:45:36.538 INFO:tasks.rados.rados.0.smithi122.stdout:50: write initial oid 50 2024-09-15T03:45:36.538 INFO:tasks.rados.rados.0.smithi122.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2024-09-15T03:45:36.544 INFO:tasks.rados.rados.0.smithi122.stdout:50: writing smithi12223906-50 from 423543 to 1208862 tid 1 2024-09-15T03:45:36.552 INFO:tasks.rados.rados.0.smithi122.stdout:50: writing smithi12223906-50 from 1767500 to 2566508 tid 2 2024-09-15T03:45:36.552 INFO:tasks.rados.rados.0.smithi122.stdout:51: write (excl) oid 29 current snap is 0 2024-09-15T03:45:36.552 INFO:tasks.rados.rados.0.smithi122.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2024-09-15T03:45:36.556 INFO:tasks.rados.rados.0.smithi122.stdout:51: writing smithi12223906-29 from 440350 to 900918 tid 1 2024-09-15T03:45:36.562 INFO:tasks.rados.rados.0.smithi122.stdout:51: writing smithi12223906-29 from 1309629 to 1852295 tid 2 2024-09-15T03:45:36.568 INFO:tasks.rados.rados.0.smithi122.stdout:51: writing smithi12223906-29 from 2323973 to 2840350 tid 3 2024-09-15T03:45:36.568 INFO:tasks.rados.rados.0.smithi122.stdout:52: read oid 14 snap -1 2024-09-15T03:45:36.568 INFO:tasks.rados.rados.0.smithi122.stdout:52: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-15T03:45:36.568 INFO:tasks.rados.rados.0.smithi122.stdout:53: write (excl) oid 16 current snap is 0 2024-09-15T03:45:36.568 INFO:tasks.rados.rados.0.smithi122.stdout:38: finishing write tid 1 to smithi12223906-38 2024-09-15T03:45:36.568 INFO:tasks.rados.rados.0.smithi122.stdout:53: seq_num 51 ranges {457157=535817,1768111=402677} 2024-09-15T03:45:36.573 INFO:tasks.rados.rados.0.smithi122.stdout:53: writing smithi12223906-16 from 457157 to 992974 tid 1 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:53: writing smithi12223906-16 from 1768111 to 2170788 tid 2 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:38: finishing write tid 2 to smithi12223906-38 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:38: finishing write tid 3 to smithi12223906-38 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:38: finishing write tid 4 to smithi12223906-38 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 9 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:39: finishing write tid 1 to smithi12223906-39 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:38: done (15 left) 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:54: rollback oid 10 current snap is 0 2024-09-15T03:45:36.584 INFO:tasks.rados.rados.0.smithi122.stdout:54: done (15 left) 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:55: read oid 10 snap -1 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:55: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 16 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:39: finishing write tid 2 to smithi12223906-39 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:39: finishing write tid 3 to smithi12223906-39 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:39: finishing write tid 4 to smithi12223906-39 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 18 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:40: finishing write tid 1 to smithi12223906-40 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:40: finishing write tid 2 to smithi12223906-40 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:39: done (15 left) 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:56: snap_create 2024-09-15T03:45:36.585 INFO:tasks.rados.rados.0.smithi122.stdout:40: finishing write tid 3 to smithi12223906-40 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:40: finishing write tid 4 to smithi12223906-40 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:41: finishing write tid 1 to smithi12223906-41 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:41: finishing write tid 2 to smithi12223906-41 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:41: finishing write tid 3 to smithi12223906-41 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:41: finishing write tid 4 to smithi12223906-41 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 9 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-15T03:45:36.586 INFO:tasks.rados.rados.0.smithi122.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:42: finishing write tid 1 to smithi12223906-42 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:42: finishing write tid 2 to smithi12223906-42 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:42: finishing write tid 3 to smithi12223906-42 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:42: finishing write tid 4 to smithi12223906-42 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 21 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:43: finishing write tid 1 to smithi12223906-43 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:43: finishing write tid 2 to smithi12223906-43 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:43: finishing write tid 3 to smithi12223906-43 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:43: finishing write tid 4 to smithi12223906-43 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 24 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:44: finishing write tid 1 to smithi12223906-44 2024-09-15T03:45:36.587 INFO:tasks.rados.rados.0.smithi122.stdout:44: finishing write tid 2 to smithi12223906-44 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:44: finishing write tid 3 to smithi12223906-44 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:44: finishing write tid 4 to smithi12223906-44 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 21 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:45: finishing write tid 1 to smithi12223906-45 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:45: finishing write tid 2 to smithi12223906-45 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:45: finishing write tid 3 to smithi12223906-45 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:45: finishing write tid 4 to smithi12223906-45 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 9 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:46: finishing write tid 1 to smithi12223906-46 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:46: finishing write tid 2 to smithi12223906-46 2024-09-15T03:45:36.588 INFO:tasks.rados.rados.0.smithi122.stdout:46: finishing write tid 3 to smithi12223906-46 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:46: finishing write tid 4 to smithi12223906-46 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 12 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:47: finishing write tid 1 to smithi12223906-47 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:47: finishing write tid 2 to smithi12223906-47 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:47: finishing write tid 3 to smithi12223906-47 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:47: finishing write tid 4 to smithi12223906-47 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 6 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:48: finishing write tid 1 to smithi12223906-48 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:48: finishing write tid 2 to smithi12223906-48 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:48: finishing write tid 3 to smithi12223906-48 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:48: finishing write tid 4 to smithi12223906-48 2024-09-15T03:45:36.589 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 12 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:49: finishing write tid 1 to smithi12223906-49 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:49: finishing write tid 2 to smithi12223906-49 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:49: finishing write tid 3 to smithi12223906-49 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:49: finishing write tid 4 to smithi12223906-49 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:50: finishing write tid 1 to smithi12223906-50 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:50: finishing write tid 2 to smithi12223906-50 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:50: finishing write tid 3 to smithi12223906-50 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:50: finishing write tid 4 to smithi12223906-50 2024-09-15T03:45:36.590 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 24 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:51: finishing write tid 1 to smithi12223906-29 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:51: finishing write tid 2 to smithi12223906-29 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:51: finishing write tid 3 to smithi12223906-29 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:51: finishing write tid 4 to smithi12223906-29 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:51: finishing write tid 5 to smithi12223906-29 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 13 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:51: left oid 29 (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:45:36.591 INFO:tasks.rados.rados.0.smithi122.stdout:52: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-15T03:45:36.658 INFO:tasks.rados.rados.0.smithi122.stdout:53: finishing write tid 1 to smithi12223906-16 2024-09-15T03:45:36.658 INFO:tasks.rados.rados.0.smithi122.stdout:53: finishing write tid 2 to smithi12223906-16 2024-09-15T03:45:36.658 INFO:tasks.rados.rados.0.smithi122.stdout:53: finishing write tid 3 to smithi12223906-16 2024-09-15T03:45:36.658 INFO:tasks.rados.rados.0.smithi122.stdout:53: finishing write tid 4 to smithi12223906-16 2024-09-15T03:45:36.658 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 15 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-15T03:45:36.658 INFO:tasks.rados.rados.0.smithi122.stdout:53: left oid 16 (ObjNum 51 snap 0 seq_num 51) 2024-09-15T03:45:36.659 INFO:tasks.rados.rados.0.smithi122.stdout:55: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:45:36.905 INFO:tasks.rados.rados.0.smithi122.stdout:40: done (15 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:41: done (14 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:42: done (13 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:43: done (12 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:44: done (11 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:45: done (10 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:46: done (9 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:47: done (8 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:48: done (7 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:49: done (6 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:50: done (5 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:51: done (4 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:52: done (3 left) 2024-09-15T03:45:36.906 INFO:tasks.rados.rados.0.smithi122.stdout:53: done (2 left) 2024-09-15T03:45:36.907 INFO:tasks.rados.rados.0.smithi122.stdout:55: done (1 left) 2024-09-15T03:45:36.907 INFO:tasks.rados.rados.0.smithi122.stdout:56: done (0 left) 2024-09-15T03:45:36.907 INFO:tasks.rados.rados.0.smithi122.stdout:57: read oid 14 snap -1 2024-09-15T03:45:36.907 INFO:tasks.rados.rados.0.smithi122.stdout:57: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-15T03:45:36.907 INFO:tasks.rados.rados.0.smithi122.stdout:58: write (excl) oid 13 current snap is 1 2024-09-15T03:45:36.907 INFO:tasks.rados.rados.0.smithi122.stdout:58: seq_num 52 ranges {473964=611066,1826593=546335,2778819=95145} 2024-09-15T03:45:36.916 INFO:tasks.rados.rados.0.smithi122.stdout:58: writing smithi12223906-13 from 473964 to 1085030 tid 1 2024-09-15T03:45:36.922 INFO:tasks.rados.rados.0.smithi122.stdout:58: writing smithi12223906-13 from 1826593 to 2372928 tid 2 2024-09-15T03:45:36.925 INFO:tasks.rados.rados.0.smithi122.stdout:58: writing smithi12223906-13 from 2778819 to 2873964 tid 3 2024-09-15T03:45:36.925 INFO:tasks.rados.rados.0.smithi122.stdout:59: write oid 9 current snap is 1 2024-09-15T03:45:36.925 INFO:tasks.rados.rados.0.smithi122.stdout:59: seq_num 53 ranges {490771=686315,1885075=406346} 2024-09-15T03:45:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:59: writing smithi12223906-9 from 490771 to 1177086 tid 1 2024-09-15T03:45:36.938 INFO:tasks.rados.rados.0.smithi122.stdout:59: writing smithi12223906-9 from 1885075 to 2291421 tid 2 2024-09-15T03:45:36.938 INFO:tasks.rados.rados.0.smithi122.stdout:60: delete oid 33 current snap is 1 2024-09-15T03:45:36.938 INFO:tasks.rados.rados.0.smithi122.stdout:57: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-15T03:45:37.018 INFO:tasks.rados.rados.0.smithi122.stdout:58: finishing write tid 1 to smithi12223906-13 2024-09-15T03:45:37.018 INFO:tasks.rados.rados.0.smithi122.stdout:58: finishing write tid 2 to smithi12223906-13 2024-09-15T03:45:37.019 INFO:tasks.rados.rados.0.smithi122.stdout:58: finishing write tid 3 to smithi12223906-13 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:58: finishing write tid 4 to smithi12223906-13 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:58: finishing write tid 5 to smithi12223906-13 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 17 (ObjNum 52 snap 1 seq_num 52) dirty exists 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:58: left oid 13 (ObjNum 52 snap 1 seq_num 52) 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:59: finishing write tid 1 to smithi12223906-9 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:59: finishing write tid 2 to smithi12223906-9 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:59: finishing write tid 3 to smithi12223906-9 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:59: finishing write tid 4 to smithi12223906-9 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 10 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:59: left oid 9 (ObjNum 53 snap 1 seq_num 53) 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:57: done (3 left) 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:58: done (2 left) 2024-09-15T03:45:37.020 INFO:tasks.rados.rados.0.smithi122.stdout:59: done (1 left) 2024-09-15T03:45:37.021 INFO:tasks.rados.rados.0.smithi122.stdout:60: done (0 left) 2024-09-15T03:45:37.021 INFO:tasks.rados.rados.0.smithi122.stdout:61: rollback oid 8 current snap is 1 2024-09-15T03:45:37.021 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 0 2024-09-15T03:45:37.021 INFO:tasks.rados.rados.0.smithi122.stdout:62: write oid 13 current snap is 1 2024-09-15T03:45:37.021 INFO:tasks.rados.rados.0.smithi122.stdout:62: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2024-09-15T03:45:37.032 INFO:tasks.rados.rados.0.smithi122.stdout:62: writing smithi12223906-13 from 524385 to 1077551 tid 1 2024-09-15T03:45:37.037 INFO:tasks.rados.rados.0.smithi122.stdout:62: writing smithi12223906-13 from 1602039 to 2012054 tid 2 2024-09-15T03:45:37.042 INFO:tasks.rados.rados.0.smithi122.stdout:62: writing smithi12223906-13 from 2577441 to 2924385 tid 3 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:63: write (excl) oid 2 current snap is 1 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:61: finishing rollback tid 0 to smithi12223906-8 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:61: finishing rollback tid 1 to smithi12223906-8 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:61: finishing rollback tid 2 to smithi12223906-8 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 13 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:62: finishing write tid 1 to smithi12223906-13 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:62: finishing write tid 2 to smithi12223906-13 2024-09-15T03:45:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:63: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2024-09-15T03:45:37.049 INFO:tasks.rados.rados.0.smithi122.stdout:63: writing smithi12223906-2 from 541192 to 1169607 tid 1 2024-09-15T03:45:37.056 INFO:tasks.rados.rados.0.smithi122.stdout:63: writing smithi12223906-2 from 1660521 to 2214194 tid 2 2024-09-15T03:45:37.060 INFO:tasks.rados.rados.0.smithi122.stdout:63: writing smithi12223906-2 from 2888511 to 2941192 tid 3 2024-09-15T03:45:37.060 INFO:tasks.rados.rados.0.smithi122.stdout:61: done (2 left) 2024-09-15T03:45:37.060 INFO:tasks.rados.rados.0.smithi122.stdout:64: write oid 31 current snap is 1 2024-09-15T03:45:37.060 INFO:tasks.rados.rados.0.smithi122.stdout:64: seq_num 57 ranges {557999=703664,1719003=413684,2632287=325712} 2024-09-15T03:45:37.067 INFO:tasks.rados.rados.0.smithi122.stdout:64: writing smithi12223906-31 from 557999 to 1261663 tid 1 2024-09-15T03:45:37.072 INFO:tasks.rados.rados.0.smithi122.stdout:64: writing smithi12223906-31 from 1719003 to 2132687 tid 2 2024-09-15T03:45:37.079 INFO:tasks.rados.rados.0.smithi122.stdout:64: writing smithi12223906-31 from 2632287 to 2957999 tid 3 2024-09-15T03:45:37.079 INFO:tasks.rados.rados.0.smithi122.stdout:65: write oid 40 current snap is 1 2024-09-15T03:45:37.079 INFO:tasks.rados.rados.0.smithi122.stdout:62: finishing write tid 3 to smithi12223906-13 2024-09-15T03:45:37.079 INFO:tasks.rados.rados.0.smithi122.stdout:65: seq_num 58 ranges {574806=778913,2061132=557342} 2024-09-15T03:45:37.085 INFO:tasks.rados.rados.0.smithi122.stdout:65: writing smithi12223906-40 from 574806 to 1353719 tid 1 2024-09-15T03:45:37.092 INFO:tasks.rados.rados.0.smithi122.stdout:65: writing smithi12223906-40 from 2061132 to 2618474 tid 2 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:66: snap_create 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:62: finishing write tid 4 to smithi12223906-13 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:62: finishing write tid 5 to smithi12223906-13 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 21 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:62: left oid 13 (ObjNum 55 snap 1 seq_num 55) 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:63: finishing write tid 1 to smithi12223906-2 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:63: finishing write tid 2 to smithi12223906-2 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:63: finishing write tid 3 to smithi12223906-2 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:63: finishing write tid 4 to smithi12223906-2 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:63: finishing write tid 5 to smithi12223906-2 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 14 (ObjNum 56 snap 1 seq_num 56) dirty exists 2024-09-15T03:45:37.093 INFO:tasks.rados.rados.0.smithi122.stdout:63: left oid 2 (ObjNum 56 snap 1 seq_num 56) 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:64: finishing write tid 1 to smithi12223906-31 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:64: finishing write tid 2 to smithi12223906-31 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:64: finishing write tid 3 to smithi12223906-31 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:64: finishing write tid 4 to smithi12223906-31 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:64: finishing write tid 5 to smithi12223906-31 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 11 (ObjNum 57 snap 1 seq_num 57) dirty exists 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:64: left oid 31 (ObjNum 57 snap 1 seq_num 57) 2024-09-15T03:45:37.094 INFO:tasks.rados.rados.0.smithi122.stdout:65: finishing write tid 1 to smithi12223906-40 2024-09-15T03:45:37.096 INFO:tasks.rados.rados.0.smithi122.stdout:65: finishing write tid 2 to smithi12223906-40 2024-09-15T03:45:37.097 INFO:tasks.rados.rados.0.smithi122.stdout:65: finishing write tid 3 to smithi12223906-40 2024-09-15T03:45:37.097 INFO:tasks.rados.rados.0.smithi122.stdout:65: finishing write tid 4 to smithi12223906-40 2024-09-15T03:45:37.097 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 13 (ObjNum 58 snap 1 seq_num 58) dirty exists 2024-09-15T03:45:37.097 INFO:tasks.rados.rados.0.smithi122.stdout:65: left oid 40 (ObjNum 58 snap 1 seq_num 58) 2024-09-15T03:45:37.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:37.135+0000 7fa643fff700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa64412dd10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:37.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:37.135+0000 7fa648b0a700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa6441485e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:37.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:37.135+0000 7fa6437fe700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa62c0116c0 legacy=0x7fa644144eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:62: done (4 left) 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:63: done (3 left) 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:64: done (2 left) 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:65: done (1 left) 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:66: done (0 left) 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:67: rollback oid 17 current snap is 2 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 1 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:68: rollback oid 13 current snap is 2 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 1 2024-09-15T03:45:37.914 INFO:tasks.rados.rados.0.smithi122.stdout:69: snap_create 2024-09-15T03:45:37.919 INFO:tasks.rados.rados.0.smithi122.stdout:68: finishing rollback tid 0 to smithi12223906-13 2024-09-15T03:45:37.920 INFO:tasks.rados.rados.0.smithi122.stdout:68: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:45:37.920 INFO:tasks.rados.rados.0.smithi122.stdout:68: finishing rollback tid 2 to smithi12223906-13 2024-09-15T03:45:37.920 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 25 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-15T03:45:37.922 INFO:tasks.rados.rados.0.smithi122.stdout:67: finishing rollback tid 0 to smithi12223906-17 2024-09-15T03:45:37.922 INFO:tasks.rados.rados.0.smithi122.stdout:67: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:45:37.923 INFO:tasks.rados.rados.0.smithi122.stdout:67: finishing rollback tid 2 to smithi12223906-17 2024-09-15T03:45:37.924 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 18 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient(hunting): tick 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 monclient(hunting): continuing hunt 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa62c0116c0 legacy=0x7fa644144eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa62c00ac40 legacy=0x7fa64412dd10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fa62c011a90 legacy=0x7fa6441485e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient: _add_conns ranks=[3,0,5,8,4,6] 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient(hunting): picked mon.noname-g con 0x7fa6200037d0 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient(hunting): picked mon.noname-a con 0x7fa620007450 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient(hunting): picked mon.noname-h con 0x7fa62000b1d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644063cd0 con 0x7fa620007450 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644097ea0 con 0x7fa6200037d0 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64411db70 con 0x7fa62000b1d0 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63affd700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa6437fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa620007450 0x7fa620009900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40598/0 (socket says 172.21.15.26:40598) 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa6437fe700 1 -- 172.21.15.26:0/1035890052 learned_addr learned my addr 172.21.15.26:0/1035890052 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121657071 0 0) 0x7fa644063cd0 con 0x7fa620007450 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): my global_id is 4446 2024-09-15T03:45:38.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c00a950 con 0x7fa620007450 2024-09-15T03:45:38.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148202890 0 0) 0x7fa64411db70 con 0x7fa62000b1d0 2024-09-15T03:45:38.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:38.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): my global_id is 4550 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa644063cd0 con 0x7fa62000b1d0 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709123984 0 0) 0x7fa644097ea0 con 0x7fa6200037d0 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): my global_id is 4548 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa64411db70 con 0x7fa6200037d0 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4009761486 0 0) 0x7fa62c00a950 con 0x7fa620007450 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa644097ea0 con 0x7fa620007450 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa630001f70 con 0x7fa620007450 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): dump: 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 504393927 0 0) 0x7fa644063cd0 con 0x7fa62000b1d0 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa62c00a950 con 0x7fa62000b1d0 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa63c004710 con 0x7fa62000b1d0 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient(hunting): dump: 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1392742716 0 0) 0x7fa64411db70 con 0x7fa6200037d0 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa644063cd0 con 0x7fa6200037d0 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1150552608 0 0) 0x7fa644097ea0 con 0x7fa620007450 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 >> v1:172.21.15.122:6791/0 conn(0x7fa62000b1d0 legacy=0x7fa62000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 >> v1:172.21.15.122:6790/0 conn(0x7fa6200037d0 legacy=0x7fa620005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 monclient: found mon.a 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644098300 con 0x7fa620007450 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: _finish_auth 0 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa648b0a700 1 -- 172.21.15.26:0/1035890052 reap_dead start 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 5 monclient: authenticate success, global_id 4446 2024-09-15T03:45:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 10 monclient: _renew_subs 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64411cf00 con 0x7fa620007450 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 10 monclient: _renew_subs 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6440635a0 con 0x7fa620007450 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa630002440 con 0x7fa620007450 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa630002970 con 0x7fa620007450 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa6427fc700 4 set_mon_vals no callback set 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.723+0000 7fa63bfff700 10 monclient: dump: 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:38.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa630002bd0 con 0x7fa620007450 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(23..23 src has 1..23) v4 ==== 3867+0+0 (unknown 1134405633 0 0) 0x7fa630043f90 con 0x7fa620007450 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa64930b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa64930b700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa64930b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6240050b0 con 0x7fa620007450 2024-09-15T03:45:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.727+0000 7fa643fff700 10 monclient: get_auth_request con 0x7fa62c039d50 auth_method 0 2024-09-15T03:45:38.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.739+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fa630014fb0 con 0x7fa620007450 2024-09-15T03:45:38.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.739+0000 7fa63bfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:38.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.739+0000 7fa63bfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:38.920 INFO:tasks.rados.rados.0.smithi122.stdout:67: done (2 left) 2024-09-15T03:45:38.920 INFO:tasks.rados.rados.0.smithi122.stdout:68: done (1 left) 2024-09-15T03:45:38.920 INFO:tasks.rados.rados.0.smithi122.stdout:69: done (0 left) 2024-09-15T03:45:38.920 INFO:tasks.rados.rados.0.smithi122.stdout:70: read oid 27 snap -1 2024-09-15T03:45:38.920 INFO:tasks.rados.rados.0.smithi122.stdout:70: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-15T03:45:38.920 INFO:tasks.rados.rados.0.smithi122.stdout:71: snap_remove snap 0 2024-09-15T03:45:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa64930b700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:45:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa64930b700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:45:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa64930b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:45:38.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa6240058e0 con 0x7fa620007450 2024-09-15T03:45:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa63bfff700 1 -- 172.21.15.26:0/1035890052 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2676 (unknown 254746657 0 718170527) 0x7fa6300149b0 con 0x7fa620007450 2024-09-15T03:45:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa63bfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:45:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.971+0000 7fa63bfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:38.975 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":28,"quorum":[0,3,4,5,6],"quorum_names":["a","g","c","h","d"],"quorum_leader_name":"d","quorum_age":6,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 >> v1:172.21.15.122:6812/20895 conn(0x7fa62c039d50 legacy=0x7fa62c03c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 10 monclient: shutdown 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 >> v1:172.21.15.26:6789/0 conn(0x7fa620007450 legacy=0x7fa620009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 shutdown_connections 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 >> 172.21.15.26:0/1035890052 conn(0x7fa644071860 msgr2=0x7fa64409c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 shutdown_connections 2024-09-15T03:45:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:38.975+0000 7fa64930b700 1 -- 172.21.15.26:0/1035890052 wait complete. 2024-09-15T03:45:38.989 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-15T03:45:38.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-15T03:45:38.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:45:39.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.091+0000 7fdd8e393700 10 monclient: get_monmap_and_config 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.091+0000 7fdd8e393700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.091+0000 7fdd8e393700 10 monclient: monmap: 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:39.095449+0000 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:39.095449+0000 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.097 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.098 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 Processor -- start 2024-09-15T03:45:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- start start 2024-09-15T03:45:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient: init 2024-09-15T03:45:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient: _add_conns ranks=[1,7,5,4,6,8,0,3,2] 2024-09-15T03:45:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient(hunting): picked mon.noname-f con 0x7fdd881335e0 addr v1:172.21.15.122:6789/0 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient(hunting): picked mon.noname-i con 0x7fdd8809fc80 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient(hunting): picked mon.noname-h con 0x7fdd880a0050 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8804ca10 con 0x7fdd881335e0 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8806eab0 con 0x7fdd880a0050 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880a1330 con 0x7fdd8809fc80 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient(hunting): authenticate will time out at 883.524728s 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8cb90700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fdd8809fc80 legacy=0x7fdd88137ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8d391700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fdd881335e0 legacy=0x7fdd88132da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8db92700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7fdd880a0050 0x7fdd8813b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:50342/0 (socket says 172.21.15.26:50342) 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8db92700 1 -- 172.21.15.26:0/4047923750 learned_addr learned my addr 172.21.15.26:0/4047923750 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854827005 0 0) 0x7fdd8806eab0 con 0x7fdd880a0050 2024-09-15T03:45:39.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient(hunting): my global_id is 4559 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd74003570 con 0x7fdd880a0050 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378283025 0 0) 0x7fdd74003570 con 0x7fdd880a0050 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 >> v1:172.21.15.122:6792/0 conn(0x7fdd8809fc80 legacy=0x7fdd88137ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 >> v1:172.21.15.122:6789/0 conn(0x7fdd881335e0 legacy=0x7fdd88132da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 monclient: found mon.noname-h 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880a10d0 con 0x7fdd880a0050 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd80001b30 con 0x7fdd880a0050 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 5 monclient: authenticate success, global_id 4559 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: dump: 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdd80001d50 con 0x7fdd880a0050 2024-09-15T03:45:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 1 -- 172.21.15.26:0/4047923750 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd80004c70 con 0x7fdd880a0050 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 4 set_mon_vals no callback set 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd7ffff700 10 monclient: dump: 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 10 monclient: shutdown 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- 172.21.15.26:0/4047923750 >> v1:172.21.15.122:6791/0 conn(0x7fdd880a0050 legacy=0x7fdd8813b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- 172.21.15.26:0/4047923750 shutdown_connections 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- 172.21.15.26:0/4047923750 >> 172.21.15.26:0/4047923750 conn(0x7fdd8806e6e0 msgr2=0x7fdd880a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- 172.21.15.26:0/4047923750 shutdown_connections 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.095+0000 7fdd8e393700 1 -- 172.21.15.26:0/4047923750 wait complete. 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: monmap: 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 Processor -- start 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 -- start start 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: init 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: _add_conns ranks=[5,4,6,7,0,8,1,3,2] 2024-09-15T03:45:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient(hunting): picked mon.noname-h con 0x7fdd8809fc80 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient(hunting): picked mon.noname-c con 0x7fdd880a0050 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient(hunting): picked mon.noname-d con 0x7fdd881335e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880a1b70 con 0x7fdd880a0050 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8809cf10 con 0x7fdd8809fc80 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8811eae0 con 0x7fdd881335e0 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient(hunting): authenticate will time out at 883.529430s 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8cb90700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fdd880a0050 0x7fdd881421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:39392/0 (socket says 172.21.15.26:39392) 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8cb90700 1 -- 172.21.15.26:0/536853685 learned_addr learned my addr 172.21.15.26:0/536853685 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543220435 0 0) 0x7fdd880a1b70 con 0x7fdd880a0050 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): my global_id is 4468 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c003570 con 0x7fdd880a0050 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432028954 0 0) 0x7fdd8811eae0 con 0x7fdd881335e0 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): my global_id is 4515 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd880a1b70 con 0x7fdd881335e0 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607075830 0 0) 0x7fdd8809cf10 con 0x7fdd8809fc80 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): my global_id is 4568 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8811eae0 con 0x7fdd8809fc80 2024-09-15T03:45:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1986257721 0 0) 0x7fdd6c003570 con 0x7fdd880a0050 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd8809cf10 con 0x7fdd880a0050 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd78004380 con 0x7fdd880a0050 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): dump: 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3710917602 0 0) 0x7fdd880a1b70 con 0x7fdd881335e0 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd6c003570 con 0x7fdd881335e0 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd80001a30 con 0x7fdd881335e0 2024-09-15T03:45:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): dump: 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2145890124 0 0) 0x7fdd8811eae0 con 0x7fdd8809fc80 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd880a1b70 con 0x7fdd8809fc80 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd84004340 con 0x7fdd8809fc80 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient(hunting): dump: 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.123 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1336231690 0 0) 0x7fdd6c003570 con 0x7fdd881335e0 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.122:6791/0 conn(0x7fdd8809fc80 legacy=0x7fdd88141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.26:6791/0 conn(0x7fdd880a0050 legacy=0x7fdd881421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 monclient: found mon.d 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd881428b0 con 0x7fdd881335e0 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 5 monclient: authenticate success, global_id 4515 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: _renew_subs 2024-09-15T03:45:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd88120ce0 con 0x7fdd881335e0 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: _renew_subs 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8811f970 con 0x7fdd881335e0 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdd80004f30 con 0x7fdd881335e0 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd800053e0 con 0x7fdd881335e0 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7d7fa700 10 monclient: dump: 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.099+0000 7fdd7f7fe700 4 set_mon_vals no callback set 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd8e393700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdd68006590 con 0x7fdd68002840 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810786346 0 0) 0x7fdd68006590 con 0x7fdd68002840 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient(hunting): my global_id is 4455 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c003570 con 0x7fdd68002840 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdd8000fda0 con 0x7fdd881335e0 2024-09-15T03:45:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3867+0+0 (unknown 4092199876 0 0) 0x7fdd800441d0 con 0x7fdd881335e0 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 706772555 0 0) 0x7fdd6c003570 con 0x7fdd68002840 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd68006590 con 0x7fdd68002840 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd84004ad0 con 0x7fdd68002840 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient: dump: 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3130072493 0 0) 0x7fdd68006590 con 0x7fdd68002840 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd6800a130 con 0x7fdd68002840 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd8cb90700 10 monclient: get_auth_request con 0x7fdd6c0380b0 auth_method 0 2024-09-15T03:45:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdd84005080 con 0x7fdd68002840 2024-09-15T03:45:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.26:6789/0 conn(0x7fdd68002840 legacy=0x7fdd68004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.103+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.26:6789/0 conn(0x7fdd68002840 legacy=0x7fdd68004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdd68007520 con 0x7fdd680068d0 2024-09-15T03:45:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732212468 0 0) 0x7fdd68007520 con 0x7fdd680068d0 2024-09-15T03:45:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient(hunting): my global_id is 4464 2024-09-15T03:45:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd68006590 con 0x7fdd680068d0 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1064697687 0 0) 0x7fdd68006590 con 0x7fdd680068d0 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd68007520 con 0x7fdd680068d0 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd840058b0 con 0x7fdd680068d0 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient: dump: 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3338910690 0 0) 0x7fdd68007520 con 0x7fdd680068d0 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd68008760 con 0x7fdd680068d0 2024-09-15T03:45:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7524 (unknown 0 0 3772948216) 0x7fdd84005a60 con 0x7fdd680068d0 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.26:6789/0 conn(0x7fdd680068d0 legacy=0x7fdd68006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd7d7fa700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.26:6789/0 conn(0x7fdd680068d0 legacy=0x7fdd68006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:39.135 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 28, 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: 6 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.136 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.137 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:39.138 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.139 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:39.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.141 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.145 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:39.154 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.122:6812/20895 conn(0x7fdd6c0380b0 legacy=0x7fdd6c03a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 10 monclient: shutdown 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 >> v1:172.21.15.26:6792/0 conn(0x7fdd881335e0 legacy=0x7fdd88149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8db92700 1 -- 172.21.15.26:0/536853685 reap_dead start 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8db92700 1 -- 172.21.15.26:0/536853685 reap_dead start 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 shutdown_connections 2024-09-15T03:45:39.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 >> 172.21.15.26:0/536853685 conn(0x7fdd8806e6e0 msgr2=0x7fdd88130be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.111+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 shutdown_connections 2024-09-15T03:45:39.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.115+0000 7fdd8e393700 1 -- 172.21.15.26:0/536853685 wait complete. 2024-09-15T03:45:39.157 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:45:39.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon c 2024-09-15T03:45:39.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:45:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.271+0000 7ff28da40700 10 monclient: get_monmap_and_config 2024-09-15T03:45:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.271+0000 7ff28da40700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.271+0000 7ff28da40700 10 monclient: monmap: 2024-09-15T03:45:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:39.274607+0000 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:39.274607+0000 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.278 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 1 Processor -- start 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 1 -- start start 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient: init 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient: _add_conns ranks=[8,6,5,3,4,1,0,2,7] 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient(hunting): picked mon.noname-e con 0x7ff28812f270 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient(hunting): picked mon.noname-d con 0x7ff28806d5b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient(hunting): picked mon.noname-h con 0x7ff2881353b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28804ca10 con 0x7ff2881353b0 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28806aab0 con 0x7ff28806d5b0 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28809f050 con 0x7ff28812f270 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff286ffd700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7ff28812f270 legacy=0x7ff28812f640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient(hunting): authenticate will time out at 883.705784s 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff27ffff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ff28806d5b0 0x7ff28806d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54470/0 (socket says 172.21.15.26:54470) 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff27ffff700 1 -- 172.21.15.26:0/2349576643 learned_addr learned my addr 172.21.15.26:0/2349576643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893899741 0 0) 0x7ff28806aab0 con 0x7ff28806d5b0 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient(hunting): my global_id is 4524 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff274004680 con 0x7ff28806d5b0 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657504415 0 0) 0x7ff28804ca10 con 0x7ff2881353b0 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient(hunting): my global_id is 4577 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff28806aab0 con 0x7ff2881353b0 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013870142 0 0) 0x7ff274004680 con 0x7ff28806d5b0 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 >> v1:172.21.15.26:6793/0 conn(0x7ff28812f270 legacy=0x7ff28812f640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 >> v1:172.21.15.122:6791/0 conn(0x7ff2881353b0 legacy=0x7ff288137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 monclient: found mon.noname-d 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28806a7d0 con 0x7ff28806d5b0 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 5 monclient: authenticate success, global_id 4524 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff2700017b0 con 0x7ff28806d5b0 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: dump: 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff270001a90 con 0x7ff28806d5b0 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:39.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff28da40700 4 set_mon_vals no callback set 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 1 -- 172.21.15.26:0/2349576643 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff270004c30 con 0x7ff28806d5b0 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.275+0000 7ff2867fc700 10 monclient: dump: 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.287 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient: shutdown 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- 172.21.15.26:0/2349576643 >> v1:172.21.15.26:6792/0 conn(0x7ff28806d5b0 legacy=0x7ff28806d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- 172.21.15.26:0/2349576643 shutdown_connections 2024-09-15T03:45:39.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- 172.21.15.26:0/2349576643 >> 172.21.15.26:0/2349576643 conn(0x7ff288063e10 msgr2=0x7ff2880a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- 172.21.15.26:0/2349576643 shutdown_connections 2024-09-15T03:45:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- 172.21.15.26:0/2349576643 wait complete. 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient: monmap: 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 Processor -- start 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- start start 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient: init 2024-09-15T03:45:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient: _add_conns ranks=[0,6,5,4,3,7,2,8,1] 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient(hunting): picked mon.noname-a con 0x7ff28806d5b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient(hunting): picked mon.noname-d con 0x7ff28812f270 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient(hunting): picked mon.noname-h con 0x7ff2881353b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28809cd70 con 0x7ff28806d5b0 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff288069d50 con 0x7ff2881353b0 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2881349e0 con 0x7ff28812f270 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27ffff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ff28812f270 0x7ff288142a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54482/0 (socket says 172.21.15.26:54482) 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27ffff700 1 -- 172.21.15.26:0/3961525185 learned_addr learned my addr 172.21.15.26:0/3961525185 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff286ffd700 1 --1- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6789/0 conn(0x7ff28806d5b0 0x7ff28813da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40636/0 (socket says 172.21.15.26:40636) 2024-09-15T03:45:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff28da40700 10 monclient(hunting): authenticate will time out at 883.711393s 2024-09-15T03:45:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379697193 0 0) 0x7ff2881349e0 con 0x7ff28812f270 2024-09-15T03:45:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 10 monclient(hunting): my global_id is 4533 2024-09-15T03:45:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff268003570 con 0x7ff28812f270 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544643695 0 0) 0x7ff28809cd70 con 0x7ff28806d5b0 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 10 monclient(hunting): my global_id is 4473 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.279+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2881349e0 con 0x7ff28806d5b0 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301804148 0 0) 0x7ff288069d50 con 0x7ff2881353b0 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): my global_id is 4586 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff28809cd70 con 0x7ff2881353b0 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4262577212 0 0) 0x7ff268003570 con 0x7ff28812f270 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff288069d50 con 0x7ff28812f270 2024-09-15T03:45:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff270004b70 con 0x7ff28812f270 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): dump: 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2934908666 0 0) 0x7ff2881349e0 con 0x7ff28806d5b0 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff268003570 con 0x7ff28806d5b0 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff2780041f0 con 0x7ff28806d5b0 2024-09-15T03:45:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): dump: 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1758709084 0 0) 0x7ff288069d50 con 0x7ff28812f270 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.122:6791/0 conn(0x7ff2881353b0 legacy=0x7ff288146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6789/0 conn(0x7ff28806d5b0 legacy=0x7ff28813da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 monclient: found mon.d 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288132e70 con 0x7ff28812f270 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 5 monclient: authenticate success, global_id 4533 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 10 monclient: _renew_subs 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff288134420 con 0x7ff28812f270 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 10 monclient: _renew_subs 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff288133530 con 0x7ff28812f270 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff270004f00 con 0x7ff28812f270 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff2700053b0 con 0x7ff28812f270 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff285ffb700 4 set_mon_vals no callback set 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: dump: 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff27000fdf0 con 0x7ff28812f270 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff26c006590 con 0x7ff26c002840 2024-09-15T03:45:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3867+0+0 (unknown 4092199876 0 0) 0x7ff270044220 con 0x7ff28812f270 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323208601 0 0) 0x7ff26c006590 con 0x7ff26c002840 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient(hunting): my global_id is 4477 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff288069d50 con 0x7ff26c002840 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff2877fe700 10 monclient: get_auth_request con 0x7ff268039fb0 auth_method 0 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2179423859 0 0) 0x7ff288069d50 con 0x7ff26c002840 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff26c006590 con 0x7ff26c002840 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff2780049c0 con 0x7ff26c002840 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: dump: 2024-09-15T03:45:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.308 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1082143008 0 0) 0x7ff26c006590 con 0x7ff26c002840 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff26c00a130 con 0x7ff26c002840 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff278005000 con 0x7ff26c002840 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6791/0 conn(0x7ff26c002840 legacy=0x7ff26c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.283+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6791/0 conn(0x7ff26c002840 legacy=0x7ff26c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff26c007520 con 0x7ff26c0068d0 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911251511 0 0) 0x7ff26c007520 con 0x7ff26c0068d0 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient(hunting): my global_id is 4486 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff26c006590 con 0x7ff26c0068d0 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1937136767 0 0) 0x7ff26c006590 con 0x7ff26c0068d0 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff26c007520 con 0x7ff26c0068d0 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff278005810 con 0x7ff26c0068d0 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient: dump: 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2299375024 0 0) 0x7ff26c007520 con 0x7ff26c0068d0 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff26c008760 con 0x7ff26c0068d0 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7354 (unknown 0 0 2928364742) 0x7ff278005bb0 con 0x7ff26c0068d0 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6791/0 conn(0x7ff26c0068d0 legacy=0x7ff26c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff27effd700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6791/0 conn(0x7ff26c0068d0 legacy=0x7ff26c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 28, 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:39.314 INFO:teuthology.orchestra.run.smithi026.stdout: 6 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.315 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.316 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:39.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.318 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:39.319 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:39.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.322 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:39.323 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:39.324 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.325 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.328 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.329 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.330 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:39.331 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.333 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:39.333 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.122:6812/20895 conn(0x7ff268039fb0 legacy=0x7ff26803c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 10 monclient: shutdown 2024-09-15T03:45:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 >> v1:172.21.15.26:6792/0 conn(0x7ff28812f270 legacy=0x7ff288142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff2877fe700 1 -- 172.21.15.26:0/3961525185 reap_dead start 2024-09-15T03:45:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff2877fe700 1 -- 172.21.15.26:0/3961525185 reap_dead start 2024-09-15T03:45:39.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 shutdown_connections 2024-09-15T03:45:39.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 >> 172.21.15.26:0/3961525185 conn(0x7ff288063e10 msgr2=0x7ff28806c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 shutdown_connections 2024-09-15T03:45:39.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.299+0000 7ff28da40700 1 -- 172.21.15.26:0/3961525185 wait complete. 2024-09-15T03:45:39.335 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:45:39.335 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-09-15T03:45:39.335 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:45:39.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: get_monmap_and_config 2024-09-15T03:45:39.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: monmap: 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:39.452285+0000 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:39.452285+0000 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.455 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 1 Processor -- start 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 1 -- start start 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: init 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: _add_conns ranks=[0,6,3,8,1,7,2,4,5] 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient(hunting): picked mon.noname-a con 0x7fc4540a6880 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient(hunting): picked mon.noname-d con 0x7fc45409faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient(hunting): picked mon.noname-g con 0x7fc454138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45404ca10 con 0x7fc4540a6880 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540a0580 con 0x7fc454138d30 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc454133780 con 0x7fc45409faa0 2024-09-15T03:45:39.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient(hunting): authenticate will time out at 883.881688s 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc452ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fc45409faa0 0x7fc45409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54488/0 (socket says 172.21.15.26:54488) 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc452ffd700 1 -- 172.21.15.26:0/1318777439 learned_addr learned my addr 172.21.15.26:0/1318777439 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054817702 0 0) 0x7fc45404ca10 con 0x7fc4540a6880 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): my global_id is 4482 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc43c003570 con 0x7fc4540a6880 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212316928 0 0) 0x7fc454133780 con 0x7fc45409faa0 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): my global_id is 4542 2024-09-15T03:45:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc45404ca10 con 0x7fc45409faa0 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383635525 0 0) 0x7fc44c001490 con 0x7fc454138d30 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient(hunting): my global_id is 4557 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc454133780 con 0x7fc454138d30 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337450551 0 0) 0x7fc43c003570 con 0x7fc4540a6880 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 >> v1:172.21.15.26:6792/0 conn(0x7fc45409faa0 legacy=0x7fc45409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 >> v1:172.21.15.122:6790/0 conn(0x7fc454138d30 legacy=0x7fc45413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 monclient: found mon.noname-a 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:39.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4540a6c50 con 0x7fc4540a6880 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 5 monclient: authenticate success, global_id 4482 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc4480017b0 con 0x7fc4540a6880 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: dump: 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc4480019d0 con 0x7fc4540a6880 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 1 -- 172.21.15.26:0/1318777439 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc448002240 con 0x7fc4540a6880 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc458891700 4 set_mon_vals no callback set 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.451+0000 7fc4527fc700 10 monclient: dump: 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient: shutdown 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- 172.21.15.26:0/1318777439 >> v1:172.21.15.26:6789/0 conn(0x7fc4540a6880 legacy=0x7fc454133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- 172.21.15.26:0/1318777439 shutdown_connections 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- 172.21.15.26:0/1318777439 >> 172.21.15.26:0/1318777439 conn(0x7fc45406e2a0 msgr2=0x7fc4540a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- 172.21.15.26:0/1318777439 shutdown_connections 2024-09-15T03:45:39.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- 172.21.15.26:0/1318777439 wait complete. 2024-09-15T03:45:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient: monmap: 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 Processor -- start 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- start start 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient: init 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient: _add_conns ranks=[7,3,2,1,5,0,8,6,4] 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient(hunting): picked mon.noname-i con 0x7fc45409faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient(hunting): picked mon.noname-g con 0x7fc4540a6880 addr v1:172.21.15.122:6790/0 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient(hunting): picked mon.noname-b con 0x7fc454138d30 addr v1:172.21.15.26:6790/0 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540a18c0 con 0x7fc454138d30 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45409cf10 con 0x7fc4540a6880 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc454137220 con 0x7fc45409faa0 2024-09-15T03:45:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc458891700 10 monclient(hunting): authenticate will time out at 883.887390s 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc453fff700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fc454138d30 legacy=0x7fc45414a1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc4537fe700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fc45409faa0 legacy=0x7fc454135930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc452ffd700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fc4540a6880 0x7fc454136040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:52560/0 (socket says 172.21.15.26:52560) 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.455+0000 7fc452ffd700 1 -- 172.21.15.26:0/2593752649 learned_addr learned my addr 172.21.15.26:0/2593752649 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932177535 0 0) 0x7fc45409cf10 con 0x7fc4540a6880 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): my global_id is 4566 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc440003570 con 0x7fc4540a6880 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1872735271 0 0) 0x7fc440003570 con 0x7fc4540a6880 2024-09-15T03:45:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc45409cf10 con 0x7fc4540a6880 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc444001aa0 con 0x7fc4540a6880 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): dump: 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1019284516 0 0) 0x7fc45409cf10 con 0x7fc4540a6880 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.122:6792/0 conn(0x7fc45409faa0 legacy=0x7fc454135930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.26:6790/0 conn(0x7fc454138d30 legacy=0x7fc45414a1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 monclient: found mon.g 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45411c890 con 0x7fc4540a6880 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 5 monclient: authenticate success, global_id 4566 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 10 monclient: _renew_subs 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.122:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc454132980 con 0x7fc4540a6880 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 10 monclient: _renew_subs 2024-09-15T03:45:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.122:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc454137f90 con 0x7fc4540a6880 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc444001d00 con 0x7fc4540a6880 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc444004ee0 con 0x7fc4540a6880 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc451ffb700 4 set_mon_vals no callback set 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: dump: 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc434006590 con 0x7fc434002840 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485974990 0 0) 0x7fc434006590 con 0x7fc434002840 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient(hunting): my global_id is 4551 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc45409cf10 con 0x7fc434002840 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2953603599 0 0) 0x7fc45409cf10 con 0x7fc434002840 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc434006590 con 0x7fc434002840 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc448000f20 con 0x7fc434002840 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.459+0000 7fc43b7fe700 10 monclient: dump: 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.479 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc4440050c0 con 0x7fc4540a6880 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3849882795 0 0) 0x7fc434006590 con 0x7fc434002840 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc43400a130 con 0x7fc434002840 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.3 v1:172.21.15.122:6790/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3867+0+0 (unknown 4092199876 0 0) 0x7fc444043f50 con 0x7fc4540a6880 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc453fff700 10 monclient: get_auth_request con 0x7fc440037bd0 auth_method 0 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc448001420 con 0x7fc434002840 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.26:6792/0 conn(0x7fc434002840 legacy=0x7fc434004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.463+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.26:6792/0 conn(0x7fc434002840 legacy=0x7fc434004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc458891700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc434009f10 con 0x7fc434006930 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777955873 0 0) 0x7fc434009f10 con 0x7fc434006930 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient(hunting): my global_id is 4560 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc434006590 con 0x7fc434006930 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 244990395 0 0) 0x7fc434006590 con 0x7fc434006930 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc434009f10 con 0x7fc434006930 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc4480017b0 con 0x7fc434006930 2024-09-15T03:45:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient: dump: 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2054689038 0 0) 0x7fc434009f10 con 0x7fc434006930 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc434008760 con 0x7fc434006930 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7526 (unknown 0 0 664771181) 0x7fc448007720 con 0x7fc434006930 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.475+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.26:6792/0 conn(0x7fc434006930 legacy=0x7fc434006db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc43b7fe700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.26:6792/0 conn(0x7fc434006930 legacy=0x7fc434006db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 28, 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: 6 2024-09-15T03:45:39.485 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.486 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.487 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.489 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.491 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.492 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:39.493 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.494 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.495 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:39.496 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.497 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.498 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:39.500 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:39.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.502 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.503 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:39.504 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.122:6812/20895 conn(0x7fc440037bd0 legacy=0x7fc44003a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 10 monclient: shutdown 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 >> v1:172.21.15.122:6790/0 conn(0x7fc4540a6880 legacy=0x7fc454136040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc453fff700 1 -- 172.21.15.26:0/2593752649 reap_dead start 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc453fff700 1 -- 172.21.15.26:0/2593752649 reap_dead start 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 shutdown_connections 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 >> 172.21.15.26:0/2593752649 conn(0x7fc45406e2a0 msgr2=0x7fc45409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 shutdown_connections 2024-09-15T03:45:39.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.479+0000 7fc458891700 1 -- 172.21.15.26:0/2593752649 wait complete. 2024-09-15T03:45:39.507 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:45:39.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon g 2024-09-15T03:45:39.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient: get_monmap_and_config 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient: monmap: 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:39.625589+0000 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:39.625589+0000 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.628 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 1 Processor -- start 2024-09-15T03:45:39.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 1 -- start start 2024-09-15T03:45:39.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient: init 2024-09-15T03:45:39.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient: _add_conns ranks=[7,8,0,2,3,6,1,5,4] 2024-09-15T03:45:39.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient(hunting): picked mon.noname-i con 0x7f08d40a4b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient(hunting): picked mon.noname-e con 0x7f08d406b5c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient(hunting): picked mon.noname-a con 0x7f08d4139410 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d404ca10 con 0x7f08d4139410 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d4068dc0 con 0x7f08d40a4b40 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d40a06e0 con 0x7f08d406b5c0 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d95e1700 10 monclient(hunting): authenticate will time out at 884.055077s 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d2ffd700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f08d40a4b40 legacy=0x7f08d409aa90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d27fc700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f08d406b5c0 legacy=0x7f08d4137cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d37fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f08d4139410 0x7f08d413b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40648/0 (socket says 172.21.15.26:40648) 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d37fe700 1 -- 172.21.15.26:0/2646819708 learned_addr learned my addr 172.21.15.26:0/2646819708 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074788528 0 0) 0x7f08d404ca10 con 0x7f08d4139410 2024-09-15T03:45:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d1ffb700 10 monclient(hunting): my global_id is 4491 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.623+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08bc003570 con 0x7f08d4139410 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 296112295 0 0) 0x7f08bc003570 con 0x7f08d4139410 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 >> v1:172.21.15.26:6793/0 conn(0x7f08d406b5c0 legacy=0x7f08d4137cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 >> v1:172.21.15.122:6792/0 conn(0x7f08d40a4b40 legacy=0x7f08d409aa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 monclient: found mon.noname-a 2024-09-15T03:45:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:45:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40a4f10 con 0x7f08d4139410 2024-09-15T03:45:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f08c40017b0 con 0x7f08d4139410 2024-09-15T03:45:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 5 monclient: authenticate success, global_id 4491 2024-09-15T03:45:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: dump: 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f08c40019d0 con 0x7f08d4139410 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 1 -- 172.21.15.26:0/2646819708 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f08c40048f0 con 0x7f08d4139410 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 4 set_mon_vals no callback set 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d1ffb700 10 monclient: dump: 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.638 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 10 monclient: shutdown 2024-09-15T03:45:39.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 -- 172.21.15.26:0/2646819708 >> v1:172.21.15.26:6789/0 conn(0x7f08d4139410 legacy=0x7f08d413b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 -- 172.21.15.26:0/2646819708 shutdown_connections 2024-09-15T03:45:39.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 -- 172.21.15.26:0/2646819708 >> 172.21.15.26:0/2646819708 conn(0x7f08d4063a10 msgr2=0x7f08d40a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 -- 172.21.15.26:0/2646819708 shutdown_connections 2024-09-15T03:45:39.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 -- 172.21.15.26:0/2646819708 wait complete. 2024-09-15T03:45:39.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 10 monclient: monmap: 2024-09-15T03:45:39.645 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.645 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:39.645 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:39.645 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.646 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 Processor -- start 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 1 -- start start 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.627+0000 7f08d95e1700 10 monclient: init 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient: _add_conns ranks=[7,6,8,2,4,1,0,3,5] 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient(hunting): picked mon.noname-i con 0x7f08d40a4b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient(hunting): picked mon.noname-d con 0x7f08d4139410 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient(hunting): picked mon.noname-e con 0x7f08d4137300 addr v1:172.21.15.26:6793/0 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d409ca80 con 0x7f08d4139410 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d409b8d0 con 0x7f08d40a4b40 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d27fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f08d4139410 0x7f08d4146cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54522/0 (socket says 172.21.15.26:54522) 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d27fc700 1 -- 172.21.15.26:0/345621827 learned_addr learned my addr 172.21.15.26:0/345621827 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d2ffd700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f08d40a4b40 legacy=0x7f08d4131ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d37fe700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f08d4137300 legacy=0x7f08d414a3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d406e120 con 0x7f08d4137300 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient(hunting): authenticate will time out at 884.061411s 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147846876 0 0) 0x7f08d409ca80 con 0x7f08d4139410 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): my global_id is 4569 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c0003570 con 0x7f08d4139410 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 25682918 0 0) 0x7f08c0003570 con 0x7f08d4139410 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08d409ca80 con 0x7f08d4139410 2024-09-15T03:45:39.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f08c80043d0 con 0x7f08d4139410 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): dump: 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.649 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3595025980 0 0) 0x7f08d409ca80 con 0x7f08d4139410 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.26:6793/0 conn(0x7f08d4137300 legacy=0x7f08d414a3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.122:6792/0 conn(0x7f08d40a4b40 legacy=0x7f08d4131ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 monclient: found mon.d 2024-09-15T03:45:39.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d406e5e0 con 0x7f08d4139410 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 5 monclient: authenticate success, global_id 4569 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient: _renew_subs 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d4137700 con 0x7f08d4139410 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient: _renew_subs 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d41381a0 con 0x7f08d4139410 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f08c80045f0 con 0x7f08d4139410 2024-09-15T03:45:39.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f08c8004b20 con 0x7f08d4139410 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d17fa700 4 set_mon_vals no callback set 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08baffd700 10 monclient: dump: 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.652 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.653 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.631+0000 7f08d95e1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f08b4006590 con 0x7f08b4002840 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f08c8004c80 con 0x7f08d4139410 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3867+0+0 (unknown 4092199876 0 0) 0x7f08c8043ef0 con 0x7f08d4139410 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08d37fe700 10 monclient: get_auth_request con 0x7f08c0037540 auth_method 0 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672069563 0 0) 0x7f08b4006590 con 0x7f08b4002840 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient(hunting): my global_id is 4575 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d409ca80 con 0x7f08b4002840 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3225351882 0 0) 0x7f08d409ca80 con 0x7f08b4002840 2024-09-15T03:45:39.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08b4006590 con 0x7f08b4002840 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f08cc004520 con 0x7f08b4002840 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient: dump: 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.655 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2988240585 0 0) 0x7f08b4006590 con 0x7f08b4002840 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08b400a130 con 0x7f08b4002840 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f08cc0068d0 con 0x7f08b4002840 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:39.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.122:6790/0 conn(0x7f08b4002840 legacy=0x7f08b4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.635+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.122:6790/0 conn(0x7f08b4002840 legacy=0x7f08b4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.647+0000 7f08d95e1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.647+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f08b4009640 con 0x7f08b40090b0 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316251702 0 0) 0x7f08b4009640 con 0x7f08b40090b0 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient(hunting): my global_id is 4584 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b4006590 con 0x7f08b40090b0 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2238714315 0 0) 0x7f08b4006590 con 0x7f08b40090b0 2024-09-15T03:45:39.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08b4009640 con 0x7f08b40090b0 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f08cc0050c0 con 0x7f08b40090b0 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient: dump: 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.659 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1420254825 0 0) 0x7f08b4009640 con 0x7f08b40090b0 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f08b40098f0 con 0x7f08b40090b0 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7524 (unknown 0 0 2082743070) 0x7f08cc0066d0 con 0x7f08b40090b0 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.122:6790/0 conn(0x7f08b40090b0 legacy=0x7f08b4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08baffd700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.122:6790/0 conn(0x7f08b40090b0 legacy=0x7f08b4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 28, 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:39.661 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: 6 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.662 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.663 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.664 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.665 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.666 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.667 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:39.668 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.669 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.670 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.671 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:39.672 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:39.673 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.674 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.675 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:39.676 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.677 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.680 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 5 2024-09-15T03:45:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.681 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:39.681 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.122:6812/20895 conn(0x7f08c0037540 legacy=0x7f08c00399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 10 monclient: shutdown 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 >> v1:172.21.15.26:6792/0 conn(0x7f08d4139410 legacy=0x7f08d4146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d37fe700 1 -- 172.21.15.26:0/345621827 reap_dead start 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 shutdown_connections 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 >> 172.21.15.26:0/345621827 conn(0x7f08d4063a10 msgr2=0x7f08d4134f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 shutdown_connections 2024-09-15T03:45:39.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.651+0000 7f08d95e1700 1 -- 172.21.15.26:0/345621827 wait complete. 2024-09-15T03:45:39.683 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:45:39.683 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-09-15T03:45:39.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:45:39.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.795+0000 7fca61fea700 10 monclient: get_monmap_and_config 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.795+0000 7fca61fea700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.795+0000 7fca61fea700 10 monclient: monmap: 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:45:39.799455+0000 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:45:39.799455+0000 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.802 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 1 Processor -- start 2024-09-15T03:45:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 1 -- start start 2024-09-15T03:45:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient: init 2024-09-15T03:45:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient: _add_conns ranks=[4,0,5,3,7,6,2,1,8] 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient(hunting): picked mon.noname-c con 0x7fca5c067b20 addr v1:172.21.15.26:6791/0 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient(hunting): picked mon.noname-a con 0x7fca5c06e310 addr v1:172.21.15.26:6789/0 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient(hunting): picked mon.noname-h con 0x7fca5c06e6e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c04ca10 con 0x7fca5c06e310 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c09e700 con 0x7fca5c067b20 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c068e30 con 0x7fca5c06e6e0 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fca5c06e310 0x7fca5c137e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40664/0 (socket says 172.21.15.26:40664) 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5b7fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fca5c067b20 0x7fca5c067ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:39420/0 (socket says 172.21.15.26:39420) 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5affd700 1 -- 172.21.15.26:0/798464402 learned_addr learned my addr 172.21.15.26:0/798464402 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient(hunting): authenticate will time out at 884.229819s 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957788185 0 0) 0x7fca5c04ca10 con 0x7fca5c06e310 2024-09-15T03:45:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): my global_id is 4500 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca44003570 con 0x7fca5c06e310 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346307502 0 0) 0x7fca5c09e700 con 0x7fca5c067b20 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): my global_id is 4495 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5c04ca10 con 0x7fca5c067b20 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845011336 0 0) 0x7fca5c068e30 con 0x7fca5c06e6e0 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient(hunting): my global_id is 4595 2024-09-15T03:45:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5c09e700 con 0x7fca5c06e6e0 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020514540 0 0) 0x7fca5c04ca10 con 0x7fca5c067b20 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 >> v1:172.21.15.122:6791/0 conn(0x7fca5c06e6e0 legacy=0x7fca5c13b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 >> v1:172.21.15.26:6789/0 conn(0x7fca5c06e310 legacy=0x7fca5c137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 monclient: found mon.noname-c 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0a0a40 con 0x7fca5c067b20 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 5 monclient: authenticate success, global_id 4495 2024-09-15T03:45:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca4c001790 con 0x7fca5c067b20 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: dump: 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fca4c0019c0 con 0x7fca5c067b20 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 10 monclient: get_monmap_and_config success 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca61fea700 4 set_mon_vals no callback set 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 1 -- 172.21.15.26:0/798464402 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca4c004830 con 0x7fca5c067b20 2024-09-15T03:45:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.799+0000 7fca5a7fc700 10 monclient: dump: 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient: shutdown 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- 172.21.15.26:0/798464402 >> v1:172.21.15.26:6791/0 conn(0x7fca5c067b20 legacy=0x7fca5c067ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- 172.21.15.26:0/798464402 shutdown_connections 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- 172.21.15.26:0/798464402 >> 172.21.15.26:0/798464402 conn(0x7fca5c063d10 msgr2=0x7fca5c0a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- 172.21.15.26:0/798464402 shutdown_connections 2024-09-15T03:45:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- 172.21.15.26:0/798464402 wait complete. 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient: build_initial_monmap 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient: monmap: 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:45:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 Processor -- start 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- start start 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient: init 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient: _reopen_session rank -1 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient: _add_conns ranks=[6,5,7,0,1,4,2,8,3] 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient(hunting): picked mon.noname-d con 0x7fca5c067b20 addr v1:172.21.15.26:6792/0 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient(hunting): picked mon.noname-h con 0x7fca5c06e310 addr v1:172.21.15.122:6791/0 2024-09-15T03:45:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient(hunting): picked mon.noname-i con 0x7fca5c06e6e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c09af80 con 0x7fca5c06e310 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c06d140 con 0x7fca5c067b20 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c09aa90 con 0x7fca5c06e6e0 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient(hunting): _renew_subs 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca61fea700 10 monclient(hunting): authenticate will time out at 884.235063s 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca5bfff700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fca5c06e6e0 legacy=0x7fca5c14a2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca5b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fca5c067b20 0x7fca5c0994d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54524/0 (socket says 172.21.15.26:54524) 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca5b7fe700 1 -- 172.21.15.26:0/2578789860 learned_addr learned my addr 172.21.15.26:0/2578789860 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437049297 0 0) 0x7fca5c06d140 con 0x7fca5c067b20 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 10 monclient(hunting): my global_id is 4578 2024-09-15T03:45:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca48003570 con 0x7fca5c067b20 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537635703 0 0) 0x7fca5c09af80 con 0x7fca5c06e310 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 10 monclient(hunting): my global_id is 4604 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.803+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5c06d140 con 0x7fca5c06e310 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3379268045 0 0) 0x7fca48003570 con 0x7fca5c067b20 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca5c09af80 con 0x7fca5c067b20 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca4c001790 con 0x7fca5c067b20 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): dump: 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1181540176 0 0) 0x7fca5c06d140 con 0x7fca5c06e310 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca48003570 con 0x7fca5c06e310 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca50001590 con 0x7fca5c06e310 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): dump: 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 631213740 0 0) 0x7fca5c09af80 con 0x7fca5c067b20 2024-09-15T03:45:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6792/0 conn(0x7fca5c06e6e0 legacy=0x7fca5c14a2a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6791/0 conn(0x7fca5c06e310 legacy=0x7fca5c099be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: _finish_hunting 0 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 monclient: found mon.d 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c12fbf0 con 0x7fca5c067b20 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: _finish_auth 0 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 5 monclient: authenticate success, global_id 4578 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 10 monclient: _renew_subs 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c06d700 con 0x7fca5c067b20 2024-09-15T03:45:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 10 monclient: _renew_subs 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c06c250 con 0x7fca5c067b20 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fca4c004e70 con 0x7fca5c067b20 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca4c005320 con 0x7fca5c067b20 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca59ffb700 4 set_mon_vals no callback set 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient: dump: 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fca3c006590 con 0x7fca3c002840 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fca4c005500 con 0x7fca5c067b20 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(24..24 src has 1..24) v4 ==== 3867+0+0 (unknown 4092199876 0 0) 0x7fca4c044200 con 0x7fca5c067b20 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186266987 0 0) 0x7fca3c006590 con 0x7fca3c002840 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 10 monclient(hunting): my global_id is 4613 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5c09af80 con 0x7fca3c002840 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.807+0000 7fca5bfff700 10 monclient: get_auth_request con 0x7fca48037b60 auth_method 0 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 455866880 0 0) 0x7fca5c09af80 con 0x7fca3c002840 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca3c006590 con 0x7fca3c002840 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca50004e20 con 0x7fca3c002840 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 10 monclient: dump: 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.828 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1887302272 0 0) 0x7fca3c006590 con 0x7fca3c002840 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca3c00a130 con 0x7fca3c002840 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fca50007190 con 0x7fca3c002840 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6791/0 conn(0x7fca3c002840 legacy=0x7fca3c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.811+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6791/0 conn(0x7fca3c002840 legacy=0x7fca3c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca61fea700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fca3c009640 con 0x7fca3c0090b0 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067399538 0 0) 0x7fca3c009640 con 0x7fca3c0090b0 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 10 monclient(hunting): my global_id is 4622 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c006590 con 0x7fca3c0090b0 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1260587239 0 0) 0x7fca3c006590 con 0x7fca3c0090b0 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca3c009640 con 0x7fca3c0090b0 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca50005930 con 0x7fca3c0090b0 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:45:39.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 10 monclient: dump: 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:45:39.832 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2196796126 0 0) 0x7fca3c009640 con 0x7fca3c0090b0 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.819+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fca3c0098f0 con 0x7fca3c0090b0 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7354 (unknown 0 0 1357277396) 0x7fca5000a180 con 0x7fca3c0090b0 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca437fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca437fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6791/0 conn(0x7fca3c0090b0 legacy=0x7fca3c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca437fe700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6791/0 conn(0x7fca3c0090b0 legacy=0x7fca3c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:45:39.833 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 28, 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: 6 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.834 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.835 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.836 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:45:39.837 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:45:39.838 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:45:39.839 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.840 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.842 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:45:39.843 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:45:39.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.845 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.848 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:45:39.849 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.850 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.851 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:45:39.852 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.122:6812/20895 conn(0x7fca48037b60 legacy=0x7fca4803a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 10 monclient: shutdown 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 >> v1:172.21.15.26:6792/0 conn(0x7fca5c067b20 legacy=0x7fca5c0994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca5bfff700 1 -- 172.21.15.26:0/2578789860 reap_dead start 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca5bfff700 1 -- 172.21.15.26:0/2578789860 reap_dead start 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 shutdown_connections 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 >> 172.21.15.26:0/2578789860 conn(0x7fca5c063d10 msgr2=0x7fca5c06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 shutdown_connections 2024-09-15T03:45:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:45:39.823+0000 7fca61fea700 1 -- 172.21.15.26:0/2578789860 wait complete. 2024-09-15T03:45:39.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:45:39.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:45:39.933 INFO:tasks.rados.rados.0.smithi122.stdout:71: done (1 left) 2024-09-15T03:45:39.934 INFO:tasks.rados.rados.0.smithi122.stdout:72: copy_from oid 1 from oid 50 current snap is 3 2024-09-15T03:45:39.934 INFO:tasks.rados.rados.0.smithi122.stdout:70: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-15T03:45:40.017 INFO:tasks.rados.rados.0.smithi122.stdout:72: finishing copy_from racing read to smithi12223906-1 2024-09-15T03:45:40.017 INFO:tasks.rados.rados.0.smithi122.stdout:72: finishing copy_from to smithi12223906-1 2024-09-15T03:45:40.017 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 24 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T03:45:40.017 INFO:tasks.rados.rados.0.smithi122.stdout:70: done (1 left) 2024-09-15T03:45:40.017 INFO:tasks.rados.rados.0.smithi122.stdout:72: done (0 left) 2024-09-15T03:45:40.017 INFO:tasks.rados.rados.0.smithi122.stdout:73: snap_remove snap 1 2024-09-15T03:45:40.938 INFO:tasks.rados.rados.0.smithi122.stdout:73: done (0 left) 2024-09-15T03:45:40.939 INFO:tasks.rados.rados.0.smithi122.stdout:74: snap_remove snap 2 2024-09-15T03:45:41.951 INFO:tasks.rados.rados.0.smithi122.stdout:74: done (0 left) 2024-09-15T03:45:41.951 INFO:tasks.rados.rados.0.smithi122.stdout:75: snap_create 2024-09-15T03:45:42.956 INFO:tasks.rados.rados.0.smithi122.stdout:75: done (0 left) 2024-09-15T03:45:42.956 INFO:tasks.rados.rados.0.smithi122.stdout:76: write oid 32 current snap is 4 2024-09-15T03:45:42.956 INFO:tasks.rados.rados.0.smithi122.stdout:76: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2024-09-15T03:45:42.965 INFO:tasks.rados.rados.0.smithi122.stdout:76: writing smithi12223906-32 from 591613 to 1045775 tid 1 2024-09-15T03:45:42.973 INFO:tasks.rados.rados.0.smithi122.stdout:76: writing smithi12223906-32 from 1719614 to 2136967 tid 2 2024-09-15T03:45:42.978 INFO:tasks.rados.rados.0.smithi122.stdout:76: writing smithi12223906-32 from 2570780 to 2991613 tid 3 2024-09-15T03:45:42.978 INFO:tasks.rados.rados.0.smithi122.stdout:77: delete oid 8 current snap is 4 2024-09-15T03:45:42.978 INFO:tasks.rados.rados.0.smithi122.stdout:76: finishing write tid 1 to smithi12223906-32 2024-09-15T03:45:42.978 INFO:tasks.rados.rados.0.smithi122.stdout:76: finishing write tid 2 to smithi12223906-32 2024-09-15T03:45:42.982 INFO:tasks.rados.rados.0.smithi122.stdout:77: done (1 left) 2024-09-15T03:45:42.982 INFO:tasks.rados.rados.0.smithi122.stdout:78: snap_create 2024-09-15T03:45:42.982 INFO:tasks.rados.rados.0.smithi122.stdout:76: finishing write tid 3 to smithi12223906-32 2024-09-15T03:45:42.983 INFO:tasks.rados.rados.0.smithi122.stdout:76: finishing write tid 4 to smithi12223906-32 2024-09-15T03:45:42.983 INFO:tasks.rados.rados.0.smithi122.stdout:76: finishing write tid 5 to smithi12223906-32 2024-09-15T03:45:42.983 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 33 (ObjNum 59 snap 4 seq_num 59) dirty exists 2024-09-15T03:45:42.983 INFO:tasks.rados.rados.0.smithi122.stdout:76: left oid 32 (ObjNum 59 snap 4 seq_num 59) 2024-09-15T03:45:43.965 INFO:tasks.rados.rados.0.smithi122.stdout:76: done (1 left) 2024-09-15T03:45:43.965 INFO:tasks.rados.rados.0.smithi122.stdout:78: done (0 left) 2024-09-15T03:45:43.965 INFO:tasks.rados.rados.0.smithi122.stdout:79: delete oid 13 current snap is 5 2024-09-15T03:45:43.971 INFO:tasks.rados.rados.0.smithi122.stdout:79: done (0 left) 2024-09-15T03:45:43.971 INFO:tasks.rados.rados.0.smithi122.stdout:80: read oid 1 snap -1 2024-09-15T03:45:43.971 INFO:tasks.rados.rados.0.smithi122.stdout:80: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:45:43.971 INFO:tasks.rados.rados.0.smithi122.stdout:81: write oid 38 current snap is 5 2024-09-15T03:45:43.971 INFO:tasks.rados.rados.0.smithi122.stdout:81: seq_num 62 ranges {642034=796262,1895060=564680} 2024-09-15T03:45:43.986 INFO:tasks.rados.rados.0.smithi122.stdout:81: writing smithi12223906-38 from 642034 to 1438296 tid 1 2024-09-15T03:45:43.993 INFO:tasks.rados.rados.0.smithi122.stdout:81: writing smithi12223906-38 from 1895060 to 2459740 tid 2 2024-09-15T03:45:43.993 INFO:tasks.rados.rados.0.smithi122.stdout:82: write (excl) oid 40 current snap is 5 2024-09-15T03:45:43.993 INFO:tasks.rados.rados.0.smithi122.stdout:82: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2024-09-15T03:45:43.999 INFO:tasks.rados.rados.0.smithi122.stdout:82: writing smithi12223906-40 from 658841 to 1130352 tid 1 2024-09-15T03:45:44.006 INFO:tasks.rados.rados.0.smithi122.stdout:82: writing smithi12223906-40 from 1553542 to 2261880 tid 2 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:82: writing smithi12223906-40 from 2964119 to 3058841 tid 3 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:83: read oid 13 snap -1 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:83: expect deleted 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:84: read oid 28 snap -1 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:84: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:85: write oid 45 current snap is 5 2024-09-15T03:45:44.011 INFO:tasks.rados.rados.0.smithi122.stdout:80: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:45:44.098 INFO:tasks.rados.rados.0.smithi122.stdout:81: finishing write tid 1 to smithi12223906-38 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:81: finishing write tid 2 to smithi12223906-38 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:81: finishing write tid 3 to smithi12223906-38 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:81: finishing write tid 4 to smithi12223906-38 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 21 (ObjNum 62 snap 5 seq_num 62) dirty exists 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:81: left oid 38 (ObjNum 62 snap 5 seq_num 62) 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:82: finishing write tid 1 to smithi12223906-40 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:82: finishing write tid 2 to smithi12223906-40 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:82: finishing write tid 3 to smithi12223906-40 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:82: finishing write tid 4 to smithi12223906-40 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:82: finishing write tid 5 to smithi12223906-40 2024-09-15T03:45:44.099 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 20 (ObjNum 63 snap 5 seq_num 63) dirty exists 2024-09-15T03:45:44.100 INFO:tasks.rados.rados.0.smithi122.stdout:82: left oid 40 (ObjNum 63 snap 5 seq_num 63) 2024-09-15T03:45:44.100 INFO:tasks.rados.rados.0.smithi122.stdout:85: seq_num 64 ranges {675648=546760,2012024=568349} 2024-09-15T03:45:44.105 INFO:tasks.rados.rados.0.smithi122.stdout:85: writing smithi12223906-45 from 675648 to 1222408 tid 1 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:85: writing smithi12223906-45 from 2012024 to 2580373 tid 2 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:80: done (5 left) 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:81: done (4 left) 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:82: done (3 left) 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:83: done (2 left) 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:86: rollback oid 24 current snap is 5 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 3 2024-09-15T03:45:44.114 INFO:tasks.rados.rados.0.smithi122.stdout:84: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-15T03:45:44.191 INFO:tasks.rados.rados.0.smithi122.stdout:85: finishing write tid 1 to smithi12223906-45 2024-09-15T03:45:44.191 INFO:tasks.rados.rados.0.smithi122.stdout:85: finishing write tid 2 to smithi12223906-45 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:86: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:86: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:85: finishing write tid 3 to smithi12223906-45 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:86: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 28 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:85: finishing write tid 4 to smithi12223906-45 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 35 (ObjNum 64 snap 5 seq_num 64) dirty exists 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:85: left oid 45 (ObjNum 64 snap 5 seq_num 64) 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:84: done (2 left) 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:85: done (1 left) 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:86: done (0 left) 2024-09-15T03:45:44.192 INFO:tasks.rados.rados.0.smithi122.stdout:87: snap_create 2024-09-15T03:45:44.976 INFO:tasks.rados.rados.0.smithi122.stdout:87: done (0 left) 2024-09-15T03:45:44.977 INFO:tasks.rados.rados.0.smithi122.stdout:88: copy_from oid 5 from oid 19 current snap is 6 2024-09-15T03:45:44.977 INFO:tasks.rados.rados.0.smithi122.stdout:89: write (excl) oid 1 current snap is 6 2024-09-15T03:45:44.977 INFO:tasks.rados.rados.0.smithi122.stdout:89: seq_num 65 ranges {692455=622009,2070506=712007} 2024-09-15T03:45:44.990 INFO:tasks.rados.rados.0.smithi122.stdout:89: writing smithi12223906-1 from 692455 to 1314464 tid 1 2024-09-15T03:45:44.999 INFO:tasks.rados.rados.0.smithi122.stdout:89: writing smithi12223906-1 from 2070506 to 2782513 tid 2 2024-09-15T03:45:44.999 INFO:tasks.rados.rados.0.smithi122.stdout:90: read oid 24 snap -1 2024-09-15T03:45:44.999 INFO:tasks.rados.rados.0.smithi122.stdout:90: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-15T03:45:44.999 INFO:tasks.rados.rados.0.smithi122.stdout:91: read oid 8 snap 5 2024-09-15T03:45:44.999 INFO:tasks.rados.rados.0.smithi122.stdout:91: expect deleted 2024-09-15T03:45:44.999 INFO:tasks.rados.rados.0.smithi122.stdout:92: snap_remove snap 3 2024-09-15T03:45:46.000 INFO:tasks.rados.rados.0.smithi122.stdout:92: done (4 left) 2024-09-15T03:45:46.000 INFO:tasks.rados.rados.0.smithi122.stdout:93: write oid 14 current snap is 6 2024-09-15T03:45:46.000 INFO:tasks.rados.rados.0.smithi122.stdout:93: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2024-09-15T03:45:46.010 INFO:tasks.rados.rados.0.smithi122.stdout:93: writing smithi12223906-14 from 709262 to 1406520 tid 1 2024-09-15T03:45:46.017 INFO:tasks.rados.rados.0.smithi122.stdout:93: writing smithi12223906-14 from 2012635 to 2584653 tid 2 2024-09-15T03:45:46.020 INFO:tasks.rados.rados.0.smithi122.stdout:93: writing smithi12223906-14 from 3046388 to 3109262 tid 3 2024-09-15T03:45:46.020 INFO:tasks.rados.rados.0.smithi122.stdout:94: read oid 3 snap -1 2024-09-15T03:45:46.021 INFO:tasks.rados.rados.0.smithi122.stdout:94: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T03:45:46.021 INFO:tasks.rados.rados.0.smithi122.stdout:95: snap_create 2024-09-15T03:45:46.021 INFO:tasks.rados.rados.0.smithi122.stdout:89: finishing write tid 1 to smithi12223906-1 2024-09-15T03:45:46.021 INFO:tasks.rados.rados.0.smithi122.stdout:90: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:88: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:88: finishing copy_from to smithi12223906-5 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:89: finishing write tid 2 to smithi12223906-1 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:89: finishing write tid 3 to smithi12223906-1 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:89: finishing write tid 4 to smithi12223906-1 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 27 (ObjNum 65 snap 6 seq_num 65) dirty exists 2024-09-15T03:45:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:89: left oid 1 (ObjNum 65 snap 6 seq_num 65) 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:93: finishing write tid 1 to smithi12223906-14 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:93: finishing write tid 2 to smithi12223906-14 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:93: finishing write tid 3 to smithi12223906-14 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:93: finishing write tid 4 to smithi12223906-14 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:93: finishing write tid 5 to smithi12223906-14 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 35 (ObjNum 66 snap 6 seq_num 66) dirty exists 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:93: left oid 14 (ObjNum 66 snap 6 seq_num 66) 2024-09-15T03:45:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:94: expect (ObjNum 2 snap 0 seq_num 2) 2024-09-15T03:45:46.681 INFO:tasks.rados.rados.0.smithi122.stdout:88: done (6 left) 2024-09-15T03:45:46.681 INFO:tasks.rados.rados.0.smithi122.stdout:89: done (5 left) 2024-09-15T03:45:46.681 INFO:tasks.rados.rados.0.smithi122.stdout:90: done (4 left) 2024-09-15T03:45:46.681 INFO:tasks.rados.rados.0.smithi122.stdout:91: done (3 left) 2024-09-15T03:45:46.681 INFO:tasks.rados.rados.0.smithi122.stdout:93: done (2 left) 2024-09-15T03:45:46.682 INFO:tasks.rados.rados.0.smithi122.stdout:94: done (1 left) 2024-09-15T03:45:46.682 INFO:tasks.rados.rados.0.smithi122.stdout:95: done (0 left) 2024-09-15T03:45:46.682 INFO:tasks.rados.rados.0.smithi122.stdout:96: read oid 42 snap -1 2024-09-15T03:45:46.682 INFO:tasks.rados.rados.0.smithi122.stdout:96: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:45:46.682 INFO:tasks.rados.rados.0.smithi122.stdout:97: delete oid 17 current snap is 7 2024-09-15T03:45:46.691 INFO:tasks.rados.rados.0.smithi122.stdout:97: done (1 left) 2024-09-15T03:45:46.691 INFO:tasks.rados.rados.0.smithi122.stdout:98: snap_create 2024-09-15T03:45:46.698 INFO:tasks.rados.rados.0.smithi122.stdout:96: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:45:47.685 INFO:tasks.rados.rados.0.smithi122.stdout:96: done (1 left) 2024-09-15T03:45:47.685 INFO:tasks.rados.rados.0.smithi122.stdout:98: done (0 left) 2024-09-15T03:45:47.685 INFO:tasks.rados.rados.0.smithi122.stdout:99: write (excl) oid 37 current snap is 8 2024-09-15T03:45:47.685 INFO:tasks.rados.rados.0.smithi122.stdout:99: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2024-09-15T03:45:47.695 INFO:tasks.rados.rados.0.smithi122.stdout:99: writing smithi12223906-37 from 742876 to 1190632 tid 1 2024-09-15T03:45:47.703 INFO:tasks.rados.rados.0.smithi122.stdout:99: writing smithi12223906-37 from 1729599 to 2305286 tid 2 2024-09-15T03:45:47.708 INFO:tasks.rados.rados.0.smithi122.stdout:99: writing smithi12223906-37 from 3101234 to 3142876 tid 3 2024-09-15T03:45:47.708 INFO:tasks.rados.rados.0.smithi122.stdout:100: snap_remove snap 6 2024-09-15T03:45:48.688 INFO:tasks.rados.rados.0.smithi122.stdout:100: done (1 left) 2024-09-15T03:45:48.688 INFO:tasks.rados.rados.0.smithi122.stdout:101: snap_remove snap 4 2024-09-15T03:45:49.704 INFO:tasks.rados.rados.0.smithi122.stdout:101: done (1 left) 2024-09-15T03:45:49.704 INFO:tasks.rados.rados.0.smithi122.stdout:102: delete oid 40 current snap is 8 2024-09-15T03:45:49.704 INFO:tasks.rados.rados.0.smithi122.stdout:99: finishing write tid 1 to smithi12223906-37 2024-09-15T03:45:49.704 INFO:tasks.rados.rados.0.smithi122.stdout:99: finishing write tid 2 to smithi12223906-37 2024-09-15T03:45:49.704 INFO:tasks.rados.rados.0.smithi122.stdout:99: finishing write tid 3 to smithi12223906-37 2024-09-15T03:45:49.704 INFO:tasks.rados.rados.0.smithi122.stdout:99: finishing write tid 4 to smithi12223906-37 2024-09-15T03:45:49.705 INFO:tasks.rados.rados.0.smithi122.stdout:99: finishing write tid 5 to smithi12223906-37 2024-09-15T03:45:49.705 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 17 (ObjNum 68 snap 8 seq_num 68) dirty exists 2024-09-15T03:45:49.705 INFO:tasks.rados.rados.0.smithi122.stdout:99: left oid 37 (ObjNum 68 snap 8 seq_num 68) 2024-09-15T03:45:49.716 INFO:tasks.rados.rados.0.smithi122.stdout:99: done (1 left) 2024-09-15T03:45:49.716 INFO:tasks.rados.rados.0.smithi122.stdout:102: done (0 left) 2024-09-15T03:45:49.716 INFO:tasks.rados.rados.0.smithi122.stdout:103: snap_create 2024-09-15T03:45:50.708 INFO:tasks.rados.rados.0.smithi122.stdout:103: done (0 left) 2024-09-15T03:45:50.708 INFO:tasks.rados.rados.0.smithi122.stdout:104: snap_create 2024-09-15T03:45:51.719 INFO:tasks.rados.rados.0.smithi122.stdout:104: done (0 left) 2024-09-15T03:45:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:105: read oid 37 snap -1 2024-09-15T03:45:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:105: expect (ObjNum 68 snap 8 seq_num 68) 2024-09-15T03:45:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:106: write oid 26 current snap is 10 2024-09-15T03:45:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:106: seq_num 70 ranges {776490=714607,2246563=579356} 2024-09-15T03:45:51.735 INFO:tasks.rados.rados.0.smithi122.stdout:106: writing smithi12223906-26 from 776490 to 1491097 tid 1 2024-09-15T03:45:51.743 INFO:tasks.rados.rados.0.smithi122.stdout:106: writing smithi12223906-26 from 2246563 to 2825919 tid 2 2024-09-15T03:45:51.743 INFO:tasks.rados.rados.0.smithi122.stdout:107: write (excl) oid 3 current snap is 10 2024-09-15T03:45:51.743 INFO:tasks.rados.rados.0.smithi122.stdout:107: seq_num 71 ranges {793297=789856,2305045=723014} 2024-09-15T03:45:51.752 INFO:tasks.rados.rados.0.smithi122.stdout:107: writing smithi12223906-3 from 793297 to 1583153 tid 1 2024-09-15T03:45:51.760 INFO:tasks.rados.rados.0.smithi122.stdout:107: writing smithi12223906-3 from 2305045 to 3028059 tid 2 2024-09-15T03:45:51.760 INFO:tasks.rados.rados.0.smithi122.stdout:108: delete oid 14 current snap is 10 2024-09-15T03:45:51.760 INFO:tasks.rados.rados.0.smithi122.stdout:105: expect (ObjNum 68 snap 8 seq_num 68) 2024-09-15T03:45:51.908 INFO:tasks.rados.rados.0.smithi122.stdout:106: finishing write tid 1 to smithi12223906-26 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:106: finishing write tid 2 to smithi12223906-26 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:106: finishing write tid 3 to smithi12223906-26 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:106: finishing write tid 4 to smithi12223906-26 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 31 (ObjNum 70 snap 10 seq_num 70) dirty exists 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:106: left oid 26 (ObjNum 70 snap 10 seq_num 70) 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:107: finishing write tid 1 to smithi12223906-3 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:107: finishing write tid 2 to smithi12223906-3 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:107: finishing write tid 3 to smithi12223906-3 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:107: finishing write tid 4 to smithi12223906-3 2024-09-15T03:45:51.909 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 43 (ObjNum 71 snap 10 seq_num 71) dirty exists 2024-09-15T03:45:51.910 INFO:tasks.rados.rados.0.smithi122.stdout:107: left oid 3 (ObjNum 71 snap 10 seq_num 71) 2024-09-15T03:45:51.910 INFO:tasks.rados.rados.0.smithi122.stdout:105: done (3 left) 2024-09-15T03:45:51.910 INFO:tasks.rados.rados.0.smithi122.stdout:106: done (2 left) 2024-09-15T03:45:51.910 INFO:tasks.rados.rados.0.smithi122.stdout:107: done (1 left) 2024-09-15T03:45:51.910 INFO:tasks.rados.rados.0.smithi122.stdout:108: done (0 left) 2024-09-15T03:45:51.910 INFO:tasks.rados.rados.0.smithi122.stdout:109: snap_remove snap 9 2024-09-15T03:45:52.736 INFO:tasks.rados.rados.0.smithi122.stdout:109: done (0 left) 2024-09-15T03:45:52.736 INFO:tasks.rados.rados.0.smithi122.stdout:110: read oid 45 snap -1 2024-09-15T03:45:52.736 INFO:tasks.rados.rados.0.smithi122.stdout:110: expect (ObjNum 64 snap 5 seq_num 64) 2024-09-15T03:45:52.736 INFO:tasks.rados.rados.0.smithi122.stdout:111: rollback oid 17 current snap is 10 2024-09-15T03:45:52.737 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 7 2024-09-15T03:45:52.737 INFO:tasks.rados.rados.0.smithi122.stdout:112: delete oid 2 current snap is 10 2024-09-15T03:45:52.745 INFO:tasks.rados.rados.0.smithi122.stdout:111: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:45:52.745 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 26 (ObjNum 129 snap 0 seq_num 1842610368) dirty dne 2024-09-15T03:45:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:111: done (2 left) 2024-09-15T03:45:52.747 INFO:tasks.rados.rados.0.smithi122.stdout:112: done (1 left) 2024-09-15T03:45:52.747 INFO:tasks.rados.rados.0.smithi122.stdout:113: rollback oid 26 current snap is 10 2024-09-15T03:45:52.747 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 7 2024-09-15T03:45:52.747 INFO:tasks.rados.rados.0.smithi122.stdout:114: snap_remove snap 8 2024-09-15T03:45:53.744 INFO:tasks.rados.rados.0.smithi122.stdout:114: done (2 left) 2024-09-15T03:45:53.744 INFO:tasks.rados.rados.0.smithi122.stdout:115: rollback oid 5 current snap is 10 2024-09-15T03:45:53.744 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 5 to 5 2024-09-15T03:45:53.745 INFO:tasks.rados.rados.0.smithi122.stdout:116: copy_from oid 16 from oid 33 current snap is 10 2024-09-15T03:45:53.745 INFO:tasks.rados.rados.0.smithi122.stdout:117: read oid 43 snap -1 2024-09-15T03:45:53.745 INFO:tasks.rados.rados.0.smithi122.stdout:117: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:45:53.745 INFO:tasks.rados.rados.0.smithi122.stdout:118: write (excl) oid 34 current snap is 10 2024-09-15T03:45:53.745 INFO:tasks.rados.rados.0.smithi122.stdout:118: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2024-09-15T03:45:53.752 INFO:tasks.rados.rados.0.smithi122.stdout:118: writing smithi12223906-34 from 443718 to 1059321 tid 1 2024-09-15T03:45:53.758 INFO:tasks.rados.rados.0.smithi122.stdout:118: writing smithi12223906-34 from 1564138 to 2034479 tid 2 2024-09-15T03:45:53.765 INFO:tasks.rados.rados.0.smithi122.stdout:118: writing smithi12223906-34 from 2633066 to 3243718 tid 3 2024-09-15T03:45:53.765 INFO:tasks.rados.rados.0.smithi122.stdout:119: rollback oid 29 current snap is 10 2024-09-15T03:45:53.766 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 29 to 7 2024-09-15T03:45:53.766 INFO:tasks.rados.rados.0.smithi122.stdout:110: expect (ObjNum 64 snap 5 seq_num 64) 2024-09-15T03:45:53.915 INFO:tasks.rados.rados.0.smithi122.stdout:113: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:45:53.915 INFO:tasks.rados.rados.0.smithi122.stdout:113: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:45:53.915 INFO:tasks.rados.rados.0.smithi122.stdout:113: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 36 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:116: finishing copy_from to smithi12223906-16 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:116: got expected ENOENT (src dne) 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:116: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:110: done (6 left) 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:113: done (5 left) 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:116: done (4 left) 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:120: read oid 36 snap -1 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:120: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:121: write (excl) oid 4 current snap is 10 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:115: finishing rollback tid 0 to smithi12223906-5 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:115: finishing rollback tid 1 to smithi12223906-5 2024-09-15T03:45:53.916 INFO:tasks.rados.rados.0.smithi122.stdout:115: finishing rollback tid 2 to smithi12223906-5 2024-09-15T03:45:53.917 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 16 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-15T03:45:53.917 INFO:tasks.rados.rados.0.smithi122.stdout:117: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:118: finishing write tid 1 to smithi12223906-34 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:118: finishing write tid 2 to smithi12223906-34 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:118: finishing write tid 3 to smithi12223906-34 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:119: finishing rollback tid 0 to smithi12223906-29 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:118: finishing write tid 4 to smithi12223906-34 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:119: finishing rollback tid 1 to smithi12223906-29 2024-09-15T03:45:54.004 INFO:tasks.rados.rados.0.smithi122.stdout:118: finishing write tid 5 to smithi12223906-34 2024-09-15T03:45:54.005 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 11 (ObjNum 74 snap 10 seq_num 74) dirty exists 2024-09-15T03:45:54.005 INFO:tasks.rados.rados.0.smithi122.stdout:118: left oid 34 (ObjNum 74 snap 10 seq_num 74) 2024-09-15T03:45:54.005 INFO:tasks.rados.rados.0.smithi122.stdout:119: finishing rollback tid 2 to smithi12223906-29 2024-09-15T03:45:54.005 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 17 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:45:54.005 INFO:tasks.rados.rados.0.smithi122.stdout:120: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-15T03:45:54.083 INFO:tasks.rados.rados.0.smithi122.stdout:121: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2024-09-15T03:45:54.099 INFO:tasks.rados.rados.0.smithi122.stdout:121: writing smithi12223906-4 from 460525 to 1151377 tid 1 2024-09-15T03:45:54.106 INFO:tasks.rados.rados.0.smithi122.stdout:121: writing smithi12223906-4 from 1622620 to 2352972 tid 2 2024-09-15T03:45:54.111 INFO:tasks.rados.rados.0.smithi122.stdout:121: writing smithi12223906-4 from 3060489 to 3260525 tid 3 2024-09-15T03:45:54.112 INFO:tasks.rados.rados.0.smithi122.stdout:115: done (5 left) 2024-09-15T03:45:54.112 INFO:tasks.rados.rados.0.smithi122.stdout:117: done (4 left) 2024-09-15T03:45:54.112 INFO:tasks.rados.rados.0.smithi122.stdout:118: done (3 left) 2024-09-15T03:45:54.112 INFO:tasks.rados.rados.0.smithi122.stdout:119: done (2 left) 2024-09-15T03:45:54.112 INFO:tasks.rados.rados.0.smithi122.stdout:120: done (1 left) 2024-09-15T03:45:54.112 INFO:tasks.rados.rados.0.smithi122.stdout:122: snap_create 2024-09-15T03:45:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:121: finishing write tid 1 to smithi12223906-4 2024-09-15T03:45:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:121: finishing write tid 2 to smithi12223906-4 2024-09-15T03:45:54.120 INFO:tasks.rados.rados.0.smithi122.stdout:121: finishing write tid 3 to smithi12223906-4 2024-09-15T03:45:54.121 INFO:tasks.rados.rados.0.smithi122.stdout:121: finishing write tid 4 to smithi12223906-4 2024-09-15T03:45:54.121 INFO:tasks.rados.rados.0.smithi122.stdout:121: finishing write tid 5 to smithi12223906-4 2024-09-15T03:45:54.121 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 18 (ObjNum 75 snap 10 seq_num 75) dirty exists 2024-09-15T03:45:54.121 INFO:tasks.rados.rados.0.smithi122.stdout:121: left oid 4 (ObjNum 75 snap 10 seq_num 75) 2024-09-15T03:45:54.747 INFO:tasks.rados.rados.0.smithi122.stdout:121: done (1 left) 2024-09-15T03:45:54.747 INFO:tasks.rados.rados.0.smithi122.stdout:122: done (0 left) 2024-09-15T03:45:54.747 INFO:tasks.rados.rados.0.smithi122.stdout:123: write oid 13 current snap is 11 2024-09-15T03:45:54.747 INFO:tasks.rados.rados.0.smithi122.stdout:123: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2024-09-15T03:45:54.758 INFO:tasks.rados.rados.0.smithi122.stdout:123: writing smithi12223906-13 from 477332 to 1243433 tid 1 2024-09-15T03:45:54.763 INFO:tasks.rados.rados.0.smithi122.stdout:123: writing smithi12223906-13 from 1681102 to 2155112 tid 2 2024-09-15T03:45:54.770 INFO:tasks.rados.rados.0.smithi122.stdout:123: writing smithi12223906-13 from 2687912 to 3277332 tid 3 2024-09-15T03:45:54.770 INFO:tasks.rados.rados.0.smithi122.stdout:124: snap_create 2024-09-15T03:45:54.770 INFO:tasks.rados.rados.0.smithi122.stdout:123: finishing write tid 1 to smithi12223906-13 2024-09-15T03:45:54.770 INFO:tasks.rados.rados.0.smithi122.stdout:123: finishing write tid 2 to smithi12223906-13 2024-09-15T03:45:54.774 INFO:tasks.rados.rados.0.smithi122.stdout:123: finishing write tid 3 to smithi12223906-13 2024-09-15T03:45:54.774 INFO:tasks.rados.rados.0.smithi122.stdout:123: finishing write tid 4 to smithi12223906-13 2024-09-15T03:45:54.775 INFO:tasks.rados.rados.0.smithi122.stdout:123: finishing write tid 5 to smithi12223906-13 2024-09-15T03:45:54.775 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 47 (ObjNum 76 snap 11 seq_num 76) dirty exists 2024-09-15T03:45:54.775 INFO:tasks.rados.rados.0.smithi122.stdout:123: left oid 13 (ObjNum 76 snap 11 seq_num 76) 2024-09-15T03:45:55.763 INFO:tasks.rados.rados.0.smithi122.stdout:123: done (1 left) 2024-09-15T03:45:55.763 INFO:tasks.rados.rados.0.smithi122.stdout:124: done (0 left) 2024-09-15T03:45:55.764 INFO:tasks.rados.rados.0.smithi122.stdout:125: delete oid 49 current snap is 12 2024-09-15T03:45:55.770 INFO:tasks.rados.rados.0.smithi122.stdout:125: done (0 left) 2024-09-15T03:45:55.771 INFO:tasks.rados.rados.0.smithi122.stdout:126: write (excl) oid 13 current snap is 12 2024-09-15T03:45:55.771 INFO:tasks.rados.rados.0.smithi122.stdout:126: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2024-09-15T03:45:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:126: writing smithi12223906-13 from 510946 to 1143898 tid 1 2024-09-15T03:45:55.789 INFO:tasks.rados.rados.0.smithi122.stdout:126: writing smithi12223906-13 from 1798066 to 2275745 tid 2 2024-09-15T03:45:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:126: writing smithi12223906-13 from 2742758 to 3199975 tid 3 2024-09-15T03:45:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:127: read oid 29 snap -1 2024-09-15T03:45:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:127: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:45:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:128: write (excl) oid 21 current snap is 12 2024-09-15T03:45:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:126: finishing write tid 1 to smithi12223906-13 2024-09-15T03:45:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:128: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2024-09-15T03:45:55.802 INFO:tasks.rados.rados.0.smithi122.stdout:128: writing smithi12223906-21 from 527753 to 1235954 tid 1 2024-09-15T03:45:55.810 INFO:tasks.rados.rados.0.smithi122.stdout:128: writing smithi12223906-21 from 1856548 to 2594238 tid 2 2024-09-15T03:45:55.814 INFO:tasks.rados.rados.0.smithi122.stdout:128: writing smithi12223906-21 from 3286534 to 3327753 tid 3 2024-09-15T03:45:55.814 INFO:tasks.rados.rados.0.smithi122.stdout:129: write oid 35 current snap is 12 2024-09-15T03:45:55.814 INFO:tasks.rados.rados.0.smithi122.stdout:129: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2024-09-15T03:45:55.822 INFO:tasks.rados.rados.0.smithi122.stdout:129: writing smithi12223906-35 from 544560 to 1328010 tid 1 2024-09-15T03:45:55.828 INFO:tasks.rados.rados.0.smithi122.stdout:129: writing smithi12223906-35 from 1915030 to 2396378 tid 2 2024-09-15T03:45:55.833 INFO:tasks.rados.rados.0.smithi122.stdout:129: writing smithi12223906-35 from 2797604 to 3277365 tid 3 2024-09-15T03:45:55.833 INFO:tasks.rados.rados.0.smithi122.stdout:126: finishing write tid 2 to smithi12223906-13 2024-09-15T03:45:55.833 INFO:tasks.rados.rados.0.smithi122.stdout:130: read oid 23 snap 7 2024-09-15T03:45:55.833 INFO:tasks.rados.rados.0.smithi122.stdout:130: expect (ObjNum 22 snap 0 seq_num 22) 2024-09-15T03:45:55.833 INFO:tasks.rados.rados.0.smithi122.stdout:131: snap_remove snap 5 2024-09-15T03:45:56.686 INFO:tasks.rados.rados.0.smithi122.stdout:131: done (5 left) 2024-09-15T03:45:56.686 INFO:tasks.rados.rados.0.smithi122.stdout:132: delete oid 4 current snap is 12 2024-09-15T03:45:56.686 INFO:tasks.rados.rados.0.smithi122.stdout:126: finishing write tid 3 to smithi12223906-13 2024-09-15T03:45:56.687 INFO:tasks.rados.rados.0.smithi122.stdout:126: finishing write tid 4 to smithi12223906-13 2024-09-15T03:45:56.687 INFO:tasks.rados.rados.0.smithi122.stdout:126: finishing write tid 5 to smithi12223906-13 2024-09-15T03:45:56.687 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 52 (ObjNum 78 snap 12 seq_num 78) dirty exists 2024-09-15T03:45:56.687 INFO:tasks.rados.rados.0.smithi122.stdout:126: left oid 13 (ObjNum 78 snap 12 seq_num 78) 2024-09-15T03:45:56.687 INFO:tasks.rados.rados.0.smithi122.stdout:128: finishing write tid 1 to smithi12223906-21 2024-09-15T03:45:56.687 INFO:tasks.rados.rados.0.smithi122.stdout:127: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:128: finishing write tid 2 to smithi12223906-21 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:128: finishing write tid 3 to smithi12223906-21 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:128: finishing write tid 4 to smithi12223906-21 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:128: finishing write tid 5 to smithi12223906-21 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 26 (ObjNum 79 snap 12 seq_num 79) dirty exists 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:128: left oid 21 (ObjNum 79 snap 12 seq_num 79) 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:129: finishing write tid 1 to smithi12223906-35 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:129: finishing write tid 2 to smithi12223906-35 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:129: finishing write tid 3 to smithi12223906-35 2024-09-15T03:45:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:129: finishing write tid 4 to smithi12223906-35 2024-09-15T03:45:56.818 INFO:tasks.rados.rados.0.smithi122.stdout:129: finishing write tid 5 to smithi12223906-35 2024-09-15T03:45:56.818 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 8 (ObjNum 80 snap 12 seq_num 80) dirty exists 2024-09-15T03:45:56.818 INFO:tasks.rados.rados.0.smithi122.stdout:129: left oid 35 (ObjNum 80 snap 12 seq_num 80) 2024-09-15T03:45:56.818 INFO:tasks.rados.rados.0.smithi122.stdout:130: expect (ObjNum 22 snap 0 seq_num 22) 2024-09-15T03:45:56.893 INFO:tasks.rados.rados.0.smithi122.stdout:126: done (5 left) 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:127: done (4 left) 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:128: done (3 left) 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:129: done (2 left) 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:130: done (1 left) 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:132: done (0 left) 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:133: read oid 33 snap 7 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:133: expect deleted 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:134: write oid 8 current snap is 12 2024-09-15T03:45:56.894 INFO:tasks.rados.rados.0.smithi122.stdout:134: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2024-09-15T03:45:56.905 INFO:tasks.rados.rados.0.smithi122.stdout:134: writing smithi12223906-8 from 578174 to 1112122 tid 1 2024-09-15T03:45:56.912 INFO:tasks.rados.rados.0.smithi122.stdout:134: writing smithi12223906-8 from 1515641 to 2000658 tid 2 2024-09-15T03:45:56.919 INFO:tasks.rados.rados.0.smithi122.stdout:134: writing smithi12223906-8 from 2736097 to 3238402 tid 3 2024-09-15T03:45:56.919 INFO:tasks.rados.rados.0.smithi122.stdout:135: copy_from oid 23 from oid 45 current snap is 12 2024-09-15T03:45:56.919 INFO:tasks.rados.rados.0.smithi122.stdout:136: snap_remove snap 11 2024-09-15T03:45:57.770 INFO:tasks.rados.rados.0.smithi122.stdout:136: done (3 left) 2024-09-15T03:45:57.770 INFO:tasks.rados.rados.0.smithi122.stdout:137: write oid 6 current snap is 12 2024-09-15T03:45:57.770 INFO:tasks.rados.rados.0.smithi122.stdout:134: finishing write tid 1 to smithi12223906-8 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:134: finishing write tid 2 to smithi12223906-8 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:134: finishing write tid 3 to smithi12223906-8 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:134: finishing write tid 4 to smithi12223906-8 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:134: finishing write tid 5 to smithi12223906-8 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 44 (ObjNum 82 snap 12 seq_num 82) dirty exists 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:134: left oid 8 (ObjNum 82 snap 12 seq_num 82) 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:135: finishing copy_from racing read to smithi12223906-23 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:135: finishing copy_from to smithi12223906-23 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 35 (ObjNum 64 snap 5 seq_num 64) dirty exists 2024-09-15T03:45:57.771 INFO:tasks.rados.rados.0.smithi122.stdout:137: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2024-09-15T03:45:57.779 INFO:tasks.rados.rados.0.smithi122.stdout:137: writing smithi12223906-6 from 594981 to 1204178 tid 1 2024-09-15T03:45:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:137: writing smithi12223906-6 from 1974123 to 2719151 tid 2 2024-09-15T03:45:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:137: writing smithi12223906-6 from 3279873 to 3394981 tid 3 2024-09-15T03:45:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:133: done (3 left) 2024-09-15T03:45:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:134: done (2 left) 2024-09-15T03:45:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:135: done (1 left) 2024-09-15T03:45:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:138: snap_create 2024-09-15T03:45:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:137: finishing write tid 1 to smithi12223906-6 2024-09-15T03:45:57.794 INFO:tasks.rados.rados.0.smithi122.stdout:137: finishing write tid 2 to smithi12223906-6 2024-09-15T03:45:57.794 INFO:tasks.rados.rados.0.smithi122.stdout:137: finishing write tid 3 to smithi12223906-6 2024-09-15T03:45:57.795 INFO:tasks.rados.rados.0.smithi122.stdout:137: finishing write tid 4 to smithi12223906-6 2024-09-15T03:45:57.795 INFO:tasks.rados.rados.0.smithi122.stdout:137: finishing write tid 5 to smithi12223906-6 2024-09-15T03:45:57.795 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 17 (ObjNum 83 snap 12 seq_num 83) dirty exists 2024-09-15T03:45:57.795 INFO:tasks.rados.rados.0.smithi122.stdout:137: left oid 6 (ObjNum 83 snap 12 seq_num 83) 2024-09-15T03:45:58.781 INFO:tasks.rados.rados.0.smithi122.stdout:137: done (1 left) 2024-09-15T03:45:58.781 INFO:tasks.rados.rados.0.smithi122.stdout:138: done (0 left) 2024-09-15T03:45:58.781 INFO:tasks.rados.rados.0.smithi122.stdout:139: write (excl) oid 6 current snap is 13 2024-09-15T03:45:58.781 INFO:tasks.rados.rados.0.smithi122.stdout:139: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2024-09-15T03:45:58.793 INFO:tasks.rados.rados.0.smithi122.stdout:139: writing smithi12223906-6 from 611788 to 1012587 tid 1 2024-09-15T03:45:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:139: writing smithi12223906-6 from 1748958 to 2237644 tid 2 2024-09-15T03:45:58.807 INFO:tasks.rados.rados.0.smithi122.stdout:139: writing smithi12223906-6 from 2907296 to 3411788 tid 3 2024-09-15T03:45:58.807 INFO:tasks.rados.rados.0.smithi122.stdout:140: copy_from oid 28 from oid 24 current snap is 13 2024-09-15T03:45:58.807 INFO:tasks.rados.rados.0.smithi122.stdout:139: finishing write tid 1 to smithi12223906-6 2024-09-15T03:45:58.807 INFO:tasks.rados.rados.0.smithi122.stdout:139: finishing write tid 2 to smithi12223906-6 2024-09-15T03:45:58.808 INFO:tasks.rados.rados.0.smithi122.stdout:141: copy_from oid 25 from oid 15 current snap is 13 2024-09-15T03:45:58.808 INFO:tasks.rados.rados.0.smithi122.stdout:142: snap_remove snap 12 2024-09-15T03:45:59.794 INFO:tasks.rados.rados.0.smithi122.stdout:142: done (3 left) 2024-09-15T03:45:59.794 INFO:tasks.rados.rados.0.smithi122.stdout:143: write (excl) oid 48 current snap is 13 2024-09-15T03:45:59.794 INFO:tasks.rados.rados.0.smithi122.stdout:139: finishing write tid 3 to smithi12223906-6 2024-09-15T03:45:59.794 INFO:tasks.rados.rados.0.smithi122.stdout:143: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2024-09-15T03:45:59.806 INFO:tasks.rados.rados.0.smithi122.stdout:143: writing smithi12223906-48 from 628595 to 1104643 tid 1 2024-09-15T03:45:59.814 INFO:tasks.rados.rados.0.smithi122.stdout:143: writing smithi12223906-48 from 1807440 to 2439784 tid 2 2024-09-15T03:45:59.819 INFO:tasks.rados.rados.0.smithi122.stdout:143: writing smithi12223906-48 from 2934719 to 3387193 tid 3 2024-09-15T03:45:59.819 INFO:tasks.rados.rados.0.smithi122.stdout:144: read oid 11 snap -1 2024-09-15T03:45:59.819 INFO:tasks.rados.rados.0.smithi122.stdout:144: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:45:59.820 INFO:tasks.rados.rados.0.smithi122.stdout:145: read oid 47 snap 7 2024-09-15T03:45:59.820 INFO:tasks.rados.rados.0.smithi122.stdout:145: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T03:45:59.820 INFO:tasks.rados.rados.0.smithi122.stdout:146: read oid 7 snap -1 2024-09-15T03:45:59.820 INFO:tasks.rados.rados.0.smithi122.stdout:146: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-15T03:45:59.820 INFO:tasks.rados.rados.0.smithi122.stdout:147: snap_remove snap 10 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:147: done (7 left) 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:148: snap_create 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:139: finishing write tid 4 to smithi12223906-6 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:139: finishing write tid 5 to smithi12223906-6 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 22 (ObjNum 84 snap 13 seq_num 84) dirty exists 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:139: left oid 6 (ObjNum 84 snap 13 seq_num 84) 2024-09-15T03:46:00.803 INFO:tasks.rados.rados.0.smithi122.stdout:140: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:140: finishing copy_from to smithi12223906-28 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 28 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:141: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:141: finishing copy_from to smithi12223906-25 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:143: finishing write tid 1 to smithi12223906-48 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:143: finishing write tid 2 to smithi12223906-48 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:143: finishing write tid 3 to smithi12223906-48 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:143: finishing write tid 4 to smithi12223906-48 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:143: finishing write tid 5 to smithi12223906-48 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 37 (ObjNum 85 snap 13 seq_num 85) dirty exists 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:143: left oid 48 (ObjNum 85 snap 13 seq_num 85) 2024-09-15T03:46:00.804 INFO:tasks.rados.rados.0.smithi122.stdout:144: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:46:00.880 INFO:tasks.rados.rados.0.smithi122.stdout:146: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-15T03:46:00.950 INFO:tasks.rados.rados.0.smithi122.stdout:145: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:139: done (7 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:140: done (6 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:141: done (5 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:143: done (4 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:144: done (3 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:145: done (2 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:146: done (1 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:148: done (0 left) 2024-09-15T03:46:01.688 INFO:tasks.rados.rados.0.smithi122.stdout:149: delete oid 45 current snap is 14 2024-09-15T03:46:01.699 INFO:tasks.rados.rados.0.smithi122.stdout:149: done (0 left) 2024-09-15T03:46:01.699 INFO:tasks.rados.rados.0.smithi122.stdout:150: snap_create 2024-09-15T03:46:02.696 INFO:tasks.rados.rados.0.smithi122.stdout:150: done (0 left) 2024-09-15T03:46:02.696 INFO:tasks.rados.rados.0.smithi122.stdout:151: delete oid 26 current snap is 15 2024-09-15T03:46:02.704 INFO:tasks.rados.rados.0.smithi122.stdout:151: done (0 left) 2024-09-15T03:46:02.704 INFO:tasks.rados.rados.0.smithi122.stdout:152: write (excl) oid 25 current snap is 15 2024-09-15T03:46:02.704 INFO:tasks.rados.rados.0.smithi122.stdout:152: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2024-09-15T03:46:02.718 INFO:tasks.rados.rados.0.smithi122.stdout:152: writing smithi12223906-25 from 679016 to 1380811 tid 1 2024-09-15T03:46:02.723 INFO:tasks.rados.rados.0.smithi122.stdout:152: writing smithi12223906-25 from 1866533 to 2362557 tid 2 2024-09-15T03:46:02.728 INFO:tasks.rados.rados.0.smithi122.stdout:152: writing smithi12223906-25 from 2900635 to 3303278 tid 3 2024-09-15T03:46:02.729 INFO:tasks.rados.rados.0.smithi122.stdout:153: write (excl) oid 49 current snap is 15 2024-09-15T03:46:02.729 INFO:tasks.rados.rados.0.smithi122.stdout:153: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2024-09-15T03:46:02.737 INFO:tasks.rados.rados.0.smithi122.stdout:153: writing smithi12223906-49 from 695823 to 1472867 tid 1 2024-09-15T03:46:02.743 INFO:tasks.rados.rados.0.smithi122.stdout:153: writing smithi12223906-49 from 1925015 to 2564697 tid 2 2024-09-15T03:46:02.749 INFO:tasks.rados.rados.0.smithi122.stdout:153: writing smithi12223906-49 from 3328058 to 3495823 tid 3 2024-09-15T03:46:02.749 INFO:tasks.rados.rados.0.smithi122.stdout:154: read oid 15 snap 7 2024-09-15T03:46:02.749 INFO:tasks.rados.rados.0.smithi122.stdout:154: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-15T03:46:02.749 INFO:tasks.rados.rados.0.smithi122.stdout:155: write (excl) oid 48 current snap is 15 2024-09-15T03:46:02.749 INFO:tasks.rados.rados.0.smithi122.stdout:152: finishing write tid 1 to smithi12223906-25 2024-09-15T03:46:02.749 INFO:tasks.rados.rados.0.smithi122.stdout:155: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2024-09-15T03:46:02.756 INFO:tasks.rados.rados.0.smithi122.stdout:155: writing smithi12223906-48 from 712630 to 1164923 tid 1 2024-09-15T03:46:02.761 INFO:tasks.rados.rados.0.smithi122.stdout:155: writing smithi12223906-48 from 1583497 to 2083190 tid 2 2024-09-15T03:46:02.766 INFO:tasks.rados.rados.0.smithi122.stdout:155: writing smithi12223906-48 from 2555481 to 2980668 tid 3 2024-09-15T03:46:02.766 INFO:tasks.rados.rados.0.smithi122.stdout:156: copy_from oid 27 from oid 35 current snap is 15 2024-09-15T03:46:02.766 INFO:tasks.rados.rados.0.smithi122.stdout:152: finishing write tid 2 to smithi12223906-25 2024-09-15T03:46:02.766 INFO:tasks.rados.rados.0.smithi122.stdout:152: finishing write tid 3 to smithi12223906-25 2024-09-15T03:46:02.766 INFO:tasks.rados.rados.0.smithi122.stdout:152: finishing write tid 4 to smithi12223906-25 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:152: finishing write tid 5 to smithi12223906-25 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 68 (ObjNum 88 snap 15 seq_num 88) dirty exists 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:152: left oid 25 (ObjNum 88 snap 15 seq_num 88) 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:153: finishing write tid 1 to smithi12223906-49 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:153: finishing write tid 2 to smithi12223906-49 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:153: finishing write tid 3 to smithi12223906-49 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:153: finishing write tid 4 to smithi12223906-49 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:153: finishing write tid 5 to smithi12223906-49 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 51 (ObjNum 89 snap 15 seq_num 89) dirty exists 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:153: left oid 49 (ObjNum 89 snap 15 seq_num 89) 2024-09-15T03:46:02.767 INFO:tasks.rados.rados.0.smithi122.stdout:154: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:155: finishing write tid 1 to smithi12223906-48 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:155: finishing write tid 2 to smithi12223906-48 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:155: finishing write tid 3 to smithi12223906-48 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:155: finishing write tid 4 to smithi12223906-48 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:155: finishing write tid 5 to smithi12223906-48 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 46 (ObjNum 90 snap 15 seq_num 90) dirty exists 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:155: left oid 48 (ObjNum 90 snap 15 seq_num 90) 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:156: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:156: finishing copy_from to smithi12223906-27 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 8 (ObjNum 80 snap 12 seq_num 80) dirty exists 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:152: done (4 left) 2024-09-15T03:46:02.842 INFO:tasks.rados.rados.0.smithi122.stdout:153: done (3 left) 2024-09-15T03:46:02.843 INFO:tasks.rados.rados.0.smithi122.stdout:154: done (2 left) 2024-09-15T03:46:02.843 INFO:tasks.rados.rados.0.smithi122.stdout:155: done (1 left) 2024-09-15T03:46:02.843 INFO:tasks.rados.rados.0.smithi122.stdout:156: done (0 left) 2024-09-15T03:46:02.843 INFO:tasks.rados.rados.0.smithi122.stdout:157: write (excl) oid 14 current snap is 15 2024-09-15T03:46:02.843 INFO:tasks.rados.rados.0.smithi122.stdout:157: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2024-09-15T03:46:02.857 INFO:tasks.rados.rados.0.smithi122.stdout:157: writing smithi12223906-14 from 729437 to 1256979 tid 1 2024-09-15T03:46:02.864 INFO:tasks.rados.rados.0.smithi122.stdout:157: writing smithi12223906-14 from 1925626 to 2568977 tid 2 2024-09-15T03:46:02.870 INFO:tasks.rados.rados.0.smithi122.stdout:157: writing smithi12223906-14 from 3266551 to 3529437 tid 3 2024-09-15T03:46:02.870 INFO:tasks.rados.rados.0.smithi122.stdout:158: rollback oid 49 current snap is 15 2024-09-15T03:46:02.870 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 7 2024-09-15T03:46:02.870 INFO:tasks.rados.rados.0.smithi122.stdout:157: finishing write tid 1 to smithi12223906-14 2024-09-15T03:46:02.870 INFO:tasks.rados.rados.0.smithi122.stdout:159: snap_remove snap 14 2024-09-15T03:46:03.822 INFO:tasks.rados.rados.0.smithi122.stdout:159: done (2 left) 2024-09-15T03:46:03.822 INFO:tasks.rados.rados.0.smithi122.stdout:160: snap_create 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:157: finishing write tid 2 to smithi12223906-14 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:158: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:158: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:157: finishing write tid 3 to smithi12223906-14 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:157: finishing write tid 4 to smithi12223906-14 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:157: finishing write tid 5 to smithi12223906-14 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 72 (ObjNum 91 snap 15 seq_num 91) dirty exists 2024-09-15T03:46:03.823 INFO:tasks.rados.rados.0.smithi122.stdout:157: left oid 14 (ObjNum 91 snap 15 seq_num 91) 2024-09-15T03:46:03.824 INFO:tasks.rados.rados.0.smithi122.stdout:158: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:46:03.824 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 54 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-15T03:46:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:157: done (2 left) 2024-09-15T03:46:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:158: done (1 left) 2024-09-15T03:46:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:160: done (0 left) 2024-09-15T03:46:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:161: rollback oid 16 current snap is 16 2024-09-15T03:46:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 7 2024-09-15T03:46:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:162: snap_remove snap 15 2024-09-15T03:46:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:161: finishing rollback tid 0 to smithi12223906-16 2024-09-15T03:46:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:161: finishing rollback tid 1 to smithi12223906-16 2024-09-15T03:46:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:161: finishing rollback tid 2 to smithi12223906-16 2024-09-15T03:46:05.841 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 35 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-15T03:46:05.841 INFO:tasks.rados.rados.0.smithi122.stdout:161: done (1 left) 2024-09-15T03:46:05.841 INFO:tasks.rados.rados.0.smithi122.stdout:162: done (0 left) 2024-09-15T03:46:05.841 INFO:tasks.rados.rados.0.smithi122.stdout:163: copy_from oid 7 from oid 26 current snap is 16 2024-09-15T03:46:05.841 INFO:tasks.rados.rados.0.smithi122.stdout:164: write (excl) oid 14 current snap is 16 2024-09-15T03:46:05.842 INFO:tasks.rados.rados.0.smithi122.stdout:164: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2024-09-15T03:46:05.849 INFO:tasks.rados.rados.0.smithi122.stdout:164: writing smithi12223906-14 from 746244 to 1465388 tid 1 2024-09-15T03:46:05.856 INFO:tasks.rados.rados.0.smithi122.stdout:164: writing smithi12223906-14 from 2100461 to 2603823 tid 2 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:164: writing smithi12223906-14 from 3126680 to 3546244 tid 3 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:165: rollback oid 38 current snap is 16 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 7 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:163: finishing copy_from to smithi12223906-7 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:163: got expected ENOENT (src dne) 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:163: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:164: finishing write tid 1 to smithi12223906-14 2024-09-15T03:46:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:164: finishing write tid 2 to smithi12223906-14 2024-09-15T03:46:05.865 INFO:tasks.rados.rados.0.smithi122.stdout:163: done (2 left) 2024-09-15T03:46:05.865 INFO:tasks.rados.rados.0.smithi122.stdout:166: snap_create 2024-09-15T03:46:05.865 INFO:tasks.rados.rados.0.smithi122.stdout:164: finishing write tid 3 to smithi12223906-14 2024-09-15T03:46:05.865 INFO:tasks.rados.rados.0.smithi122.stdout:164: finishing write tid 4 to smithi12223906-14 2024-09-15T03:46:05.865 INFO:tasks.rados.rados.0.smithi122.stdout:164: finishing write tid 5 to smithi12223906-14 2024-09-15T03:46:05.866 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 78 (ObjNum 92 snap 16 seq_num 92) dirty exists 2024-09-15T03:46:05.866 INFO:tasks.rados.rados.0.smithi122.stdout:164: left oid 14 (ObjNum 92 snap 16 seq_num 92) 2024-09-15T03:46:05.866 INFO:tasks.rados.rados.0.smithi122.stdout:165: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:46:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:165: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:46:05.868 INFO:tasks.rados.rados.0.smithi122.stdout:165: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:46:05.868 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 52 (ObjNum 62 snap 5 seq_num 62) dirty exists 2024-09-15T03:46:06.690 INFO:tasks.rados.rados.0.smithi122.stdout:164: done (2 left) 2024-09-15T03:46:06.690 INFO:tasks.rados.rados.0.smithi122.stdout:165: done (1 left) 2024-09-15T03:46:06.690 INFO:tasks.rados.rados.0.smithi122.stdout:166: done (0 left) 2024-09-15T03:46:06.690 INFO:tasks.rados.rados.0.smithi122.stdout:167: snap_create 2024-09-15T03:46:07.691 INFO:tasks.rados.rados.0.smithi122.stdout:167: done (0 left) 2024-09-15T03:46:07.691 INFO:tasks.rados.rados.0.smithi122.stdout:168: read oid 43 snap 13 2024-09-15T03:46:07.691 INFO:tasks.rados.rados.0.smithi122.stdout:168: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:46:07.691 INFO:tasks.rados.rados.0.smithi122.stdout:169: write (excl) oid 12 current snap is 18 2024-09-15T03:46:07.692 INFO:tasks.rados.rados.0.smithi122.stdout:169: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2024-09-15T03:46:07.705 INFO:tasks.rados.rados.0.smithi122.stdout:169: writing smithi12223906-12 from 763051 to 1557444 tid 1 2024-09-15T03:46:07.711 INFO:tasks.rados.rados.0.smithi122.stdout:169: writing smithi12223906-12 from 2158943 to 2805963 tid 2 2024-09-15T03:46:07.716 INFO:tasks.rados.rados.0.smithi122.stdout:169: writing smithi12223906-12 from 3437750 to 3563051 tid 3 2024-09-15T03:46:07.716 INFO:tasks.rados.rados.0.smithi122.stdout:170: delete oid 23 current snap is 18 2024-09-15T03:46:07.716 INFO:tasks.rados.rados.0.smithi122.stdout:168: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: finishing write tid 1 to smithi12223906-12 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: finishing write tid 2 to smithi12223906-12 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: finishing write tid 3 to smithi12223906-12 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: finishing write tid 4 to smithi12223906-12 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: finishing write tid 5 to smithi12223906-12 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 18 (ObjNum 93 snap 18 seq_num 93) dirty exists 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: left oid 12 (ObjNum 93 snap 18 seq_num 93) 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:168: done (2 left) 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:169: done (1 left) 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:170: done (0 left) 2024-09-15T03:46:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:171: delete oid 38 current snap is 18 2024-09-15T03:46:07.822 INFO:tasks.rados.rados.0.smithi122.stdout:171: done (0 left) 2024-09-15T03:46:07.822 INFO:tasks.rados.rados.0.smithi122.stdout:172: copy_from oid 3 from oid 25 current snap is 18 2024-09-15T03:46:07.822 INFO:tasks.rados.rados.0.smithi122.stdout:173: write (excl) oid 7 current snap is 18 2024-09-15T03:46:07.822 INFO:tasks.rados.rados.0.smithi122.stdout:173: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2024-09-15T03:46:07.831 INFO:tasks.rados.rados.0.smithi122.stdout:173: writing smithi12223906-7 from 413472 to 1033612 tid 1 2024-09-15T03:46:07.839 INFO:tasks.rados.rados.0.smithi122.stdout:173: writing smithi12223906-7 from 1818036 to 2328736 tid 2 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:173: writing smithi12223906-7 from 3120019 to 3613472 tid 3 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:174: copy_from oid 45 from oid 43 current snap is 18 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:173: finishing write tid 1 to smithi12223906-7 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:173: finishing write tid 2 to smithi12223906-7 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:172: finishing copy_from racing read to smithi12223906-3 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:172: finishing copy_from to smithi12223906-3 2024-09-15T03:46:07.847 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 68 (ObjNum 88 snap 15 seq_num 88) dirty exists 2024-09-15T03:46:07.848 INFO:tasks.rados.rados.0.smithi122.stdout:172: done (2 left) 2024-09-15T03:46:07.848 INFO:tasks.rados.rados.0.smithi122.stdout:175: delete oid 8 current snap is 18 2024-09-15T03:46:07.850 INFO:tasks.rados.rados.0.smithi122.stdout:173: finishing write tid 3 to smithi12223906-7 2024-09-15T03:46:07.852 INFO:tasks.rados.rados.0.smithi122.stdout:173: finishing write tid 4 to smithi12223906-7 2024-09-15T03:46:07.852 INFO:tasks.rados.rados.0.smithi122.stdout:175: done (2 left) 2024-09-15T03:46:07.852 INFO:tasks.rados.rados.0.smithi122.stdout:176: write oid 24 current snap is 18 2024-09-15T03:46:07.852 INFO:tasks.rados.rados.0.smithi122.stdout:176: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2024-09-15T03:46:07.858 INFO:tasks.rados.rados.0.smithi122.stdout:176: writing smithi12223906-24 from 447086 to 1217724 tid 1 2024-09-15T03:46:07.867 INFO:tasks.rados.rados.0.smithi122.stdout:176: writing smithi12223906-24 from 1935000 to 2733016 tid 2 2024-09-15T03:46:07.873 INFO:tasks.rados.rados.0.smithi122.stdout:176: writing smithi12223906-24 from 3458512 to 3647086 tid 3 2024-09-15T03:46:07.873 INFO:tasks.rados.rados.0.smithi122.stdout:177: write oid 41 current snap is 18 2024-09-15T03:46:07.873 INFO:tasks.rados.rados.0.smithi122.stdout:173: finishing write tid 5 to smithi12223906-7 2024-09-15T03:46:07.873 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 32 (ObjNum 96 snap 18 seq_num 96) dirty exists 2024-09-15T03:46:07.873 INFO:tasks.rados.rados.0.smithi122.stdout:173: left oid 7 (ObjNum 96 snap 18 seq_num 96) 2024-09-15T03:46:07.873 INFO:tasks.rados.rados.0.smithi122.stdout:177: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2024-09-15T03:46:07.878 INFO:tasks.rados.rados.0.smithi122.stdout:177: writing smithi12223906-41 from 463893 to 1026133 tid 1 2024-09-15T03:46:07.885 INFO:tasks.rados.rados.0.smithi122.stdout:177: writing smithi12223906-41 from 1593482 to 2251509 tid 2 2024-09-15T03:46:07.892 INFO:tasks.rados.rados.0.smithi122.stdout:177: writing smithi12223906-41 from 2685935 to 3245276 tid 3 2024-09-15T03:46:07.892 INFO:tasks.rados.rados.0.smithi122.stdout:173: done (3 left) 2024-09-15T03:46:07.892 INFO:tasks.rados.rados.0.smithi122.stdout:178: delete oid 39 current snap is 18 2024-09-15T03:46:07.892 INFO:tasks.rados.rados.0.smithi122.stdout:174: finishing copy_from racing read to smithi12223906-45 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:174: finishing copy_from to smithi12223906-45 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 24 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:176: finishing write tid 1 to smithi12223906-24 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:176: finishing write tid 2 to smithi12223906-24 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:176: finishing write tid 3 to smithi12223906-24 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:176: finishing write tid 4 to smithi12223906-24 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:176: finishing write tid 5 to smithi12223906-24 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 85 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:176: left oid 24 (ObjNum 98 snap 18 seq_num 98) 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:177: finishing write tid 1 to smithi12223906-41 2024-09-15T03:46:07.893 INFO:tasks.rados.rados.0.smithi122.stdout:177: finishing write tid 2 to smithi12223906-41 2024-09-15T03:46:07.896 INFO:tasks.rados.rados.0.smithi122.stdout:174: done (3 left) 2024-09-15T03:46:07.897 INFO:tasks.rados.rados.0.smithi122.stdout:176: done (2 left) 2024-09-15T03:46:07.897 INFO:tasks.rados.rados.0.smithi122.stdout:178: done (1 left) 2024-09-15T03:46:07.897 INFO:tasks.rados.rados.0.smithi122.stdout:179: write oid 12 current snap is 18 2024-09-15T03:46:07.897 INFO:tasks.rados.rados.0.smithi122.stdout:179: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2024-09-15T03:46:07.904 INFO:tasks.rados.rados.0.smithi122.stdout:179: writing smithi12223906-12 from 497507 to 1210245 tid 1 2024-09-15T03:46:07.911 INFO:tasks.rados.rados.0.smithi122.stdout:179: writing smithi12223906-12 from 1710446 to 2372142 tid 2 2024-09-15T03:46:07.919 INFO:tasks.rados.rados.0.smithi122.stdout:179: writing smithi12223906-12 from 3140781 to 3697507 tid 3 2024-09-15T03:46:07.919 INFO:tasks.rados.rados.0.smithi122.stdout:180: snap_remove snap 16 2024-09-15T03:46:08.849 INFO:tasks.rados.rados.0.smithi122.stdout:180: done (2 left) 2024-09-15T03:46:08.849 INFO:tasks.rados.rados.0.smithi122.stdout:181: rollback oid 17 current snap is 18 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 17 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:177: finishing write tid 3 to smithi12223906-41 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:177: finishing write tid 4 to smithi12223906-41 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:177: finishing write tid 5 to smithi12223906-41 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 33 (ObjNum 99 snap 18 seq_num 99) dirty exists 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:177: left oid 41 (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:179: finishing write tid 1 to smithi12223906-12 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:179: finishing write tid 2 to smithi12223906-12 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:179: finishing write tid 3 to smithi12223906-12 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:179: finishing write tid 4 to smithi12223906-12 2024-09-15T03:46:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:179: finishing write tid 5 to smithi12223906-12 2024-09-15T03:46:08.851 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 22 (ObjNum 101 snap 18 seq_num 101) dirty exists 2024-09-15T03:46:08.851 INFO:tasks.rados.rados.0.smithi122.stdout:179: left oid 12 (ObjNum 101 snap 18 seq_num 101) 2024-09-15T03:46:08.851 INFO:tasks.rados.rados.0.smithi122.stdout:177: done (2 left) 2024-09-15T03:46:08.851 INFO:tasks.rados.rados.0.smithi122.stdout:179: done (1 left) 2024-09-15T03:46:08.851 INFO:tasks.rados.rados.0.smithi122.stdout:182: snap_remove snap 7 2024-09-15T03:46:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:182: done (1 left) 2024-09-15T03:46:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:183: rollback oid 4 current snap is 18 2024-09-15T03:46:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:181: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:46:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 0 (ObjNum 433 snap 0 seq_num 1842656016) dirty dne 2024-09-15T03:46:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 13 2024-09-15T03:46:09.862 INFO:tasks.rados.rados.0.smithi122.stdout:181: done (1 left) 2024-09-15T03:46:09.862 INFO:tasks.rados.rados.0.smithi122.stdout:184: write (excl) oid 19 current snap is 18 2024-09-15T03:46:09.862 INFO:tasks.rados.rados.0.smithi122.stdout:184: seq_num 102 ranges {514314=787987,1768928=405354,2768204=509510} 2024-09-15T03:46:09.872 INFO:tasks.rados.rados.0.smithi122.stdout:184: writing smithi12223906-19 from 514314 to 1302301 tid 1 2024-09-15T03:46:09.877 INFO:tasks.rados.rados.0.smithi122.stdout:184: writing smithi12223906-19 from 1768928 to 2174282 tid 2 2024-09-15T03:46:09.884 INFO:tasks.rados.rados.0.smithi122.stdout:184: writing smithi12223906-19 from 2768204 to 3277714 tid 3 2024-09-15T03:46:09.884 INFO:tasks.rados.rados.0.smithi122.stdout:185: read oid 10 snap -1 2024-09-15T03:46:09.884 INFO:tasks.rados.rados.0.smithi122.stdout:185: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:186: read oid 5 snap -1 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:186: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:183: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 20 (ObjNum 129 snap 0 seq_num 1843204144) dirty dne 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:184: finishing write tid 1 to smithi12223906-19 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:184: finishing write tid 2 to smithi12223906-19 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:183: done (3 left) 2024-09-15T03:46:09.885 INFO:tasks.rados.rados.0.smithi122.stdout:187: snap_create 2024-09-15T03:46:09.888 INFO:tasks.rados.rados.0.smithi122.stdout:184: finishing write tid 3 to smithi12223906-19 2024-09-15T03:46:09.888 INFO:tasks.rados.rados.0.smithi122.stdout:184: finishing write tid 4 to smithi12223906-19 2024-09-15T03:46:09.888 INFO:tasks.rados.rados.0.smithi122.stdout:184: finishing write tid 5 to smithi12223906-19 2024-09-15T03:46:09.888 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 64 (ObjNum 102 snap 18 seq_num 102) dirty exists 2024-09-15T03:46:09.888 INFO:tasks.rados.rados.0.smithi122.stdout:184: left oid 19 (ObjNum 102 snap 18 seq_num 102) 2024-09-15T03:46:09.890 INFO:tasks.rados.rados.0.smithi122.stdout:185: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:46:09.961 INFO:tasks.rados.rados.0.smithi122.stdout:186: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-15T03:46:10.870 INFO:tasks.rados.rados.0.smithi122.stdout:184: done (3 left) 2024-09-15T03:46:10.870 INFO:tasks.rados.rados.0.smithi122.stdout:185: done (2 left) 2024-09-15T03:46:10.870 INFO:tasks.rados.rados.0.smithi122.stdout:186: done (1 left) 2024-09-15T03:46:10.871 INFO:tasks.rados.rados.0.smithi122.stdout:187: done (0 left) 2024-09-15T03:46:10.871 INFO:tasks.rados.rados.0.smithi122.stdout:188: snap_remove snap 13 2024-09-15T03:46:11.692 INFO:tasks.rados.rados.0.smithi122.stdout:188: done (0 left) 2024-09-15T03:46:11.692 INFO:tasks.rados.rados.0.smithi122.stdout:189: write (excl) oid 17 current snap is 19 2024-09-15T03:46:11.692 INFO:tasks.rados.rados.0.smithi122.stdout:189: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2024-09-15T03:46:11.698 INFO:tasks.rados.rados.0.smithi122.stdout:189: writing smithi12223906-17 from 531121 to 994357 tid 1 2024-09-15T03:46:11.707 INFO:tasks.rados.rados.0.smithi122.stdout:189: writing smithi12223906-17 from 1711057 to 2376422 tid 2 2024-09-15T03:46:11.715 INFO:tasks.rados.rados.0.smithi122.stdout:189: writing smithi12223906-17 from 3079274 to 3731121 tid 3 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:190: rollback oid 24 current snap is 19 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 17 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:189: finishing write tid 1 to smithi12223906-17 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:189: finishing write tid 2 to smithi12223906-17 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:191: rollback oid 42 current snap is 19 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 18 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:192: read oid 4 snap 17 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:192: expect deleted 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:193: rollback oid 41 current snap is 19 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 18 2024-09-15T03:46:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:194: snap_create 2024-09-15T03:46:11.721 INFO:tasks.rados.rados.0.smithi122.stdout:193: finishing rollback tid 0 to smithi12223906-41 2024-09-15T03:46:11.723 INFO:tasks.rados.rados.0.smithi122.stdout:193: finishing rollback tid 1 to smithi12223906-41 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:193: finishing rollback tid 2 to smithi12223906-41 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 45 (ObjNum 99 snap 18 seq_num 99) dirty exists 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:190: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:189: finishing write tid 3 to smithi12223906-17 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:189: finishing write tid 4 to smithi12223906-17 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:189: finishing write tid 5 to smithi12223906-17 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 62 (ObjNum 103 snap 19 seq_num 103) dirty exists 2024-09-15T03:46:11.724 INFO:tasks.rados.rados.0.smithi122.stdout:189: left oid 17 (ObjNum 103 snap 19 seq_num 103) 2024-09-15T03:46:11.725 INFO:tasks.rados.rados.0.smithi122.stdout:190: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:46:11.725 INFO:tasks.rados.rados.0.smithi122.stdout:190: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:46:11.725 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 115 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-15T03:46:11.726 INFO:tasks.rados.rados.0.smithi122.stdout:191: finishing rollback tid 0 to smithi12223906-42 2024-09-15T03:46:11.726 INFO:tasks.rados.rados.0.smithi122.stdout:191: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:46:11.726 INFO:tasks.rados.rados.0.smithi122.stdout:191: finishing rollback tid 2 to smithi12223906-42 2024-09-15T03:46:11.726 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 119 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:46:12.703 INFO:tasks.rados.rados.0.smithi122.stdout:189: done (5 left) 2024-09-15T03:46:12.703 INFO:tasks.rados.rados.0.smithi122.stdout:190: done (4 left) 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:191: done (3 left) 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:192: done (2 left) 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:193: done (1 left) 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:194: done (0 left) 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:195: rollback oid 17 current snap is 20 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 17 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:196: read oid 33 snap 18 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:196: expect deleted 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:197: rollback oid 40 current snap is 20 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 18 2024-09-15T03:46:12.704 INFO:tasks.rados.rados.0.smithi122.stdout:198: delete oid 7 current snap is 20 2024-09-15T03:46:12.709 INFO:tasks.rados.rados.0.smithi122.stdout:197: finishing rollback tid 1 to smithi12223906-40 2024-09-15T03:46:12.709 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 0 (ObjNum 129 snap 0 seq_num 1842611832) dirty dne 2024-09-15T03:46:12.710 INFO:tasks.rados.rados.0.smithi122.stdout:196: done (3 left) 2024-09-15T03:46:12.710 INFO:tasks.rados.rados.0.smithi122.stdout:197: done (2 left) 2024-09-15T03:46:12.710 INFO:tasks.rados.rados.0.smithi122.stdout:198: done (1 left) 2024-09-15T03:46:12.710 INFO:tasks.rados.rados.0.smithi122.stdout:199: delete oid 46 current snap is 20 2024-09-15T03:46:12.715 INFO:tasks.rados.rados.0.smithi122.stdout:199: done (1 left) 2024-09-15T03:46:12.715 INFO:tasks.rados.rados.0.smithi122.stdout:200: rollback oid 29 current snap is 20 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 29 to 17 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:201: rollback oid 25 current snap is 20 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 17 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:202: rollback oid 50 current snap is 20 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 17 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:203: read oid 10 snap 19 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:203: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:204: write (excl) oid 33 current snap is 20 2024-09-15T03:46:12.716 INFO:tasks.rados.rados.0.smithi122.stdout:204: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2024-09-15T03:46:12.724 INFO:tasks.rados.rados.0.smithi122.stdout:204: writing smithi12223906-33 from 581542 to 1270525 tid 1 2024-09-15T03:46:12.730 INFO:tasks.rados.rados.0.smithi122.stdout:204: writing smithi12223906-33 from 1886503 to 2299195 tid 2 2024-09-15T03:46:12.737 INFO:tasks.rados.rados.0.smithi122.stdout:204: writing smithi12223906-33 from 2761543 to 3432494 tid 3 2024-09-15T03:46:12.737 INFO:tasks.rados.rados.0.smithi122.stdout:205: rollback oid 7 current snap is 20 2024-09-15T03:46:12.737 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 17 2024-09-15T03:46:12.737 INFO:tasks.rados.rados.0.smithi122.stdout:202: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:46:12.738 INFO:tasks.rados.rados.0.smithi122.stdout:201: finishing rollback tid 0 to smithi12223906-25 2024-09-15T03:46:12.738 INFO:tasks.rados.rados.0.smithi122.stdout:203: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:202: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:201: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:202: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 61 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:201: finishing rollback tid 2 to smithi12223906-25 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 123 (ObjNum 88 snap 15 seq_num 88) dirty exists 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:195: finishing rollback tid 0 to smithi12223906-17 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:195: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 65 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:200: finishing rollback tid 0 to smithi12223906-29 2024-09-15T03:46:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:200: finishing rollback tid 1 to smithi12223906-29 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:200: finishing rollback tid 2 to smithi12223906-29 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 46 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:204: finishing write tid 1 to smithi12223906-33 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:204: finishing write tid 2 to smithi12223906-33 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:205: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:205: finishing rollback tid 2 to smithi12223906-7 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 48 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:204: finishing write tid 3 to smithi12223906-33 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:204: finishing write tid 4 to smithi12223906-33 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:204: finishing write tid 5 to smithi12223906-33 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 65 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:46:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:204: left oid 33 (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:195: done (6 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:200: done (5 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:201: done (4 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:202: done (3 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:203: done (2 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:204: done (1 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:205: done (0 left) 2024-09-15T03:46:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:206: snap_create 2024-09-15T03:46:13.714 INFO:tasks.rados.rados.0.smithi122.stdout:206: done (0 left) 2024-09-15T03:46:13.714 INFO:tasks.rados.rados.0.smithi122.stdout:207: read oid 43 snap -1 2024-09-15T03:46:13.714 INFO:tasks.rados.rados.0.smithi122.stdout:207: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:46:13.714 INFO:tasks.rados.rados.0.smithi122.stdout:208: delete oid 7 current snap is 21 2024-09-15T03:46:13.718 INFO:tasks.rados.rados.0.smithi122.stdout:208: done (1 left) 2024-09-15T03:46:13.718 INFO:tasks.rados.rados.0.smithi122.stdout:209: snap_create 2024-09-15T03:46:13.722 INFO:tasks.rados.rados.0.smithi122.stdout:207: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-15T03:46:14.720 INFO:tasks.rados.rados.0.smithi122.stdout:207: done (1 left) 2024-09-15T03:46:14.721 INFO:tasks.rados.rados.0.smithi122.stdout:209: done (0 left) 2024-09-15T03:46:14.721 INFO:tasks.rados.rados.0.smithi122.stdout:210: rollback oid 18 current snap is 22 2024-09-15T03:46:14.721 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 20 2024-09-15T03:46:14.721 INFO:tasks.rados.rados.0.smithi122.stdout:211: snap_remove snap 18 2024-09-15T03:46:15.732 INFO:tasks.rados.rados.0.smithi122.stdout:211: done (1 left) 2024-09-15T03:46:15.732 INFO:tasks.rados.rados.0.smithi122.stdout:212: snap_create 2024-09-15T03:46:15.732 INFO:tasks.rados.rados.0.smithi122.stdout:210: finishing rollback tid 0 to smithi12223906-18 2024-09-15T03:46:15.732 INFO:tasks.rados.rados.0.smithi122.stdout:210: finishing rollback tid 1 to smithi12223906-18 2024-09-15T03:46:15.732 INFO:tasks.rados.rados.0.smithi122.stdout:210: finishing rollback tid 2 to smithi12223906-18 2024-09-15T03:46:15.732 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 69 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-15T03:46:16.694 INFO:tasks.rados.rados.0.smithi122.stdout:210: done (1 left) 2024-09-15T03:46:16.694 INFO:tasks.rados.rados.0.smithi122.stdout:212: done (0 left) 2024-09-15T03:46:16.694 INFO:tasks.rados.rados.0.smithi122.stdout:213: write oid 18 current snap is 23 2024-09-15T03:46:16.694 INFO:tasks.rados.rados.0.smithi122.stdout:213: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2024-09-15T03:46:16.706 INFO:tasks.rados.rados.0.smithi122.stdout:213: writing smithi12223906-18 from 615156 to 1170990 tid 1 2024-09-15T03:46:16.713 INFO:tasks.rados.rados.0.smithi122.stdout:213: writing smithi12223906-18 from 1603467 to 2019828 tid 2 2024-09-15T03:46:16.721 INFO:tasks.rados.rados.0.smithi122.stdout:213: writing smithi12223906-18 from 2816389 to 3509884 tid 3 2024-09-15T03:46:16.721 INFO:tasks.rados.rados.0.smithi122.stdout:214: write oid 45 current snap is 23 2024-09-15T03:46:16.722 INFO:tasks.rados.rados.0.smithi122.stdout:214: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2024-09-15T03:46:16.728 INFO:tasks.rados.rados.0.smithi122.stdout:214: writing smithi12223906-45 from 631963 to 1263046 tid 1 2024-09-15T03:46:16.736 INFO:tasks.rados.rados.0.smithi122.stdout:214: writing smithi12223906-45 from 2061949 to 2621968 tid 2 2024-09-15T03:46:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:214: writing smithi12223906-45 from 3243812 to 3748579 tid 3 2024-09-15T03:46:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:215: write oid 7 current snap is 23 2024-09-15T03:46:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:215: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2024-09-15T03:46:16.750 INFO:tasks.rados.rados.0.smithi122.stdout:215: writing smithi12223906-7 from 648770 to 1355102 tid 1 2024-09-15T03:46:16.757 INFO:tasks.rados.rados.0.smithi122.stdout:215: writing smithi12223906-7 from 2120431 to 2540461 tid 2 2024-09-15T03:46:16.764 INFO:tasks.rados.rados.0.smithi122.stdout:215: writing smithi12223906-7 from 3271235 to 3703627 tid 3 2024-09-15T03:46:16.764 INFO:tasks.rados.rados.0.smithi122.stdout:216: rollback oid 17 current snap is 23 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 20 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:213: finishing write tid 1 to smithi12223906-18 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:213: finishing write tid 2 to smithi12223906-18 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:213: finishing write tid 3 to smithi12223906-18 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:213: finishing write tid 4 to smithi12223906-18 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:213: finishing write tid 5 to smithi12223906-18 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 78 (ObjNum 108 snap 23 seq_num 108) dirty exists 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:213: left oid 18 (ObjNum 108 snap 23 seq_num 108) 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:214: finishing write tid 1 to smithi12223906-45 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:214: finishing write tid 2 to smithi12223906-45 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:214: finishing write tid 3 to smithi12223906-45 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:214: finishing write tid 4 to smithi12223906-45 2024-09-15T03:46:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:214: finishing write tid 5 to smithi12223906-45 2024-09-15T03:46:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 77 (ObjNum 109 snap 23 seq_num 109) dirty exists 2024-09-15T03:46:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:214: left oid 45 (ObjNum 109 snap 23 seq_num 109) 2024-09-15T03:46:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:213: done (3 left) 2024-09-15T03:46:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:214: done (2 left) 2024-09-15T03:46:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:217: snap_remove snap 19 2024-09-15T03:46:17.747 INFO:tasks.rados.rados.0.smithi122.stdout:217: done (2 left) 2024-09-15T03:46:17.747 INFO:tasks.rados.rados.0.smithi122.stdout:218: snap_remove snap 21 2024-09-15T03:46:18.749 INFO:tasks.rados.rados.0.smithi122.stdout:218: done (2 left) 2024-09-15T03:46:18.749 INFO:tasks.rados.rados.0.smithi122.stdout:219: write (excl) oid 47 current snap is 23 2024-09-15T03:46:18.749 INFO:tasks.rados.rados.0.smithi122.stdout:215: finishing write tid 1 to smithi12223906-7 2024-09-15T03:46:18.749 INFO:tasks.rados.rados.0.smithi122.stdout:219: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2024-09-15T03:46:18.758 INFO:tasks.rados.rados.0.smithi122.stdout:219: writing smithi12223906-47 from 665577 to 1447158 tid 1 2024-09-15T03:46:18.764 INFO:tasks.rados.rados.0.smithi122.stdout:219: writing smithi12223906-47 from 2062560 to 2626248 tid 2 2024-09-15T03:46:18.771 INFO:tasks.rados.rados.0.smithi122.stdout:219: writing smithi12223906-47 from 3182305 to 3825969 tid 3 2024-09-15T03:46:18.771 INFO:tasks.rados.rados.0.smithi122.stdout:220: read oid 42 snap -1 2024-09-15T03:46:18.771 INFO:tasks.rados.rados.0.smithi122.stdout:220: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:46:18.772 INFO:tasks.rados.rados.0.smithi122.stdout:221: snap_remove snap 22 2024-09-15T03:46:19.770 INFO:tasks.rados.rados.0.smithi122.stdout:215: finishing write tid 2 to smithi12223906-7 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:215: finishing write tid 3 to smithi12223906-7 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:215: finishing write tid 4 to smithi12223906-7 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:216: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 65 (ObjNum 33 snap 0 seq_num 1843174576) dirty dne 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:215: finishing write tid 5 to smithi12223906-7 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 58 (ObjNum 110 snap 23 seq_num 110) dirty exists 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:215: left oid 7 (ObjNum 110 snap 23 seq_num 110) 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:219: finishing write tid 1 to smithi12223906-47 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:219: finishing write tid 2 to smithi12223906-47 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:219: finishing write tid 3 to smithi12223906-47 2024-09-15T03:46:19.771 INFO:tasks.rados.rados.0.smithi122.stdout:220: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:46:19.860 INFO:tasks.rados.rados.0.smithi122.stdout:219: finishing write tid 4 to smithi12223906-47 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:219: finishing write tid 5 to smithi12223906-47 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 25 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:219: left oid 47 (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:215: done (4 left) 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:216: done (3 left) 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:219: done (2 left) 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:220: done (1 left) 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:221: done (0 left) 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:222: write oid 26 current snap is 23 2024-09-15T03:46:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:222: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2024-09-15T03:46:19.873 INFO:tasks.rados.rados.0.smithi122.stdout:222: writing smithi12223906-26 from 682384 to 1139214 tid 1 2024-09-15T03:46:19.879 INFO:tasks.rados.rados.0.smithi122.stdout:222: writing smithi12223906-26 from 1721042 to 2144741 tid 2 2024-09-15T03:46:19.885 INFO:tasks.rados.rados.0.smithi122.stdout:222: writing smithi12223906-26 from 2809728 to 3264664 tid 3 2024-09-15T03:46:19.889 INFO:tasks.rados.rados.0.smithi122.stdout:222: writing smithi12223906-26 from 3731357 to 3882384 tid 4 2024-09-15T03:46:19.889 INFO:tasks.rados.rados.0.smithi122.stdout:223: read oid 41 snap -1 2024-09-15T03:46:19.889 INFO:tasks.rados.rados.0.smithi122.stdout:223: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:19.889 INFO:tasks.rados.rados.0.smithi122.stdout:224: write (excl) oid 23 current snap is 23 2024-09-15T03:46:19.889 INFO:tasks.rados.rados.0.smithi122.stdout:224: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2024-09-15T03:46:19.896 INFO:tasks.rados.rados.0.smithi122.stdout:224: writing smithi12223906-23 from 699191 to 1231270 tid 1 2024-09-15T03:46:19.902 INFO:tasks.rados.rados.0.smithi122.stdout:224: writing smithi12223906-23 from 1779524 to 2346881 tid 2 2024-09-15T03:46:19.909 INFO:tasks.rados.rados.0.smithi122.stdout:224: writing smithi12223906-23 from 2837151 to 3503359 tid 3 2024-09-15T03:46:19.909 INFO:tasks.rados.rados.0.smithi122.stdout:225: write oid 43 current snap is 23 2024-09-15T03:46:19.909 INFO:tasks.rados.rados.0.smithi122.stdout:222: finishing write tid 1 to smithi12223906-26 2024-09-15T03:46:19.910 INFO:tasks.rados.rados.0.smithi122.stdout:225: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2024-09-15T03:46:19.916 INFO:tasks.rados.rados.0.smithi122.stdout:225: writing smithi12223906-43 from 715998 to 1323326 tid 1 2024-09-15T03:46:19.922 INFO:tasks.rados.rados.0.smithi122.stdout:225: writing smithi12223906-43 from 1838006 to 2265374 tid 2 2024-09-15T03:46:19.928 INFO:tasks.rados.rados.0.smithi122.stdout:225: writing smithi12223906-43 from 2864574 to 3342054 tid 3 2024-09-15T03:46:19.931 INFO:tasks.rados.rados.0.smithi122.stdout:225: writing smithi12223906-43 from 3863835 to 3915998 tid 4 2024-09-15T03:46:19.931 INFO:tasks.rados.rados.0.smithi122.stdout:226: snap_remove snap 17 2024-09-15T03:46:20.773 INFO:tasks.rados.rados.0.smithi122.stdout:226: done (4 left) 2024-09-15T03:46:20.773 INFO:tasks.rados.rados.0.smithi122.stdout:227: copy_from oid 50 from oid 8 current snap is 23 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:222: finishing write tid 2 to smithi12223906-26 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:222: finishing write tid 3 to smithi12223906-26 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:222: finishing write tid 4 to smithi12223906-26 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:222: finishing write tid 5 to smithi12223906-26 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:222: finishing write tid 6 to smithi12223906-26 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 75 (ObjNum 112 snap 23 seq_num 112) dirty exists 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:222: left oid 26 (ObjNum 112 snap 23 seq_num 112) 2024-09-15T03:46:20.774 INFO:tasks.rados.rados.0.smithi122.stdout:223: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:20.944 INFO:tasks.rados.rados.0.smithi122.stdout:224: finishing write tid 1 to smithi12223906-23 2024-09-15T03:46:20.944 INFO:tasks.rados.rados.0.smithi122.stdout:224: finishing write tid 2 to smithi12223906-23 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:224: finishing write tid 3 to smithi12223906-23 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:224: finishing write tid 4 to smithi12223906-23 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:224: finishing write tid 5 to smithi12223906-23 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 90 (ObjNum 113 snap 23 seq_num 113) dirty exists 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:224: left oid 23 (ObjNum 113 snap 23 seq_num 113) 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: finishing write tid 1 to smithi12223906-43 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: finishing write tid 2 to smithi12223906-43 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: finishing write tid 3 to smithi12223906-43 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: finishing write tid 4 to smithi12223906-43 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: finishing write tid 5 to smithi12223906-43 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: finishing write tid 6 to smithi12223906-43 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 137 (ObjNum 114 snap 23 seq_num 114) dirty exists 2024-09-15T03:46:20.945 INFO:tasks.rados.rados.0.smithi122.stdout:225: left oid 43 (ObjNum 114 snap 23 seq_num 114) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:227: finishing copy_from to smithi12223906-50 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:227: got expected ENOENT (src dne) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:227: finishing copy_from racing read to smithi12223906-50 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:222: done (4 left) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:223: done (3 left) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:224: done (2 left) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:225: done (1 left) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:227: done (0 left) 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:228: write oid 20 current snap is 23 2024-09-15T03:46:20.946 INFO:tasks.rados.rados.0.smithi122.stdout:228: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2024-09-15T03:46:20.962 INFO:tasks.rados.rados.0.smithi122.stdout:228: writing smithi12223906-20 from 732805 to 1531735 tid 1 2024-09-15T03:46:20.969 INFO:tasks.rados.rados.0.smithi122.stdout:228: writing smithi12223906-20 from 2296488 to 2867514 tid 2 2024-09-15T03:46:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:228: writing smithi12223906-20 from 3291997 to 3697102 tid 3 2024-09-15T03:46:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:229: snap_remove snap 20 2024-09-15T03:46:21.696 INFO:tasks.rados.rados.0.smithi122.stdout:229: done (1 left) 2024-09-15T03:46:21.697 INFO:tasks.rados.rados.0.smithi122.stdout:230: copy_from oid 35 from oid 47 current snap is 23 2024-09-15T03:46:21.697 INFO:tasks.rados.rados.0.smithi122.stdout:231: write oid 45 current snap is 23 2024-09-15T03:46:21.697 INFO:tasks.rados.rados.0.smithi122.stdout:231: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2024-09-15T03:46:21.707 INFO:tasks.rados.rados.0.smithi122.stdout:231: writing smithi12223906-45 from 749612 to 1223791 tid 1 2024-09-15T03:46:21.714 INFO:tasks.rados.rados.0.smithi122.stdout:231: writing smithi12223906-45 from 1954970 to 2386007 tid 2 2024-09-15T03:46:21.721 INFO:tasks.rados.rados.0.smithi122.stdout:231: writing smithi12223906-45 from 2919420 to 3535797 tid 3 2024-09-15T03:46:21.721 INFO:tasks.rados.rados.0.smithi122.stdout:232: read oid 44 snap -1 2024-09-15T03:46:21.721 INFO:tasks.rados.rados.0.smithi122.stdout:232: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:46:21.721 INFO:tasks.rados.rados.0.smithi122.stdout:233: snap_create 2024-09-15T03:46:21.721 INFO:tasks.rados.rados.0.smithi122.stdout:228: finishing write tid 1 to smithi12223906-20 2024-09-15T03:46:21.721 INFO:tasks.rados.rados.0.smithi122.stdout:228: finishing write tid 2 to smithi12223906-20 2024-09-15T03:46:21.722 INFO:tasks.rados.rados.0.smithi122.stdout:228: finishing write tid 3 to smithi12223906-20 2024-09-15T03:46:21.722 INFO:tasks.rados.rados.0.smithi122.stdout:228: finishing write tid 4 to smithi12223906-20 2024-09-15T03:46:21.722 INFO:tasks.rados.rados.0.smithi122.stdout:228: finishing write tid 5 to smithi12223906-20 2024-09-15T03:46:21.722 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 8 (ObjNum 115 snap 23 seq_num 115) dirty exists 2024-09-15T03:46:21.722 INFO:tasks.rados.rados.0.smithi122.stdout:228: left oid 20 (ObjNum 115 snap 23 seq_num 115) 2024-09-15T03:46:21.723 INFO:tasks.rados.rados.0.smithi122.stdout:230: finishing copy_from racing read to smithi12223906-35 2024-09-15T03:46:21.727 INFO:tasks.rados.rados.0.smithi122.stdout:230: finishing copy_from to smithi12223906-35 2024-09-15T03:46:21.727 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 25 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:46:21.728 INFO:tasks.rados.rados.0.smithi122.stdout:231: finishing write tid 1 to smithi12223906-45 2024-09-15T03:46:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:231: finishing write tid 2 to smithi12223906-45 2024-09-15T03:46:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:231: finishing write tid 3 to smithi12223906-45 2024-09-15T03:46:21.733 INFO:tasks.rados.rados.0.smithi122.stdout:232: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:46:21.827 INFO:tasks.rados.rados.0.smithi122.stdout:231: finishing write tid 4 to smithi12223906-45 2024-09-15T03:46:21.827 INFO:tasks.rados.rados.0.smithi122.stdout:231: finishing write tid 5 to smithi12223906-45 2024-09-15T03:46:21.827 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 95 (ObjNum 116 snap 23 seq_num 116) dirty exists 2024-09-15T03:46:21.827 INFO:tasks.rados.rados.0.smithi122.stdout:231: left oid 45 (ObjNum 116 snap 23 seq_num 116) 2024-09-15T03:46:22.709 INFO:tasks.rados.rados.0.smithi122.stdout:228: done (4 left) 2024-09-15T03:46:22.709 INFO:tasks.rados.rados.0.smithi122.stdout:230: done (3 left) 2024-09-15T03:46:22.709 INFO:tasks.rados.rados.0.smithi122.stdout:231: done (2 left) 2024-09-15T03:46:22.709 INFO:tasks.rados.rados.0.smithi122.stdout:232: done (1 left) 2024-09-15T03:46:22.710 INFO:tasks.rados.rados.0.smithi122.stdout:233: done (0 left) 2024-09-15T03:46:22.710 INFO:tasks.rados.rados.0.smithi122.stdout:234: snap_remove snap 23 2024-09-15T03:46:23.712 INFO:tasks.rados.rados.0.smithi122.stdout:234: done (0 left) 2024-09-15T03:46:23.712 INFO:tasks.rados.rados.0.smithi122.stdout:235: write (excl) oid 39 current snap is 24 2024-09-15T03:46:23.712 INFO:tasks.rados.rados.0.smithi122.stdout:235: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2024-09-15T03:46:23.721 INFO:tasks.rados.rados.0.smithi122.stdout:235: writing smithi12223906-39 from 766419 to 1315847 tid 1 2024-09-15T03:46:23.729 INFO:tasks.rados.rados.0.smithi122.stdout:235: writing smithi12223906-39 from 2013452 to 2588147 tid 2 2024-09-15T03:46:23.736 INFO:tasks.rados.rados.0.smithi122.stdout:235: writing smithi12223906-39 from 3346843 to 3774492 tid 3 2024-09-15T03:46:23.736 INFO:tasks.rados.rados.0.smithi122.stdout:236: write oid 49 current snap is 24 2024-09-15T03:46:23.736 INFO:tasks.rados.rados.0.smithi122.stdout:236: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2024-09-15T03:46:23.744 INFO:tasks.rados.rados.0.smithi122.stdout:236: writing smithi12223906-49 from 783226 to 1407903 tid 1 2024-09-15T03:46:23.750 INFO:tasks.rados.rados.0.smithi122.stdout:236: writing smithi12223906-49 from 2071934 to 2506640 tid 2 2024-09-15T03:46:23.757 INFO:tasks.rados.rados.0.smithi122.stdout:236: writing smithi12223906-49 from 2974266 to 3613187 tid 3 2024-09-15T03:46:23.757 INFO:tasks.rados.rados.0.smithi122.stdout:237: write (excl) oid 5 current snap is 24 2024-09-15T03:46:23.757 INFO:tasks.rados.rados.0.smithi122.stdout:237: seq_num 119 ranges {400033=699926,1614063=385970} 2024-09-15T03:46:23.763 INFO:tasks.rados.rados.0.smithi122.stdout:237: writing smithi12223906-5 from 400033 to 1099959 tid 1 2024-09-15T03:46:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:237: writing smithi12223906-5 from 1614063 to 2000033 tid 2 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:238: copy_from oid 27 from oid 33 current snap is 24 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: finishing write tid 1 to smithi12223906-39 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: finishing write tid 2 to smithi12223906-39 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: finishing write tid 3 to smithi12223906-39 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: finishing write tid 4 to smithi12223906-39 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: finishing write tid 5 to smithi12223906-39 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 155 (ObjNum 117 snap 24 seq_num 117) dirty exists 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: left oid 39 (ObjNum 117 snap 24 seq_num 117) 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:236: finishing write tid 1 to smithi12223906-49 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:236: finishing write tid 2 to smithi12223906-49 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:235: done (3 left) 2024-09-15T03:46:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:239: snap_create 2024-09-15T03:46:23.771 INFO:tasks.rados.rados.0.smithi122.stdout:236: finishing write tid 3 to smithi12223906-49 2024-09-15T03:46:23.771 INFO:tasks.rados.rados.0.smithi122.stdout:236: finishing write tid 4 to smithi12223906-49 2024-09-15T03:46:23.771 INFO:tasks.rados.rados.0.smithi122.stdout:236: finishing write tid 5 to smithi12223906-49 2024-09-15T03:46:23.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 72 (ObjNum 118 snap 24 seq_num 118) dirty exists 2024-09-15T03:46:23.771 INFO:tasks.rados.rados.0.smithi122.stdout:236: left oid 49 (ObjNum 118 snap 24 seq_num 118) 2024-09-15T03:46:23.771 INFO:tasks.rados.rados.0.smithi122.stdout:237: finishing write tid 1 to smithi12223906-5 2024-09-15T03:46:23.775 INFO:tasks.rados.rados.0.smithi122.stdout:237: finishing write tid 2 to smithi12223906-5 2024-09-15T03:46:23.776 INFO:tasks.rados.rados.0.smithi122.stdout:237: finishing write tid 3 to smithi12223906-5 2024-09-15T03:46:23.776 INFO:tasks.rados.rados.0.smithi122.stdout:237: finishing write tid 4 to smithi12223906-5 2024-09-15T03:46:23.776 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 34 (ObjNum 119 snap 24 seq_num 119) dirty exists 2024-09-15T03:46:23.776 INFO:tasks.rados.rados.0.smithi122.stdout:237: left oid 5 (ObjNum 119 snap 24 seq_num 119) 2024-09-15T03:46:23.786 INFO:tasks.rados.rados.0.smithi122.stdout:238: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:46:23.787 INFO:tasks.rados.rados.0.smithi122.stdout:238: finishing copy_from to smithi12223906-27 2024-09-15T03:46:23.787 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 65 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:236: done (3 left) 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:237: done (2 left) 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:238: done (1 left) 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:239: done (0 left) 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:240: copy_from oid 25 from oid 1 current snap is 25 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:241: write oid 26 current snap is 25 2024-09-15T03:46:24.713 INFO:tasks.rados.rados.0.smithi122.stdout:241: seq_num 120 ranges {416840=775175,1672545=344295} 2024-09-15T03:46:24.724 INFO:tasks.rados.rados.0.smithi122.stdout:241: writing smithi12223906-26 from 416840 to 1192015 tid 1 2024-09-15T03:46:24.730 INFO:tasks.rados.rados.0.smithi122.stdout:241: writing smithi12223906-26 from 1672545 to 2016840 tid 2 2024-09-15T03:46:24.731 INFO:tasks.rados.rados.0.smithi122.stdout:242: write oid 24 current snap is 25 2024-09-15T03:46:24.731 INFO:tasks.rados.rados.0.smithi122.stdout:242: seq_num 121 ranges {433647=450424,1331027=582033} 2024-09-15T03:46:24.735 INFO:tasks.rados.rados.0.smithi122.stdout:242: writing smithi12223906-24 from 433647 to 884071 tid 1 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:242: writing smithi12223906-24 from 1331027 to 1913060 tid 2 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:243: snap_create 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:241: finishing write tid 1 to smithi12223906-26 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:241: finishing write tid 2 to smithi12223906-26 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:241: finishing write tid 3 to smithi12223906-26 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:241: finishing write tid 4 to smithi12223906-26 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 83 (ObjNum 120 snap 25 seq_num 120) dirty exists 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:241: left oid 26 (ObjNum 120 snap 25 seq_num 120) 2024-09-15T03:46:24.742 INFO:tasks.rados.rados.0.smithi122.stdout:240: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:46:24.743 INFO:tasks.rados.rados.0.smithi122.stdout:240: finishing copy_from to smithi12223906-25 2024-09-15T03:46:24.743 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 27 (ObjNum 65 snap 6 seq_num 65) dirty exists 2024-09-15T03:46:24.743 INFO:tasks.rados.rados.0.smithi122.stdout:242: finishing write tid 1 to smithi12223906-24 2024-09-15T03:46:24.745 INFO:tasks.rados.rados.0.smithi122.stdout:242: finishing write tid 2 to smithi12223906-24 2024-09-15T03:46:24.746 INFO:tasks.rados.rados.0.smithi122.stdout:242: finishing write tid 3 to smithi12223906-24 2024-09-15T03:46:24.746 INFO:tasks.rados.rados.0.smithi122.stdout:242: finishing write tid 4 to smithi12223906-24 2024-09-15T03:46:24.746 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 161 (ObjNum 121 snap 25 seq_num 121) dirty exists 2024-09-15T03:46:24.746 INFO:tasks.rados.rados.0.smithi122.stdout:242: left oid 24 (ObjNum 121 snap 25 seq_num 121) 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:240: done (3 left) 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:241: done (2 left) 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:242: done (1 left) 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:243: done (0 left) 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:244: copy_from oid 43 from oid 33 current snap is 26 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:245: write oid 13 current snap is 26 2024-09-15T03:46:25.732 INFO:tasks.rados.rados.0.smithi122.stdout:245: seq_num 122 ranges {450454=642026,1505862=544592} 2024-09-15T03:46:25.744 INFO:tasks.rados.rados.0.smithi122.stdout:245: writing smithi12223906-13 from 450454 to 1092480 tid 1 2024-09-15T03:46:25.749 INFO:tasks.rados.rados.0.smithi122.stdout:245: writing smithi12223906-13 from 1505862 to 2050454 tid 2 2024-09-15T03:46:25.749 INFO:tasks.rados.rados.0.smithi122.stdout:246: rollback oid 16 current snap is 26 2024-09-15T03:46:25.750 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 24 2024-09-15T03:46:25.750 INFO:tasks.rados.rados.0.smithi122.stdout:247: delete oid 24 current snap is 26 2024-09-15T03:46:25.758 INFO:tasks.rados.rados.0.smithi122.stdout:244: finishing copy_from racing read to smithi12223906-43 2024-09-15T03:46:25.758 INFO:tasks.rados.rados.0.smithi122.stdout:245: finishing write tid 1 to smithi12223906-13 2024-09-15T03:46:25.759 INFO:tasks.rados.rados.0.smithi122.stdout:246: finishing rollback tid 0 to smithi12223906-16 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:245: finishing write tid 2 to smithi12223906-13 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:245: finishing write tid 3 to smithi12223906-13 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:246: finishing rollback tid 1 to smithi12223906-16 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:245: finishing write tid 4 to smithi12223906-13 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 99 (ObjNum 122 snap 26 seq_num 122) dirty exists 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:245: left oid 13 (ObjNum 122 snap 26 seq_num 122) 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:246: finishing rollback tid 2 to smithi12223906-16 2024-09-15T03:46:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 45 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-15T03:46:25.761 INFO:tasks.rados.rados.0.smithi122.stdout:244: finishing copy_from to smithi12223906-43 2024-09-15T03:46:25.761 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 65 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:46:25.761 INFO:tasks.rados.rados.0.smithi122.stdout:244: done (3 left) 2024-09-15T03:46:25.762 INFO:tasks.rados.rados.0.smithi122.stdout:245: done (2 left) 2024-09-15T03:46:25.762 INFO:tasks.rados.rados.0.smithi122.stdout:246: done (1 left) 2024-09-15T03:46:25.762 INFO:tasks.rados.rados.0.smithi122.stdout:247: done (0 left) 2024-09-15T03:46:25.762 INFO:tasks.rados.rados.0.smithi122.stdout:248: write oid 1 current snap is 26 2024-09-15T03:46:25.762 INFO:tasks.rados.rados.0.smithi122.stdout:248: seq_num 124 ranges {484068=792524,1906473=177595} 2024-09-15T03:46:25.770 INFO:tasks.rados.rados.0.smithi122.stdout:248: writing smithi12223906-1 from 484068 to 1276592 tid 1 2024-09-15T03:46:25.774 INFO:tasks.rados.rados.0.smithi122.stdout:248: writing smithi12223906-1 from 1906473 to 2084068 tid 2 2024-09-15T03:46:25.775 INFO:tasks.rados.rados.0.smithi122.stdout:249: write (excl) oid 12 current snap is 26 2024-09-15T03:46:25.775 INFO:tasks.rados.rados.0.smithi122.stdout:249: seq_num 125 ranges {500875=467773,1564955=535920} 2024-09-15T03:46:25.780 INFO:tasks.rados.rados.0.smithi122.stdout:249: writing smithi12223906-12 from 500875 to 968648 tid 1 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:249: writing smithi12223906-12 from 1564955 to 2100875 tid 2 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:250: delete oid 14 current snap is 26 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:248: finishing write tid 1 to smithi12223906-1 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:248: finishing write tid 2 to smithi12223906-1 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:248: finishing write tid 3 to smithi12223906-1 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:248: finishing write tid 4 to smithi12223906-1 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 49 (ObjNum 124 snap 26 seq_num 124) dirty exists 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:248: left oid 1 (ObjNum 124 snap 26 seq_num 124) 2024-09-15T03:46:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:249: finishing write tid 1 to smithi12223906-12 2024-09-15T03:46:25.792 INFO:tasks.rados.rados.0.smithi122.stdout:248: done (2 left) 2024-09-15T03:46:25.792 INFO:tasks.rados.rados.0.smithi122.stdout:250: done (1 left) 2024-09-15T03:46:25.792 INFO:tasks.rados.rados.0.smithi122.stdout:251: snap_remove snap 24 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:251: done (1 left) 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:252: snap_create 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:249: finishing write tid 2 to smithi12223906-12 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:249: finishing write tid 3 to smithi12223906-12 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:249: finishing write tid 4 to smithi12223906-12 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 36 (ObjNum 125 snap 26 seq_num 125) dirty exists 2024-09-15T03:46:26.705 INFO:tasks.rados.rados.0.smithi122.stdout:249: left oid 12 (ObjNum 125 snap 26 seq_num 125) 2024-09-15T03:46:27.702 INFO:tasks.rados.rados.0.smithi122.stdout:249: done (1 left) 2024-09-15T03:46:27.702 INFO:tasks.rados.rados.0.smithi122.stdout:252: done (0 left) 2024-09-15T03:46:27.702 INFO:tasks.rados.rados.0.smithi122.stdout:253: write (excl) oid 6 current snap is 27 2024-09-15T03:46:27.702 INFO:tasks.rados.rados.0.smithi122.stdout:253: seq_num 127 ranges {534489=618271,1565566=568923} 2024-09-15T03:46:27.711 INFO:tasks.rados.rados.0.smithi122.stdout:253: writing smithi12223906-6 from 534489 to 1152760 tid 1 2024-09-15T03:46:27.717 INFO:tasks.rados.rados.0.smithi122.stdout:253: writing smithi12223906-6 from 1565566 to 2134489 tid 2 2024-09-15T03:46:27.717 INFO:tasks.rados.rados.0.smithi122.stdout:254: read oid 38 snap -1 2024-09-15T03:46:27.717 INFO:tasks.rados.rados.0.smithi122.stdout:254: expect deleted 2024-09-15T03:46:27.717 INFO:tasks.rados.rados.0.smithi122.stdout:255: snap_remove snap 26 2024-09-15T03:46:28.705 INFO:tasks.rados.rados.0.smithi122.stdout:255: done (2 left) 2024-09-15T03:46:28.705 INFO:tasks.rados.rados.0.smithi122.stdout:256: snap_remove snap 25 2024-09-15T03:46:29.723 INFO:tasks.rados.rados.0.smithi122.stdout:256: done (2 left) 2024-09-15T03:46:29.723 INFO:tasks.rados.rados.0.smithi122.stdout:257: write oid 15 current snap is 27 2024-09-15T03:46:29.723 INFO:tasks.rados.rados.0.smithi122.stdout:257: seq_num 128 ranges {551296=693520,2024048=127248} 2024-09-15T03:46:29.735 INFO:tasks.rados.rados.0.smithi122.stdout:257: writing smithi12223906-15 from 551296 to 1244816 tid 1 2024-09-15T03:46:29.741 INFO:tasks.rados.rados.0.smithi122.stdout:257: writing smithi12223906-15 from 2024048 to 2151296 tid 2 2024-09-15T03:46:29.741 INFO:tasks.rados.rados.0.smithi122.stdout:258: copy_from oid 22 from oid 42 current snap is 27 2024-09-15T03:46:29.741 INFO:tasks.rados.rados.0.smithi122.stdout:253: finishing write tid 1 to smithi12223906-6 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:253: finishing write tid 2 to smithi12223906-6 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:253: finishing write tid 3 to smithi12223906-6 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:253: finishing write tid 4 to smithi12223906-6 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 32 (ObjNum 127 snap 27 seq_num 127) dirty exists 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:253: left oid 6 (ObjNum 127 snap 27 seq_num 127) 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:253: done (3 left) 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:254: done (2 left) 2024-09-15T03:46:29.742 INFO:tasks.rados.rados.0.smithi122.stdout:259: snap_create 2024-09-15T03:46:29.744 INFO:tasks.rados.rados.0.smithi122.stdout:257: finishing write tid 1 to smithi12223906-15 2024-09-15T03:46:29.744 INFO:tasks.rados.rados.0.smithi122.stdout:257: finishing write tid 2 to smithi12223906-15 2024-09-15T03:46:29.745 INFO:tasks.rados.rados.0.smithi122.stdout:257: finishing write tid 3 to smithi12223906-15 2024-09-15T03:46:29.746 INFO:tasks.rados.rados.0.smithi122.stdout:257: finishing write tid 4 to smithi12223906-15 2024-09-15T03:46:29.746 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 60 (ObjNum 128 snap 27 seq_num 128) dirty exists 2024-09-15T03:46:29.746 INFO:tasks.rados.rados.0.smithi122.stdout:257: left oid 15 (ObjNum 128 snap 27 seq_num 128) 2024-09-15T03:46:29.754 INFO:tasks.rados.rados.0.smithi122.stdout:258: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:46:29.755 INFO:tasks.rados.rados.0.smithi122.stdout:258: finishing copy_from to smithi12223906-22 2024-09-15T03:46:29.755 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 119 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:46:30.731 INFO:tasks.rados.rados.0.smithi122.stdout:257: done (2 left) 2024-09-15T03:46:30.731 INFO:tasks.rados.rados.0.smithi122.stdout:258: done (1 left) 2024-09-15T03:46:30.731 INFO:tasks.rados.rados.0.smithi122.stdout:259: done (0 left) 2024-09-15T03:46:30.731 INFO:tasks.rados.rados.0.smithi122.stdout:260: read oid 41 snap -1 2024-09-15T03:46:30.731 INFO:tasks.rados.rados.0.smithi122.stdout:260: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:30.731 INFO:tasks.rados.rados.0.smithi122.stdout:261: snap_create 2024-09-15T03:46:30.736 INFO:tasks.rados.rados.0.smithi122.stdout:260: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:31.702 INFO:tasks.rados.rados.0.smithi122.stdout:260: done (1 left) 2024-09-15T03:46:31.703 INFO:tasks.rados.rados.0.smithi122.stdout:261: done (0 left) 2024-09-15T03:46:31.703 INFO:tasks.rados.rados.0.smithi122.stdout:262: delete oid 16 current snap is 29 2024-09-15T03:46:31.715 INFO:tasks.rados.rados.0.smithi122.stdout:262: done (0 left) 2024-09-15T03:46:31.715 INFO:tasks.rados.rados.0.smithi122.stdout:263: read oid 26 snap -1 2024-09-15T03:46:31.715 INFO:tasks.rados.rados.0.smithi122.stdout:263: expect (ObjNum 120 snap 25 seq_num 120) 2024-09-15T03:46:31.715 INFO:tasks.rados.rados.0.smithi122.stdout:264: delete oid 18 current snap is 29 2024-09-15T03:46:31.719 INFO:tasks.rados.rados.0.smithi122.stdout:264: done (1 left) 2024-09-15T03:46:31.719 INFO:tasks.rados.rados.0.smithi122.stdout:265: read oid 38 snap -1 2024-09-15T03:46:31.719 INFO:tasks.rados.rados.0.smithi122.stdout:265: expect deleted 2024-09-15T03:46:31.719 INFO:tasks.rados.rados.0.smithi122.stdout:266: copy_from oid 35 from oid 19 current snap is 29 2024-09-15T03:46:31.719 INFO:tasks.rados.rados.0.smithi122.stdout:267: delete oid 12 current snap is 29 2024-09-15T03:46:31.721 INFO:tasks.rados.rados.0.smithi122.stdout:263: expect (ObjNum 120 snap 25 seq_num 120) 2024-09-15T03:46:31.789 INFO:tasks.rados.rados.0.smithi122.stdout:266: finishing copy_from racing read to smithi12223906-35 2024-09-15T03:46:31.789 INFO:tasks.rados.rados.0.smithi122.stdout:266: finishing copy_from to smithi12223906-35 2024-09-15T03:46:31.790 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 64 (ObjNum 102 snap 18 seq_num 102) dirty exists 2024-09-15T03:46:31.790 INFO:tasks.rados.rados.0.smithi122.stdout:263: done (3 left) 2024-09-15T03:46:31.790 INFO:tasks.rados.rados.0.smithi122.stdout:265: done (2 left) 2024-09-15T03:46:31.790 INFO:tasks.rados.rados.0.smithi122.stdout:266: done (1 left) 2024-09-15T03:46:31.790 INFO:tasks.rados.rados.0.smithi122.stdout:267: done (0 left) 2024-09-15T03:46:31.790 INFO:tasks.rados.rados.0.smithi122.stdout:268: snap_remove snap 28 2024-09-15T03:46:32.744 INFO:tasks.rados.rados.0.smithi122.stdout:268: done (0 left) 2024-09-15T03:46:32.744 INFO:tasks.rados.rados.0.smithi122.stdout:269: snap_remove snap 27 2024-09-15T03:46:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:269: done (0 left) 2024-09-15T03:46:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:270: copy_from oid 2 from oid 29 current snap is 29 2024-09-15T03:46:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:271: copy_from oid 42 from oid 17 current snap is 29 2024-09-15T03:46:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:272: delete oid 4 current snap is 29 2024-09-15T03:46:33.760 INFO:tasks.rados.rados.0.smithi122.stdout:271: finishing copy_from to smithi12223906-42 2024-09-15T03:46:33.760 INFO:tasks.rados.rados.0.smithi122.stdout:271: got expected ENOENT (src dne) 2024-09-15T03:46:33.760 INFO:tasks.rados.rados.0.smithi122.stdout:271: finishing copy_from racing read to smithi12223906-42 2024-09-15T03:46:33.761 INFO:tasks.rados.rados.0.smithi122.stdout:271: done (2 left) 2024-09-15T03:46:33.761 INFO:tasks.rados.rados.0.smithi122.stdout:272: done (1 left) 2024-09-15T03:46:33.761 INFO:tasks.rados.rados.0.smithi122.stdout:273: write (excl) oid 37 current snap is 29 2024-09-15T03:46:33.761 INFO:tasks.rados.rados.0.smithi122.stdout:273: seq_num 133 ranges {635331=786118,1916458=318873} 2024-09-15T03:46:33.772 INFO:tasks.rados.rados.0.smithi122.stdout:273: writing smithi12223906-37 from 635331 to 1421449 tid 1 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:273: writing smithi12223906-37 from 1916458 to 2235331 tid 2 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:274: copy_from oid 14 from oid 25 current snap is 29 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:270: finishing copy_from racing read to smithi12223906-2 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:270: finishing copy_from to smithi12223906-2 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 46 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:270: done (2 left) 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:275: read oid 45 snap -1 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:275: expect (ObjNum 116 snap 23 seq_num 116) 2024-09-15T03:46:33.778 INFO:tasks.rados.rados.0.smithi122.stdout:276: snap_create 2024-09-15T03:46:33.781 INFO:tasks.rados.rados.0.smithi122.stdout:273: finishing write tid 1 to smithi12223906-37 2024-09-15T03:46:33.782 INFO:tasks.rados.rados.0.smithi122.stdout:273: finishing write tid 2 to smithi12223906-37 2024-09-15T03:46:33.782 INFO:tasks.rados.rados.0.smithi122.stdout:273: finishing write tid 3 to smithi12223906-37 2024-09-15T03:46:33.783 INFO:tasks.rados.rados.0.smithi122.stdout:273: finishing write tid 4 to smithi12223906-37 2024-09-15T03:46:33.783 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 59 (ObjNum 133 snap 29 seq_num 133) dirty exists 2024-09-15T03:46:33.783 INFO:tasks.rados.rados.0.smithi122.stdout:273: left oid 37 (ObjNum 133 snap 29 seq_num 133) 2024-09-15T03:46:33.795 INFO:tasks.rados.rados.0.smithi122.stdout:274: finishing copy_from racing read to smithi12223906-14 2024-09-15T03:46:33.795 INFO:tasks.rados.rados.0.smithi122.stdout:275: expect (ObjNum 116 snap 23 seq_num 116) 2024-09-15T03:46:34.050 INFO:tasks.rados.rados.0.smithi122.stdout:274: finishing copy_from to smithi12223906-14 2024-09-15T03:46:34.050 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 27 (ObjNum 65 snap 6 seq_num 65) dirty exists 2024-09-15T03:46:34.771 INFO:tasks.rados.rados.0.smithi122.stdout:273: done (3 left) 2024-09-15T03:46:34.771 INFO:tasks.rados.rados.0.smithi122.stdout:274: done (2 left) 2024-09-15T03:46:34.771 INFO:tasks.rados.rados.0.smithi122.stdout:275: done (1 left) 2024-09-15T03:46:34.771 INFO:tasks.rados.rados.0.smithi122.stdout:276: done (0 left) 2024-09-15T03:46:34.771 INFO:tasks.rados.rados.0.smithi122.stdout:277: write oid 31 current snap is 30 2024-09-15T03:46:34.771 INFO:tasks.rados.rados.0.smithi122.stdout:277: seq_num 134 ranges {652138=461367,1574940=677198} 2024-09-15T03:46:34.781 INFO:tasks.rados.rados.0.smithi122.stdout:277: writing smithi12223906-31 from 652138 to 1113505 tid 1 2024-09-15T03:46:34.788 INFO:tasks.rados.rados.0.smithi122.stdout:277: writing smithi12223906-31 from 1574940 to 2252138 tid 2 2024-09-15T03:46:34.788 INFO:tasks.rados.rados.0.smithi122.stdout:278: write (excl) oid 21 current snap is 30 2024-09-15T03:46:34.788 INFO:tasks.rados.rados.0.smithi122.stdout:278: seq_num 135 ranges {668945=536616,1633422=491363} 2024-09-15T03:46:34.795 INFO:tasks.rados.rados.0.smithi122.stdout:278: writing smithi12223906-21 from 668945 to 1205561 tid 1 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:278: writing smithi12223906-21 from 1633422 to 2124785 tid 2 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:279: snap_create 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:277: finishing write tid 1 to smithi12223906-31 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:277: finishing write tid 2 to smithi12223906-31 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:277: finishing write tid 3 to smithi12223906-31 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:277: finishing write tid 4 to smithi12223906-31 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 34 (ObjNum 134 snap 30 seq_num 134) dirty exists 2024-09-15T03:46:34.801 INFO:tasks.rados.rados.0.smithi122.stdout:277: left oid 31 (ObjNum 134 snap 30 seq_num 134) 2024-09-15T03:46:34.804 INFO:tasks.rados.rados.0.smithi122.stdout:278: finishing write tid 1 to smithi12223906-21 2024-09-15T03:46:34.804 INFO:tasks.rados.rados.0.smithi122.stdout:278: finishing write tid 2 to smithi12223906-21 2024-09-15T03:46:34.805 INFO:tasks.rados.rados.0.smithi122.stdout:278: finishing write tid 3 to smithi12223906-21 2024-09-15T03:46:34.805 INFO:tasks.rados.rados.0.smithi122.stdout:278: finishing write tid 4 to smithi12223906-21 2024-09-15T03:46:34.806 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 63 (ObjNum 135 snap 30 seq_num 135) dirty exists 2024-09-15T03:46:34.806 INFO:tasks.rados.rados.0.smithi122.stdout:278: left oid 21 (ObjNum 135 snap 30 seq_num 135) 2024-09-15T03:46:35.784 INFO:tasks.rados.rados.0.smithi122.stdout:277: done (2 left) 2024-09-15T03:46:35.784 INFO:tasks.rados.rados.0.smithi122.stdout:278: done (1 left) 2024-09-15T03:46:35.784 INFO:tasks.rados.rados.0.smithi122.stdout:279: done (0 left) 2024-09-15T03:46:35.784 INFO:tasks.rados.rados.0.smithi122.stdout:280: write (excl) oid 38 current snap is 31 2024-09-15T03:46:35.784 INFO:tasks.rados.rados.0.smithi122.stdout:280: seq_num 136 ranges {685752=611865,1975551=310201} 2024-09-15T03:46:35.797 INFO:tasks.rados.rados.0.smithi122.stdout:280: writing smithi12223906-38 from 685752 to 1297617 tid 1 2024-09-15T03:46:35.802 INFO:tasks.rados.rados.0.smithi122.stdout:280: writing smithi12223906-38 from 1975551 to 2285752 tid 2 2024-09-15T03:46:35.802 INFO:tasks.rados.rados.0.smithi122.stdout:281: snap_create 2024-09-15T03:46:35.806 INFO:tasks.rados.rados.0.smithi122.stdout:280: finishing write tid 1 to smithi12223906-38 2024-09-15T03:46:35.808 INFO:tasks.rados.rados.0.smithi122.stdout:280: finishing write tid 2 to smithi12223906-38 2024-09-15T03:46:35.809 INFO:tasks.rados.rados.0.smithi122.stdout:280: finishing write tid 3 to smithi12223906-38 2024-09-15T03:46:35.809 INFO:tasks.rados.rados.0.smithi122.stdout:280: finishing write tid 4 to smithi12223906-38 2024-09-15T03:46:35.809 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 88 (ObjNum 136 snap 31 seq_num 136) dirty exists 2024-09-15T03:46:35.809 INFO:tasks.rados.rados.0.smithi122.stdout:280: left oid 38 (ObjNum 136 snap 31 seq_num 136) 2024-09-15T03:46:36.704 INFO:tasks.rados.rados.0.smithi122.stdout:280: done (1 left) 2024-09-15T03:46:36.704 INFO:tasks.rados.rados.0.smithi122.stdout:281: done (0 left) 2024-09-15T03:46:36.704 INFO:tasks.rados.rados.0.smithi122.stdout:282: copy_from oid 33 from oid 23 current snap is 32 2024-09-15T03:46:36.704 INFO:tasks.rados.rados.0.smithi122.stdout:283: delete oid 28 current snap is 32 2024-09-15T03:46:36.711 INFO:tasks.rados.rados.0.smithi122.stdout:283: done (1 left) 2024-09-15T03:46:36.712 INFO:tasks.rados.rados.0.smithi122.stdout:284: delete oid 34 current snap is 32 2024-09-15T03:46:36.720 INFO:tasks.rados.rados.0.smithi122.stdout:284: done (1 left) 2024-09-15T03:46:36.720 INFO:tasks.rados.rados.0.smithi122.stdout:285: read oid 3 snap -1 2024-09-15T03:46:36.720 INFO:tasks.rados.rados.0.smithi122.stdout:285: expect (ObjNum 88 snap 15 seq_num 88) 2024-09-15T03:46:36.720 INFO:tasks.rados.rados.0.smithi122.stdout:286: delete oid 9 current snap is 32 2024-09-15T03:46:36.726 INFO:tasks.rados.rados.0.smithi122.stdout:286: done (2 left) 2024-09-15T03:46:36.726 INFO:tasks.rados.rados.0.smithi122.stdout:287: write oid 17 current snap is 32 2024-09-15T03:46:36.726 INFO:tasks.rados.rados.0.smithi122.stdout:287: seq_num 140 ranges {752980=629214,1809479=543501} 2024-09-15T03:46:36.737 INFO:tasks.rados.rados.0.smithi122.stdout:287: writing smithi12223906-17 from 752980 to 1382194 tid 1 2024-09-15T03:46:36.744 INFO:tasks.rados.rados.0.smithi122.stdout:287: writing smithi12223906-17 from 1809479 to 2352980 tid 2 2024-09-15T03:46:36.744 INFO:tasks.rados.rados.0.smithi122.stdout:288: write (excl) oid 13 current snap is 32 2024-09-15T03:46:36.745 INFO:tasks.rados.rados.0.smithi122.stdout:288: seq_num 141 ranges {769787=704463,2267961=101826} 2024-09-15T03:46:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:288: writing smithi12223906-13 from 769787 to 1474250 tid 1 2024-09-15T03:46:36.758 INFO:tasks.rados.rados.0.smithi122.stdout:288: writing smithi12223906-13 from 2267961 to 2369787 tid 2 2024-09-15T03:46:36.758 INFO:tasks.rados.rados.0.smithi122.stdout:289: read oid 9 snap 31 2024-09-15T03:46:36.758 INFO:tasks.rados.rados.0.smithi122.stdout:289: expect (ObjNum 53 snap 1 seq_num 53) 2024-09-15T03:46:36.758 INFO:tasks.rados.rados.0.smithi122.stdout:290: copy_from oid 18 from oid 29 current snap is 32 2024-09-15T03:46:36.758 INFO:tasks.rados.rados.0.smithi122.stdout:285: expect (ObjNum 88 snap 15 seq_num 88) 2024-09-15T03:46:36.931 INFO:tasks.rados.rados.0.smithi122.stdout:282: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:282: finishing copy_from to smithi12223906-33 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 90 (ObjNum 113 snap 23 seq_num 113) dirty exists 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:287: finishing write tid 1 to smithi12223906-17 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:287: finishing write tid 2 to smithi12223906-17 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:287: finishing write tid 3 to smithi12223906-17 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:287: finishing write tid 4 to smithi12223906-17 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 76 (ObjNum 140 snap 32 seq_num 140) dirty exists 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:287: left oid 17 (ObjNum 140 snap 32 seq_num 140) 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:288: finishing write tid 1 to smithi12223906-13 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:288: finishing write tid 2 to smithi12223906-13 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:288: finishing write tid 3 to smithi12223906-13 2024-09-15T03:46:36.932 INFO:tasks.rados.rados.0.smithi122.stdout:289: expect (ObjNum 53 snap 1 seq_num 53) 2024-09-15T03:46:37.065 INFO:tasks.rados.rados.0.smithi122.stdout:288: finishing write tid 4 to smithi12223906-13 2024-09-15T03:46:37.065 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 107 (ObjNum 141 snap 32 seq_num 141) dirty exists 2024-09-15T03:46:37.065 INFO:tasks.rados.rados.0.smithi122.stdout:288: left oid 13 (ObjNum 141 snap 32 seq_num 141) 2024-09-15T03:46:37.065 INFO:tasks.rados.rados.0.smithi122.stdout:290: finishing copy_from racing read to smithi12223906-18 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:290: finishing copy_from to smithi12223906-18 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 46 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:282: done (5 left) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:285: done (4 left) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:287: done (3 left) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:288: done (2 left) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:289: done (1 left) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:290: done (0 left) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:291: read oid 5 snap 29 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:291: expect (ObjNum 119 snap 24 seq_num 119) 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:292: copy_from oid 12 from oid 47 current snap is 32 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:293: write (excl) oid 3 current snap is 32 2024-09-15T03:46:37.066 INFO:tasks.rados.rados.0.smithi122.stdout:293: seq_num 142 ranges {786594=779712,2326443=60151} 2024-09-15T03:46:37.082 INFO:tasks.rados.rados.0.smithi122.stdout:293: writing smithi12223906-3 from 786594 to 1566306 tid 1 2024-09-15T03:46:37.087 INFO:tasks.rados.rados.0.smithi122.stdout:293: writing smithi12223906-3 from 2326443 to 2386594 tid 2 2024-09-15T03:46:37.087 INFO:tasks.rados.rados.0.smithi122.stdout:294: copy_from oid 48 from oid 42 current snap is 32 2024-09-15T03:46:37.087 INFO:tasks.rados.rados.0.smithi122.stdout:291: expect (ObjNum 119 snap 24 seq_num 119) 2024-09-15T03:46:37.169 INFO:tasks.rados.rados.0.smithi122.stdout:292: finishing copy_from racing read to smithi12223906-12 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:293: finishing write tid 1 to smithi12223906-3 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:292: finishing copy_from to smithi12223906-12 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 25 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:293: finishing write tid 2 to smithi12223906-3 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:293: finishing write tid 3 to smithi12223906-3 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:293: finishing write tid 4 to smithi12223906-3 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 186 (ObjNum 142 snap 32 seq_num 142) dirty exists 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:293: left oid 3 (ObjNum 142 snap 32 seq_num 142) 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:294: finishing copy_from racing read to smithi12223906-48 2024-09-15T03:46:37.170 INFO:tasks.rados.rados.0.smithi122.stdout:294: finishing copy_from to smithi12223906-48 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 119 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:291: done (3 left) 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:292: done (2 left) 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:293: done (1 left) 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:294: done (0 left) 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:295: write (excl) oid 31 current snap is 32 2024-09-15T03:46:37.171 INFO:tasks.rados.rados.0.smithi122.stdout:295: seq_num 143 ranges {403401=454961,1584925=506039} 2024-09-15T03:46:37.174 INFO:tasks.rados.rados.0.smithi122.stdout:295: writing smithi12223906-31 from 403401 to 858362 tid 1 2024-09-15T03:46:37.181 INFO:tasks.rados.rados.0.smithi122.stdout:295: writing smithi12223906-31 from 1584925 to 2090964 tid 2 2024-09-15T03:46:37.181 INFO:tasks.rados.rados.0.smithi122.stdout:296: write oid 40 current snap is 32 2024-09-15T03:46:37.181 INFO:tasks.rados.rados.0.smithi122.stdout:295: finishing write tid 1 to smithi12223906-31 2024-09-15T03:46:37.182 INFO:tasks.rados.rados.0.smithi122.stdout:296: seq_num 144 ranges {420208=530210,1527054=766050} 2024-09-15T03:46:37.187 INFO:tasks.rados.rados.0.smithi122.stdout:296: writing smithi12223906-40 from 420208 to 950418 tid 1 2024-09-15T03:46:37.195 INFO:tasks.rados.rados.0.smithi122.stdout:296: writing smithi12223906-40 from 1527054 to 2293104 tid 2 2024-09-15T03:46:37.195 INFO:tasks.rados.rados.0.smithi122.stdout:297: rollback oid 23 current snap is 32 2024-09-15T03:46:37.195 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 30 2024-09-15T03:46:37.195 INFO:tasks.rados.rados.0.smithi122.stdout:295: finishing write tid 2 to smithi12223906-31 2024-09-15T03:46:37.195 INFO:tasks.rados.rados.0.smithi122.stdout:298: write (excl) oid 35 current snap is 32 2024-09-15T03:46:37.195 INFO:tasks.rados.rados.0.smithi122.stdout:298: seq_num 145 ranges {437015=721812,1701889=509708} 2024-09-15T03:46:37.201 INFO:tasks.rados.rados.0.smithi122.stdout:298: writing smithi12223906-35 from 437015 to 1158827 tid 1 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:298: writing smithi12223906-35 from 1701889 to 2211597 tid 2 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:299: rollback oid 39 current snap is 32 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 39 to 29 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:295: finishing write tid 3 to smithi12223906-31 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:300: delete oid 38 current snap is 32 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:295: finishing write tid 4 to smithi12223906-31 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 38 (ObjNum 143 snap 32 seq_num 143) dirty exists 2024-09-15T03:46:37.208 INFO:tasks.rados.rados.0.smithi122.stdout:295: left oid 31 (ObjNum 143 snap 32 seq_num 143) 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:296: finishing write tid 1 to smithi12223906-40 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:296: finishing write tid 2 to smithi12223906-40 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:296: finishing write tid 3 to smithi12223906-40 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:296: finishing write tid 4 to smithi12223906-40 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 52 (ObjNum 144 snap 32 seq_num 144) dirty exists 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:296: left oid 40 (ObjNum 144 snap 32 seq_num 144) 2024-09-15T03:46:37.209 INFO:tasks.rados.rados.0.smithi122.stdout:297: finishing rollback tid 0 to smithi12223906-23 2024-09-15T03:46:37.211 INFO:tasks.rados.rados.0.smithi122.stdout:297: finishing rollback tid 1 to smithi12223906-23 2024-09-15T03:46:37.211 INFO:tasks.rados.rados.0.smithi122.stdout:297: finishing rollback tid 2 to smithi12223906-23 2024-09-15T03:46:37.211 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 123 (ObjNum 113 snap 23 seq_num 113) dirty exists 2024-09-15T03:46:37.213 INFO:tasks.rados.rados.0.smithi122.stdout:298: finishing write tid 1 to smithi12223906-35 2024-09-15T03:46:37.213 INFO:tasks.rados.rados.0.smithi122.stdout:295: done (5 left) 2024-09-15T03:46:37.213 INFO:tasks.rados.rados.0.smithi122.stdout:296: done (4 left) 2024-09-15T03:46:37.213 INFO:tasks.rados.rados.0.smithi122.stdout:297: done (3 left) 2024-09-15T03:46:37.214 INFO:tasks.rados.rados.0.smithi122.stdout:300: done (2 left) 2024-09-15T03:46:37.214 INFO:tasks.rados.rados.0.smithi122.stdout:301: write (excl) oid 7 current snap is 32 2024-09-15T03:46:37.214 INFO:tasks.rados.rados.0.smithi122.stdout:301: seq_num 147 ranges {470629=472310,1418853=513377} 2024-09-15T03:46:37.219 INFO:tasks.rados.rados.0.smithi122.stdout:301: writing smithi12223906-7 from 470629 to 942939 tid 1 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:301: writing smithi12223906-7 from 1418853 to 1932230 tid 2 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:302: delete oid 40 current snap is 32 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:298: finishing write tid 2 to smithi12223906-35 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:299: finishing rollback tid 0 to smithi12223906-39 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:298: finishing write tid 3 to smithi12223906-35 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:299: finishing rollback tid 1 to smithi12223906-39 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:298: finishing write tid 4 to smithi12223906-35 2024-09-15T03:46:37.227 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 67 (ObjNum 145 snap 32 seq_num 145) dirty exists 2024-09-15T03:46:37.228 INFO:tasks.rados.rados.0.smithi122.stdout:298: left oid 35 (ObjNum 145 snap 32 seq_num 145) 2024-09-15T03:46:37.228 INFO:tasks.rados.rados.0.smithi122.stdout:299: finishing rollback tid 2 to smithi12223906-39 2024-09-15T03:46:37.228 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 190 (ObjNum 117 snap 24 seq_num 117) dirty exists 2024-09-15T03:46:37.228 INFO:tasks.rados.rados.0.smithi122.stdout:301: finishing write tid 1 to smithi12223906-7 2024-09-15T03:46:37.230 INFO:tasks.rados.rados.0.smithi122.stdout:298: done (3 left) 2024-09-15T03:46:37.230 INFO:tasks.rados.rados.0.smithi122.stdout:299: done (2 left) 2024-09-15T03:46:37.230 INFO:tasks.rados.rados.0.smithi122.stdout:302: done (1 left) 2024-09-15T03:46:37.230 INFO:tasks.rados.rados.0.smithi122.stdout:303: snap_create 2024-09-15T03:46:37.231 INFO:tasks.rados.rados.0.smithi122.stdout:301: finishing write tid 2 to smithi12223906-7 2024-09-15T03:46:37.231 INFO:tasks.rados.rados.0.smithi122.stdout:301: finishing write tid 3 to smithi12223906-7 2024-09-15T03:46:37.231 INFO:tasks.rados.rados.0.smithi122.stdout:301: finishing write tid 4 to smithi12223906-7 2024-09-15T03:46:37.232 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 74 (ObjNum 147 snap 32 seq_num 147) dirty exists 2024-09-15T03:46:37.232 INFO:tasks.rados.rados.0.smithi122.stdout:301: left oid 7 (ObjNum 147 snap 32 seq_num 147) 2024-09-15T03:46:37.795 INFO:tasks.rados.rados.0.smithi122.stdout:301: done (1 left) 2024-09-15T03:46:37.795 INFO:tasks.rados.rados.0.smithi122.stdout:303: done (0 left) 2024-09-15T03:46:37.795 INFO:tasks.rados.rados.0.smithi122.stdout:304: write oid 1 current snap is 33 2024-09-15T03:46:37.795 INFO:tasks.rados.rados.0.smithi122.stdout:304: seq_num 149 ranges {504243=622808,1819464=517046} 2024-09-15T03:46:37.802 INFO:tasks.rados.rados.0.smithi122.stdout:304: writing smithi12223906-1 from 504243 to 1127051 tid 1 2024-09-15T03:46:37.811 INFO:tasks.rados.rados.0.smithi122.stdout:304: writing smithi12223906-1 from 1819464 to 2336510 tid 2 2024-09-15T03:46:37.811 INFO:tasks.rados.rados.0.smithi122.stdout:305: copy_from oid 45 from oid 44 current snap is 33 2024-09-15T03:46:37.811 INFO:tasks.rados.rados.0.smithi122.stdout:306: snap_create 2024-09-15T03:46:37.811 INFO:tasks.rados.rados.0.smithi122.stdout:304: finishing write tid 1 to smithi12223906-1 2024-09-15T03:46:37.817 INFO:tasks.rados.rados.0.smithi122.stdout:304: finishing write tid 2 to smithi12223906-1 2024-09-15T03:46:37.819 INFO:tasks.rados.rados.0.smithi122.stdout:304: finishing write tid 3 to smithi12223906-1 2024-09-15T03:46:37.820 INFO:tasks.rados.rados.0.smithi122.stdout:304: finishing write tid 4 to smithi12223906-1 2024-09-15T03:46:37.820 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 70 (ObjNum 149 snap 33 seq_num 149) dirty exists 2024-09-15T03:46:37.820 INFO:tasks.rados.rados.0.smithi122.stdout:304: left oid 1 (ObjNum 149 snap 33 seq_num 149) 2024-09-15T03:46:37.829 INFO:tasks.rados.rados.0.smithi122.stdout:305: finishing copy_from racing read to smithi12223906-45 2024-09-15T03:46:37.830 INFO:tasks.rados.rados.0.smithi122.stdout:305: finishing copy_from to smithi12223906-45 2024-09-15T03:46:37.830 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 21 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-15T03:46:38.798 INFO:tasks.rados.rados.0.smithi122.stdout:304: done (2 left) 2024-09-15T03:46:38.798 INFO:tasks.rados.rados.0.smithi122.stdout:305: done (1 left) 2024-09-15T03:46:38.799 INFO:tasks.rados.rados.0.smithi122.stdout:306: done (0 left) 2024-09-15T03:46:38.799 INFO:tasks.rados.rados.0.smithi122.stdout:307: write oid 23 current snap is 34 2024-09-15T03:46:38.799 INFO:tasks.rados.rados.0.smithi122.stdout:307: seq_num 150 ranges {521050=698057,1877946=643104} 2024-09-15T03:46:38.810 INFO:tasks.rados.rados.0.smithi122.stdout:307: writing smithi12223906-23 from 521050 to 1219107 tid 1 2024-09-15T03:46:38.818 INFO:tasks.rados.rados.0.smithi122.stdout:307: writing smithi12223906-23 from 1877946 to 2521050 tid 2 2024-09-15T03:46:38.818 INFO:tasks.rados.rados.0.smithi122.stdout:308: snap_remove snap 33 2024-09-15T03:46:39.820 INFO:tasks.rados.rados.0.smithi122.stdout:308: done (1 left) 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:309: copy_from oid 39 from oid 13 current snap is 34 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:307: finishing write tid 1 to smithi12223906-23 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:307: finishing write tid 2 to smithi12223906-23 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:307: finishing write tid 3 to smithi12223906-23 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:307: finishing write tid 4 to smithi12223906-23 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 126 (ObjNum 150 snap 34 seq_num 150) dirty exists 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:307: left oid 23 (ObjNum 150 snap 34 seq_num 150) 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:307: done (1 left) 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:310: read oid 8 snap -1 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:310: expect deleted 2024-09-15T03:46:39.821 INFO:tasks.rados.rados.0.smithi122.stdout:311: snap_create 2024-09-15T03:46:39.831 INFO:tasks.rados.rados.0.smithi122.stdout:309: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:46:39.833 INFO:tasks.rados.rados.0.smithi122.stdout:309: finishing copy_from to smithi12223906-39 2024-09-15T03:46:39.833 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 107 (ObjNum 141 snap 32 seq_num 141) dirty exists 2024-09-15T03:46:40.828 INFO:tasks.rados.rados.0.smithi122.stdout:309: done (2 left) 2024-09-15T03:46:40.828 INFO:tasks.rados.rados.0.smithi122.stdout:310: done (1 left) 2024-09-15T03:46:40.828 INFO:tasks.rados.rados.0.smithi122.stdout:311: done (0 left) 2024-09-15T03:46:40.828 INFO:tasks.rados.rados.0.smithi122.stdout:312: rollback oid 2 current snap is 35 2024-09-15T03:46:40.829 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 29 2024-09-15T03:46:40.829 INFO:tasks.rados.rados.0.smithi122.stdout:313: snap_remove snap 32 2024-09-15T03:46:41.707 INFO:tasks.rados.rados.0.smithi122.stdout:313: done (1 left) 2024-09-15T03:46:41.707 INFO:tasks.rados.rados.0.smithi122.stdout:314: delete oid 42 current snap is 35 2024-09-15T03:46:41.707 INFO:tasks.rados.rados.0.smithi122.stdout:312: finishing rollback tid 0 to smithi12223906-2 2024-09-15T03:46:41.707 INFO:tasks.rados.rados.0.smithi122.stdout:312: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:46:41.707 INFO:tasks.rados.rados.0.smithi122.stdout:312: finishing rollback tid 2 to smithi12223906-2 2024-09-15T03:46:41.707 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 80 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:312: done (1 left) 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:314: done (0 left) 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:315: read oid 32 snap -1 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:315: expect (ObjNum 59 snap 4 seq_num 59) 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:316: copy_from oid 13 from oid 16 current snap is 35 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:317: copy_from oid 38 from oid 35 current snap is 35 2024-09-15T03:46:41.713 INFO:tasks.rados.rados.0.smithi122.stdout:318: write (excl) oid 19 current snap is 35 2024-09-15T03:46:41.714 INFO:tasks.rados.rados.0.smithi122.stdout:318: seq_num 152 ranges {554664=448555,1478557=664373} 2024-09-15T03:46:41.724 INFO:tasks.rados.rados.0.smithi122.stdout:318: writing smithi12223906-19 from 554664 to 1003219 tid 1 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:318: writing smithi12223906-19 from 1478557 to 2142930 tid 2 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:319: copy_from oid 34 from oid 29 current snap is 35 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:316: finishing copy_from to smithi12223906-13 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:316: got expected ENOENT (src dne) 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:316: finishing copy_from racing read to smithi12223906-13 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:317: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:46:41.732 INFO:tasks.rados.rados.0.smithi122.stdout:316: done (4 left) 2024-09-15T03:46:41.733 INFO:tasks.rados.rados.0.smithi122.stdout:320: snap_remove snap 34 2024-09-15T03:46:42.718 INFO:tasks.rados.rados.0.smithi122.stdout:315: expect (ObjNum 59 snap 4 seq_num 59) 2024-09-15T03:46:42.857 INFO:tasks.rados.rados.0.smithi122.stdout:318: finishing write tid 1 to smithi12223906-19 2024-09-15T03:46:42.857 INFO:tasks.rados.rados.0.smithi122.stdout:318: finishing write tid 2 to smithi12223906-19 2024-09-15T03:46:42.857 INFO:tasks.rados.rados.0.smithi122.stdout:318: finishing write tid 3 to smithi12223906-19 2024-09-15T03:46:42.857 INFO:tasks.rados.rados.0.smithi122.stdout:318: finishing write tid 4 to smithi12223906-19 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 115 (ObjNum 152 snap 35 seq_num 152) dirty exists 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:318: left oid 19 (ObjNum 152 snap 35 seq_num 152) 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:317: finishing copy_from to smithi12223906-38 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 67 (ObjNum 145 snap 32 seq_num 145) dirty exists 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:319: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:319: finishing copy_from to smithi12223906-34 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 46 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:315: done (4 left) 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:317: done (3 left) 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:318: done (2 left) 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:319: done (1 left) 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:320: done (0 left) 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:321: write (excl) oid 36 current snap is 35 2024-09-15T03:46:42.858 INFO:tasks.rados.rados.0.smithi122.stdout:321: seq_num 153 ranges {571471=640157,1653392=524384} 2024-09-15T03:46:42.873 INFO:tasks.rados.rados.0.smithi122.stdout:321: writing smithi12223906-36 from 571471 to 1211628 tid 1 2024-09-15T03:46:42.878 INFO:tasks.rados.rados.0.smithi122.stdout:321: writing smithi12223906-36 from 1653392 to 2177776 tid 2 2024-09-15T03:46:42.878 INFO:tasks.rados.rados.0.smithi122.stdout:322: read oid 39 snap -1 2024-09-15T03:46:42.878 INFO:tasks.rados.rados.0.smithi122.stdout:322: expect (ObjNum 141 snap 32 seq_num 141) 2024-09-15T03:46:42.878 INFO:tasks.rados.rados.0.smithi122.stdout:323: snap_remove snap 29 2024-09-15T03:46:43.850 INFO:tasks.rados.rados.0.smithi122.stdout:323: done (2 left) 2024-09-15T03:46:43.850 INFO:tasks.rados.rados.0.smithi122.stdout:321: finishing write tid 1 to smithi12223906-36 2024-09-15T03:46:43.850 INFO:tasks.rados.rados.0.smithi122.stdout:321: finishing write tid 2 to smithi12223906-36 2024-09-15T03:46:43.851 INFO:tasks.rados.rados.0.smithi122.stdout:321: finishing write tid 3 to smithi12223906-36 2024-09-15T03:46:43.851 INFO:tasks.rados.rados.0.smithi122.stdout:321: finishing write tid 4 to smithi12223906-36 2024-09-15T03:46:43.851 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 200 (ObjNum 153 snap 35 seq_num 153) dirty exists 2024-09-15T03:46:43.851 INFO:tasks.rados.rados.0.smithi122.stdout:321: left oid 36 (ObjNum 153 snap 35 seq_num 153) 2024-09-15T03:46:43.851 INFO:tasks.rados.rados.0.smithi122.stdout:322: expect (ObjNum 141 snap 32 seq_num 141) 2024-09-15T03:46:43.958 INFO:tasks.rados.rados.0.smithi122.stdout:324: read oid 24 snap -1 2024-09-15T03:46:43.958 INFO:tasks.rados.rados.0.smithi122.stdout:324: expect deleted 2024-09-15T03:46:43.958 INFO:tasks.rados.rados.0.smithi122.stdout:321: done (2 left) 2024-09-15T03:46:43.958 INFO:tasks.rados.rados.0.smithi122.stdout:322: done (1 left) 2024-09-15T03:46:43.958 INFO:tasks.rados.rados.0.smithi122.stdout:325: delete oid 18 current snap is 35 2024-09-15T03:46:43.962 INFO:tasks.rados.rados.0.smithi122.stdout:324: done (1 left) 2024-09-15T03:46:43.962 INFO:tasks.rados.rados.0.smithi122.stdout:325: done (0 left) 2024-09-15T03:46:43.962 INFO:tasks.rados.rados.0.smithi122.stdout:326: snap_create 2024-09-15T03:46:44.853 INFO:tasks.rados.rados.0.smithi122.stdout:326: done (0 left) 2024-09-15T03:46:44.853 INFO:tasks.rados.rados.0.smithi122.stdout:327: snap_create 2024-09-15T03:46:45.865 INFO:tasks.rados.rados.0.smithi122.stdout:327: done (0 left) 2024-09-15T03:46:45.865 INFO:tasks.rados.rados.0.smithi122.stdout:328: rollback oid 10 current snap is 37 2024-09-15T03:46:45.865 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 35 2024-09-15T03:46:45.865 INFO:tasks.rados.rados.0.smithi122.stdout:329: copy_from oid 27 from oid 20 current snap is 37 2024-09-15T03:46:45.866 INFO:tasks.rados.rados.0.smithi122.stdout:330: copy_from oid 7 from oid 6 current snap is 37 2024-09-15T03:46:45.866 INFO:tasks.rados.rados.0.smithi122.stdout:331: write (excl) oid 24 current snap is 37 2024-09-15T03:46:45.866 INFO:tasks.rados.rados.0.smithi122.stdout:331: seq_num 155 ranges {605085=790655,2170356=434729} 2024-09-15T03:46:45.879 INFO:tasks.rados.rados.0.smithi122.stdout:331: writing smithi12223906-24 from 605085 to 1395740 tid 1 2024-09-15T03:46:45.887 INFO:tasks.rados.rados.0.smithi122.stdout:331: writing smithi12223906-24 from 2170356 to 2605085 tid 2 2024-09-15T03:46:45.887 INFO:tasks.rados.rados.0.smithi122.stdout:332: rollback oid 34 current snap is 37 2024-09-15T03:46:45.887 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 35 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:328: finishing rollback tid 0 to smithi12223906-10 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:328: finishing rollback tid 1 to smithi12223906-10 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:328: finishing rollback tid 2 to smithi12223906-10 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 122 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:330: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:330: finishing copy_from to smithi12223906-7 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 32 (ObjNum 127 snap 27 seq_num 127) dirty exists 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:328: done (4 left) 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:330: done (3 left) 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:333: rollback oid 43 current snap is 37 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 36 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:334: write (excl) oid 31 current snap is 37 2024-09-15T03:46:45.888 INFO:tasks.rados.rados.0.smithi122.stdout:334: seq_num 156 ranges {621892=465904,1712485=671711} 2024-09-15T03:46:45.893 INFO:tasks.rados.rados.0.smithi122.stdout:334: writing smithi12223906-31 from 621892 to 1087796 tid 1 2024-09-15T03:46:45.900 INFO:tasks.rados.rados.0.smithi122.stdout:334: writing smithi12223906-31 from 1712485 to 2384196 tid 2 2024-09-15T03:46:45.900 INFO:tasks.rados.rados.0.smithi122.stdout:335: snap_remove snap 30 2024-09-15T03:46:46.710 INFO:tasks.rados.rados.0.smithi122.stdout:331: finishing write tid 1 to smithi12223906-24 2024-09-15T03:46:46.710 INFO:tasks.rados.rados.0.smithi122.stdout:329: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:46:46.710 INFO:tasks.rados.rados.0.smithi122.stdout:329: finishing copy_from to smithi12223906-27 2024-09-15T03:46:46.710 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 8 (ObjNum 115 snap 23 seq_num 115) dirty exists 2024-09-15T03:46:46.710 INFO:tasks.rados.rados.0.smithi122.stdout:331: finishing write tid 2 to smithi12223906-24 2024-09-15T03:46:46.710 INFO:tasks.rados.rados.0.smithi122.stdout:331: finishing write tid 3 to smithi12223906-24 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:332: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:331: finishing write tid 4 to smithi12223906-24 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 213 (ObjNum 155 snap 37 seq_num 155) dirty exists 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:331: left oid 24 (ObjNum 155 snap 37 seq_num 155) 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:333: finishing rollback tid 0 to smithi12223906-43 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:333: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:332: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:333: finishing rollback tid 2 to smithi12223906-43 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 217 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:332: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:46:46.711 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 56 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:334: finishing write tid 1 to smithi12223906-31 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:334: finishing write tid 2 to smithi12223906-31 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:334: finishing write tid 3 to smithi12223906-31 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:334: finishing write tid 4 to smithi12223906-31 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 44 (ObjNum 156 snap 37 seq_num 156) dirty exists 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:334: left oid 31 (ObjNum 156 snap 37 seq_num 156) 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:329: done (5 left) 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:331: done (4 left) 2024-09-15T03:46:46.712 INFO:tasks.rados.rados.0.smithi122.stdout:332: done (3 left) 2024-09-15T03:46:46.713 INFO:tasks.rados.rados.0.smithi122.stdout:333: done (2 left) 2024-09-15T03:46:46.713 INFO:tasks.rados.rados.0.smithi122.stdout:334: done (1 left) 2024-09-15T03:46:46.713 INFO:tasks.rados.rados.0.smithi122.stdout:335: done (0 left) 2024-09-15T03:46:46.713 INFO:tasks.rados.rados.0.smithi122.stdout:336: snap_remove snap 36 2024-09-15T03:46:47.713 INFO:tasks.rados.rados.0.smithi122.stdout:336: done (0 left) 2024-09-15T03:46:47.713 INFO:tasks.rados.rados.0.smithi122.stdout:337: snap_create 2024-09-15T03:46:48.713 INFO:tasks.rados.rados.0.smithi122.stdout:337: done (0 left) 2024-09-15T03:46:48.713 INFO:tasks.rados.rados.0.smithi122.stdout:338: read oid 19 snap 31 2024-09-15T03:46:48.713 INFO:tasks.rados.rados.0.smithi122.stdout:338: expect (ObjNum 102 snap 18 seq_num 102) 2024-09-15T03:46:48.713 INFO:tasks.rados.rados.0.smithi122.stdout:339: snap_create 2024-09-15T03:46:48.731 INFO:tasks.rados.rados.0.smithi122.stdout:338: expect (ObjNum 102 snap 18 seq_num 102) 2024-09-15T03:46:49.733 INFO:tasks.rados.rados.0.smithi122.stdout:338: done (1 left) 2024-09-15T03:46:49.733 INFO:tasks.rados.rados.0.smithi122.stdout:339: done (0 left) 2024-09-15T03:46:49.734 INFO:tasks.rados.rados.0.smithi122.stdout:340: snap_create 2024-09-15T03:46:50.737 INFO:tasks.rados.rados.0.smithi122.stdout:340: done (0 left) 2024-09-15T03:46:50.737 INFO:tasks.rados.rados.0.smithi122.stdout:341: write oid 45 current snap is 40 2024-09-15T03:46:50.737 INFO:tasks.rados.rados.0.smithi122.stdout:341: seq_num 157 ranges {638699=541153,1770967=531722} 2024-09-15T03:46:50.749 INFO:tasks.rados.rados.0.smithi122.stdout:341: writing smithi12223906-45 from 638699 to 1179852 tid 1 2024-09-15T03:46:50.758 INFO:tasks.rados.rados.0.smithi122.stdout:341: writing smithi12223906-45 from 1770967 to 2302689 tid 2 2024-09-15T03:46:50.758 INFO:tasks.rados.rados.0.smithi122.stdout:342: snap_create 2024-09-15T03:46:50.758 INFO:tasks.rados.rados.0.smithi122.stdout:341: finishing write tid 1 to smithi12223906-45 2024-09-15T03:46:50.762 INFO:tasks.rados.rados.0.smithi122.stdout:341: finishing write tid 2 to smithi12223906-45 2024-09-15T03:46:50.762 INFO:tasks.rados.rados.0.smithi122.stdout:341: finishing write tid 3 to smithi12223906-45 2024-09-15T03:46:50.762 INFO:tasks.rados.rados.0.smithi122.stdout:341: finishing write tid 4 to smithi12223906-45 2024-09-15T03:46:50.763 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 133 (ObjNum 157 snap 40 seq_num 157) dirty exists 2024-09-15T03:46:50.763 INFO:tasks.rados.rados.0.smithi122.stdout:341: left oid 45 (ObjNum 157 snap 40 seq_num 157) 2024-09-15T03:46:51.713 INFO:tasks.rados.rados.0.smithi122.stdout:341: done (1 left) 2024-09-15T03:46:51.713 INFO:tasks.rados.rados.0.smithi122.stdout:342: done (0 left) 2024-09-15T03:46:51.713 INFO:tasks.rados.rados.0.smithi122.stdout:343: delete oid 39 current snap is 41 2024-09-15T03:46:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:343: done (0 left) 2024-09-15T03:46:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:344: read oid 14 snap -1 2024-09-15T03:46:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:344: expect (ObjNum 65 snap 6 seq_num 65) 2024-09-15T03:46:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:345: write oid 5 current snap is 41 2024-09-15T03:46:51.720 INFO:tasks.rados.rados.0.smithi122.stdout:345: seq_num 159 ranges {672313=691651,1887931=419038} 2024-09-15T03:46:51.729 INFO:tasks.rados.rados.0.smithi122.stdout:345: writing smithi12223906-5 from 672313 to 1363964 tid 1 2024-09-15T03:46:51.734 INFO:tasks.rados.rados.0.smithi122.stdout:345: writing smithi12223906-5 from 1887931 to 2306969 tid 2 2024-09-15T03:46:51.734 INFO:tasks.rados.rados.0.smithi122.stdout:346: delete oid 21 current snap is 41 2024-09-15T03:46:51.734 INFO:tasks.rados.rados.0.smithi122.stdout:344: expect (ObjNum 65 snap 6 seq_num 65) 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:345: finishing write tid 1 to smithi12223906-5 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:345: finishing write tid 2 to smithi12223906-5 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:344: done (2 left) 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:346: done (1 left) 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:347: read oid 41 snap 39 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:347: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:348: copy_from oid 16 from oid 19 current snap is 41 2024-09-15T03:46:51.877 INFO:tasks.rados.rados.0.smithi122.stdout:345: finishing write tid 3 to smithi12223906-5 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:345: finishing write tid 4 to smithi12223906-5 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 38 (ObjNum 159 snap 41 seq_num 159) dirty exists 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:345: left oid 5 (ObjNum 159 snap 41 seq_num 159) 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:345: done (2 left) 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:349: read oid 30 snap 35 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:349: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-15T03:46:51.878 INFO:tasks.rados.rados.0.smithi122.stdout:350: snap_create 2024-09-15T03:46:51.886 INFO:tasks.rados.rados.0.smithi122.stdout:347: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-15T03:46:52.047 INFO:tasks.rados.rados.0.smithi122.stdout:349: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-15T03:46:52.124 INFO:tasks.rados.rados.0.smithi122.stdout:348: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:46:52.124 INFO:tasks.rados.rados.0.smithi122.stdout:348: finishing copy_from to smithi12223906-16 2024-09-15T03:46:52.124 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 115 (ObjNum 152 snap 35 seq_num 152) dirty exists 2024-09-15T03:46:52.759 INFO:tasks.rados.rados.0.smithi122.stdout:347: done (3 left) 2024-09-15T03:46:52.759 INFO:tasks.rados.rados.0.smithi122.stdout:348: done (2 left) 2024-09-15T03:46:52.760 INFO:tasks.rados.rados.0.smithi122.stdout:349: done (1 left) 2024-09-15T03:46:52.760 INFO:tasks.rados.rados.0.smithi122.stdout:350: done (0 left) 2024-09-15T03:46:52.760 INFO:tasks.rados.rados.0.smithi122.stdout:351: write oid 23 current snap is 42 2024-09-15T03:46:52.760 INFO:tasks.rados.rados.0.smithi122.stdout:351: seq_num 161 ranges {705927=558502,2004895=422707} 2024-09-15T03:46:52.773 INFO:tasks.rados.rados.0.smithi122.stdout:351: writing smithi12223906-23 from 705927 to 1264429 tid 1 2024-09-15T03:46:52.780 INFO:tasks.rados.rados.0.smithi122.stdout:351: writing smithi12223906-23 from 2004895 to 2427602 tid 2 2024-09-15T03:46:52.780 INFO:tasks.rados.rados.0.smithi122.stdout:352: snap_remove snap 31 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:352: done (1 left) 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:353: copy_from oid 41 from oid 17 current snap is 42 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:351: finishing write tid 1 to smithi12223906-23 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:351: finishing write tid 2 to smithi12223906-23 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:351: finishing write tid 3 to smithi12223906-23 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:351: finishing write tid 4 to smithi12223906-23 2024-09-15T03:46:53.773 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 140 (ObjNum 161 snap 42 seq_num 161) dirty exists 2024-09-15T03:46:53.774 INFO:tasks.rados.rados.0.smithi122.stdout:351: left oid 23 (ObjNum 161 snap 42 seq_num 161) 2024-09-15T03:46:53.774 INFO:tasks.rados.rados.0.smithi122.stdout:351: done (1 left) 2024-09-15T03:46:53.774 INFO:tasks.rados.rados.0.smithi122.stdout:354: snap_create 2024-09-15T03:46:53.789 INFO:tasks.rados.rados.0.smithi122.stdout:353: finishing copy_from racing read to smithi12223906-41 2024-09-15T03:46:53.793 INFO:tasks.rados.rados.0.smithi122.stdout:353: finishing copy_from to smithi12223906-41 2024-09-15T03:46:53.793 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 76 (ObjNum 140 snap 32 seq_num 140) dirty exists 2024-09-15T03:46:54.784 INFO:tasks.rados.rados.0.smithi122.stdout:353: done (1 left) 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:354: done (0 left) 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:355: read oid 46 snap -1 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:355: expect deleted 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:356: read oid 50 snap -1 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:356: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:357: read oid 29 snap -1 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:357: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:46:54.785 INFO:tasks.rados.rados.0.smithi122.stdout:358: snap_remove snap 40 2024-09-15T03:46:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:358: done (3 left) 2024-09-15T03:46:55.795 INFO:tasks.rados.rados.0.smithi122.stdout:359: read oid 14 snap -1 2024-09-15T03:46:55.796 INFO:tasks.rados.rados.0.smithi122.stdout:359: expect (ObjNum 65 snap 6 seq_num 65) 2024-09-15T03:46:55.796 INFO:tasks.rados.rados.0.smithi122.stdout:360: read oid 24 snap -1 2024-09-15T03:46:55.796 INFO:tasks.rados.rados.0.smithi122.stdout:360: expect (ObjNum 155 snap 37 seq_num 155) 2024-09-15T03:46:55.796 INFO:tasks.rados.rados.0.smithi122.stdout:361: copy_from oid 6 from oid 28 current snap is 43 2024-09-15T03:46:55.796 INFO:tasks.rados.rados.0.smithi122.stdout:357: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:46:55.921 INFO:tasks.rados.rados.0.smithi122.stdout:356: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:46:56.012 INFO:tasks.rados.rados.0.smithi122.stdout:361: finishing copy_from to smithi12223906-6 2024-09-15T03:46:56.013 INFO:tasks.rados.rados.0.smithi122.stdout:361: got expected ENOENT (src dne) 2024-09-15T03:46:56.013 INFO:tasks.rados.rados.0.smithi122.stdout:361: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:46:56.013 INFO:tasks.rados.rados.0.smithi122.stdout:359: expect (ObjNum 65 snap 6 seq_num 65) 2024-09-15T03:46:56.144 INFO:tasks.rados.rados.0.smithi122.stdout:360: expect (ObjNum 155 snap 37 seq_num 155) 2024-09-15T03:46:56.221 INFO:tasks.rados.rados.0.smithi122.stdout:355: done (5 left) 2024-09-15T03:46:56.221 INFO:tasks.rados.rados.0.smithi122.stdout:356: done (4 left) 2024-09-15T03:46:56.221 INFO:tasks.rados.rados.0.smithi122.stdout:357: done (3 left) 2024-09-15T03:46:56.222 INFO:tasks.rados.rados.0.smithi122.stdout:359: done (2 left) 2024-09-15T03:46:56.222 INFO:tasks.rados.rados.0.smithi122.stdout:360: done (1 left) 2024-09-15T03:46:56.222 INFO:tasks.rados.rados.0.smithi122.stdout:361: done (0 left) 2024-09-15T03:46:56.222 INFO:tasks.rados.rados.0.smithi122.stdout:362: read oid 11 snap 42 2024-09-15T03:46:56.222 INFO:tasks.rados.rados.0.smithi122.stdout:362: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:46:56.223 INFO:tasks.rados.rados.0.smithi122.stdout:363: snap_remove snap 41 2024-09-15T03:46:56.715 INFO:tasks.rados.rados.0.smithi122.stdout:363: done (1 left) 2024-09-15T03:46:56.715 INFO:tasks.rados.rados.0.smithi122.stdout:364: read oid 24 snap -1 2024-09-15T03:46:56.715 INFO:tasks.rados.rados.0.smithi122.stdout:364: expect (ObjNum 155 snap 37 seq_num 155) 2024-09-15T03:46:56.716 INFO:tasks.rados.rados.0.smithi122.stdout:365: read oid 19 snap -1 2024-09-15T03:46:56.716 INFO:tasks.rados.rados.0.smithi122.stdout:365: expect (ObjNum 152 snap 35 seq_num 152) 2024-09-15T03:46:56.716 INFO:tasks.rados.rados.0.smithi122.stdout:366: snap_remove snap 37 2024-09-15T03:46:57.719 INFO:tasks.rados.rados.0.smithi122.stdout:366: done (3 left) 2024-09-15T03:46:57.719 INFO:tasks.rados.rados.0.smithi122.stdout:367: delete oid 47 current snap is 43 2024-09-15T03:46:57.719 INFO:tasks.rados.rados.0.smithi122.stdout:362: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:46:57.792 INFO:tasks.rados.rados.0.smithi122.stdout:364: expect (ObjNum 155 snap 37 seq_num 155) 2024-09-15T03:46:57.875 INFO:tasks.rados.rados.0.smithi122.stdout:365: expect (ObjNum 152 snap 35 seq_num 152) 2024-09-15T03:46:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:362: done (3 left) 2024-09-15T03:46:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:364: done (2 left) 2024-09-15T03:46:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:365: done (1 left) 2024-09-15T03:46:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:367: done (0 left) 2024-09-15T03:46:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:368: read oid 16 snap -1 2024-09-15T03:46:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:368: expect (ObjNum 152 snap 35 seq_num 152) 2024-09-15T03:46:58.021 INFO:tasks.rados.rados.0.smithi122.stdout:369: snap_create 2024-09-15T03:46:58.026 INFO:tasks.rados.rados.0.smithi122.stdout:368: expect (ObjNum 152 snap 35 seq_num 152) 2024-09-15T03:46:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:368: done (1 left) 2024-09-15T03:46:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:369: done (0 left) 2024-09-15T03:46:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:370: read oid 30 snap -1 2024-09-15T03:46:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:370: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-15T03:46:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:371: write oid 1 current snap is 44 2024-09-15T03:46:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:371: seq_num 163 ranges {739541=709000,2121859=426376} 2024-09-15T03:46:58.812 INFO:tasks.rados.rados.0.smithi122.stdout:371: writing smithi12223906-1 from 739541 to 1448541 tid 1 2024-09-15T03:46:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:371: writing smithi12223906-1 from 2121859 to 2548235 tid 2 2024-09-15T03:46:58.820 INFO:tasks.rados.rados.0.smithi122.stdout:372: write (excl) oid 33 current snap is 44 2024-09-15T03:46:58.820 INFO:tasks.rados.rados.0.smithi122.stdout:372: seq_num 164 ranges {756348=784249,2063988=686387} 2024-09-15T03:46:58.828 INFO:tasks.rados.rados.0.smithi122.stdout:372: writing smithi12223906-33 from 756348 to 1540597 tid 1 2024-09-15T03:46:58.835 INFO:tasks.rados.rados.0.smithi122.stdout:372: writing smithi12223906-33 from 2063988 to 2750375 tid 2 2024-09-15T03:46:58.835 INFO:tasks.rados.rados.0.smithi122.stdout:373: snap_remove snap 38 2024-09-15T03:46:59.817 INFO:tasks.rados.rados.0.smithi122.stdout:373: done (3 left) 2024-09-15T03:46:59.817 INFO:tasks.rados.rados.0.smithi122.stdout:374: snap_remove snap 35 2024-09-15T03:47:00.817 INFO:tasks.rados.rados.0.smithi122.stdout:374: done (3 left) 2024-09-15T03:47:00.817 INFO:tasks.rados.rados.0.smithi122.stdout:375: write (excl) oid 2 current snap is 44 2024-09-15T03:47:00.817 INFO:tasks.rados.rados.0.smithi122.stdout:375: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2024-09-15T03:47:00.833 INFO:tasks.rados.rados.0.smithi122.stdout:375: writing smithi12223906-2 from 773155 to 1232653 tid 1 2024-09-15T03:47:00.838 INFO:tasks.rados.rados.0.smithi122.stdout:375: writing smithi12223906-2 from 1722470 to 2152515 tid 2 2024-09-15T03:47:00.842 INFO:tasks.rados.rados.0.smithi122.stdout:375: writing smithi12223906-2 from 2648676 to 2773155 tid 3 2024-09-15T03:47:00.842 INFO:tasks.rados.rados.0.smithi122.stdout:376: write oid 35 current snap is 44 2024-09-15T03:47:00.842 INFO:tasks.rados.rados.0.smithi122.stdout:376: seq_num 166 ranges {789962=534747,1780952=690056} 2024-09-15T03:47:00.850 INFO:tasks.rados.rados.0.smithi122.stdout:376: writing smithi12223906-35 from 789962 to 1324709 tid 1 2024-09-15T03:47:00.857 INFO:tasks.rados.rados.0.smithi122.stdout:376: writing smithi12223906-35 from 1780952 to 2471008 tid 2 2024-09-15T03:47:00.857 INFO:tasks.rados.rados.0.smithi122.stdout:377: snap_create 2024-09-15T03:47:00.857 INFO:tasks.rados.rados.0.smithi122.stdout:370: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:371: finishing write tid 1 to smithi12223906-1 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:371: finishing write tid 2 to smithi12223906-1 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:371: finishing write tid 3 to smithi12223906-1 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:371: finishing write tid 4 to smithi12223906-1 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 118 (ObjNum 163 snap 44 seq_num 163) dirty exists 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:371: left oid 1 (ObjNum 163 snap 44 seq_num 163) 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:372: finishing write tid 1 to smithi12223906-33 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:372: finishing write tid 2 to smithi12223906-33 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:372: finishing write tid 3 to smithi12223906-33 2024-09-15T03:47:00.944 INFO:tasks.rados.rados.0.smithi122.stdout:372: finishing write tid 4 to smithi12223906-33 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 156 (ObjNum 164 snap 44 seq_num 164) dirty exists 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:372: left oid 33 (ObjNum 164 snap 44 seq_num 164) 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:375: finishing write tid 1 to smithi12223906-2 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:375: finishing write tid 2 to smithi12223906-2 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:375: finishing write tid 3 to smithi12223906-2 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:375: finishing write tid 4 to smithi12223906-2 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:375: finishing write tid 5 to smithi12223906-2 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 95 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:375: left oid 2 (ObjNum 165 snap 44 seq_num 165) 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:376: finishing write tid 1 to smithi12223906-35 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:376: finishing write tid 2 to smithi12223906-35 2024-09-15T03:47:00.945 INFO:tasks.rados.rados.0.smithi122.stdout:376: finishing write tid 3 to smithi12223906-35 2024-09-15T03:47:00.946 INFO:tasks.rados.rados.0.smithi122.stdout:376: finishing write tid 4 to smithi12223906-35 2024-09-15T03:47:00.946 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 70 (ObjNum 166 snap 44 seq_num 166) dirty exists 2024-09-15T03:47:00.946 INFO:tasks.rados.rados.0.smithi122.stdout:376: left oid 35 (ObjNum 166 snap 44 seq_num 166) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:370: done (5 left) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:371: done (4 left) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:372: done (3 left) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:375: done (2 left) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:376: done (1 left) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:377: done (0 left) 2024-09-15T03:47:01.715 INFO:tasks.rados.rados.0.smithi122.stdout:378: write (excl) oid 45 current snap is 45 2024-09-15T03:47:01.716 INFO:tasks.rados.rados.0.smithi122.stdout:378: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2024-09-15T03:47:01.725 INFO:tasks.rados.rados.0.smithi122.stdout:378: writing smithi12223906-45 from 406769 to 1016765 tid 1 2024-09-15T03:47:01.731 INFO:tasks.rados.rados.0.smithi122.stdout:378: writing smithi12223906-45 from 1439434 to 1873148 tid 2 2024-09-15T03:47:01.737 INFO:tasks.rados.rados.0.smithi122.stdout:378: writing smithi12223906-45 from 2303522 to 2806769 tid 3 2024-09-15T03:47:01.737 INFO:tasks.rados.rados.0.smithi122.stdout:379: snap_remove snap 42 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:379: done (1 left) 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:380: read oid 44 snap 39 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:380: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:378: finishing write tid 1 to smithi12223906-45 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:378: finishing write tid 2 to smithi12223906-45 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:378: finishing write tid 3 to smithi12223906-45 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:378: finishing write tid 4 to smithi12223906-45 2024-09-15T03:47:02.726 INFO:tasks.rados.rados.0.smithi122.stdout:378: finishing write tid 5 to smithi12223906-45 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 150 (ObjNum 167 snap 45 seq_num 167) dirty exists 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:378: left oid 45 (ObjNum 167 snap 45 seq_num 167) 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:378: done (1 left) 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:381: read oid 28 snap -1 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:381: expect deleted 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:382: rollback oid 27 current snap is 45 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 43 2024-09-15T03:47:02.727 INFO:tasks.rados.rados.0.smithi122.stdout:383: delete oid 21 current snap is 45 2024-09-15T03:47:02.730 INFO:tasks.rados.rados.0.smithi122.stdout:383: done (3 left) 2024-09-15T03:47:02.730 INFO:tasks.rados.rados.0.smithi122.stdout:384: rollback oid 22 current snap is 45 2024-09-15T03:47:02.730 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 22 to 43 2024-09-15T03:47:02.730 INFO:tasks.rados.rados.0.smithi122.stdout:385: read oid 15 snap -1 2024-09-15T03:47:02.730 INFO:tasks.rados.rados.0.smithi122.stdout:385: expect (ObjNum 128 snap 27 seq_num 128) 2024-09-15T03:47:02.730 INFO:tasks.rados.rados.0.smithi122.stdout:386: rollback oid 10 current snap is 45 2024-09-15T03:47:02.731 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 43 2024-09-15T03:47:02.731 INFO:tasks.rados.rados.0.smithi122.stdout:387: snap_remove snap 44 2024-09-15T03:47:03.736 INFO:tasks.rados.rados.0.smithi122.stdout:387: done (6 left) 2024-09-15T03:47:03.737 INFO:tasks.rados.rados.0.smithi122.stdout:388: delete oid 16 current snap is 45 2024-09-15T03:47:03.737 INFO:tasks.rados.rados.0.smithi122.stdout:386: finishing rollback tid 0 to smithi12223906-10 2024-09-15T03:47:03.737 INFO:tasks.rados.rados.0.smithi122.stdout:386: finishing rollback tid 1 to smithi12223906-10 2024-09-15T03:47:03.737 INFO:tasks.rados.rados.0.smithi122.stdout:386: finishing rollback tid 2 to smithi12223906-10 2024-09-15T03:47:03.737 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 126 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T03:47:03.737 INFO:tasks.rados.rados.0.smithi122.stdout:385: expect (ObjNum 128 snap 27 seq_num 128) 2024-09-15T03:47:03.835 INFO:tasks.rados.rados.0.smithi122.stdout:380: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:382: finishing rollback tid 0 to smithi12223906-27 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:382: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:382: finishing rollback tid 2 to smithi12223906-27 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 170 (ObjNum 115 snap 23 seq_num 115) dirty exists 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:384: finishing rollback tid 0 to smithi12223906-22 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:384: finishing rollback tid 1 to smithi12223906-22 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:384: finishing rollback tid 2 to smithi12223906-22 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 174 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:47:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:380: done (6 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:381: done (5 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:382: done (4 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:384: done (3 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:385: done (2 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:386: done (1 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:388: done (0 left) 2024-09-15T03:47:03.921 INFO:tasks.rados.rados.0.smithi122.stdout:389: delete oid 32 current snap is 45 2024-09-15T03:47:03.929 INFO:tasks.rados.rados.0.smithi122.stdout:389: done (0 left) 2024-09-15T03:47:03.929 INFO:tasks.rados.rados.0.smithi122.stdout:390: write oid 45 current snap is 45 2024-09-15T03:47:03.929 INFO:tasks.rados.rados.0.smithi122.stdout:390: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2024-09-15T03:47:03.941 INFO:tasks.rados.rados.0.smithi122.stdout:390: writing smithi12223906-45 from 473997 to 1101342 tid 1 2024-09-15T03:47:03.949 INFO:tasks.rados.rados.0.smithi122.stdout:390: writing smithi12223906-45 from 1673362 to 2114414 tid 2 2024-09-15T03:47:03.953 INFO:tasks.rados.rados.0.smithi122.stdout:390: writing smithi12223906-45 from 2529567 to 2873997 tid 3 2024-09-15T03:47:03.953 INFO:tasks.rados.rados.0.smithi122.stdout:391: read oid 28 snap -1 2024-09-15T03:47:03.953 INFO:tasks.rados.rados.0.smithi122.stdout:391: expect deleted 2024-09-15T03:47:03.954 INFO:tasks.rados.rados.0.smithi122.stdout:392: write (excl) oid 18 current snap is 45 2024-09-15T03:47:03.954 INFO:tasks.rados.rados.0.smithi122.stdout:392: seq_num 172 ranges {490804=702594,1615491=584710,2724284=166520} 2024-09-15T03:47:03.960 INFO:tasks.rados.rados.0.smithi122.stdout:392: writing smithi12223906-18 from 490804 to 1193398 tid 1 2024-09-15T03:47:03.965 INFO:tasks.rados.rados.0.smithi122.stdout:392: writing smithi12223906-18 from 1615491 to 2200201 tid 2 2024-09-15T03:47:03.970 INFO:tasks.rados.rados.0.smithi122.stdout:392: writing smithi12223906-18 from 2724284 to 2890804 tid 3 2024-09-15T03:47:03.970 INFO:tasks.rados.rados.0.smithi122.stdout:393: copy_from oid 38 from oid 34 current snap is 45 2024-09-15T03:47:03.970 INFO:tasks.rados.rados.0.smithi122.stdout:394: copy_from oid 41 from oid 31 current snap is 45 2024-09-15T03:47:03.970 INFO:tasks.rados.rados.0.smithi122.stdout:390: finishing write tid 1 to smithi12223906-45 2024-09-15T03:47:03.970 INFO:tasks.rados.rados.0.smithi122.stdout:390: finishing write tid 2 to smithi12223906-45 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:390: finishing write tid 3 to smithi12223906-45 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:390: finishing write tid 4 to smithi12223906-45 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:390: finishing write tid 5 to smithi12223906-45 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 158 (ObjNum 171 snap 45 seq_num 171) dirty exists 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:390: left oid 45 (ObjNum 171 snap 45 seq_num 171) 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:392: finishing write tid 1 to smithi12223906-18 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:390: done (4 left) 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:391: done (3 left) 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:395: rollback oid 35 current snap is 45 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 39 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:396: write oid 28 current snap is 45 2024-09-15T03:47:03.971 INFO:tasks.rados.rados.0.smithi122.stdout:396: seq_num 173 ranges {507611=777843,2073973=444721} 2024-09-15T03:47:03.977 INFO:tasks.rados.rados.0.smithi122.stdout:396: writing smithi12223906-28 from 507611 to 1285454 tid 1 2024-09-15T03:47:03.985 INFO:tasks.rados.rados.0.smithi122.stdout:396: writing smithi12223906-28 from 2073973 to 2518694 tid 2 2024-09-15T03:47:03.985 INFO:tasks.rados.rados.0.smithi122.stdout:397: snap_create 2024-09-15T03:47:03.985 INFO:tasks.rados.rados.0.smithi122.stdout:392: finishing write tid 2 to smithi12223906-18 2024-09-15T03:47:03.985 INFO:tasks.rados.rados.0.smithi122.stdout:395: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:47:03.985 INFO:tasks.rados.rados.0.smithi122.stdout:395: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:47:03.985 INFO:tasks.rados.rados.0.smithi122.stdout:392: finishing write tid 3 to smithi12223906-18 2024-09-15T03:47:03.986 INFO:tasks.rados.rados.0.smithi122.stdout:392: finishing write tid 4 to smithi12223906-18 2024-09-15T03:47:03.986 INFO:tasks.rados.rados.0.smithi122.stdout:392: finishing write tid 5 to smithi12223906-18 2024-09-15T03:47:03.986 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 186 (ObjNum 172 snap 45 seq_num 172) dirty exists 2024-09-15T03:47:03.986 INFO:tasks.rados.rados.0.smithi122.stdout:392: left oid 18 (ObjNum 172 snap 45 seq_num 172) 2024-09-15T03:47:03.986 INFO:tasks.rados.rados.0.smithi122.stdout:395: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:47:03.986 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 73 (ObjNum 145 snap 32 seq_num 145) dirty exists 2024-09-15T03:47:03.989 INFO:tasks.rados.rados.0.smithi122.stdout:396: finishing write tid 1 to smithi12223906-28 2024-09-15T03:47:03.989 INFO:tasks.rados.rados.0.smithi122.stdout:396: finishing write tid 2 to smithi12223906-28 2024-09-15T03:47:03.989 INFO:tasks.rados.rados.0.smithi122.stdout:396: finishing write tid 3 to smithi12223906-28 2024-09-15T03:47:03.989 INFO:tasks.rados.rados.0.smithi122.stdout:396: finishing write tid 4 to smithi12223906-28 2024-09-15T03:47:03.989 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 95 (ObjNum 173 snap 45 seq_num 173) dirty exists 2024-09-15T03:47:03.989 INFO:tasks.rados.rados.0.smithi122.stdout:396: left oid 28 (ObjNum 173 snap 45 seq_num 173) 2024-09-15T03:47:03.993 INFO:tasks.rados.rados.0.smithi122.stdout:394: finishing copy_from racing read to smithi12223906-41 2024-09-15T03:47:03.993 INFO:tasks.rados.rados.0.smithi122.stdout:393: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:47:03.993 INFO:tasks.rados.rados.0.smithi122.stdout:394: finishing copy_from to smithi12223906-41 2024-09-15T03:47:03.993 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 44 (ObjNum 156 snap 37 seq_num 156) dirty exists 2024-09-15T03:47:03.994 INFO:tasks.rados.rados.0.smithi122.stdout:393: finishing copy_from to smithi12223906-38 2024-09-15T03:47:03.994 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 56 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:47:04.848 INFO:tasks.rados.rados.0.smithi122.stdout:392: done (5 left) 2024-09-15T03:47:04.848 INFO:tasks.rados.rados.0.smithi122.stdout:393: done (4 left) 2024-09-15T03:47:04.848 INFO:tasks.rados.rados.0.smithi122.stdout:394: done (3 left) 2024-09-15T03:47:04.848 INFO:tasks.rados.rados.0.smithi122.stdout:395: done (2 left) 2024-09-15T03:47:04.849 INFO:tasks.rados.rados.0.smithi122.stdout:396: done (1 left) 2024-09-15T03:47:04.849 INFO:tasks.rados.rados.0.smithi122.stdout:397: done (0 left) 2024-09-15T03:47:04.849 INFO:tasks.rados.rados.0.smithi122.stdout:398: snap_remove snap 39 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:398: done (0 left) 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:399: read oid 23 snap -1 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:399: expect (ObjNum 161 snap 42 seq_num 161) 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:400: rollback oid 39 current snap is 46 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 39 to 45 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:401: write (excl) oid 17 current snap is 46 2024-09-15T03:47:05.862 INFO:tasks.rados.rados.0.smithi122.stdout:401: seq_num 174 ranges {524418=453092,1732455=588379,2779130=145288} 2024-09-15T03:47:05.872 INFO:tasks.rados.rados.0.smithi122.stdout:401: writing smithi12223906-17 from 524418 to 977510 tid 1 2024-09-15T03:47:05.881 INFO:tasks.rados.rados.0.smithi122.stdout:401: writing smithi12223906-17 from 1732455 to 2320834 tid 2 2024-09-15T03:47:05.885 INFO:tasks.rados.rados.0.smithi122.stdout:401: writing smithi12223906-17 from 2779130 to 2924418 tid 3 2024-09-15T03:47:05.885 INFO:tasks.rados.rados.0.smithi122.stdout:402: rollback oid 7 current snap is 46 2024-09-15T03:47:05.885 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 43 2024-09-15T03:47:05.885 INFO:tasks.rados.rados.0.smithi122.stdout:400: finishing rollback tid 1 to smithi12223906-39 2024-09-15T03:47:05.885 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 231 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:47:05.885 INFO:tasks.rados.rados.0.smithi122.stdout:399: expect (ObjNum 161 snap 42 seq_num 161) 2024-09-15T03:47:06.045 INFO:tasks.rados.rados.0.smithi122.stdout:401: finishing write tid 1 to smithi12223906-17 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:401: finishing write tid 2 to smithi12223906-17 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:401: finishing write tid 3 to smithi12223906-17 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:402: finishing rollback tid 0 to smithi12223906-7 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:401: finishing write tid 4 to smithi12223906-17 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:402: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:401: finishing write tid 5 to smithi12223906-17 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 104 (ObjNum 174 snap 46 seq_num 174) dirty exists 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:401: left oid 17 (ObjNum 174 snap 46 seq_num 174) 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:399: done (3 left) 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:400: done (2 left) 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:401: done (1 left) 2024-09-15T03:47:06.046 INFO:tasks.rados.rados.0.smithi122.stdout:403: delete oid 9 current snap is 46 2024-09-15T03:47:06.047 INFO:tasks.rados.rados.0.smithi122.stdout:402: finishing rollback tid 2 to smithi12223906-7 2024-09-15T03:47:06.047 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 99 (ObjNum 127 snap 27 seq_num 127) dirty exists 2024-09-15T03:47:06.048 INFO:tasks.rados.rados.0.smithi122.stdout:402: done (1 left) 2024-09-15T03:47:06.048 INFO:tasks.rados.rados.0.smithi122.stdout:403: done (0 left) 2024-09-15T03:47:06.048 INFO:tasks.rados.rados.0.smithi122.stdout:404: copy_from oid 30 from oid 11 current snap is 46 2024-09-15T03:47:06.048 INFO:tasks.rados.rados.0.smithi122.stdout:405: read oid 49 snap -1 2024-09-15T03:47:06.048 INFO:tasks.rados.rados.0.smithi122.stdout:405: expect (ObjNum 118 snap 24 seq_num 118) 2024-09-15T03:47:06.048 INFO:tasks.rados.rados.0.smithi122.stdout:406: write oid 20 current snap is 46 2024-09-15T03:47:06.049 INFO:tasks.rados.rados.0.smithi122.stdout:406: seq_num 176 ranges {558032=719943,1849419=592048} 2024-09-15T03:47:06.063 INFO:tasks.rados.rados.0.smithi122.stdout:406: writing smithi12223906-20 from 558032 to 1277975 tid 1 2024-09-15T03:47:06.070 INFO:tasks.rados.rados.0.smithi122.stdout:406: writing smithi12223906-20 from 1849419 to 2441467 tid 2 2024-09-15T03:47:06.070 INFO:tasks.rados.rados.0.smithi122.stdout:407: snap_remove snap 43 2024-09-15T03:47:06.719 INFO:tasks.rados.rados.0.smithi122.stdout:407: done (3 left) 2024-09-15T03:47:06.719 INFO:tasks.rados.rados.0.smithi122.stdout:408: snap_remove snap 45 2024-09-15T03:47:06.719 INFO:tasks.rados.rados.0.smithi122.stdout:404: finishing copy_from racing read to smithi12223906-30 2024-09-15T03:47:06.719 INFO:tasks.rados.rados.0.smithi122.stdout:405: expect (ObjNum 118 snap 24 seq_num 118) 2024-09-15T03:47:06.905 INFO:tasks.rados.rados.0.smithi122.stdout:404: finishing copy_from to smithi12223906-30 2024-09-15T03:47:06.905 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-15T03:47:06.905 INFO:tasks.rados.rados.0.smithi122.stdout:406: finishing write tid 1 to smithi12223906-20 2024-09-15T03:47:06.905 INFO:tasks.rados.rados.0.smithi122.stdout:406: finishing write tid 2 to smithi12223906-20 2024-09-15T03:47:06.905 INFO:tasks.rados.rados.0.smithi122.stdout:406: finishing write tid 3 to smithi12223906-20 2024-09-15T03:47:06.905 INFO:tasks.rados.rados.0.smithi122.stdout:406: finishing write tid 4 to smithi12223906-20 2024-09-15T03:47:06.906 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 14 (ObjNum 176 snap 46 seq_num 176) dirty exists 2024-09-15T03:47:06.906 INFO:tasks.rados.rados.0.smithi122.stdout:406: left oid 20 (ObjNum 176 snap 46 seq_num 176) 2024-09-15T03:47:07.864 INFO:tasks.rados.rados.0.smithi122.stdout:404: done (3 left) 2024-09-15T03:47:07.864 INFO:tasks.rados.rados.0.smithi122.stdout:405: done (2 left) 2024-09-15T03:47:07.864 INFO:tasks.rados.rados.0.smithi122.stdout:406: done (1 left) 2024-09-15T03:47:07.864 INFO:tasks.rados.rados.0.smithi122.stdout:408: done (0 left) 2024-09-15T03:47:07.864 INFO:tasks.rados.rados.0.smithi122.stdout:409: delete oid 25 current snap is 46 2024-09-15T03:47:07.870 INFO:tasks.rados.rados.0.smithi122.stdout:409: done (0 left) 2024-09-15T03:47:07.870 INFO:tasks.rados.rados.0.smithi122.stdout:410: delete oid 47 current snap is 46 2024-09-15T03:47:07.871 INFO:tasks.rados.rados.0.smithi122.stdout:410: done (0 left) 2024-09-15T03:47:07.871 INFO:tasks.rados.rados.0.smithi122.stdout:411: snap_create 2024-09-15T03:47:08.867 INFO:tasks.rados.rados.0.smithi122.stdout:411: done (0 left) 2024-09-15T03:47:08.868 INFO:tasks.rados.rados.0.smithi122.stdout:412: snap_create 2024-09-15T03:47:09.855 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T03:47:09.856 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T03:47:09.856 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T03:47:09.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T03:47:09.859 INFO:tasks.ceph.mon.f:Started 2024-09-15T03:47:09.859 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T03:47:09.860 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-15T03:47:09.860 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T03:47:09.860 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T03:47:09.862 INFO:tasks.ceph.mon.i:Started 2024-09-15T03:47:09.862 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T03:47:09.862 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-15T03:47:09.862 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-15T03:47:09.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-15T03:47:09.865 INFO:tasks.ceph.mon.e:Started 2024-09-15T03:47:09.865 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T03:47:09.865 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T03:47:09.865 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T03:47:09.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T03:47:09.867 INFO:tasks.ceph.mon.b:Started 2024-09-15T03:47:09.867 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-15T03:47:09.868 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-15T03:47:09.868 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-09-15T03:47:09.868 INFO:tasks.ceph.mon.g:Sent signal 19 2024-09-15T03:47:09.868 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:47:24.869 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-15T03:47:24.869 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-15T03:47:24.869 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-09-15T03:47:24.869 INFO:tasks.ceph.mon.g:Sent signal 18 2024-09-15T03:47:24.869 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:47:24.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:47:24.975 INFO:tasks.rados.rados.0.smithi122.stdout:412: done (0 left) 2024-09-15T03:47:24.975 INFO:tasks.rados.rados.0.smithi122.stdout:413: snap_remove snap 46 2024-09-15T03:47:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.978+0000 7fbd1dbcd700 10 monclient: get_monmap_and_config 2024-09-15T03:47:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.978+0000 7fbd1dbcd700 10 monclient: build_initial_monmap 2024-09-15T03:47:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.978+0000 7fbd1dbcd700 10 monclient: monmap: 2024-09-15T03:47:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:24.983425+0000 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:24.983425+0000 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:24.985 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:24.985 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:24.985 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:24.985 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:24.985 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:24.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 1 Processor -- start 2024-09-15T03:47:24.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 1 -- start start 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient: init 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient: _add_conns ranks=[8,2,4,6,5,0,7,3,1] 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient(hunting): picked mon.noname-e con 0x7fbd18070f80 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient(hunting): picked mon.noname-b con 0x7fbd1809dee0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient(hunting): picked mon.noname-c con 0x7fbd181391e0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1804ca10 con 0x7fbd1809dee0 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1809edc0 con 0x7fbd181391e0 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1809eae0 con 0x7fbd18070f80 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:24.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1dbcd700 10 monclient(hunting): authenticate will time out at 989.413490s 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1d3cc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fbd181391e0 0x7fbd1813b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42726/0 (socket says 172.21.15.26:42726) 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1cbcb700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fbd18070f80 0x7fbd180950b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56904/0 (socket says 172.21.15.26:56904) 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd1d3cc700 1 -- 172.21.15.26:0/18299200 learned_addr learned my addr 172.21.15.26:0/18299200 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346983705 0 0) 0x7fbd1809edc0 con 0x7fbd181391e0 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:24.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 10 monclient(hunting): my global_id is 4504 2024-09-15T03:47:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd08003570 con 0x7fbd181391e0 2024-09-15T03:47:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325868542 0 0) 0x7fbd1804ca10 con 0x7fbd1809dee0 2024-09-15T03:47:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 10 monclient(hunting): my global_id is 134102 2024-09-15T03:47:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.982+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd1809edc0 con 0x7fbd1809dee0 2024-09-15T03:47:24.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.986+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256182582 0 0) 0x7fbd1809eae0 con 0x7fbd18070f80 2024-09-15T03:47:24.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.986+0000 7fbd177fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:24.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.986+0000 7fbd177fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:24.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.986+0000 7fbd177fe700 10 monclient(hunting): my global_id is 134108 2024-09-15T03:47:24.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.986+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd1804ca10 con 0x7fbd18070f80 2024-09-15T03:47:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153180590 0 0) 0x7fbd08003570 con 0x7fbd181391e0 2024-09-15T03:47:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 >> v1:172.21.15.26:6793/0 conn(0x7fbd18070f80 legacy=0x7fbd180950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 >> v1:172.21.15.26:6790/0 conn(0x7fbd1809dee0 legacy=0x7fbd180957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 monclient: found mon.noname-c 2024-09-15T03:47:24.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1809e800 con 0x7fbd181391e0 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd10001b30 con 0x7fbd181391e0 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: dump: 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:24.997 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:24.998 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 5 monclient: authenticate success, global_id 4504 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd10001d50 con 0x7fbd181391e0 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:24.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 1 -- 172.21.15.26:0/18299200 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd10001f40 con 0x7fbd181391e0 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd177fe700 10 monclient: dump: 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.000 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 4 set_mon_vals no callback set 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 10 monclient: shutdown 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/18299200 >> v1:172.21.15.26:6791/0 conn(0x7fbd181391e0 legacy=0x7fbd1813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/18299200 shutdown_connections 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/18299200 >> 172.21.15.26:0/18299200 conn(0x7fbd180a24e0 msgr2=0x7fbd180a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/18299200 shutdown_connections 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/18299200 wait complete. 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.990+0000 7fbd1dbcd700 10 monclient: monmap: 2024-09-15T03:47:25.002 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.003 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.004 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.004 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.004 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.994+0000 7fbd1dbcd700 1 Processor -- start 2024-09-15T03:47:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.994+0000 7fbd1dbcd700 1 -- start start 2024-09-15T03:47:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.994+0000 7fbd1dbcd700 10 monclient: init 2024-09-15T03:47:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.994+0000 7fbd1dbcd700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.994+0000 7fbd1dbcd700 10 monclient: _add_conns ranks=[4,5,7,3,6,0,2,8,1] 2024-09-15T03:47:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.994+0000 7fbd1dbcd700 10 monclient(hunting): picked mon.noname-c con 0x7fbd18070f80 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:25.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient(hunting): picked mon.noname-h con 0x7fbd1809dee0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient(hunting): picked mon.noname-i con 0x7fbd181391e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd18099250 con 0x7fbd18070f80 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1806b700 con 0x7fbd1809dee0 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1806b9e0 con 0x7fbd181391e0 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient(hunting): authenticate will time out at 989.428098s 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1cbcb700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fbd18070f80 0x7fbd1806d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42742/0 (socket says 172.21.15.26:42742) 2024-09-15T03:47:25.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1cbcb700 1 -- 172.21.15.26:0/2338700253 learned_addr learned my addr 172.21.15.26:0/2338700253 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879960008 0 0) 0x7fbd18099250 con 0x7fbd18070f80 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): my global_id is 4513 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc003570 con 0x7fbd18070f80 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3548419420 0 0) 0x7fbcfc003570 con 0x7fbd18070f80 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd18099250 con 0x7fbd18070f80 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd0c001a60 con 0x7fbd18070f80 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:25.011 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:25.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138109331 0 0) 0x7fbd1806b700 con 0x7fbd1809dee0 2024-09-15T03:47:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient(hunting): my global_id is 4631 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc003570 con 0x7fbd1809dee0 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2282981298 0 0) 0x7fbd18099250 con 0x7fbd18070f80 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 >> v1:172.21.15.122:6792/0 conn(0x7fbd181391e0 legacy=0x7fbd18156650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 >> v1:172.21.15.122:6791/0 conn(0x7fbd1809dee0 legacy=0x7fbd18095b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 monclient: found mon.c 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1806a7a0 con 0x7fbd18070f80 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 5 monclient: authenticate success, global_id 4513 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient: _renew_subs 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd18157d70 con 0x7fbd18070f80 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient: _renew_subs 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd181580a0 con 0x7fbd18070f80 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd0c004be0 con 0x7fbd18070f80 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd0c005110 con 0x7fbd18070f80 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd167fc700 4 set_mon_vals no callback set 2024-09-15T03:47:25.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 10 monclient: dump: 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbd0c0052f0 con 0x7fbd18070f80 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf80050b0 con 0x7fbd18070f80 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (unknown 2280859119 0 0) 0x7fbd0c0440e0 con 0x7fbd18070f80 2024-09-15T03:47:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:24.998+0000 7fbd17fff700 10 monclient: get_auth_request con 0x7fbcfc0374f0 auth_method 0 2024-09-15T03:47:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.002+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fbd0c011550 con 0x7fbd18070f80 2024-09-15T03:47:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.002+0000 7fbd14ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.002+0000 7fbd14ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd1dbcd700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:47:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd1dbcd700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:47:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd1dbcd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbcf80058e0 con 0x7fbd18070f80 2024-09-15T03:47:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd14ff9700 1 -- 172.21.15.26:0/2338700253 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 1237693022) 0x7fbd0c0148e0 con 0x7fbd18070f80 2024-09-15T03:47:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd14ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:47:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.226+0000 7fbd14ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:25.229 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":34,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:47:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 >> v1:172.21.15.122:6812/20895 conn(0x7fbcfc0374f0 legacy=0x7fbcfc0399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 10 monclient: shutdown 2024-09-15T03:47:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 >> v1:172.21.15.26:6791/0 conn(0x7fbd18070f80 legacy=0x7fbd1806d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 shutdown_connections 2024-09-15T03:47:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 >> 172.21.15.26:0/2338700253 conn(0x7fbd180a24e0 msgr2=0x7fbd1809b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 shutdown_connections 2024-09-15T03:47:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.230+0000 7fbd1dbcd700 1 -- 172.21.15.26:0/2338700253 wait complete. 2024-09-15T03:47:25.249 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:47:25.249 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:47:25.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient: get_monmap_and_config 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient: monmap: 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:25.353957+0000 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:25.353957+0000 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.355 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 1 Processor -- start 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 1 -- start start 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient: init 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient: _add_conns ranks=[0,5,7,4,3,6,8,2,1] 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient(hunting): picked mon.noname-a con 0x7f3bfc132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient(hunting): picked mon.noname-h con 0x7f3bfc09faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient(hunting): picked mon.noname-i con 0x7f3bfc138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc04ca10 con 0x7f3bfc132b40 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc09d5d0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc133620 con 0x7f3bfc138d40 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.350+0000 7f3c01ebe700 10 monclient(hunting): authenticate will time out at 989.783919s 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c00ebc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3bfc132b40 0x7f3bfc132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36124/0 (socket says 172.21.15.26:36124) 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c00ebc700 1 -- 172.21.15.26:0/2547770688 learned_addr learned my addr 172.21.15.26:0/2547770688 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104249242 0 0) 0x7f3bfc04ca10 con 0x7f3bfc132b40 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): my global_id is 4509 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be4003570 con 0x7f3bfc132b40 2024-09-15T03:47:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988627586 0 0) 0x7f3bfc09d5d0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): my global_id is 4640 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bfc04ca10 con 0x7f3bfc09faa0 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126349046 0 0) 0x7f3bfc133620 con 0x7f3bfc138d40 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient(hunting): my global_id is 134107 2024-09-15T03:47:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bfc09d5d0 con 0x7f3bfc138d40 2024-09-15T03:47:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3422920715 0 0) 0x7f3be4003570 con 0x7f3bfc132b40 2024-09-15T03:47:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 >> v1:172.21.15.122:6792/0 conn(0x7f3bfc138d40 legacy=0x7f3bfc13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 >> v1:172.21.15.122:6791/0 conn(0x7f3bfc09faa0 legacy=0x7f3bfc09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 monclient: found mon.noname-a 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0a0cc0 con 0x7f3bfc132b40 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 5 monclient: authenticate success, global_id 4509 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bec001ab0 con 0x7f3bfc132b40 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: dump: 2024-09-15T03:47:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3bec001cd0 con 0x7f3bfc132b40 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 1 -- 172.21.15.26:0/2547770688 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bec004c70 con 0x7f3bfc132b40 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 4 set_mon_vals no callback set 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3bfb7fe700 10 monclient: dump: 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 10 monclient: shutdown 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2547770688 >> v1:172.21.15.26:6789/0 conn(0x7f3bfc132b40 legacy=0x7f3bfc132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2547770688 shutdown_connections 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2547770688 >> 172.21.15.26:0/2547770688 conn(0x7f3bfc06e370 msgr2=0x7f3bfc0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2547770688 shutdown_connections 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2547770688 wait complete. 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.354+0000 7f3c01ebe700 10 monclient: monmap: 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 Processor -- start 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 -- start start 2024-09-15T03:47:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: init 2024-09-15T03:47:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: _add_conns ranks=[4,8,3,1,5,7,0,6,2] 2024-09-15T03:47:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient(hunting): picked mon.noname-c con 0x7f3bfc09faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient(hunting): picked mon.noname-e con 0x7f3bfc132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient(hunting): picked mon.noname-g con 0x7f3bfc138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc1349d0 con 0x7f3bfc138d40 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc134cb0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc134f90 con 0x7f3bfc132b40 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient(hunting): authenticate will time out at 989.789350s 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c00ebc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f3bfc09faa0 0x7f3bfc130470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42758/0 (socket says 172.21.15.26:42758) 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c00ebc700 1 -- 172.21.15.26:0/2831426071 learned_addr learned my addr 172.21.15.26:0/2831426071 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bfbfff700 1 --1- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6793/0 conn(0x7f3bfc132b40 0x7f3bfc130b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56906/0 (socket says 172.21.15.26:56906) 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678599495 0 0) 0x7f3bfc134cb0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): my global_id is 4522 2024-09-15T03:47:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be80035a0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558491915 0 0) 0x7f3bfc134f90 con 0x7f3bfc132b40 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): my global_id is 134117 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bfc134cb0 con 0x7f3bfc132b40 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042364679 0 0) 0x7f3bfc1349d0 con 0x7f3bfc138d40 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): my global_id is 4593 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bfc134f90 con 0x7f3bfc138d40 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2285797940 0 0) 0x7f3bfc134cb0 con 0x7f3bfc132b40 2024-09-15T03:47:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bfc1349d0 con 0x7f3bfc132b40 2024-09-15T03:47:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bf0001bf0 con 0x7f3bfc132b40 2024-09-15T03:47:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.379 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2131566772 0 0) 0x7f3be80035a0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bfc134cb0 con 0x7f3bfc09faa0 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2481839183 0 0) 0x7f3bfc134f90 con 0x7f3bfc138d40 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3be80035a0 con 0x7f3bfc138d40 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bf40019c0 con 0x7f3bfc138d40 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.383 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2974019922 0 0) 0x7f3bfc1349d0 con 0x7f3bfc132b40 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6791/0 conn(0x7f3bfc09faa0 legacy=0x7f3bfc130470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.122:6790/0 conn(0x7f3bfc138d40 legacy=0x7f3bfc14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 monclient: found mon.e 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc135b80 con 0x7f3bfc132b40 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 5 monclient: authenticate success, global_id 134117 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: _renew_subs 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc134620 con 0x7f3bfc132b40 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: _renew_subs 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc131290 con 0x7f3bfc132b40 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3bf0002210 con 0x7f3bfc132b40 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bf0002740 con 0x7f3bfc132b40 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bfaffd700 4 set_mon_vals no callback set 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.358+0000 7f3bf8ff9700 10 monclient: dump: 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.386 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.387 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.388 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3c01ebe700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3bdc006590 con 0x7f3bdc002840 2024-09-15T03:47:25.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3bf0002920 con 0x7f3bfc132b40 2024-09-15T03:47:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (unknown 2280859119 0 0) 0x7f3bf0043f50 con 0x7f3bfc132b40 2024-09-15T03:47:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873991418 0 0) 0x7f3bdc006590 con 0x7f3bdc002840 2024-09-15T03:47:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient(hunting): my global_id is 4518 2024-09-15T03:47:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bfc1349d0 con 0x7f3bdc002840 2024-09-15T03:47:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3c016bd700 10 monclient: get_auth_request con 0x7f3be8037ef0 auth_method 0 2024-09-15T03:47:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2637891296 0 0) 0x7f3bfc1349d0 con 0x7f3bdc002840 2024-09-15T03:47:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bdc006590 con 0x7f3bdc002840 2024-09-15T03:47:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bec005280 con 0x7f3bdc002840 2024-09-15T03:47:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient: dump: 2024-09-15T03:47:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.394 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.395 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1120343832 0 0) 0x7f3bdc006590 con 0x7f3bdc002840 2024-09-15T03:47:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bdc00a130 con 0x7f3bdc002840 2024-09-15T03:47:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3bec007480 con 0x7f3bdc002840 2024-09-15T03:47:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:25.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6789/0 conn(0x7f3bdc002840 legacy=0x7f3bdc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.362+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6789/0 conn(0x7f3bdc002840 legacy=0x7f3bdc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:25.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3bdc007520 con 0x7f3bdc0068d0 2024-09-15T03:47:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854680637 0 0) 0x7f3bdc007520 con 0x7f3bdc0068d0 2024-09-15T03:47:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient(hunting): my global_id is 4527 2024-09-15T03:47:25.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc006590 con 0x7f3bdc0068d0 2024-09-15T03:47:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1572798151 0 0) 0x7f3bdc006590 con 0x7f3bdc0068d0 2024-09-15T03:47:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3bdc007520 con 0x7f3bdc0068d0 2024-09-15T03:47:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3bec005c20 con 0x7f3bdc0068d0 2024-09-15T03:47:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:25.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient: dump: 2024-09-15T03:47:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.404 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.405 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.406 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 246677705 0 0) 0x7f3bdc007520 con 0x7f3bdc0068d0 2024-09-15T03:47:25.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3bdc008760 con 0x7f3bdc0068d0 2024-09-15T03:47:25.407 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:25.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:25.407 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:25.407 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:25.407 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:25.408 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:25.408 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:25.408 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:25.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:25.408 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:25.409 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:25.409 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:25.409 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:25.409 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:25.409 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:25.410 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.410 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:25.410 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:25.410 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:25.411 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:25.411 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.411 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.412 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.412 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.412 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.413 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.413 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.413 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.413 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:25.413 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:25.413 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.414 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.414 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.414 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.414 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.414 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.415 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.415 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.415 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.416 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:25.416 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:25.416 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:25.417 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:25.417 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:25.417 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:25.417 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:25.417 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:25.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:25.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:25.418 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:25.418 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:25.418 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:25.418 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:25.419 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:25.419 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:25.419 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:25.419 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.419 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.420 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.420 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.420 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.420 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.421 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.421 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.421 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.421 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.422 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:25.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.422 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:25.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.422 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:25.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:25.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:25.424 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:25.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:25.425 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.425 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.425 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:25.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:25.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:25.427 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.428 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:25.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:25.428 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.429 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.429 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.429 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:25.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:25.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:25.431 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.431 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.431 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:25.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:25.432 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.432 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.432 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.433 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:25.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:25.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:25.434 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.435 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:25.435 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:25.435 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.436 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.436 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.436 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:25.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:25.437 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.437 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.437 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:25.438 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.438 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.438 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:25.438 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:25.439 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.439 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.439 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.439 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:25.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:25.440 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.440 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.441 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:25.441 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.441 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.441 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.441 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:25.442 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:25.442 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.442 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.442 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.443 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:25.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:25.443 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.443 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.446 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:25.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:25.446 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.446 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:25.447 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.447 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.448 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.448 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:25.448 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:25.448 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.448 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:25.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:25.450 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.451 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.451 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.451 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:25.452 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:25.452 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.452 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.452 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.452 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.453 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.453 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:25.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:25.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.453 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.454 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.454 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.454 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.454 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:25.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.456 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:47:25.456 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.456 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.456 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.456 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.456 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.457 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.457 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:25.457 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2687016259) 0x7f3bec0076b0 con 0x7f3bdc0068d0 2024-09-15T03:47:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6789/0 conn(0x7f3bdc0068d0 legacy=0x7f3bdc006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3bf8ff9700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6789/0 conn(0x7f3bdc0068d0 legacy=0x7f3bdc006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.122:6812/20895 conn(0x7f3be8037ef0 legacy=0x7f3be803a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 10 monclient: shutdown 2024-09-15T03:47:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 >> v1:172.21.15.26:6793/0 conn(0x7f3bfc132b40 legacy=0x7f3bfc130b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c016bd700 1 -- 172.21.15.26:0/2831426071 reap_dead start 2024-09-15T03:47:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 shutdown_connections 2024-09-15T03:47:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 >> 172.21.15.26:0/2831426071 conn(0x7f3bfc06e370 msgr2=0x7f3bfc12eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 shutdown_connections 2024-09-15T03:47:25.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.374+0000 7f3c01ebe700 1 -- 172.21.15.26:0/2831426071 wait complete. 2024-09-15T03:47:25.463 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:25.463 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:47:25.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:47:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.574+0000 7f8fd352a700 10 monclient: get_monmap_and_config 2024-09-15T03:47:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.574+0000 7f8fd352a700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.574+0000 7f8fd352a700 10 monclient: monmap: 2024-09-15T03:47:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:25.579365+0000 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:25.579365+0000 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.581 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 Processor -- start 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- start start 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient: init 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient: _add_conns ranks=[8,5,2,6,3,4,1,0,7] 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient(hunting): picked mon.noname-e con 0x7f8fcc0992d0 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient(hunting): picked mon.noname-h con 0x7f8fcc0996a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient(hunting): picked mon.noname-b con 0x7f8fcc099a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc04ca10 con 0x7f8fcc099a70 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc0a2760 con 0x7f8fcc0996a0 2024-09-15T03:47:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc133950 con 0x7f8fcc0992d0 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient(hunting): authenticate will time out at 990.009007s 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd12c6700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8fcc0992d0 0x7f8fcc133240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56914/0 (socket says 172.21.15.26:56914) 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd12c6700 1 -- 172.21.15.26:0/2485171349 learned_addr learned my addr 172.21.15.26:0/2485171349 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391897858 0 0) 0x7f8fcc04ca10 con 0x7f8fcc099a70 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): my global_id is 134111 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb8003570 con 0x7f8fcc099a70 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476460096 0 0) 0x7f8fcc133950 con 0x7f8fcc0992d0 2024-09-15T03:47:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): my global_id is 134126 2024-09-15T03:47:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc04ca10 con 0x7f8fcc0992d0 2024-09-15T03:47:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263127952 0 0) 0x7f8fcc0a2760 con 0x7f8fcc0996a0 2024-09-15T03:47:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient(hunting): my global_id is 4649 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc133950 con 0x7f8fcc0996a0 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895749702 0 0) 0x7f8fcc04ca10 con 0x7f8fcc0992d0 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 >> v1:172.21.15.122:6791/0 conn(0x7f8fcc0996a0 legacy=0x7f8fcc137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 >> v1:172.21.15.26:6790/0 conn(0x7f8fcc099a70 legacy=0x7f8fcc13b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 monclient: found mon.noname-e 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc099ef0 con 0x7f8fcc0992d0 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 5 monclient: authenticate success, global_id 134126 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fc8001790 con 0x7f8fcc0992d0 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: dump: 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8fc8001ab0 con 0x7f8fcc0992d0 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 1 -- 172.21.15.26:0/2485171349 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fc8004c30 con 0x7f8fcc0992d0 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 4 set_mon_vals no callback set 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fc3fff700 10 monclient: dump: 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 10 monclient: shutdown 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- 172.21.15.26:0/2485171349 >> v1:172.21.15.26:6793/0 conn(0x7f8fcc0992d0 legacy=0x7f8fcc133240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- 172.21.15.26:0/2485171349 shutdown_connections 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- 172.21.15.26:0/2485171349 >> 172.21.15.26:0/2485171349 conn(0x7f8fcc06e470 msgr2=0x7f8fcc0a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- 172.21.15.26:0/2485171349 shutdown_connections 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.578+0000 7f8fd352a700 1 -- 172.21.15.26:0/2485171349 wait complete. 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient: monmap: 2024-09-15T03:47:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.595 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 1 Processor -- start 2024-09-15T03:47:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 1 -- start start 2024-09-15T03:47:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient: init 2024-09-15T03:47:25.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient: _add_conns ranks=[5,0,2,6,8,3,1,7,4] 2024-09-15T03:47:25.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient(hunting): picked mon.noname-h con 0x7f8fcc0992d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient(hunting): picked mon.noname-a con 0x7f8fcc0996a0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient(hunting): picked mon.noname-b con 0x7f8fcc099a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc11f100 con 0x7f8fcc0996a0 2024-09-15T03:47:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc13ff30 con 0x7f8fcc099a70 2024-09-15T03:47:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc11f3e0 con 0x7f8fcc0992d0 2024-09-15T03:47:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 10 monclient(hunting): authenticate will time out at 990.014141s 2024-09-15T03:47:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd0ac5700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8fcc0996a0 0x7f8fcc146db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36146/0 (socket says 172.21.15.26:36146) 2024-09-15T03:47:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd0ac5700 1 -- 172.21.15.26:0/2871998322 learned_addr learned my addr 172.21.15.26:0/2871998322 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162204504 0 0) 0x7f8fcc11f100 con 0x7f8fcc0996a0 2024-09-15T03:47:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): my global_id is 4536 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb0003570 con 0x7f8fcc0996a0 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106070774 0 0) 0x7f8fcc13ff30 con 0x7f8fcc099a70 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): my global_id is 134120 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc11f100 con 0x7f8fcc099a70 2024-09-15T03:47:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086165766 0 0) 0x7f8fcc11f3e0 con 0x7f8fcc0992d0 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): my global_id is 4658 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc13ff30 con 0x7f8fcc0992d0 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 351707646 0 0) 0x7f8fb0003570 con 0x7f8fcc0996a0 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fcc11f3e0 con 0x7f8fcc0996a0 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fbc001d10 con 0x7f8fcc0996a0 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): dump: 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3427228786 0 0) 0x7f8fcc11f100 con 0x7f8fcc099a70 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fb0003570 con 0x7f8fcc099a70 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fc4001c60 con 0x7f8fcc099a70 2024-09-15T03:47:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): dump: 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 845102518 0 0) 0x7f8fcc13ff30 con 0x7f8fcc0992d0 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fcc11f100 con 0x7f8fcc0992d0 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fc8001750 con 0x7f8fcc0992d0 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient(hunting): dump: 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.608 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4203845512 0 0) 0x7f8fcc11f3e0 con 0x7f8fcc0996a0 2024-09-15T03:47:25.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.122:6791/0 conn(0x7f8fcc0992d0 legacy=0x7f8fcc131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.26:6790/0 conn(0x7f8fcc099a70 legacy=0x7f8fcc14a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 monclient: found mon.a 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc11ee20 con 0x7f8fcc0996a0 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fc17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.582+0000 7f8fd352a700 5 monclient: authenticate success, global_id 4536 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 10 monclient: _renew_subs 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc11df30 con 0x7f8fcc0996a0 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 10 monclient: _renew_subs 2024-09-15T03:47:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8fbc002230 con 0x7f8fcc0996a0 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0a1b90 con 0x7f8fcc0996a0 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fbc002760 con 0x7f8fcc0996a0 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc37fe700 4 set_mon_vals no callback set 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: dump: 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8fbc00fb20 con 0x7f8fcc0996a0 2024-09-15T03:47:25.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8fcc04bbf0 con 0x7f8fcc09c350 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (unknown 2280859119 0 0) 0x7f8fbc043ef0 con 0x7f8fcc0996a0 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822171600 0 0) 0x7f8fcc04bbf0 con 0x7f8fcc09c350 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient(hunting): my global_id is 134129 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc11f3e0 con 0x7f8fcc09c350 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fd1ac7700 10 monclient: get_auth_request con 0x7f8fb0037ff0 auth_method 0 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2353171869 0 0) 0x7f8fcc11f3e0 con 0x7f8fcc09c350 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fcc04bbf0 con 0x7f8fcc09c350 2024-09-15T03:47:25.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fc8001210 con 0x7f8fcc09c350 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: dump: 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.614 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3712774163 0 0) 0x7f8fcc04bbf0 con 0x7f8fcc09c350 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fcc09c720 con 0x7f8fcc09c350 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8fc80052c0 con 0x7f8fcc09c350 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.26:6790/0 conn(0x7f8fcc09c350 legacy=0x7f8fcc09e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.586+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.26:6790/0 conn(0x7f8fcc09c350 legacy=0x7f8fcc09e5b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fd352a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:25.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8fcc11f7a0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540501223 0 0) 0x7f8fcc11f7a0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient(hunting): my global_id is 134138 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fcc04bbf0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1206946027 0 0) 0x7f8fcc04bbf0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8fcc11f7a0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8fc8005a50 con 0x7f8fcc02ad10 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient: dump: 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.618 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3635957723 0 0) 0x7f8fcc11f7a0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8fcc06f230 con 0x7f8fcc02ad10 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 587400346) 0x7f8fc80082b0 con 0x7f8fcc02ad10 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.598+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.26:6790/0 conn(0x7f8fcc02ad10 legacy=0x7f8fcc09ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fc17fa700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.26:6790/0 conn(0x7f8fcc02ad10 legacy=0x7f8fcc09ecc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:25.620 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:25.621 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.622 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:25.623 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:25.624 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.625 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.627 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:25.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:25.628 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.628 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.628 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.629 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:25.632 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.633 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.637 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.638 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.639 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:25.640 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:25.640 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.640 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.640 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.640 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.640 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.641 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.642 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.642 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.642 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.642 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:25.642 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.122:6812/20895 conn(0x7f8fb0037ff0 legacy=0x7f8fb003a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 10 monclient: shutdown 2024-09-15T03:47:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 >> v1:172.21.15.26:6789/0 conn(0x7f8fcc0996a0 legacy=0x7f8fcc146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd1ac7700 1 -- 172.21.15.26:0/2871998322 reap_dead start 2024-09-15T03:47:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 shutdown_connections 2024-09-15T03:47:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 >> 172.21.15.26:0/2871998322 conn(0x7f8fcc06e470 msgr2=0x7f8fcc135460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 shutdown_connections 2024-09-15T03:47:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.602+0000 7f8fd352a700 1 -- 172.21.15.26:0/2871998322 wait complete. 2024-09-15T03:47:25.644 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:25.644 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:47:25.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:47:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient: get_monmap_and_config 2024-09-15T03:47:25.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient: monmap: 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:25.761422+0000 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:25.761422+0000 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.762 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.763 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.763 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 1 Processor -- start 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 1 -- start start 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient: init 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient: _add_conns ranks=[3,4,8,2,0,5,6,1,7] 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient(hunting): picked mon.noname-g con 0x7f86fc09ff60 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:25.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient(hunting): picked mon.noname-c con 0x7f86fc09caf0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient(hunting): picked mon.noname-e con 0x7f86fc09cee0 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc04ca10 con 0x7f86fc09ff60 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc09de50 con 0x7f86fc09caf0 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc09dc10 con 0x7f86fc09cee0 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86fbfff700 10 monclient(hunting): authenticate will time out at 990.191579s 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86f3fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f86fc09caf0 0x7f86fc135ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42772/0 (socket says 172.21.15.26:42772) 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.758+0000 7f86f3fff700 1 -- 172.21.15.26:0/3298858796 learned_addr learned my addr 172.21.15.26:0/3298858796 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785716906 0 0) 0x7f86fc09de50 con 0x7f86fc09caf0 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): my global_id is 4531 2024-09-15T03:47:25.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e8003570 con 0x7f86fc09caf0 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300532478 0 0) 0x7f86fc09dc10 con 0x7f86fc09cee0 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): my global_id is 134135 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc09de50 con 0x7f86fc09cee0 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410980178 0 0) 0x7f86fc04ca10 con 0x7f86fc09ff60 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient(hunting): my global_id is 4602 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc09dc10 con 0x7f86fc09ff60 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2736889303 0 0) 0x7f86e8003570 con 0x7f86fc09caf0 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 >> v1:172.21.15.26:6793/0 conn(0x7f86fc09cee0 legacy=0x7f86fc1392d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 >> v1:172.21.15.122:6790/0 conn(0x7f86fc09ff60 legacy=0x7f86fc130e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 monclient: found mon.noname-c 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc09d360 con 0x7f86fc09caf0 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 5 monclient: authenticate success, global_id 4531 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86e40017b0 con 0x7f86fc09caf0 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:25.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: dump: 2024-09-15T03:47:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.771 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86e4001a20 con 0x7f86fc09caf0 2024-09-15T03:47:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 1 -- 172.21.15.26:0/3298858796 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86e4004c30 con 0x7f86fc09caf0 2024-09-15T03:47:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 4 set_mon_vals no callback set 2024-09-15T03:47:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fa7fc700 10 monclient: dump: 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.775 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.775 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.775 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.775 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 10 monclient: shutdown 2024-09-15T03:47:25.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 -- 172.21.15.26:0/3298858796 >> v1:172.21.15.26:6791/0 conn(0x7f86fc09caf0 legacy=0x7f86fc135ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 -- 172.21.15.26:0/3298858796 shutdown_connections 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 -- 172.21.15.26:0/3298858796 >> 172.21.15.26:0/3298858796 conn(0x7f86fc0a3430 msgr2=0x7f86fc0a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 -- 172.21.15.26:0/3298858796 shutdown_connections 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 -- 172.21.15.26:0/3298858796 wait complete. 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 10 monclient: monmap: 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.788 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.789 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.790 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 Processor -- start 2024-09-15T03:47:25.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 1 -- start start 2024-09-15T03:47:25.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.762+0000 7f86fbfff700 10 monclient: init 2024-09-15T03:47:25.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient: _add_conns ranks=[5,6,8,7,4,0,2,3,1] 2024-09-15T03:47:25.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient(hunting): picked mon.noname-h con 0x7f86fc09caf0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:25.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient(hunting): picked mon.noname-d con 0x7f86fc09cee0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:25.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient(hunting): picked mon.noname-e con 0x7f86fc09ff60 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:25.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc09e8b0 con 0x7f86fc09caf0 2024-09-15T03:47:25.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc09e580 con 0x7f86fc09cee0 2024-09-15T03:47:25.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86fc11f7b0 con 0x7f86fc09ff60 2024-09-15T03:47:25.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient(hunting): authenticate will time out at 990.196727s 2024-09-15T03:47:25.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fb7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f86fc09ff60 0x7f86fc1482e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56922/0 (socket says 172.21.15.26:56922) 2024-09-15T03:47:25.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fb7fe700 1 -- 172.21.15.26:0/2921083543 learned_addr learned my addr 172.21.15.26:0/2921083543 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187879520 0 0) 0x7f86fc11f7b0 con 0x7f86fc09ff60 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): my global_id is 134144 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86dc003570 con 0x7f86fc09ff60 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754693079 0 0) 0x7f86fc09e580 con 0x7f86fc09cee0 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): my global_id is 4587 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc11f7b0 con 0x7f86fc09cee0 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161870463 0 0) 0x7f86fc09e8b0 con 0x7f86fc09caf0 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): my global_id is 4667 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc09e580 con 0x7f86fc09caf0 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 958807964 0 0) 0x7f86dc003570 con 0x7f86fc09ff60 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86fc09e8b0 con 0x7f86fc09ff60 2024-09-15T03:47:25.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86f4002e50 con 0x7f86fc09ff60 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.796 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.797 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.797 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.797 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.797 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.797 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.797 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.798 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.798 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.798 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.798 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.798 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.798 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3532401543 0 0) 0x7f86fc11f7b0 con 0x7f86fc09cee0 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86dc003570 con 0x7f86fc09cee0 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86e4001770 con 0x7f86fc09cee0 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.799 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.800 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 140015155 0 0) 0x7f86fc09e8b0 con 0x7f86fc09ff60 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.26:6792/0 conn(0x7f86fc09cee0 legacy=0x7f86fc144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.122:6791/0 conn(0x7f86fc09caf0 legacy=0x7f86fc12f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 monclient: found mon.e 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc131d60 con 0x7f86fc09ff60 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 5 monclient: authenticate success, global_id 134144 2024-09-15T03:47:25.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient: _renew_subs 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86fc11fd40 con 0x7f86fc09ff60 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient: _renew_subs 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f86fc140730 con 0x7f86fc09ff60 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f86f40030f0 con 0x7f86fc09ff60 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86f4001d20 con 0x7f86fc09ff60 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86fbfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f9ffb700 4 set_mon_vals no callback set 2024-09-15T03:47:25.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 10 monclient: dump: 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.803 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.766+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f86f4001e80 con 0x7f86fc09ff60 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (unknown 2280859119 0 0) 0x7f86f4044250 con 0x7f86fc09ff60 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f86fc11d5c0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86faffd700 10 monclient: get_auth_request con 0x7f86dc037ed0 auth_method 0 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251027944 0 0) 0x7f86fc11d5c0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient(hunting): my global_id is 4540 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc09e8b0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 189464381 0 0) 0x7f86fc09e8b0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86fc11d5c0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86e4001200 con 0x7f86fc11d1f0 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient: dump: 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.805 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.806 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4056522580 0 0) 0x7f86fc11d5c0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86fc04bbf0 con 0x7f86fc11d1f0 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f86e4005240 con 0x7f86fc11d1f0 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.26:6791/0 conn(0x7f86fc11d1f0 legacy=0x7f86fc119520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.770+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.26:6791/0 conn(0x7f86fc11d1f0 legacy=0x7f86fc119520 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86fbfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f86fc11f960 con 0x7f86fc02ad10 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448165315 0 0) 0x7f86dc003570 con 0x7f86fc02ad10 2024-09-15T03:47:25.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 10 monclient(hunting): my global_id is 4549 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc11d5c0 con 0x7f86fc02ad10 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 974991049 0 0) 0x7f86fc11d5c0 con 0x7f86fc02ad10 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86dc003570 con 0x7f86fc02ad10 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f86e4005af0 con 0x7f86fc02ad10 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 10 monclient: dump: 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.808 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.809 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1949441405 0 0) 0x7f86dc003570 con 0x7f86fc02ad10 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.778+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f86fc0008d0 con 0x7f86fc02ad10 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1574011265) 0x7f86e4008230 con 0x7f86fc02ad10 2024-09-15T03:47:25.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86f2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:25.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86f2ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:25.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.26:6791/0 conn(0x7f86fc02ad10 legacy=0x7f86fc119c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86f2ffd700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.26:6791/0 conn(0x7f86fc02ad10 legacy=0x7f86fc119c30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:25.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:25.812 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:25.812 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:25.812 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:25.812 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:25.813 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:25.813 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:25.813 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:25.813 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:25.813 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:25.813 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:25.814 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:25.814 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:25.814 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:25.814 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.815 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.816 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:25.817 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:25.818 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:25.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.820 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:25.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.824 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:25.827 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.829 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:25.830 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:25.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:25.832 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:25.833 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.122:6812/20895 conn(0x7f86dc037ed0 legacy=0x7f86dc03a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 10 monclient: shutdown 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 >> v1:172.21.15.26:6793/0 conn(0x7f86fc09ff60 legacy=0x7f86fc1482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fb7fe700 1 -- 172.21.15.26:0/2921083543 reap_dead start 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 shutdown_connections 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 >> 172.21.15.26:0/2921083543 conn(0x7f86fc0a3430 msgr2=0x7f86fc09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 shutdown_connections 2024-09-15T03:47:25.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.782+0000 7f86fbfff700 1 -- 172.21.15.26:0/2921083543 wait complete. 2024-09-15T03:47:25.835 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:25.835 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:47:25.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:47:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient: get_monmap_and_config 2024-09-15T03:47:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient: monmap: 2024-09-15T03:47:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:25.953610+0000 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:25.953610+0000 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.955 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.957 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.957 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 1 Processor -- start 2024-09-15T03:47:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 1 -- start start 2024-09-15T03:47:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient: init 2024-09-15T03:47:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient: _add_conns ranks=[7,5,2,1,3,6,0,4,8] 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient(hunting): picked mon.noname-i con 0x7f4d58132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient(hunting): picked mon.noname-h con 0x7f4d5809faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient(hunting): picked mon.noname-b con 0x7f4d58138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5804ca10 con 0x7f4d58138d40 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5809d5d0 con 0x7f4d5809faa0 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d58133620 con 0x7f4d58132b40 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e8f5700 10 monclient(hunting): authenticate will time out at 990.383656s 2024-09-15T03:47:25.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e0f4700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f4d58138d40 0x7f4d5813b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54434/0 (socket says 172.21.15.26:54434) 2024-09-15T03:47:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.950+0000 7f4d5e0f4700 1 -- 172.21.15.26:0/2668011500 learned_addr learned my addr 172.21.15.26:0/2668011500 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388596296 0 0) 0x7f4d5804ca10 con 0x7f4d58138d40 2024-09-15T03:47:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): my global_id is 134147 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d44003570 con 0x7f4d58138d40 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679992777 0 0) 0x7f4d58133620 con 0x7f4d58132b40 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): my global_id is 134116 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d5804ca10 con 0x7f4d58132b40 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788179287 0 0) 0x7f4d5809d5d0 con 0x7f4d5809faa0 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient(hunting): my global_id is 4676 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d58133620 con 0x7f4d5809faa0 2024-09-15T03:47:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218539152 0 0) 0x7f4d44003570 con 0x7f4d58138d40 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 >> v1:172.21.15.122:6792/0 conn(0x7f4d58132b40 legacy=0x7f4d58132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 >> v1:172.21.15.122:6791/0 conn(0x7f4d5809faa0 legacy=0x7f4d5809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 monclient: found mon.noname-b 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580a0cc0 con 0x7f4d58138d40 2024-09-15T03:47:25.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 5 monclient: authenticate success, global_id 134147 2024-09-15T03:47:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d50001b30 con 0x7f4d58138d40 2024-09-15T03:47:25.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: dump: 2024-09-15T03:47:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.974 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.975 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.976 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:25.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d50001d20 con 0x7f4d58138d40 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 4 set_mon_vals no callback set 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 1 -- 172.21.15.26:0/2668011500 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d50004c70 con 0x7f4d58138d40 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5c8f1700 10 monclient: dump: 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.977 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.978 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 10 monclient: shutdown 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/2668011500 >> v1:172.21.15.26:6790/0 conn(0x7f4d58138d40 legacy=0x7f4d5813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/2668011500 shutdown_connections 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/2668011500 >> 172.21.15.26:0/2668011500 conn(0x7f4d5806e370 msgr2=0x7f4d580a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/2668011500 shutdown_connections 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/2668011500 wait complete. 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 10 monclient: build_initial_monmap 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 10 monclient: monmap: 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:25.979 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:25.980 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 Processor -- start 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 1 -- start start 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.954+0000 7f4d5e8f5700 10 monclient: init 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient: _add_conns ranks=[6,4,2,3,5,7,8,1,0] 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient(hunting): picked mon.noname-d con 0x7f4d5809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient(hunting): picked mon.noname-c con 0x7f4d58132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient(hunting): picked mon.noname-b con 0x7f4d58138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d58137810 con 0x7f4d58138d40 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5809cfa0 con 0x7f4d58132b40 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d58137050 con 0x7f4d5809faa0 2024-09-15T03:47:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient(hunting): authenticate will time out at 990.388405s 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5d8f3700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f4d5809faa0 0x7f4d581355a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60476/0 (socket says 172.21.15.26:60476) 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5d0f2700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f4d58132b40 0x7f4d58135cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42784/0 (socket says 172.21.15.26:42784) 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5d8f3700 1 -- 172.21.15.26:0/1119914172 learned_addr learned my addr 172.21.15.26:0/1119914172 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e0f4700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f4d58138d40 0x7f4d581363c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54448/0 (socket says 172.21.15.26:54448) 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775777119 0 0) 0x7f4d58137810 con 0x7f4d58138d40 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): my global_id is 134156 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d3c003570 con 0x7f4d58138d40 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373053532 0 0) 0x7f4d5809cfa0 con 0x7f4d58132b40 2024-09-15T03:47:25.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): my global_id is 4558 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d58137810 con 0x7f4d58132b40 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158453877 0 0) 0x7f4d58137050 con 0x7f4d5809faa0 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): my global_id is 4596 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d5809cfa0 con 0x7f4d5809faa0 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1294612390 0 0) 0x7f4d58137810 con 0x7f4d58132b40 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d58137050 con 0x7f4d58132b40 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d48001a60 con 0x7f4d58132b40 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:25.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): dump: 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2392428930 0 0) 0x7f4d3c003570 con 0x7f4d58138d40 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d58137810 con 0x7f4d58138d40 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d50001af0 con 0x7f4d58138d40 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): dump: 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.986 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1057737083 0 0) 0x7f4d5809cfa0 con 0x7f4d5809faa0 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d3c003570 con 0x7f4d5809faa0 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d54001d90 con 0x7f4d5809faa0 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient(hunting): dump: 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.987 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.988 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 575181356 0 0) 0x7f4d58137050 con 0x7f4d58132b40 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6792/0 conn(0x7f4d5809faa0 legacy=0x7f4d581355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6790/0 conn(0x7f4d58138d40 legacy=0x7f4d581363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 monclient: found mon.c 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d58137560 con 0x7f4d58132b40 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: _finish_auth 0 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 5 monclient: authenticate success, global_id 4558 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient: _renew_subs 2024-09-15T03:47:25.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d58137c20 con 0x7f4d58132b40 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient: _renew_subs 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5814b200 con 0x7f4d58132b40 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d48002030 con 0x7f4d58132b40 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d48002560 con 0x7f4d58132b40 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 10 monclient: dump: 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.991 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4ffff700 4 set_mon_vals no callback set 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d38006590 con 0x7f4d38002840 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.958+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4d48002740 con 0x7f4d58132b40 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(116..116 src has 1..116) v4 ==== 3959+0+0 (unknown 2280859119 0 0) 0x7f4d48043e20 con 0x7f4d58132b40 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172209482 0 0) 0x7f4d38006590 con 0x7f4d38002840 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient(hunting): my global_id is 4605 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d58137050 con 0x7f4d38002840 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d5e0f4700 10 monclient: get_auth_request con 0x7f4d3c037f20 auth_method 0 2024-09-15T03:47:25.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2657589655 0 0) 0x7f4d58137050 con 0x7f4d38002840 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d38006590 con 0x7f4d38002840 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d540022b0 con 0x7f4d38002840 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient: dump: 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3615393359 0 0) 0x7f4d38006590 con 0x7f4d38002840 2024-09-15T03:47:25.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d3800a130 con 0x7f4d38002840 2024-09-15T03:47:25.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4d54002860 con 0x7f4d38002840 2024-09-15T03:47:25.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:25.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:25.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6792/0 conn(0x7f4d38002840 legacy=0x7f4d38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:25.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.962+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6792/0 conn(0x7f4d38002840 legacy=0x7f4d38004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:25.996 INFO:tasks.rados.rados.0.smithi122.stdout:413: done (0 left) 2024-09-15T03:47:25.997 INFO:tasks.rados.rados.0.smithi122.stdout:414: write oid 15 current snap is 48 2024-09-15T03:47:25.997 INFO:tasks.rados.rados.0.smithi122.stdout:414: seq_num 179 ranges {608453=545690,1624865=455728,2632598=375855} 2024-09-15T03:47:25.997 INFO:tasks.rados.rados.0.smithi122.stdout:414: writing smithi12223906-15 from 608453 to 1154143 tid 1 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.970+0000 7f4d5e8f5700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.970+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4d38007520 con 0x7f4d380068d0 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489353034 0 0) 0x7f4d38007520 con 0x7f4d380068d0 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 10 monclient(hunting): my global_id is 4614 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d38006590 con 0x7f4d380068d0 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4039449528 0 0) 0x7f4d38006590 con 0x7f4d380068d0 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d38007520 con 0x7f4d380068d0 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d54005a20 con 0x7f4d380068d0 2024-09-15T03:47:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 10 monclient: dump: 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2747288605 0 0) 0x7f4d38007520 con 0x7f4d380068d0 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.974+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4d38008760 con 0x7f4d380068d0 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 3885418151) 0x7f4d54008330 con 0x7f4d380068d0 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d4dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d4dffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6792/0 conn(0x7f4d380068d0 legacy=0x7f4d38006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d4dffb700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6792/0 conn(0x7f4d380068d0 legacy=0x7f4d38006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:26.002 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:26.003 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.004 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:26.004 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.004 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:26.004 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:26.004 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.005 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.005 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.005 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.005 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.005 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.006 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.006 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.006 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.006 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.006 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:26.006 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:26.007 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.007 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.007 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.007 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.008 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:26.009 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:26.009 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:26.009 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:26.009 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:26.009 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:26.010 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.010 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:26.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:26.010 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:26.010 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:26.011 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:26.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:26.011 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:26.011 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.011 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:26.011 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.012 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.012 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.012 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.012 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.013 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.013 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.013 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.013 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.013 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.014 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.016 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.017 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:26.018 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.018 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.024 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:26.026 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.029 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:26.030 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.122:6812/20895 conn(0x7f4d3c037f20 legacy=0x7f4d3c03a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 10 monclient: shutdown 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e0f4700 1 -- 172.21.15.26:0/1119914172 reap_dead start 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 >> v1:172.21.15.26:6791/0 conn(0x7f4d58132b40 legacy=0x7f4d58135cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 shutdown_connections 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 >> 172.21.15.26:0/1119914172 conn(0x7f4d5806e370 msgr2=0x7f4d580a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 shutdown_connections 2024-09-15T03:47:26.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:25.978+0000 7f4d5e8f5700 1 -- 172.21.15.26:0/1119914172 wait complete. 2024-09-15T03:47:26.033 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:26.033 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:47:26.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:47:26.035 INFO:tasks.rados.rados.0.smithi122.stdout:414: writing smithi12223906-15 from 1624865 to 2080593 tid 2 2024-09-15T03:47:26.035 INFO:tasks.rados.rados.0.smithi122.stdout:414: writing smithi12223906-15 from 2632598 to 3008453 tid 3 2024-09-15T03:47:26.035 INFO:tasks.rados.rados.0.smithi122.stdout:415: read oid 40 snap -1 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:415: expect deleted 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:416: rollback oid 38 current snap is 48 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 47 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:414: finishing write tid 1 to smithi12223906-15 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:414: finishing write tid 2 to smithi12223906-15 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:417: read oid 30 snap -1 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:417: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:418: rollback oid 23 current snap is 48 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 47 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:419: read oid 2 snap -1 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:419: expect (ObjNum 165 snap 44 seq_num 165) 2024-09-15T03:47:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:420: write oid 6 current snap is 48 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:420: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:420: writing smithi12223906-6 from 625260 to 1246199 tid 1 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:420: writing smithi12223906-6 from 1683347 to 2282733 tid 2 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:420: writing smithi12223906-6 from 2943668 to 3025260 tid 3 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:421: read oid 9 snap -1 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:421: expect deleted 2024-09-15T03:47:26.037 INFO:tasks.rados.rados.0.smithi122.stdout:422: copy_from oid 33 from oid 48 current snap is 48 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:414: finishing write tid 3 to smithi12223906-15 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:414: finishing write tid 4 to smithi12223906-15 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:414: finishing write tid 5 to smithi12223906-15 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 123 (ObjNum 179 snap 48 seq_num 179) dirty exists 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:414: left oid 15 (ObjNum 179 snap 48 seq_num 179) 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:414: done (8 left) 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:415: done (7 left) 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:423: delete oid 40 current snap is 48 2024-09-15T03:47:26.038 INFO:tasks.rados.rados.0.smithi122.stdout:419: expect (ObjNum 165 snap 44 seq_num 165) 2024-09-15T03:47:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.142+0000 7f7b4b59e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.142+0000 7f7b4b59e700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.142+0000 7f7b4b59e700 10 monclient: monmap: 2024-09-15T03:47:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:26.147151+0000 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:26.147151+0000 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.148 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 Processor -- start 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- start start 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient: init 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient: _add_conns ranks=[8,3,7,6,4,2,5,1,0] 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient(hunting): picked mon.noname-e con 0x7f7b4c132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient(hunting): picked mon.noname-g con 0x7f7b4c09faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient(hunting): picked mon.noname-i con 0x7f7b4c138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c04ca10 con 0x7f7b4c09faa0 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c09d5d0 con 0x7f7b4c138d40 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c133620 con 0x7f7b4c132b40 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient(hunting): authenticate will time out at 990.576931s 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4a59c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f7b4c132b40 0x7f7b4c132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56932/0 (socket says 172.21.15.26:56932) 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4a59c700 1 -- 172.21.15.26:0/4051905653 learned_addr learned my addr 172.21.15.26:0/4051905653 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513683457 0 0) 0x7f7b4c133620 con 0x7f7b4c132b40 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): my global_id is 134153 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b34003570 con 0x7f7b4c132b40 2024-09-15T03:47:26.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019184392 0 0) 0x7f7b4c09d5d0 con 0x7f7b4c138d40 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): my global_id is 134125 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c133620 con 0x7f7b4c138d40 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089964537 0 0) 0x7f7b4c04ca10 con 0x7f7b4c09faa0 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient(hunting): my global_id is 4611 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c09d5d0 con 0x7f7b4c09faa0 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 333859159 0 0) 0x7f7b4c133620 con 0x7f7b4c138d40 2024-09-15T03:47:26.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 >> v1:172.21.15.26:6793/0 conn(0x7f7b4c132b40 legacy=0x7f7b4c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 >> v1:172.21.15.122:6790/0 conn(0x7f7b4c09faa0 legacy=0x7f7b4c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 monclient: found mon.noname-i 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0a0cc0 con 0x7f7b4c138d40 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b44001b10 con 0x7f7b4c138d40 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: dump: 2024-09-15T03:47:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 5 monclient: authenticate success, global_id 134125 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b44001dc0 con 0x7f7b4c138d40 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 1 -- 172.21.15.26:0/4051905653 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b440022f0 con 0x7f7b4c138d40 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 4 set_mon_vals no callback set 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4959a700 10 monclient: dump: 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 10 monclient: shutdown 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- 172.21.15.26:0/4051905653 >> v1:172.21.15.122:6792/0 conn(0x7f7b4c138d40 legacy=0x7f7b4c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- 172.21.15.26:0/4051905653 shutdown_connections 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- 172.21.15.26:0/4051905653 >> 172.21.15.26:0/4051905653 conn(0x7f7b4c06e370 msgr2=0x7f7b4c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- 172.21.15.26:0/4051905653 shutdown_connections 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.146+0000 7f7b4b59e700 1 -- 172.21.15.26:0/4051905653 wait complete. 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient: monmap: 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 1 Processor -- start 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 1 -- start start 2024-09-15T03:47:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient: init 2024-09-15T03:47:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient: _add_conns ranks=[6,2,8,1,4,0,5,7,3] 2024-09-15T03:47:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient(hunting): picked mon.noname-d con 0x7f7b4c09faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient(hunting): picked mon.noname-b con 0x7f7b4c132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient(hunting): picked mon.noname-e con 0x7f7b4c138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c09cfa0 con 0x7f7b4c132b40 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c09ccc0 con 0x7f7b4c09faa0 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c09be10 con 0x7f7b4c138d40 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4b59e700 10 monclient(hunting): authenticate will time out at 990.581960s 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4ad9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f7b4c138d40 0x7f7b4c14a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56940/0 (socket says 172.21.15.26:56940) 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4a59c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f7b4c09faa0 0x7f7b4c0987f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60502/0 (socket says 172.21.15.26:60502) 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b4ad9d700 1 -- 172.21.15.26:0/750022299 learned_addr learned my addr 172.21.15.26:0/750022299 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b49d9b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f7b4c132b40 0x7f7b4c098f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54458/0 (socket says 172.21.15.26:54458) 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755796 0 0) 0x7f7b4c09be10 con 0x7f7b4c138d40 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): my global_id is 134162 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b2c003570 con 0x7f7b4c138d40 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513635809 0 0) 0x7f7b4c09cfa0 con 0x7f7b4c132b40 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): my global_id is 134165 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c09be10 con 0x7f7b4c132b40 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828273665 0 0) 0x7f7b4c09ccc0 con 0x7f7b4c09faa0 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): my global_id is 4623 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c09cfa0 con 0x7f7b4c09faa0 2024-09-15T03:47:26.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 550491808 0 0) 0x7f7b4c09be10 con 0x7f7b4c132b40 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b4c09ccc0 con 0x7f7b4c132b40 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b40001c50 con 0x7f7b4c132b40 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2379268437 0 0) 0x7f7b2c003570 con 0x7f7b4c138d40 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b4c09be10 con 0x7f7b4c138d40 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b44001ad0 con 0x7f7b4c138d40 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:26.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.150+0000 7f7b3a7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.172 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3904151282 0 0) 0x7f7b4c09cfa0 con 0x7f7b4c09faa0 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b2c003570 con 0x7f7b4c09faa0 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3378446387 0 0) 0x7f7b4c09ccc0 con 0x7f7b4c132b40 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6793/0 conn(0x7f7b4c138d40 legacy=0x7f7b4c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6792/0 conn(0x7f7b4c09faa0 legacy=0x7f7b4c0987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 monclient: found mon.b 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c09a4a0 con 0x7f7b4c132b40 2024-09-15T03:47:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 5 monclient: authenticate success, global_id 134165 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 10 monclient: _renew_subs 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c09bb30 con 0x7f7b4c132b40 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 10 monclient: _renew_subs 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c09abd0 con 0x7f7b4c132b40 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7b4c04bbf0 con 0x7f7b4c099610 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7b400016a0 con 0x7f7b4c132b40 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b400024f0 con 0x7f7b4c132b40 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b48d99700 4 set_mon_vals no callback set 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: dump: 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.175 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507117555 0 0) 0x7f7b4c04bbf0 con 0x7f7b4c099610 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient(hunting): my global_id is 134171 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c09ccc0 con 0x7f7b4c099610 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2577660499 0 0) 0x7f7b4c09ccc0 con 0x7f7b4c099610 2024-09-15T03:47:26.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b4c04bbf0 con 0x7f7b4c099610 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b3c001d50 con 0x7f7b4c099610 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: dump: 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f7b4000fa60 con 0x7f7b4c132b40 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(117..117 src has 1..117) v4 ==== 4003+0+0 (unknown 3408743952 0 0) 0x7f7b40043d70 con 0x7f7b4c132b40 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 704366783 0 0) 0x7f7b4c04bbf0 con 0x7f7b4c099610 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b4c09bfd0 con 0x7f7b4c099610 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b4ad9d700 10 monclient: get_auth_request con 0x7f7b2c038170 auth_method 0 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7b3c005090 con 0x7f7b4c099610 2024-09-15T03:47:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:26.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6793/0 conn(0x7f7b4c099610 legacy=0x7f7b4c130010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.154+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6793/0 conn(0x7f7b4c099610 legacy=0x7f7b4c130010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:26.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7b4c06fac0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810782915 0 0) 0x7f7b4c06fac0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient(hunting): my global_id is 134180 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b4c04bbf0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2242598734 0 0) 0x7f7b4c04bbf0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7b4c06fac0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7b3c005980 con 0x7f7b4c02ad10 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient: dump: 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4112510627 0 0) 0x7f7b4c06fac0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b4c066f50 con 0x7f7b4c02ad10 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1867724401) 0x7f7b3c005bb0 con 0x7f7b4c02ad10 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6793/0 conn(0x7f7b4c02ad10 legacy=0x7f7b4c130720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b3a7fc700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6793/0 conn(0x7f7b4c02ad10 legacy=0x7f7b4c130720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:26.183 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.184 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.185 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.186 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:26.187 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.188 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.189 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.190 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.191 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.194 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:26.197 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.199 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.200 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.200 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.200 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.201 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:26.202 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.202 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.202 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.204 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:26.204 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.205 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.205 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.205 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.205 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.205 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.206 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.206 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.206 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.206 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.206 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:26.207 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:26.207 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.207 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.207 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.207 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.209 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.209 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.209 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:26.209 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.122:6812/20895 conn(0x7f7b2c038170 legacy=0x7f7b2c03a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 10 monclient: shutdown 2024-09-15T03:47:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 >> v1:172.21.15.26:6790/0 conn(0x7f7b4c132b40 legacy=0x7f7b4c098f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4ad9d700 1 -- 172.21.15.26:0/750022299 reap_dead start 2024-09-15T03:47:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4ad9d700 1 -- 172.21.15.26:0/750022299 reap_dead start 2024-09-15T03:47:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 shutdown_connections 2024-09-15T03:47:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 >> 172.21.15.26:0/750022299 conn(0x7f7b4c06e370 msgr2=0x7f7b4c12ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 shutdown_connections 2024-09-15T03:47:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.166+0000 7f7b4b59e700 1 -- 172.21.15.26:0/750022299 wait complete. 2024-09-15T03:47:26.214 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:26.215 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:47:26.215 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:47:26.216 INFO:tasks.rados.rados.0.smithi122.stdout:418: finishing rollback tid 0 to smithi12223906-23 2024-09-15T03:47:26.216 INFO:tasks.rados.rados.0.smithi122.stdout:418: finishing rollback tid 1 to smithi12223906-23 2024-09-15T03:47:26.216 INFO:tasks.rados.rados.0.smithi122.stdout:418: finishing rollback tid 2 to smithi12223906-23 2024-09-15T03:47:26.216 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 208 (ObjNum 161 snap 42 seq_num 161) dirty exists 2024-09-15T03:47:26.217 INFO:tasks.rados.rados.0.smithi122.stdout:416: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:47:26.217 INFO:tasks.rados.rados.0.smithi122.stdout:416: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:47:26.217 INFO:tasks.rados.rados.0.smithi122.stdout:416: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:47:26.217 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 103 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:47:26.217 INFO:tasks.rados.rados.0.smithi122.stdout:417: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:420: finishing write tid 1 to smithi12223906-6 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:420: finishing write tid 2 to smithi12223906-6 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:420: finishing write tid 3 to smithi12223906-6 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:420: finishing write tid 4 to smithi12223906-6 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:420: finishing write tid 5 to smithi12223906-6 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 50 (ObjNum 180 snap 48 seq_num 180) dirty exists 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:420: left oid 6 (ObjNum 180 snap 48 seq_num 180) 2024-09-15T03:47:26.253 INFO:tasks.rados.rados.0.smithi122.stdout:422: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:422: finishing copy_from to smithi12223906-33 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 119 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:416: done (7 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:417: done (6 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:418: done (5 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:419: done (4 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:420: done (3 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:421: done (2 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:422: done (1 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:423: done (0 left) 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:424: write oid 8 current snap is 48 2024-09-15T03:47:26.254 INFO:tasks.rados.rados.0.smithi122.stdout:424: seq_num 182 ranges {658874=771437,2083958=603055} 2024-09-15T03:47:26.270 INFO:tasks.rados.rados.0.smithi122.stdout:424: writing smithi12223906-8 from 658874 to 1430311 tid 1 2024-09-15T03:47:26.277 INFO:tasks.rados.rados.0.smithi122.stdout:424: writing smithi12223906-8 from 2083958 to 2687013 tid 2 2024-09-15T03:47:26.277 INFO:tasks.rados.rados.0.smithi122.stdout:425: snap_remove snap 47 2024-09-15T03:47:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.330+0000 7f27d375e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.330+0000 7f27d375e700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.330+0000 7f27d375e700 10 monclient: monmap: 2024-09-15T03:47:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:26.335663+0000 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:26.335663+0000 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.338 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 Processor -- start 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- start start 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient: init 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient: _add_conns ranks=[0,7,6,4,3,1,8,5,2] 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient(hunting): picked mon.noname-a con 0x7f27cc09e440 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient(hunting): picked mon.noname-i con 0x7f27cc12ffc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient(hunting): picked mon.noname-d con 0x7f27cc130390 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc04ca10 con 0x7f27cc09e440 2024-09-15T03:47:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc09a0f0 con 0x7f27cc130390 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc09e810 con 0x7f27cc12ffc0 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient(hunting): authenticate will time out at 990.765294s 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d14fa700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f27cc09e440 0x7f27cc12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36152/0 (socket says 172.21.15.26:36152) 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d14fa700 1 -- 172.21.15.26:0/2148617580 learned_addr learned my addr 172.21.15.26:0/2148617580 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802181831 0 0) 0x7f27cc09a0f0 con 0x7f27cc130390 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): my global_id is 4632 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8003570 con 0x7f27cc130390 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534032933 0 0) 0x7f27cc04ca10 con 0x7f27cc09e440 2024-09-15T03:47:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): my global_id is 4545 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc09a0f0 con 0x7f27cc09e440 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732149322 0 0) 0x7f27cc09e810 con 0x7f27cc12ffc0 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient(hunting): my global_id is 134134 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc04ca10 con 0x7f27cc12ffc0 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2813107690 0 0) 0x7f27b8003570 con 0x7f27cc130390 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 >> v1:172.21.15.122:6792/0 conn(0x7f27cc12ffc0 legacy=0x7f27cc134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 >> v1:172.21.15.26:6789/0 conn(0x7f27cc09e440 legacy=0x7f27cc12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 monclient: found mon.noname-d 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc09c2a0 con 0x7f27cc130390 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27c4001b30 con 0x7f27cc130390 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: dump: 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 5 monclient: authenticate success, global_id 4632 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27c4001e10 con 0x7f27cc130390 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 1 -- 172.21.15.26:0/2148617580 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27c40025c0 con 0x7f27cc130390 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 4 set_mon_vals no callback set 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27c3fff700 10 monclient: dump: 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 10 monclient: shutdown 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- 172.21.15.26:0/2148617580 >> v1:172.21.15.26:6792/0 conn(0x7f27cc130390 legacy=0x7f27cc138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- 172.21.15.26:0/2148617580 shutdown_connections 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- 172.21.15.26:0/2148617580 >> 172.21.15.26:0/2148617580 conn(0x7f27cc063f30 msgr2=0x7f27cc0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- 172.21.15.26:0/2148617580 shutdown_connections 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.334+0000 7f27d375e700 1 -- 172.21.15.26:0/2148617580 wait complete. 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: monmap: 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 Processor -- start 2024-09-15T03:47:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- start start 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: init 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: _add_conns ranks=[2,7,8,4,6,0,3,1,5] 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient(hunting): picked mon.noname-b con 0x7f27cc09e440 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient(hunting): picked mon.noname-i con 0x7f27cc12ffc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient(hunting): picked mon.noname-e con 0x7f27cc130390 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc11f420 con 0x7f27cc09e440 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc1195e0 con 0x7f27cc12ffc0 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc119a90 con 0x7f27cc130390 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient(hunting): authenticate will time out at 990.769111s 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d14fa700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f27cc09e440 0x7f27cc12ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:54470/0 (socket says 172.21.15.26:54470) 2024-09-15T03:47:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d14fa700 1 -- 172.21.15.26:0/3313560007 learned_addr learned my addr 172.21.15.26:0/3313560007 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518540388 0 0) 0x7f27cc11f420 con 0x7f27cc09e440 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): my global_id is 134174 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b0003570 con 0x7f27cc09e440 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321244392 0 0) 0x7f27cc119a90 con 0x7f27cc130390 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): my global_id is 134189 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc11f420 con 0x7f27cc130390 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574405269 0 0) 0x7f27cc1195e0 con 0x7f27cc12ffc0 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): my global_id is 134143 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc119a90 con 0x7f27cc12ffc0 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4204311999 0 0) 0x7f27cc11f420 con 0x7f27cc130390 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27cc1195e0 con 0x7f27cc130390 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27c4001be0 con 0x7f27cc130390 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): dump: 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3440943237 0 0) 0x7f27b0003570 con 0x7f27cc09e440 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27cc11f420 con 0x7f27cc09e440 2024-09-15T03:47:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 925593713 0 0) 0x7f27cc119a90 con 0x7f27cc12ffc0 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27b0003570 con 0x7f27cc12ffc0 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 276582938 0 0) 0x7f27cc1195e0 con 0x7f27cc130390 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.122:6792/0 conn(0x7f27cc12ffc0 legacy=0x7f27cc147200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.26:6790/0 conn(0x7f27cc09e440 legacy=0x7f27cc12ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 monclient: found mon.e 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc11ee60 con 0x7f27cc130390 2024-09-15T03:47:26.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 5 monclient: authenticate success, global_id 134189 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: _renew_subs 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc11f140 con 0x7f27cc130390 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: _renew_subs 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc12f5c0 con 0x7f27cc130390 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27c4001580 con 0x7f27cc130390 2024-09-15T03:47:26.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27c40028b0 con 0x7f27cc130390 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 10 monclient: dump: 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.362 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c37fe700 4 set_mon_vals no callback set 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f27c4002a90 con 0x7f27cc130390 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f27ac006590 con 0x7f27ac002840 2024-09-15T03:47:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.338+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(117..117 src has 1..117) v4 ==== 4003+0+0 (unknown 3408743952 0 0) 0x7f27c4041880 con 0x7f27cc130390 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27d0cf9700 10 monclient: get_auth_request con 0x7f27b0037c30 auth_method 0 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935609318 0 0) 0x7f27ac006590 con 0x7f27ac002840 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient(hunting): my global_id is 134101 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc1195e0 con 0x7f27ac002840 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2858159433 0 0) 0x7f27cc1195e0 con 0x7f27ac002840 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27ac006590 con 0x7f27ac002840 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27c8004bf0 con 0x7f27ac002840 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient: dump: 2024-09-15T03:47:26.364 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.365 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2265856577 0 0) 0x7f27ac006590 con 0x7f27ac002840 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27ac00a130 con 0x7f27ac002840 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f27c8005920 con 0x7f27ac002840 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.122:6789/0 conn(0x7f27ac002840 legacy=0x7f27ac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.342+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.122:6789/0 conn(0x7f27ac002840 legacy=0x7f27ac004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27d375e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f27ac007520 con 0x7f27ac0068d0 2024-09-15T03:47:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420638511 0 0) 0x7f27ac007520 con 0x7f27ac0068d0 2024-09-15T03:47:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 10 monclient(hunting): my global_id is 134110 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac006590 con 0x7f27ac0068d0 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4000803544 0 0) 0x7f27ac006590 con 0x7f27ac0068d0 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27ac007520 con 0x7f27ac0068d0 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27c8005730 con 0x7f27ac0068d0 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.350+0000 7f27c17fa700 10 monclient: dump: 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.369 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3215939793 0 0) 0x7f27ac007520 con 0x7f27ac0068d0 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f27ac008760 con 0x7f27ac0068d0 2024-09-15T03:47:26.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1420208710) 0x7f27c8005e40 con 0x7f27ac0068d0 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.122:6789/0 conn(0x7f27ac0068d0 legacy=0x7f27ac006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27c17fa700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.122:6789/0 conn(0x7f27ac0068d0 legacy=0x7f27ac006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:26.371 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.372 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.373 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.374 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:26.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.377 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.379 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.381 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.382 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.382 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.382 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:26.383 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.384 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.384 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:26.386 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.386 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.387 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.388 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.388 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.388 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.391 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.393 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.396 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.397 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.397 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.397 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:26.397 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.122:6812/20895 conn(0x7f27b0037c30 legacy=0x7f27b003a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 10 monclient: shutdown 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 >> v1:172.21.15.26:6793/0 conn(0x7f27cc130390 legacy=0x7f27cc14a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d1cfb700 1 -- 172.21.15.26:0/3313560007 reap_dead start 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d1cfb700 1 -- 172.21.15.26:0/3313560007 reap_dead start 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 shutdown_connections 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 >> 172.21.15.26:0/3313560007 conn(0x7f27cc063f30 msgr2=0x7f27cc09abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 shutdown_connections 2024-09-15T03:47:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.354+0000 7f27d375e700 1 -- 172.21.15.26:0/3313560007 wait complete. 2024-09-15T03:47:26.400 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:26.400 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:47:26.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:47:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient: get_monmap_and_config 2024-09-15T03:47:26.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient: monmap: 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:26.517317+0000 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:26.517317+0000 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.518 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.519 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.519 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.519 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 1 Processor -- start 2024-09-15T03:47:26.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 1 -- start start 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient: init 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient: _add_conns ranks=[7,5,8,6,2,3,4,1,0] 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient(hunting): picked mon.noname-i con 0x7f9f00132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient(hunting): picked mon.noname-h con 0x7f9f0009faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient(hunting): picked mon.noname-e con 0x7f9f00138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0004ca10 con 0x7f9f0009faa0 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0009d5d0 con 0x7f9f00132b40 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f00133620 con 0x7f9f00138d40 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9f049d5700 10 monclient(hunting): authenticate will time out at 990.947578s 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9eff59e700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f9f00138d40 0x7f9f0013b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56990/0 (socket says 172.21.15.26:56990) 2024-09-15T03:47:26.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.514+0000 7f9eff59e700 1 -- 172.21.15.26:0/3582737605 learned_addr learned my addr 172.21.15.26:0/3582737605 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995230141 0 0) 0x7f9f00133620 con 0x7f9f00138d40 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): my global_id is 134198 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eec003570 con 0x7f9f00138d40 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893354960 0 0) 0x7f9f0004ca10 con 0x7f9f0009faa0 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): my global_id is 4685 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f00133620 con 0x7f9f0009faa0 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424812383 0 0) 0x7f9f0009d5d0 con 0x7f9f00132b40 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient(hunting): my global_id is 134152 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0004ca10 con 0x7f9f00132b40 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517711487 0 0) 0x7f9eec003570 con 0x7f9f00138d40 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 >> v1:172.21.15.122:6792/0 conn(0x7f9f00132b40 legacy=0x7f9f00132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 >> v1:172.21.15.122:6791/0 conn(0x7f9f0009faa0 legacy=0x7f9f0009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 monclient: found mon.noname-e 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000a0cc0 con 0x7f9f00138d40 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 5 monclient: authenticate success, global_id 134198 2024-09-15T03:47:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ef8001b30 con 0x7f9f00138d40 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: dump: 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.525 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9ef8001da0 con 0x7f9f00138d40 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 4 set_mon_vals no callback set 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 1 -- 172.21.15.26:0/3582737605 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ef8004fb0 con 0x7f9f00138d40 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9efe59c700 10 monclient: dump: 2024-09-15T03:47:26.526 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.527 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 10 monclient: shutdown 2024-09-15T03:47:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 1 -- 172.21.15.26:0/3582737605 >> v1:172.21.15.26:6793/0 conn(0x7f9f00138d40 legacy=0x7f9f0013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 1 -- 172.21.15.26:0/3582737605 shutdown_connections 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 1 -- 172.21.15.26:0/3582737605 >> 172.21.15.26:0/3582737605 conn(0x7f9f0006e370 msgr2=0x7f9f000a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 1 -- 172.21.15.26:0/3582737605 shutdown_connections 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 1 -- 172.21.15.26:0/3582737605 wait complete. 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.518+0000 7f9f049d5700 10 monclient: monmap: 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:26.533 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 Processor -- start 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 -- start start 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: init 2024-09-15T03:47:26.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: _add_conns ranks=[7,6,0,3,5,4,1,8,2] 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient(hunting): picked mon.noname-i con 0x7f9f0009faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient(hunting): picked mon.noname-d con 0x7f9f00132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient(hunting): picked mon.noname-a con 0x7f9f00138d40 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0009cfa0 con 0x7f9f00138d40 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0009ccc0 con 0x7f9f00132b40 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f00137620 con 0x7f9f0009faa0 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient(hunting): authenticate will time out at 990.953406s 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9eff59e700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9f00138d40 0x7f9f0014a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36154/0 (socket says 172.21.15.26:36154) 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9eff59e700 1 -- 172.21.15.26:0/642146628 learned_addr learned my addr 172.21.15.26:0/642146628 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977021978 0 0) 0x7f9f0009cfa0 con 0x7f9f00138d40 2024-09-15T03:47:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): my global_id is 4554 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee0003570 con 0x7f9f00138d40 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466984469 0 0) 0x7f9f0009ccc0 con 0x7f9f00132b40 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): my global_id is 4641 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0009cfa0 con 0x7f9f00132b40 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607574144 0 0) 0x7f9f00137620 con 0x7f9f0009faa0 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): my global_id is 134161 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0009ccc0 con 0x7f9f0009faa0 2024-09-15T03:47:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 760882735 0 0) 0x7f9f0009cfa0 con 0x7f9f00132b40 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f00137620 con 0x7f9f00132b40 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1076631920 0 0) 0x7f9f0009ccc0 con 0x7f9f0009faa0 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f0009cfa0 con 0x7f9f0009faa0 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ee8001a60 con 0x7f9f00132b40 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.537 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 439436631 0 0) 0x7f9ee0003570 con 0x7f9f00138d40 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f0009ccc0 con 0x7f9f00138d40 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ef0001a60 con 0x7f9f0009faa0 2024-09-15T03:47:26.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.539 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ef8001af0 con 0x7f9f00138d40 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:26.540 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 389813471 0 0) 0x7f9f00137620 con 0x7f9f00132b40 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.122:6792/0 conn(0x7f9f0009faa0 legacy=0x7f9f00135960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.26:6789/0 conn(0x7f9f00138d40 legacy=0x7f9f0014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 monclient: found mon.d 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f00137110 con 0x7f9f00132b40 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 5 monclient: authenticate success, global_id 4641 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: _renew_subs 2024-09-15T03:47:26.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0011c870 con 0x7f9f00132b40 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: _renew_subs 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f00137ce0 con 0x7f9f00132b40 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9ee8001f70 con 0x7f9f00132b40 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ee80024a0 con 0x7f9f00132b40 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9ef6ffd700 10 monclient: dump: 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.543 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.522+0000 7f9efdd9b700 4 set_mon_vals no callback set 2024-09-15T03:47:26.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9f049d5700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9ee8002680 con 0x7f9f00132b40 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9ee4006590 con 0x7f9ee4002840 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(117..117 src has 1..117) v4 ==== 4003+0+0 (unknown 3408743952 0 0) 0x7f9ee8041600 con 0x7f9f00132b40 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9eff59e700 10 monclient: get_auth_request con 0x7f9ee00380c0 auth_method 0 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684022427 0 0) 0x7f9ee4006590 con 0x7f9ee4002840 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 10 monclient(hunting): my global_id is 4620 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f00137620 con 0x7f9ee4002840 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1504436786 0 0) 0x7f9f00137620 con 0x7f9ee4002840 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ee4006590 con 0x7f9ee4002840 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ef0001f00 con 0x7f9ee4002840 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 10 monclient: dump: 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.546 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2519666330 0 0) 0x7f9ee4006590 con 0x7f9ee4002840 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.526+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ee400a130 con 0x7f9ee4002840 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.530+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9ef00023e0 con 0x7f9ee4002840 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.530+0000 7f9ef6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.530+0000 7f9ef6ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.530+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.122:6790/0 conn(0x7f9ee4002840 legacy=0x7f9ee4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.530+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.122:6790/0 conn(0x7f9ee4002840 legacy=0x7f9ee4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9f049d5700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:26.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9ee4009640 con 0x7f9ee40090b0 2024-09-15T03:47:26.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512298538 0 0) 0x7f9ee4009640 con 0x7f9ee40090b0 2024-09-15T03:47:26.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 10 monclient(hunting): my global_id is 4629 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee4006590 con 0x7f9ee40090b0 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2057254160 0 0) 0x7f9ee4006590 con 0x7f9ee40090b0 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ee4009640 con 0x7f9ee40090b0 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9ef0003000 con 0x7f9ee40090b0 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 10 monclient: dump: 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.550 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.551 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2935830145 0 0) 0x7f9ee4009640 con 0x7f9ee40090b0 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.542+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9ee40098f0 con 0x7f9ee40090b0 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 847130948) 0x7f9ef00052d0 con 0x7f9ee40090b0 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9ef6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9ef6ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.122:6790/0 conn(0x7f9ee40090b0 legacy=0x7f9ee4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9ef6ffd700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.122:6790/0 conn(0x7f9ee40090b0 legacy=0x7f9ee4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:26.553 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:26.554 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.555 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:26.556 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.557 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:26.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.559 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:26.563 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:26.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:26.566 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.571 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.572 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 6 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:26.573 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:26.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.122:6812/20895 conn(0x7f9ee00380c0 legacy=0x7f9ee003a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 10 monclient: shutdown 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 >> v1:172.21.15.26:6792/0 conn(0x7f9f00132b40 legacy=0x7f9f00136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9eff59e700 1 -- 172.21.15.26:0/642146628 reap_dead start 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 shutdown_connections 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 >> 172.21.15.26:0/642146628 conn(0x7f9f0006e370 msgr2=0x7f9f0009da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 shutdown_connections 2024-09-15T03:47:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.546+0000 7f9f049d5700 1 -- 172.21.15.26:0/642146628 wait complete. 2024-09-15T03:47:26.575 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:26.575 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:47:26.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.686+0000 7f3e88409700 10 monclient: get_monmap_and_config 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.686+0000 7f3e88409700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.686+0000 7f3e88409700 10 monclient: monmap: 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:26.690048+0000 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:26.690048+0000 2024-09-15T03:47:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.686+0000 7f3e88409700 1 Processor -- start 2024-09-15T03:47:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.686+0000 7f3e88409700 1 -- start start 2024-09-15T03:47:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.686+0000 7f3e88409700 10 monclient: init 2024-09-15T03:47:26.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient: _add_conns ranks=[6,3,4,7,0,5,2,8,1] 2024-09-15T03:47:26.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient(hunting): picked mon.noname-d con 0x7f3e800a3860 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient(hunting): picked mon.noname-g con 0x7f3e8009e0b0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient(hunting): picked mon.noname-c con 0x7f3e8009e480 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8004ca10 con 0x7f3e8009e0b0 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e80098c50 con 0x7f3e8009e480 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e80098970 con 0x7f3e800a3860 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient(hunting): authenticate will time out at 991.121121s 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e861a5700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f3e800a3860 0x7f3e8009d9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41388/0 (socket says 172.21.15.26:41388) 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e861a5700 1 -- 172.21.15.26:0/1154129083 learned_addr learned my addr 172.21.15.26:0/1154129083 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e869a6700 1 --1- 172.21.15.26:0/1154129083 >> v1:172.21.15.26:6791/0 conn(0x7f3e8009e480 0x7f3e801382d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42414/0 (socket says 172.21.15.26:42414) 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596675729 0 0) 0x7f3e80098c50 con 0x7f3e8009e480 2024-09-15T03:47:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): my global_id is 4567 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e6c003570 con 0x7f3e8009e480 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273200312 0 0) 0x7f3e80098970 con 0x7f3e800a3860 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): my global_id is 4650 2024-09-15T03:47:26.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e80098c50 con 0x7f3e800a3860 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626938573 0 0) 0x7f3e8004ca10 con 0x7f3e8009e0b0 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient(hunting): my global_id is 4638 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e80098970 con 0x7f3e8009e0b0 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2502067988 0 0) 0x7f3e6c003570 con 0x7f3e8009e480 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 >> v1:172.21.15.26:6792/0 conn(0x7f3e800a3860 legacy=0x7f3e8009d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 >> v1:172.21.15.122:6790/0 conn(0x7f3e8009e0b0 legacy=0x7f3e80134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 monclient: found mon.noname-c 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80098f30 con 0x7f3e8009e480 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 5 monclient: authenticate success, global_id 4567 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e78001b30 con 0x7f3e8009e480 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: dump: 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.702 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3e78001d20 con 0x7f3e8009e480 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 4 set_mon_vals no callback set 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 1 -- 172.21.15.26:0/1154129083 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e78004bb0 con 0x7f3e8009e480 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e851a3700 10 monclient: dump: 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.704 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 10 monclient: shutdown 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- 172.21.15.26:0/1154129083 >> v1:172.21.15.26:6791/0 conn(0x7f3e8009e480 legacy=0x7f3e801382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- 172.21.15.26:0/1154129083 shutdown_connections 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- 172.21.15.26:0/1154129083 >> 172.21.15.26:0/1154129083 conn(0x7f3e80071350 msgr2=0x7f3e800a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- 172.21.15.26:0/1154129083 shutdown_connections 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.690+0000 7f3e88409700 1 -- 172.21.15.26:0/1154129083 wait complete. 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: monmap: 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 Processor -- start 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 -- start start 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: init 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: _add_conns ranks=[3,6,2,4,7,5,0,1,8] 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient(hunting): picked mon.noname-g con 0x7f3e8009e0b0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient(hunting): picked mon.noname-d con 0x7f3e8009e480 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient(hunting): picked mon.noname-b con 0x7f3e800a3860 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e80122ff0 con 0x7f3e800a3860 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8011c8d0 con 0x7f3e8009e0b0 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e80122d10 con 0x7f3e8009e480 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient(hunting): authenticate will time out at 991.126130s 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e869a6700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f3e800a3860 0x7f3e8014a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36320/0 (socket says 172.21.15.26:36320) 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e869a6700 1 -- 172.21.15.26:0/2370097394 learned_addr learned my addr 172.21.15.26:0/2370097394 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962058887 0 0) 0x7f3e80122ff0 con 0x7f3e800a3860 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): my global_id is 134183 2024-09-15T03:47:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e64003570 con 0x7f3e800a3860 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237431687 0 0) 0x7f3e80122d10 con 0x7f3e8009e480 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): my global_id is 4659 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e80122ff0 con 0x7f3e8009e480 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552987467 0 0) 0x7f3e8011c8d0 con 0x7f3e8009e0b0 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): my global_id is 4647 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e80122d10 con 0x7f3e8009e0b0 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1461027276 0 0) 0x7f3e64003570 con 0x7f3e800a3860 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e8011c8d0 con 0x7f3e800a3860 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e78001af0 con 0x7f3e800a3860 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): dump: 2024-09-15T03:47:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1074211865 0 0) 0x7f3e80122ff0 con 0x7f3e8009e480 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e64003570 con 0x7f3e8009e480 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e70001b70 con 0x7f3e8009e480 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient(hunting): dump: 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.713 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 929105748 0 0) 0x7f3e8011c8d0 con 0x7f3e800a3860 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.26:6792/0 conn(0x7f3e8009e480 legacy=0x7f3e80147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.122:6790/0 conn(0x7f3e8009e0b0 legacy=0x7f3e8011cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 monclient: found mon.b 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80122930 con 0x7f3e800a3860 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e767fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 5 monclient: authenticate success, global_id 134183 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: _renew_subs 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e80063e10 con 0x7f3e800a3860 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: _renew_subs 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:26.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.694+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e801332d0 con 0x7f3e800a3860 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3e780051f0 con 0x7f3e800a3860 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e78005720 con 0x7f3e800a3860 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e849a2700 4 set_mon_vals no callback set 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: dump: 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.716 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.717 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e88409700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:47:26.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3e78005900 con 0x7f3e800a3860 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(117..117 src has 1..117) v4 ==== 4003+0+0 (unknown 3408743952 0 0) 0x7f3e780445c0 con 0x7f3e800a3860 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e859a4700 10 monclient: get_auth_request con 0x7f3e64037cf0 auth_method 0 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114005418 0 0) 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient(hunting): my global_id is 4694 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8011c8d0 con 0x7f3e68002840 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4167926179 0 0) 0x7f3e8011c8d0 con 0x7f3e68002840 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e7c001cb0 con 0x7f3e68002840 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:26.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 10 monclient: dump: 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.721 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4215687769 0 0) 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.698+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e6800a130 con 0x7f3e68002840 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.702+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3e7c006980 con 0x7f3e68002840 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.702+0000 7f3e767fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.702+0000 7f3e767fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.702+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.122:6791/0 conn(0x7f3e68002840 legacy=0x7f3e68004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.702+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.122:6791/0 conn(0x7f3e68002840 legacy=0x7f3e68004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.710+0000 7f3e88409700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:26.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.714+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3e68009640 con 0x7f3e680090b0 2024-09-15T03:47:26.723 INFO:tasks.rados.rados.0.smithi122.stdout:424: finishing write tid 1 to smithi12223906-8 2024-09-15T03:47:26.723 INFO:tasks.rados.rados.0.smithi122.stdout:424: finishing write tid 2 to smithi12223906-8 2024-09-15T03:47:26.723 INFO:tasks.rados.rados.0.smithi122.stdout:424: finishing write tid 3 to smithi12223906-8 2024-09-15T03:47:26.723 INFO:tasks.rados.rados.0.smithi122.stdout:424: finishing write tid 4 to smithi12223906-8 2024-09-15T03:47:26.723 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 106 (ObjNum 182 snap 48 seq_num 182) dirty exists 2024-09-15T03:47:26.723 INFO:tasks.rados.rados.0.smithi122.stdout:424: left oid 8 (ObjNum 182 snap 48 seq_num 182) 2024-09-15T03:47:26.724 INFO:tasks.rados.rados.0.smithi122.stdout:424: done (1 left) 2024-09-15T03:47:26.724 INFO:tasks.rados.rados.0.smithi122.stdout:425: done (0 left) 2024-09-15T03:47:26.724 INFO:tasks.rados.rados.0.smithi122.stdout:426: delete oid 31 current snap is 48 2024-09-15T03:47:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295624192 0 0) 0x7f3e68009640 con 0x7f3e680090b0 2024-09-15T03:47:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 10 monclient(hunting): my global_id is 4703 2024-09-15T03:47:26.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e68006590 con 0x7f3e680090b0 2024-09-15T03:47:26.738 INFO:tasks.rados.rados.0.smithi122.stdout:426: done (0 left) 2024-09-15T03:47:26.738 INFO:tasks.rados.rados.0.smithi122.stdout:427: read oid 26 snap -1 2024-09-15T03:47:26.738 INFO:tasks.rados.rados.0.smithi122.stdout:427: expect (ObjNum 120 snap 25 seq_num 120) 2024-09-15T03:47:26.738 INFO:tasks.rados.rados.0.smithi122.stdout:428: copy_from oid 17 from oid 10 current snap is 48 2024-09-15T03:47:26.738 INFO:tasks.rados.rados.0.smithi122.stdout:429: delete oid 32 current snap is 48 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1409774015 0 0) 0x7f3e68006590 con 0x7f3e680090b0 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e68009640 con 0x7f3e680090b0 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e7c0052b0 con 0x7f3e680090b0 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 10 monclient: dump: 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1838273022 0 0) 0x7f3e68009640 con 0x7f3e680090b0 2024-09-15T03:47:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.734+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3e680098f0 con 0x7f3e680090b0 2024-09-15T03:47:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1633589492) 0x7f3e7c006af0 con 0x7f3e680090b0 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e767fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e767fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.122:6791/0 conn(0x7f3e680090b0 legacy=0x7f3e68008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e767fc700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.122:6791/0 conn(0x7f3e680090b0 legacy=0x7f3e68008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.742 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.744 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.745 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:26.746 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.747 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.751 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.752 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.759 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:26.761 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.122:6812/20895 conn(0x7f3e64037cf0 legacy=0x7f3e6403a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e869a6700 1 -- 172.21.15.26:0/2370097394 reap_dead start 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 10 monclient: shutdown 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 >> v1:172.21.15.26:6790/0 conn(0x7f3e800a3860 legacy=0x7f3e8014a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e869a6700 1 -- 172.21.15.26:0/2370097394 reap_dead start 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 shutdown_connections 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 >> 172.21.15.26:0/2370097394 conn(0x7f3e80071350 msgr2=0x7f3e8009b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 shutdown_connections 2024-09-15T03:47:26.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.738+0000 7f3e88409700 1 -- 172.21.15.26:0/2370097394 wait complete. 2024-09-15T03:47:26.764 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:26.764 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:47:26.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:47:26.765 INFO:tasks.rados.rados.0.smithi122.stdout:429: done (2 left) 2024-09-15T03:47:26.765 INFO:tasks.rados.rados.0.smithi122.stdout:430: read oid 2 snap -1 2024-09-15T03:47:26.765 INFO:tasks.rados.rados.0.smithi122.stdout:430: expect (ObjNum 165 snap 44 seq_num 165) 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:431: copy_from oid 36 from oid 4 current snap is 48 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:432: write (excl) oid 49 current snap is 48 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:432: seq_num 185 ranges {709295=713537,1859404=750382} 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:432: writing smithi12223906-49 from 709295 to 1422832 tid 1 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:432: writing smithi12223906-49 from 1859404 to 2609786 tid 2 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:433: write (excl) oid 8 current snap is 48 2024-09-15T03:47:26.766 INFO:tasks.rados.rados.0.smithi122.stdout:433: seq_num 186 ranges {726102=788786,1917886=610393,3108206=17896} 2024-09-15T03:47:26.768 INFO:tasks.rados.rados.0.smithi122.stdout:433: writing smithi12223906-8 from 726102 to 1514888 tid 1 2024-09-15T03:47:26.774 INFO:tasks.rados.rados.0.smithi122.stdout:433: writing smithi12223906-8 from 1917886 to 2528279 tid 2 2024-09-15T03:47:26.777 INFO:tasks.rados.rados.0.smithi122.stdout:433: writing smithi12223906-8 from 3108206 to 3126102 tid 3 2024-09-15T03:47:26.778 INFO:tasks.rados.rados.0.smithi122.stdout:434: copy_from oid 13 from oid 43 current snap is 48 2024-09-15T03:47:26.778 INFO:tasks.rados.rados.0.smithi122.stdout:427: expect (ObjNum 120 snap 25 seq_num 120) 2024-09-15T03:47:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:431: finishing copy_from to smithi12223906-36 2024-09-15T03:47:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:431: got expected ENOENT (src dne) 2024-09-15T03:47:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:431: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:47:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:428: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:47:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:430: expect (ObjNum 165 snap 44 seq_num 165) 2024-09-15T03:47:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.870+0000 7fdd8930e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.870+0000 7fdd8930e700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.870+0000 7fdd8930e700 10 monclient: monmap: 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:26.874624+0000 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:26.874624+0000 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.876 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.870+0000 7fdd8930e700 1 Processor -- start 2024-09-15T03:47:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.870+0000 7fdd8930e700 1 -- start start 2024-09-15T03:47:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.870+0000 7fdd8930e700 10 monclient: init 2024-09-15T03:47:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient: _add_conns ranks=[3,6,7,8,4,5,0,1,2] 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient(hunting): picked mon.noname-g con 0x7fdd8406e530 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient(hunting): picked mon.noname-d con 0x7fdd84099240 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient(hunting): picked mon.noname-i con 0x7fdd84138d70 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8404ca10 con 0x7fdd8406e530 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8409e310 con 0x7fdd84099240 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd84133730 con 0x7fdd84138d70 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient(hunting): authenticate will time out at 991.304188s 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd7b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fdd84099240 0x7fdd84099610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41412/0 (socket says 172.21.15.26:41412) 2024-09-15T03:47:26.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd7b7fe700 1 -- 172.21.15.26:0/3824570518 learned_addr learned my addr 172.21.15.26:0/3824570518 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370752519 0 0) 0x7fdd8409e310 con 0x7fdd84099240 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): my global_id is 4668 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd70003570 con 0x7fdd84099240 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046537712 0 0) 0x7fdd8404ca10 con 0x7fdd8406e530 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): my global_id is 4656 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8409e310 con 0x7fdd8406e530 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608561145 0 0) 0x7fdd84133730 con 0x7fdd84138d70 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient(hunting): my global_id is 134170 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8404ca10 con 0x7fdd84138d70 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751356564 0 0) 0x7fdd70003570 con 0x7fdd84099240 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 >> v1:172.21.15.122:6792/0 conn(0x7fdd84138d70 legacy=0x7fdd8413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 >> v1:172.21.15.122:6790/0 conn(0x7fdd8406e530 legacy=0x7fdd84133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 monclient: found mon.noname-d 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84097b00 con 0x7fdd84099240 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 5 monclient: authenticate success, global_id 4668 2024-09-15T03:47:26.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd6c0017b0 con 0x7fdd84099240 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: dump: 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.883 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdd6c001a90 con 0x7fdd84099240 2024-09-15T03:47:26.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 1 -- 172.21.15.26:0/3824570518 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd6c004c30 con 0x7fdd84099240 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 4 set_mon_vals no callback set 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd837fe700 10 monclient: dump: 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient: shutdown 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- 172.21.15.26:0/3824570518 >> v1:172.21.15.26:6792/0 conn(0x7fdd84099240 legacy=0x7fdd84099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- 172.21.15.26:0/3824570518 shutdown_connections 2024-09-15T03:47:26.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- 172.21.15.26:0/3824570518 >> 172.21.15.26:0/3824570518 conn(0x7fdd84070de0 msgr2=0x7fdd840711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- 172.21.15.26:0/3824570518 shutdown_connections 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 -- 172.21.15.26:0/3824570518 wait complete. 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient: build_initial_monmap 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 10 monclient: monmap: 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:26.887 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.874+0000 7fdd8930e700 1 Processor -- start 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- start start 2024-09-15T03:47:26.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: init 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: _add_conns ranks=[1,6,0,4,5,7,2,3,8] 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient(hunting): picked mon.noname-f con 0x7fdd8406e530 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient(hunting): picked mon.noname-d con 0x7fdd84099240 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient(hunting): picked mon.noname-a con 0x7fdd84138d70 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8409e790 con 0x7fdd84138d70 2024-09-15T03:47:26.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd84137a30 con 0x7fdd8406e530 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8412fd80 con 0x7fdd84099240 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient(hunting): authenticate will time out at 991.308889s 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd7b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fdd84099240 0x7fdd8412f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41418/0 (socket says 172.21.15.26:41418) 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd7b7fe700 1 -- 172.21.15.26:0/2575598660 learned_addr learned my addr 172.21.15.26:0/2575598660 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605059770 0 0) 0x7fdd8409e790 con 0x7fdd84138d70 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): my global_id is 4563 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd64003570 con 0x7fdd84138d70 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328291892 0 0) 0x7fdd8412fd80 con 0x7fdd84099240 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): my global_id is 4677 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8409e790 con 0x7fdd84099240 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114574512 0 0) 0x7fdd84137a30 con 0x7fdd8406e530 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): my global_id is 134119 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8412fd80 con 0x7fdd8406e530 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3439834838 0 0) 0x7fdd8409e790 con 0x7fdd84099240 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd84137a30 con 0x7fdd84099240 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd6c004b70 con 0x7fdd84099240 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:26.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.896 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2727540176 0 0) 0x7fdd64003570 con 0x7fdd84138d70 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd8409e790 con 0x7fdd84138d70 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd7c001cc0 con 0x7fdd84138d70 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.897 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3214560192 0 0) 0x7fdd8412fd80 con 0x7fdd8406e530 2024-09-15T03:47:26.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd64003570 con 0x7fdd8406e530 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd74001a60 con 0x7fdd8406e530 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.899 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 650614318 0 0) 0x7fdd84137a30 con 0x7fdd84099240 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.122:6789/0 conn(0x7fdd8406e530 legacy=0x7fdd8412ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.26:6789/0 conn(0x7fdd84138d70 legacy=0x7fdd8414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 monclient: found mon.d 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:26.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84131160 con 0x7fdd84099240 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 5 monclient: authenticate success, global_id 4677 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: _renew_subs 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd84130ec0 con 0x7fdd84099240 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: _renew_subs 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd84134210 con 0x7fdd84099240 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdd6c004f00 con 0x7fdd84099240 2024-09-15T03:47:26.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdd8404bbf0 con 0x7fdd841315e0 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd6c0053b0 con 0x7fdd84099240 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd80ff9700 10 monclient: dump: 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.902 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.878+0000 7fdd82ffd700 4 set_mon_vals no callback set 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdd6c00fd70 con 0x7fdd84099240 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(118..118 src has 1..118) v4 ==== 4031+0+0 (unknown 3757952977 0 0) 0x7fdd6c0440b0 con 0x7fdd84099240 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240679355 0 0) 0x7fdd8404bbf0 con 0x7fdd841315e0 2024-09-15T03:47:26.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient(hunting): my global_id is 134179 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd84137a30 con 0x7fdd841315e0 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd88b0d700 10 monclient: get_auth_request con 0x7fdd64038000 auth_method 0 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1546111289 0 0) 0x7fdd84137a30 con 0x7fdd841315e0 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd8404bbf0 con 0x7fdd841315e0 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd74002270 con 0x7fdd841315e0 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient: dump: 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.904 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.905 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3007368008 0 0) 0x7fdd8404bbf0 con 0x7fdd841315e0 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd84066de0 con 0x7fdd841315e0 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdd74004640 con 0x7fdd841315e0 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.122:6792/0 conn(0x7fdd841315e0 legacy=0x7fdd8409a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.882+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.122:6792/0 conn(0x7fdd841315e0 legacy=0x7fdd8409a310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.890+0000 7fdd8930e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.890+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdd84138800 con 0x7fdd84138430 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 416818537 0 0) 0x7fdd84138800 con 0x7fdd84138430 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient(hunting): my global_id is 134188 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8404bbf0 con 0x7fdd84138430 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3060289640 0 0) 0x7fdd8404bbf0 con 0x7fdd84138430 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd84138800 con 0x7fdd84138430 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd74002dc0 con 0x7fdd84138430 2024-09-15T03:47:26.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient: dump: 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:26.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1853022364 0 0) 0x7fdd84138800 con 0x7fdd84138430 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdd84068340 con 0x7fdd84138430 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1195146601) 0x7fdd740041b0 con 0x7fdd84138430 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.122:6792/0 conn(0x7fdd84138430 legacy=0x7fdd8402ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd80ff9700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.122:6792/0 conn(0x7fdd84138430 legacy=0x7fdd8402ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:26.911 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.912 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.913 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.914 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:26.915 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:26.916 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.917 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:26.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.921 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.922 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.925 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.926 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:26.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:26.929 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:26.930 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:26.931 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:26.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.122:6812/20895 conn(0x7fdd64038000 legacy=0x7fdd6403a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 10 monclient: shutdown 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 >> v1:172.21.15.26:6792/0 conn(0x7fdd84099240 legacy=0x7fdd8412f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd88b0d700 1 -- 172.21.15.26:0/2575598660 reap_dead start 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd88b0d700 1 -- 172.21.15.26:0/2575598660 reap_dead start 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 shutdown_connections 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 >> 172.21.15.26:0/2575598660 conn(0x7fdd84070de0 msgr2=0x7fdd84098a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 shutdown_connections 2024-09-15T03:47:26.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:26.894+0000 7fdd8930e700 1 -- 172.21.15.26:0/2575598660 wait complete. 2024-09-15T03:47:26.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:26.933 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T03:47:26.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T03:47:27.008 INFO:tasks.rados.rados.0.smithi122.stdout:428: finishing copy_from to smithi12223906-17 2024-09-15T03:47:27.008 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 126 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-15T03:47:27.008 INFO:tasks.rados.rados.0.smithi122.stdout:432: finishing write tid 1 to smithi12223906-49 2024-09-15T03:47:27.008 INFO:tasks.rados.rados.0.smithi122.stdout:432: finishing write tid 2 to smithi12223906-49 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:432: finishing write tid 3 to smithi12223906-49 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:432: finishing write tid 4 to smithi12223906-49 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 129 (ObjNum 185 snap 48 seq_num 185) dirty exists 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:432: left oid 49 (ObjNum 185 snap 48 seq_num 185) 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:433: finishing write tid 1 to smithi12223906-8 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:433: finishing write tid 2 to smithi12223906-8 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:433: finishing write tid 3 to smithi12223906-8 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:433: finishing write tid 4 to smithi12223906-8 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:433: finishing write tid 5 to smithi12223906-8 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 112 (ObjNum 186 snap 48 seq_num 186) dirty exists 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:433: left oid 8 (ObjNum 186 snap 48 seq_num 186) 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:434: finishing copy_from racing read to smithi12223906-13 2024-09-15T03:47:27.009 INFO:tasks.rados.rados.0.smithi122.stdout:427: done (6 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:428: done (5 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:430: done (4 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:431: done (3 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:432: done (2 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:433: done (1 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:435: rollback oid 16 current snap is 48 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:435: done (1 left) 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:436: delete oid 40 current snap is 48 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:434: finishing copy_from to smithi12223906-13 2024-09-15T03:47:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 217 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:47:27.011 INFO:tasks.rados.rados.0.smithi122.stdout:434: done (1 left) 2024-09-15T03:47:27.011 INFO:tasks.rados.rados.0.smithi122.stdout:436: done (0 left) 2024-09-15T03:47:27.011 INFO:tasks.rados.rados.0.smithi122.stdout:437: snap_create 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient: get_monmap_and_config 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient: build_initial_monmap 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient: monmap: 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:27.053119+0000 2024-09-15T03:47:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:27.053119+0000 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:27.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 1 Processor -- start 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 1 -- start start 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient: init 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient: _add_conns ranks=[3,5,1,8,6,2,0,4,7] 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient(hunting): picked mon.noname-g con 0x7fb5dc0a6a80 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient(hunting): picked mon.noname-h con 0x7fb5dc09fb80 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient(hunting): picked mon.noname-f con 0x7fb5dc138d30 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc04ca10 con 0x7fb5dc138d30 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc0a0f50 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc133680 con 0x7fb5dc09fb80 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 10 monclient(hunting): authenticate will time out at 991.482516s 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e287f700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb5dc138d30 0x7fb5dc13b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:47052/0 (socket says 172.21.15.26:47052) 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e287f700 1 -- 172.21.15.26:0/318815682 learned_addr learned my addr 172.21.15.26:0/318815682 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e207e700 1 --1- 172.21.15.26:0/318815682 >> v1:172.21.15.122:6790/0 conn(0x7fb5dc0a6a80 0x7fb5dc132f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:36276/0 (socket says 172.21.15.26:36276) 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879465912 0 0) 0x7fb5dc04ca10 con 0x7fb5dc138d30 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): my global_id is 134128 2024-09-15T03:47:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c8003570 con 0x7fb5dc138d30 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775435258 0 0) 0x7fb5dc133680 con 0x7fb5dc09fb80 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): my global_id is 4712 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc04ca10 con 0x7fb5dc09fb80 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848721650 0 0) 0x7fb5dc0a0f50 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient(hunting): my global_id is 4665 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc133680 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638998137 0 0) 0x7fb5dc04ca10 con 0x7fb5dc09fb80 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 >> v1:172.21.15.122:6790/0 conn(0x7fb5dc0a6a80 legacy=0x7fb5dc132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 >> v1:172.21.15.122:6789/0 conn(0x7fb5dc138d30 legacy=0x7fb5dc13b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient: _finish_hunting 0 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 monclient: found mon.noname-h 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc0a6e50 con 0x7fb5dc09fb80 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient: _finish_auth 0 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5d8001b10 con 0x7fb5dc09fb80 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e107c700 10 monclient: dump: 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:27.061 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 5 monclient: authenticate success, global_id 4712 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.050+0000 7fb5e3080700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5d8001e30 con 0x7fb5dc09fb80 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e107c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e107c700 1 -- 172.21.15.26:0/318815682 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5d80025c0 con 0x7fb5dc09fb80 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e107c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e107c700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e107c700 10 monclient: dump: 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 4 set_mon_vals no callback set 2024-09-15T03:47:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: shutdown 2024-09-15T03:47:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- 172.21.15.26:0/318815682 >> v1:172.21.15.122:6791/0 conn(0x7fb5dc09fb80 legacy=0x7fb5dc09ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- 172.21.15.26:0/318815682 shutdown_connections 2024-09-15T03:47:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- 172.21.15.26:0/318815682 >> 172.21.15.26:0/318815682 conn(0x7fb5dc06e380 msgr2=0x7fb5dc0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- 172.21.15.26:0/318815682 shutdown_connections 2024-09-15T03:47:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- 172.21.15.26:0/318815682 wait complete. 2024-09-15T03:47:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: build_initial_monmap 2024-09-15T03:47:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: monmap: 2024-09-15T03:47:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 Processor -- start 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- start start 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: init 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient: _add_conns ranks=[6,2,3,8,4,1,0,7,5] 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient(hunting): picked mon.noname-d con 0x7fb5dc09fb80 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient(hunting): picked mon.noname-b con 0x7fb5dc0a6a80 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient(hunting): picked mon.noname-g con 0x7fb5dc138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc09b640 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc11cdf0 con 0x7fb5dc138d30 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5dc0a1920 con 0x7fb5dc09fb80 2024-09-15T03:47:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.054+0000 7fb5e3080700 10 monclient(hunting): authenticate will time out at 991.487878s 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e187d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb5dc0a6a80 0x7fb5dc099d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36324/0 (socket says 172.21.15.26:36324) 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e187d700 1 -- 172.21.15.26:0/2926663600 learned_addr learned my addr 172.21.15.26:0/2926663600 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549819546 0 0) 0x7fb5dc09b640 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): my global_id is 134192 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c0003570 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858399774 0 0) 0x7fb5dc11cdf0 con 0x7fb5dc138d30 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): my global_id is 4674 2024-09-15T03:47:27.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc09b640 con 0x7fb5dc138d30 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367386446 0 0) 0x7fb5dc0a1920 con 0x7fb5dc09fb80 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): my global_id is 4686 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5dc11cdf0 con 0x7fb5dc09fb80 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1744992989 0 0) 0x7fb5c0003570 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5dc0a1920 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5d80022e0 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): dump: 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:27.071 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:27.072 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3429811826 0 0) 0x7fb5dc09b640 con 0x7fb5dc138d30 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5c0003570 con 0x7fb5dc138d30 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5d40019a0 con 0x7fb5dc138d30 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient(hunting): dump: 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:27.073 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3310312768 0 0) 0x7fb5dc0a1920 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 >> v1:172.21.15.26:6792/0 conn(0x7fb5dc09fb80 legacy=0x7fb5dc099650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 >> v1:172.21.15.122:6790/0 conn(0x7fb5dc138d30 legacy=0x7fb5dc09a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:27.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 monclient: found mon.b 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5dc09cfc0 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 5 monclient: authenticate success, global_id 134192 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 monclient: _renew_subs 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc09b0c0 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 monclient: _renew_subs 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5dc09be30 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5d8002550 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5d8002e80 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e087b700 4 set_mon_vals no callback set 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 10 monclient: dump: 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:27.076 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb5d8003060 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:27.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c40050b0 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.058+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(118..118 src has 1..118) v4 ==== 4031+0+0 (unknown 3757952977 0 0) 0x7fb5d8044580 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.062+0000 7fb5e207e700 10 monclient: get_auth_request con 0x7fb5c0037d20 auth_method 0 2024-09-15T03:47:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.066+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fb5d80151f0 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.066+0000 7fb5d27fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:27.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.066+0000 7fb5d27fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.290+0000 7fb5e3080700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T03:47:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.290+0000 7fb5e3080700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T03:47:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.290+0000 7fb5e3080700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.290+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fb5c4005b70 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.294+0000 7fb5d27fc700 1 -- 172.21.15.26:0/2926663600 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fb5d80151f0 con 0x7fb5dc0a6a80 2024-09-15T03:47:27.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.294+0000 7fb5d27fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T03:47:27.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.294+0000 7fb5d27fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:27.301 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.301 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.301 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=21,config=2,health=12,kv=7,logm=58} crc {auth=3444041218,config=3140514009,health=3977944326,kv=2862677453,logm=1138154972}) 2024-09-15T03:47:27.301 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.301 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=22,config=2,health=11,kv=8,logm=57} crc {auth=2548551785,config=3140514009,health=3926953126,kv=2519098299,logm=4108116995}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=20,config=2,health=12,kv=7,logm=59} crc {auth=469229390,config=3140514009,health=3977944326,kv=2862677453,logm=1290215441}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=3838255296,config=3140514009,health=3977944326,kv=2519098299,logm=1310830968}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=19,config=2,health=12,kv=8,logm=59} crc {auth=304614414,config=3140514009,health=3977944326,kv=2519098299,logm=3092651318}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=21,config=2,health=10,kv=8,logm=59} crc {auth=3058325791,config=3140514009,health=1151464652,kv=2519098299,logm=3065130720}) 2024-09-15T03:47:27.302 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.303 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.303 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=23,config=2,health=12,kv=8,logm=55} crc {auth=1055432702,config=3140514009,health=3977944326,kv=2519098299,logm=354073240}) 2024-09-15T03:47:27.303 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.303 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=23,config=2,health=11,kv=7,logm=57} crc {auth=1055432702,config=3140514009,health=3716924378,kv=2862677453,logm=1764285624}) 2024-09-15T03:47:27.303 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.298+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=21,config=1,health=12,kv=8,logm=58} crc {auth=2890258977,config=1228700967,health=3977944326,kv=2519098299,logm=3422746801}) 2024-09-15T03:47:27.304 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.304 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3701911917}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1582458507}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2903785402}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=962998982}) 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.305 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 >> v1:172.21.15.122:6812/20895 conn(0x7fb5c0037d20 legacy=0x7fb5c003a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 10 monclient: shutdown 2024-09-15T03:47:27.306 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2106228263}) 2024-09-15T03:47:27.306 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.306 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.306 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3386434591}) 2024-09-15T03:47:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 >> v1:172.21.15.26:6790/0 conn(0x7fb5dc0a6a80 legacy=0x7fb5dc099d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:27.307 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.307 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 shutdown_connections 2024-09-15T03:47:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 >> 172.21.15.26:0/2926663600 conn(0x7fb5dc06e380 msgr2=0x7fb5dc09db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:27.308 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1876790462}) 2024-09-15T03:47:27.308 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.308 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2909359803}) 2024-09-15T03:47:27.308 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2067244434}) 2024-09-15T03:47:27.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 shutdown_connections 2024-09-15T03:47:27.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:27.302+0000 7fb5e3080700 1 -- 172.21.15.26:0/2926663600 wait complete. 2024-09-15T03:47:27.314 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.314 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2203863056}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.302+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2461929948}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2212336704}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1976886367}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=226278106}) 2024-09-15T03:47:27.315 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1434665485}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2640601722}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4292485841}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1971581152}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1199528033}) 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.316 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3152739148}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=275268923}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1850369926}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1792362641}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.317 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1627017935}) 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4206757037}) 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=867723023}) 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.306+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=241914792}) 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.318 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1951441584}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2672985244}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4035761640}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3836939406}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2861363337}) 2024-09-15T03:47:27.319 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3871946058}) 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1885007562}) 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=746779919}) 2024-09-15T03:47:27.320 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2503959586}) 2024-09-15T03:47:27.327 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1831730286}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3662181405}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3953043970}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1353721692}) 2024-09-15T03:47:27.328 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1336662626}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.310+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=808741074}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2044760095}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2076419565}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4200115633}) 2024-09-15T03:47:27.329 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=741734291}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3832000884}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3995726090}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1959489906}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.330 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2346708132}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443959690}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3752468957}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1106340796}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.314+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=646763789}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20,osdmap=35} crc {logm=2832249934,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=3327777643,osdmap=105002926}) 2024-09-15T03:47:27.331 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=8,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20,osdmap=37} crc {logm=2726937385,mdsmap=1531151339,mgr=3669162908,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=3327777643,osdmap=748644345}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=37} crc {logm=2832249934,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=3406880948,osdmap=3082488178}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=9,mdsmap=3,mgr=7,mgr_metadata=1,mgrstat=7,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20,osdmap=39} crc {logm=2832249934,mdsmap=1531151339,mgr=1568706250,mgr_metadata=4176136816,mgrstat=1767483693,mon_config_key=64957642,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=3327777643,osdmap=3424615193}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=9,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=39} crc {logm=2832249934,mdsmap=3140514009,mgr=3669162908,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=2173287178,osd_metadata=686778778,osd_pg_creating=1099985362,osd_snap=2019928798,osdmap=4193204328}) 2024-09-15T03:47:27.332 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=9,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=18,osdmap=39} crc {logm=2832249934,mdsmap=1531151339,mgr=4187012829,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=157028094,osdmap=204221553}) 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=9,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=17,osdmap=42} crc {logm=2832249934,mdsmap=3585761405,mgr=73913752,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=2113949861,monmap=4273209975,osd_metadata=2459566930,osd_pg_creating=1099985362,osd_snap=4047960313,osdmap=1449345746}) 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.333 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=9,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=19,osdmap=36} crc {logm=2100066390,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=693233123,osdmap=3829611677}) 2024-09-15T03:47:27.334 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T03:47:27.337 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.318+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=20,osdmap=36} crc {logm=990635478,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1553731658,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=1099985362,osd_snap=3327777643,osdmap=556234612}) 2024-09-15T03:47:27.337 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.337 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.337 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=500508131}) 2024-09-15T03:47:27.337 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.337 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3711059460}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1167366623}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3139474089}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4198977965}) 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.338 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=512051390}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1569999503}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4270607843}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3626190525}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=44} crc {osdmap=2357402837}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.339 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=44} crc {osdmap=1403360872}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=48} crc {osdmap=2251214624}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=48} crc {osdmap=2251214624}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=46} crc {osdmap=740387102}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=48} crc {osdmap=2251214624}) 2024-09-15T03:47:27.340 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.341 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.341 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=46} crc {osdmap=3559197665}) 2024-09-15T03:47:27.341 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:47:27.341 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=41} crc {osdmap=772193761}) 2024-09-15T03:47:27.341 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:47:27.322+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=45} crc {osdmap=2404407714}) 2024-09-15T03:47:28.000 INFO:tasks.rados.rados.0.smithi122.stdout:437: done (0 left) 2024-09-15T03:47:28.000 INFO:tasks.rados.rados.0.smithi122.stdout:438: rollback oid 42 current snap is 49 2024-09-15T03:47:28.001 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 48 2024-09-15T03:47:28.001 INFO:tasks.rados.rados.0.smithi122.stdout:439: read oid 41 snap 48 2024-09-15T03:47:28.001 INFO:tasks.rados.rados.0.smithi122.stdout:439: expect (ObjNum 156 snap 37 seq_num 156) 2024-09-15T03:47:28.001 INFO:tasks.rados.rados.0.smithi122.stdout:440: read oid 30 snap -1 2024-09-15T03:47:28.001 INFO:tasks.rados.rados.0.smithi122.stdout:440: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:47:28.001 INFO:tasks.rados.rados.0.smithi122.stdout:441: delete oid 22 current snap is 49 2024-09-15T03:47:28.003 INFO:tasks.rados.rados.0.smithi122.stdout:438: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:47:28.003 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:47:28.003 INFO:tasks.rados.rados.0.smithi122.stdout:438: done (3 left) 2024-09-15T03:47:28.003 INFO:tasks.rados.rados.0.smithi122.stdout:441: done (2 left) 2024-09-15T03:47:28.004 INFO:tasks.rados.rados.0.smithi122.stdout:442: snap_create 2024-09-15T03:47:28.011 INFO:tasks.rados.rados.0.smithi122.stdout:439: expect (ObjNum 156 snap 37 seq_num 156) 2024-09-15T03:47:28.219 INFO:tasks.rados.rados.0.smithi122.stdout:440: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:47:28.335 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:47:28.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:47:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.442+0000 7fb45e16b700 10 monclient: get_monmap_and_config 2024-09-15T03:47:28.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.442+0000 7fb45e16b700 10 monclient: build_initial_monmap 2024-09-15T03:47:28.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.442+0000 7fb45e16b700 10 monclient: monmap: 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:28.447663+0000 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:28.447663+0000 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:28.516 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 Processor -- start 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- start start 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient: init 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient: _add_conns ranks=[1,2,6,4,5,3,7,8,0] 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient(hunting): picked mon.noname-f con 0x7fb4580a4a30 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient(hunting): picked mon.noname-b con 0x7fb458067f60 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient(hunting): picked mon.noname-d con 0x7fb4581393a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45804ca10 con 0x7fb4580a4a30 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb458068aa0 con 0x7fb458067f60 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580a4e20 con 0x7fb4581393a0 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient(hunting): authenticate will time out at 992.877028s 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45c968700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb458067f60 0x7fb458137c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36328/0 (socket says 172.21.15.26:36328) 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45c968700 1 -- 172.21.15.26:0/3998043622 learned_addr learned my addr 172.21.15.26:0/3998043622 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139069050 0 0) 0x7fb450001490 con 0x7fb458067f60 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): my global_id is 134201 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb444003570 con 0x7fb458067f60 2024-09-15T03:47:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433024760 0 0) 0x7fb4580a4e20 con 0x7fb4581393a0 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): my global_id is 4695 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb450001490 con 0x7fb4581393a0 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409163759 0 0) 0x7fb45804ca10 con 0x7fb4580a4a30 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient(hunting): my global_id is 134137 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4580a4e20 con 0x7fb4580a4a30 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376033023 0 0) 0x7fb444003570 con 0x7fb458067f60 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 >> v1:172.21.15.26:6792/0 conn(0x7fb4581393a0 legacy=0x7fb45813b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 >> v1:172.21.15.122:6789/0 conn(0x7fb4580a4a30 legacy=0x7fb45809aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 monclient: found mon.noname-b 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580a0610 con 0x7fb458067f60 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: _finish_auth 0 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb450001c40 con 0x7fb458067f60 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 5 monclient: authenticate success, global_id 134201 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: dump: 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb450001e60 con 0x7fb458067f60 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 4 set_mon_vals no callback set 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 1 -- 172.21.15.26:0/3998043622 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb450004fb0 con 0x7fb458067f60 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb44bfff700 10 monclient: dump: 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 10 monclient: shutdown 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- 172.21.15.26:0/3998043622 >> v1:172.21.15.26:6790/0 conn(0x7fb458067f60 legacy=0x7fb458137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- 172.21.15.26:0/3998043622 shutdown_connections 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- 172.21.15.26:0/3998043622 >> 172.21.15.26:0/3998043622 conn(0x7fb458063b90 msgr2=0x7fb4580a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- 172.21.15.26:0/3998043622 shutdown_connections 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.446+0000 7fb45e16b700 1 -- 172.21.15.26:0/3998043622 wait complete. 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient: build_initial_monmap 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient: monmap: 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 1 Processor -- start 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 1 -- start start 2024-09-15T03:47:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient: init 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient: _add_conns ranks=[2,4,3,7,5,6,8,0,1] 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient(hunting): picked mon.noname-b con 0x7fb458067f60 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient(hunting): picked mon.noname-c con 0x7fb4580a4a30 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient(hunting): picked mon.noname-g con 0x7fb4581393a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45806e700 con 0x7fb458067f60 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45806e220 con 0x7fb4581393a0 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb458122b20 con 0x7fb4580a4a30 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45e16b700 10 monclient(hunting): authenticate will time out at 992.882889s 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45d169700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb458067f60 0x7fb458141ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36334/0 (socket says 172.21.15.26:36334) 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb45d169700 1 -- 172.21.15.26:0/1802703208 learned_addr learned my addr 172.21.15.26:0/1802703208 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393982442 0 0) 0x7fb45806e700 con 0x7fb458067f60 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): my global_id is 134210 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb43c003570 con 0x7fb458067f60 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047432927 0 0) 0x7fb458122b20 con 0x7fb4580a4a30 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): my global_id is 4576 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45806e700 con 0x7fb4580a4a30 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459726055 0 0) 0x7fb45806e220 con 0x7fb4581393a0 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): my global_id is 4683 2024-09-15T03:47:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb458122b20 con 0x7fb4581393a0 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3078339131 0 0) 0x7fb43c003570 con 0x7fb458067f60 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb45806e220 con 0x7fb458067f60 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb44c001b70 con 0x7fb458067f60 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.450+0000 7fb4497fa700 10 monclient(hunting): dump: 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3395617438 0 0) 0x7fb45806e700 con 0x7fb4580a4a30 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb43c003570 con 0x7fb4580a4a30 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb450004ef0 con 0x7fb4580a4a30 2024-09-15T03:47:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient(hunting): dump: 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1558144862 0 0) 0x7fb45806e220 con 0x7fb458067f60 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 >> v1:172.21.15.26:6791/0 conn(0x7fb4580a4a30 legacy=0x7fb458138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 >> v1:172.21.15.122:6790/0 conn(0x7fb4581393a0 legacy=0x7fb45814a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 monclient: found mon.b 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458139140 con 0x7fb458067f60 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 5 monclient: authenticate success, global_id 134210 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 monclient: _renew_subs 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb458120ff0 con 0x7fb458067f60 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 monclient: _renew_subs 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb458142b80 con 0x7fb458067f60 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb44c002040 con 0x7fb458067f60 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb44c0024f0 con 0x7fb458067f60 2024-09-15T03:47:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb44b7fe700 4 set_mon_vals no callback set 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 10 monclient: dump: 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb44c002650 con 0x7fb458067f60 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4380050b0 con 0x7fb458067f60 2024-09-15T03:47:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(119..119 src has 1..119) v4 ==== 3911+0+0 (unknown 3148561556 0 0) 0x7fb44c043e40 con 0x7fb458067f60 2024-09-15T03:47:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.454+0000 7fb45c968700 10 monclient: get_auth_request con 0x7fb43c037cd0 auth_method 0 2024-09-15T03:47:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.462+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fb44c014c10 con 0x7fb458067f60 2024-09-15T03:47:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.462+0000 7fb4497fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.462+0000 7fb4497fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:28.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb45e16b700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:47:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb45e16b700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:47:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb45e16b700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb4380058e0 con 0x7fb458067f60 2024-09-15T03:47:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb4497fa700 1 -- 172.21.15.26:0/1802703208 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 3304543425) 0x7fb44c014610 con 0x7fb458067f60 2024-09-15T03:47:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb4497fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:47:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.686+0000 7fb4497fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:28.691 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":34,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:47:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 >> v1:172.21.15.122:6812/20895 conn(0x7fb43c037cd0 legacy=0x7fb43c03a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 10 monclient: shutdown 2024-09-15T03:47:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 >> v1:172.21.15.26:6790/0 conn(0x7fb458067f60 legacy=0x7fb458141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 shutdown_connections 2024-09-15T03:47:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 >> 172.21.15.26:0/1802703208 conn(0x7fb458063b90 msgr2=0x7fb45806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 shutdown_connections 2024-09-15T03:47:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.690+0000 7fb45e16b700 1 -- 172.21.15.26:0/1802703208 wait complete. 2024-09-15T03:47:28.704 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:47:28.704 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:47:28.704 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:47:28.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:47:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient: build_initial_monmap 2024-09-15T03:47:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient: monmap: 2024-09-15T03:47:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.809 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:28.809349+0000 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:28.809349+0000 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:28.810 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 1 Processor -- start 2024-09-15T03:47:28.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 1 -- start start 2024-09-15T03:47:28.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient: init 2024-09-15T03:47:28.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:28.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient: _add_conns ranks=[0,4,2,6,8,5,3,1,7] 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient(hunting): picked mon.noname-a con 0x7f245c0a48e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient(hunting): picked mon.noname-c con 0x7f245c09aca0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient(hunting): picked mon.noname-b con 0x7f245c09a5e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c04ca10 con 0x7f245c0a48e0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c0696a0 con 0x7f245c09a5e0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c068d20 con 0x7f245c09aca0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246191e700 10 monclient(hunting): authenticate will time out at 993.238682s 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245bfff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f245c09aca0 0x7f245c137c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42422/0 (socket says 172.21.15.26:42422) 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f246111d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f245c09a5e0 0x7f245c13b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36346/0 (socket says 172.21.15.26:36346) 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245bfff700 1 -- 172.21.15.26:0/2791046872 learned_addr learned my addr 172.21.15.26:0/2791046872 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627955079 0 0) 0x7f245c0696a0 con 0x7f245c09a5e0 2024-09-15T03:47:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): my global_id is 134219 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003570 con 0x7f245c09a5e0 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845192216 0 0) 0x7f245c068d20 con 0x7f245c09aca0 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): my global_id is 4585 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245c0696a0 con 0x7f245c09aca0 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124758813 0 0) 0x7f245c04ca10 con 0x7f245c0a48e0 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient(hunting): my global_id is 4572 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245c068d20 con 0x7f245c0a48e0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1782359665 0 0) 0x7f244c003570 con 0x7f245c09a5e0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 >> v1:172.21.15.26:6791/0 conn(0x7f245c09aca0 legacy=0x7f245c137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 >> v1:172.21.15.26:6789/0 conn(0x7f245c0a48e0 legacy=0x7f245c099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 monclient: found mon.noname-b 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c067640 con 0x7f245c09a5e0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2454001b30 con 0x7f245c09a5e0 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.806+0000 7f245b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 10 monclient: dump: 2024-09-15T03:47:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.819 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 5 monclient: authenticate success, global_id 134219 2024-09-15T03:47:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2454001da0 con 0x7f245c09a5e0 2024-09-15T03:47:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 1 -- 172.21.15.26:0/2791046872 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2454004ef0 con 0x7f245c09a5e0 2024-09-15T03:47:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 4 set_mon_vals no callback set 2024-09-15T03:47:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f245b7fe700 10 monclient: dump: 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: shutdown 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- 172.21.15.26:0/2791046872 >> v1:172.21.15.26:6790/0 conn(0x7f245c09a5e0 legacy=0x7f245c13b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- 172.21.15.26:0/2791046872 shutdown_connections 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- 172.21.15.26:0/2791046872 >> 172.21.15.26:0/2791046872 conn(0x7f245c0638b0 msgr2=0x7f245c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- 172.21.15.26:0/2791046872 shutdown_connections 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- 172.21.15.26:0/2791046872 wait complete. 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: build_initial_monmap 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: monmap: 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 Processor -- start 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- start start 2024-09-15T03:47:28.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: init 2024-09-15T03:47:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient: _add_conns ranks=[2,8,5,6,1,0,3,7,4] 2024-09-15T03:47:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient(hunting): picked mon.noname-b con 0x7f245c09a5e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient(hunting): picked mon.noname-e con 0x7f245c09aca0 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:28.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient(hunting): picked mon.noname-h con 0x7f245c0a48e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c122b90 con 0x7f245c09a5e0 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246091c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f245c09a5e0 0x7f245c13d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36352/0 (socket says 172.21.15.26:36352) 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c13df20 con 0x7f245c0a48e0 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246091c700 1 -- 172.21.15.26:0/3532438445 learned_addr learned my addr 172.21.15.26:0/3532438445 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c13ec20 con 0x7f245c09aca0 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.810+0000 7f246191e700 10 monclient(hunting): authenticate will time out at 993.243934s 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298769587 0 0) 0x7f245c122b90 con 0x7f245c09a5e0 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): my global_id is 134228 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2440003570 con 0x7f245c09a5e0 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309409610 0 0) 0x7f245c13ec20 con 0x7f245c09aca0 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): my global_id is 134207 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245c122b90 con 0x7f245c09aca0 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223487897 0 0) 0x7f245c13df20 con 0x7f245c0a48e0 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): my global_id is 4721 2024-09-15T03:47:28.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245c13ec20 con 0x7f245c0a48e0 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1605615660 0 0) 0x7f2440003570 con 0x7f245c09a5e0 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f245c13df20 con 0x7f245c09a5e0 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2448001b60 con 0x7f245c09a5e0 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4238346879 0 0) 0x7f245c122b90 con 0x7f245c09aca0 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2440003570 con 0x7f245c09aca0 2024-09-15T03:47:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2450001510 con 0x7f245c09aca0 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.838 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 829307224 0 0) 0x7f245c13ec20 con 0x7f245c0a48e0 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f245c122b90 con 0x7f245c0a48e0 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2454004ef0 con 0x7f245c0a48e0 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.840 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1798285999 0 0) 0x7f245c13df20 con 0x7f245c09a5e0 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.26:6793/0 conn(0x7f245c09aca0 legacy=0x7f245c146a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.122:6791/0 conn(0x7f245c0a48e0 legacy=0x7f245c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 monclient: found mon.b 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c1228b0 con 0x7f245c09a5e0 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 5 monclient: authenticate success, global_id 134228 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 10 monclient: _renew_subs 2024-09-15T03:47:28.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c122e70 con 0x7f245c09a5e0 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 10 monclient: _renew_subs 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c121b80 con 0x7f245c09a5e0 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2448002070 con 0x7f245c09a5e0 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f24480022a0 con 0x7f245c09a5e0 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f245affd700 4 set_mon_vals no callback set 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f2458ff9700 10 monclient: dump: 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.814+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f245c06e320 con 0x7f245c120480 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007652787 0 0) 0x7f2450004ae0 con 0x7f245c120480 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient(hunting): my global_id is 4581 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f245c13df20 con 0x7f245c120480 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f244800f8c0 con 0x7f245c09a5e0 2024-09-15T03:47:28.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(119..119 src has 1..119) v4 ==== 3911+0+0 (unknown 3148561556 0 0) 0x7f2448043e10 con 0x7f245c09a5e0 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2154406307 0 0) 0x7f245c13df20 con 0x7f245c120480 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2450004ae0 con 0x7f245c120480 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f24500017b0 con 0x7f245c120480 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient: dump: 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f246111d700 10 monclient: get_auth_request con 0x7f24400380a0 auth_method 0 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3420565195 0 0) 0x7f2450004ae0 con 0x7f245c120480 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f245c120ea0 con 0x7f245c120480 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2450007320 con 0x7f245c120480 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:28.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.26:6789/0 conn(0x7f245c120480 legacy=0x7f245c09b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.818+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.26:6789/0 conn(0x7f245c120480 legacy=0x7f245c09b410 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f246191e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f245c13e530 con 0x7f245c13e160 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074596595 0 0) 0x7f245c13e530 con 0x7f245c13e160 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 10 monclient(hunting): my global_id is 4590 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2450004ae0 con 0x7f245c13e160 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 207664018 0 0) 0x7f2450004ae0 con 0x7f245c13e160 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f245c13e530 con 0x7f245c13e160 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2450005b50 con 0x7f245c13e160 2024-09-15T03:47:28.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.826+0000 7f2458ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 10 monclient: dump: 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 545942016 0 0) 0x7f245c13e530 con 0x7f245c13e160 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f245c04bbf0 con 0x7f245c13e160 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2857784585) 0x7f2450006fb0 con 0x7f245c13e160 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.26:6789/0 conn(0x7f245c13e160 legacy=0x7f245c02ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f2458ff9700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.26:6789/0 conn(0x7f245c13e160 legacy=0x7f245c02ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:28.849 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:28.850 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:28.851 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:28.852 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:28.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:28.854 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.855 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.857 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:28.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.860 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.861 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:28.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:28.866 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:28.867 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:28.868 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:28.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:28.870 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:28.870 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.122:6812/20895 conn(0x7f24400380a0 legacy=0x7f244003a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 10 monclient: shutdown 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 >> v1:172.21.15.26:6790/0 conn(0x7f245c09a5e0 legacy=0x7f245c13d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246111d700 1 -- 172.21.15.26:0/3532438445 reap_dead start 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246111d700 1 -- 172.21.15.26:0/3532438445 reap_dead start 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 shutdown_connections 2024-09-15T03:47:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 >> 172.21.15.26:0/3532438445 conn(0x7f245c0638b0 msgr2=0x7f245c098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:28.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 shutdown_connections 2024-09-15T03:47:28.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.830+0000 7f246191e700 1 -- 172.21.15.26:0/3532438445 wait complete. 2024-09-15T03:47:28.872 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:28.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:47:28.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.986+0000 7fe683702700 10 monclient: get_monmap_and_config 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.986+0000 7fe683702700 10 monclient: build_initial_monmap 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.986+0000 7fe683702700 10 monclient: monmap: 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:28.991716+0000 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:28.991716+0000 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:28.993 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.990+0000 7fe683702700 1 Processor -- start 2024-09-15T03:47:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.990+0000 7fe683702700 1 -- start start 2024-09-15T03:47:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.990+0000 7fe683702700 10 monclient: init 2024-09-15T03:47:28.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient: _add_conns ranks=[1,7,6,3,5,8,2,4,0] 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient(hunting): picked mon.noname-f con 0x7fe67c0990c0 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient(hunting): picked mon.noname-i con 0x7fe67c0a1310 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient(hunting): picked mon.noname-d con 0x7fe67c1335a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c04ca10 con 0x7fe67c0990c0 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0a1b20 con 0x7fe67c1335a0 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0a10b0 con 0x7fe67c0a1310 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient(hunting): authenticate will time out at 993.424775s 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe681c9f700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fe67c1335a0 0x7fe67c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41442/0 (socket says 172.21.15.26:41442) 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe681c9f700 1 -- 172.21.15.26:0/2402592253 learned_addr learned my addr 172.21.15.26:0/2402592253 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874698046 0 0) 0x7fe67c0a1b20 con 0x7fe67c1335a0 2024-09-15T03:47:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient(hunting): my global_id is 4704 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe668003570 con 0x7fe67c1335a0 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355874064 0 0) 0x7fe668003570 con 0x7fe67c1335a0 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 >> v1:172.21.15.122:6792/0 conn(0x7fe67c0a1310 legacy=0x7fe67c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 >> v1:172.21.15.122:6789/0 conn(0x7fe67c0990c0 legacy=0x7fe67c132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 monclient: found mon.noname-d 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c099ba0 con 0x7fe67c1335a0 2024-09-15T03:47:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 5 monclient: authenticate success, global_id 4704 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe6780017b0 con 0x7fe67c1335a0 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: dump: 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe678001ad0 con 0x7fe67c1335a0 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 1 -- 172.21.15.26:0/2402592253 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe678004c30 con 0x7fe67c1335a0 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 4 set_mon_vals no callback set 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe673fff700 10 monclient: dump: 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 10 monclient: shutdown 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- 172.21.15.26:0/2402592253 >> v1:172.21.15.26:6792/0 conn(0x7fe67c1335a0 legacy=0x7fe67c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- 172.21.15.26:0/2402592253 shutdown_connections 2024-09-15T03:47:29.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- 172.21.15.26:0/2402592253 >> 172.21.15.26:0/2402592253 conn(0x7fe67c06e370 msgr2=0x7fe67c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:439: done (2 left) 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:440: done (1 left) 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:442: done (0 left) 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:443: copy_from oid 32 from oid 12 current snap is 50 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:444: delete oid 31 current snap is 50 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:444: done (1 left) 2024-09-15T03:47:29.006 INFO:tasks.rados.rados.0.smithi122.stdout:445: snap_create 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- 172.21.15.26:0/2402592253 shutdown_connections 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.994+0000 7fe683702700 1 -- 172.21.15.26:0/2402592253 wait complete. 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient: monmap: 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 1 Processor -- start 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 1 -- start start 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient: init 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient: _add_conns ranks=[1,2,5,3,4,0,7,6,8] 2024-09-15T03:47:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient(hunting): picked mon.noname-f con 0x7fe67c0990c0 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient(hunting): picked mon.noname-b con 0x7fe67c0a1310 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient(hunting): picked mon.noname-h con 0x7fe67c1335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0a1ce0 con 0x7fe67c0990c0 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c09a010 con 0x7fe67c0a1310 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c12e650 con 0x7fe67c1335a0 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe683702700 10 monclient(hunting): authenticate will time out at 993.430312s 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe681c9f700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7fe67c1335a0 0x7fe67c14a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:46728/0 (socket says 172.21.15.26:46728) 2024-09-15T03:47:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:28.998+0000 7fe681c9f700 1 -- 172.21.15.26:0/2404531521 learned_addr learned my addr 172.21.15.26:0/2404531521 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.002+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464559579 0 0) 0x7fe67c09a010 con 0x7fe67c0a1310 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.002+0000 7fe6717fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.002+0000 7fe6717fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.002+0000 7fe6717fa700 10 monclient(hunting): my global_id is 134237 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.002+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe660003570 con 0x7fe67c0a1310 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2170111898 0 0) 0x7fe660003570 con 0x7fe67c0a1310 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe67c09a010 con 0x7fe67c0a1310 2024-09-15T03:47:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe674001cd0 con 0x7fe67c0a1310 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient(hunting): dump: 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4072056721 0 0) 0x7fe67c09a010 con 0x7fe67c0a1310 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.122:6791/0 conn(0x7fe67c1335a0 legacy=0x7fe67c14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.122:6789/0 conn(0x7fe67c0990c0 legacy=0x7fe67c09b630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 monclient: found mon.b 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c09c450 con 0x7fe67c0a1310 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 5 monclient: authenticate success, global_id 134237 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 10 monclient: _renew_subs 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c12eef0 con 0x7fe67c0a1310 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 10 monclient: _renew_subs 2024-09-15T03:47:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c132750 con 0x7fe67c0a1310 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe674003020 con 0x7fe67c0a1310 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe6740025a0 con 0x7fe67c0a1310 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6737fe700 4 set_mon_vals no callback set 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 10 monclient: dump: 2024-09-15T03:47:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.006+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe674002700 con 0x7fe67c0a1310 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe683702700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe65c006590 con 0x7fe65c002840 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 3911+0+0 (unknown 2439909402 0 0) 0x7fe674041890 con 0x7fe67c0a1310 2024-09-15T03:47:29.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe68149e700 10 monclient: get_auth_request con 0x7fe6600375d0 auth_method 0 2024-09-15T03:47:29.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392445920 0 0) 0x7fe65c006590 con 0x7fe65c002840 2024-09-15T03:47:29.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient(hunting): my global_id is 134246 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67c09a010 con 0x7fe65c002840 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3372671711 0 0) 0x7fe67c09a010 con 0x7fe65c002840 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe65c006590 con 0x7fe65c002840 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe678004b70 con 0x7fe65c002840 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient: dump: 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.020 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3432140180 0 0) 0x7fe65c006590 con 0x7fe65c002840 2024-09-15T03:47:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe65c00a130 con 0x7fe65c002840 2024-09-15T03:47:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe678001670 con 0x7fe65c002840 2024-09-15T03:47:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.26:6790/0 conn(0x7fe65c002840 legacy=0x7fe65c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.010+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.26:6790/0 conn(0x7fe65c002840 legacy=0x7fe65c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.022 INFO:tasks.rados.rados.0.smithi122.stdout:443: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:47:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.018+0000 7fe683702700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:29.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe65c009620 con 0x7fe65c009090 2024-09-15T03:47:29.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423203947 0 0) 0x7fe65c009620 con 0x7fe65c009090 2024-09-15T03:47:29.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient(hunting): my global_id is 134255 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe65c006590 con 0x7fe65c009090 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1898190222 0 0) 0x7fe65c006590 con 0x7fe65c009090 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe65c009620 con 0x7fe65c009090 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe678005530 con 0x7fe65c009090 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient: dump: 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.027 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1880759059 0 0) 0x7fe65c009620 con 0x7fe65c009090 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe65c0098d0 con 0x7fe65c009090 2024-09-15T03:47:29.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 4001504596) 0x7fe678007710 con 0x7fe65c009090 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.26:6790/0 conn(0x7fe65c009090 legacy=0x7fe65c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe6717fa700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.26:6790/0 conn(0x7fe65c009090 legacy=0x7fe65c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.030 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.031 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.032 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.033 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.034 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.034 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.034 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:29.034 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:29.034 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:29.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:29.036 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.037 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.044 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.047 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.047 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.047 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.047 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.048 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.048 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:29.052 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.052 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.053 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.053 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.053 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.053 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.055 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.055 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.055 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:29.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.057 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.057 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.057 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.057 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:47:29.058 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.059 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.059 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:29.059 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.122:6812/20895 conn(0x7fe6600375d0 legacy=0x7fe660039a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 10 monclient: shutdown 2024-09-15T03:47:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 >> v1:172.21.15.26:6790/0 conn(0x7fe67c0a1310 legacy=0x7fe67c09bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe681c9f700 1 -- 172.21.15.26:0/2404531521 reap_dead start 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe681c9f700 1 -- 172.21.15.26:0/2404531521 reap_dead start 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 shutdown_connections 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 >> 172.21.15.26:0/2404531521 conn(0x7fe67c06e370 msgr2=0x7fe67c12be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 shutdown_connections 2024-09-15T03:47:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.022+0000 7fe683702700 1 -- 172.21.15.26:0/2404531521 wait complete. 2024-09-15T03:47:29.063 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:29.063 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:47:29.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:47:29.064 INFO:tasks.rados.rados.0.smithi122.stdout:443: finishing copy_from to smithi12223906-32 2024-09-15T03:47:29.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 25 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:47:29.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:29.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient: monmap: 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:29.173317+0000 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:29.173317+0000 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.174 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.175 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.175 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.175 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 1 Processor -- start 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 1 -- start start 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient: init 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient: _add_conns ranks=[3,5,6,8,1,0,4,2,7] 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient(hunting): picked mon.noname-g con 0x7f7cac132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient(hunting): picked mon.noname-h con 0x7f7cac09faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient(hunting): picked mon.noname-d con 0x7f7cac138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac04ca10 con 0x7f7cac132b40 2024-09-15T03:47:29.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac09d5d0 con 0x7f7cac09faa0 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac133620 con 0x7f7cac138d40 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.170+0000 7f7cab59e700 10 monclient(hunting): authenticate will time out at 993.603909s 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7caad9d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f7cac138d40 0x7f7cac13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41444/0 (socket says 172.21.15.26:41444) 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7caad9d700 1 -- 172.21.15.26:0/401138837 learned_addr learned my addr 172.21.15.26:0/401138837 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874304006 0 0) 0x7f7cac133620 con 0x7f7cac138d40 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient(hunting): my global_id is 4713 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c94003570 con 0x7f7cac138d40 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 461793239 0 0) 0x7f7c94003570 con 0x7f7cac138d40 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 >> v1:172.21.15.122:6791/0 conn(0x7f7cac09faa0 legacy=0x7f7cac09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 >> v1:172.21.15.122:6790/0 conn(0x7f7cac132b40 legacy=0x7f7cac132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 monclient: found mon.noname-d 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0a0cc0 con 0x7f7cac138d40 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 5 monclient: authenticate success, global_id 4713 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7ca4001b30 con 0x7f7cac138d40 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: dump: 2024-09-15T03:47:29.181 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.182 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ca4001de0 con 0x7f7cac138d40 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 1 -- 172.21.15.26:0/401138837 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7ca4004fb0 con 0x7f7cac138d40 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 4 set_mon_vals no callback set 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7ca959a700 10 monclient: dump: 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.184 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 10 monclient: shutdown 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 -- 172.21.15.26:0/401138837 >> v1:172.21.15.26:6792/0 conn(0x7f7cac138d40 legacy=0x7f7cac13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 -- 172.21.15.26:0/401138837 shutdown_connections 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 -- 172.21.15.26:0/401138837 >> 172.21.15.26:0/401138837 conn(0x7f7cac06e370 msgr2=0x7f7cac0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 -- 172.21.15.26:0/401138837 shutdown_connections 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 -- 172.21.15.26:0/401138837 wait complete. 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 10 monclient: monmap: 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.185 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.186 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.187 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 Processor -- start 2024-09-15T03:47:29.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 1 -- start start 2024-09-15T03:47:29.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.174+0000 7f7cab59e700 10 monclient: init 2024-09-15T03:47:29.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient: _add_conns ranks=[1,6,8,4,3,5,2,7,0] 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient(hunting): picked mon.noname-f con 0x7f7cac09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient(hunting): picked mon.noname-d con 0x7f7cac132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient(hunting): picked mon.noname-e con 0x7f7cac138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac138a20 con 0x7f7cac09faa0 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac09ccc0 con 0x7f7cac132b40 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac09cfa0 con 0x7f7cac138d40 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient(hunting): authenticate will time out at 993.608332s 2024-09-15T03:47:29.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7ca9d9b700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f7cac132b40 0x7f7cac0987e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41446/0 (socket says 172.21.15.26:41446) 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7ca9d9b700 1 -- 172.21.15.26:0/2884895749 learned_addr learned my addr 172.21.15.26:0/2884895749 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779675762 0 0) 0x7f7cac09cfa0 con 0x7f7cac138d40 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): my global_id is 134216 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c8c003570 con 0x7f7cac138d40 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367538254 0 0) 0x7f7cac09ccc0 con 0x7f7cac132b40 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): my global_id is 4722 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cac09cfa0 con 0x7f7cac132b40 2024-09-15T03:47:29.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925929675 0 0) 0x7f7cac138a20 con 0x7f7cac09faa0 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): my global_id is 134146 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cac09ccc0 con 0x7f7cac09faa0 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4068650816 0 0) 0x7f7cac09cfa0 con 0x7f7cac132b40 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7cac138a20 con 0x7f7cac132b40 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7ca0001aa0 con 0x7f7cac132b40 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.195 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2929862829 0 0) 0x7f7c8c003570 con 0x7f7cac138d40 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7cac09cfa0 con 0x7f7cac138d40 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7ca4001ba0 con 0x7f7cac138d40 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2485987972 0 0) 0x7f7cac09ccc0 con 0x7f7cac09faa0 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7c8c003570 con 0x7f7cac09faa0 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7c9c001a20 con 0x7f7cac09faa0 2024-09-15T03:47:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 350205806 0 0) 0x7f7cac138a20 con 0x7f7cac132b40 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.26:6793/0 conn(0x7f7cac138d40 legacy=0x7f7cac14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.122:6789/0 conn(0x7f7cac09faa0 legacy=0x7f7cac0980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 monclient: found mon.d 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac09b690 con 0x7f7cac132b40 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 5 monclient: authenticate success, global_id 4722 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient: _renew_subs 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac099bd0 con 0x7f7cac132b40 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient: _renew_subs 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac099090 con 0x7f7cac132b40 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7ca0001d80 con 0x7f7cac132b40 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7ca00024f0 con 0x7f7cac132b40 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7ca8d99700 4 set_mon_vals no callback set 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 10 monclient: dump: 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7c90006590 con 0x7f7c90002840 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.178+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f7ca0002650 con 0x7f7cac132b40 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 3911+0+0 (unknown 2439909402 0 0) 0x7f7ca0043da0 con 0x7f7cac132b40 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435642284 0 0) 0x7f7c90006590 con 0x7f7c90002840 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient(hunting): my global_id is 4594 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cac138a20 con 0x7f7c90002840 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7caad9d700 10 monclient: get_auth_request con 0x7f7c8c038100 auth_method 0 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3011946177 0 0) 0x7f7cac138a20 con 0x7f7c90002840 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7c90006590 con 0x7f7c90002840 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7c9c0021b0 con 0x7f7c90002840 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient: dump: 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1436569809 0 0) 0x7f7c90006590 con 0x7f7c90002840 2024-09-15T03:47:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c9000a130 con 0x7f7c90002840 2024-09-15T03:47:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7c9c004540 con 0x7f7c90002840 2024-09-15T03:47:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.26:6791/0 conn(0x7f7c90002840 legacy=0x7f7c90004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.182+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.26:6791/0 conn(0x7f7c90002840 legacy=0x7f7c90004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7cab59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7c90009640 con 0x7f7c900090b0 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493665385 0 0) 0x7f7c90009640 con 0x7f7c900090b0 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7c9a7fc700 10 monclient(hunting): my global_id is 4603 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.190+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c90006590 con 0x7f7c900090b0 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 342352914 0 0) 0x7f7c90006590 con 0x7f7c900090b0 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7c90009640 con 0x7f7c900090b0 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7c9c002e10 con 0x7f7c900090b0 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 10 monclient: dump: 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2554498379 0 0) 0x7f7c90009640 con 0x7f7c900090b0 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7c900098f0 con 0x7f7c900090b0 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3436621930) 0x7f7c9c0043a0 con 0x7f7c900090b0 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.26:6791/0 conn(0x7f7c900090b0 legacy=0x7f7c90008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7c9a7fc700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.26:6791/0 conn(0x7f7c900090b0 legacy=0x7f7c90008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:29.211 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:47:29.212 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.213 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:29.215 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.216 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.217 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.218 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.220 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.223 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:29.224 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.225 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.226 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.227 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.228 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.229 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.230 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.231 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.231 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.231 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:29.231 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.122:6812/20895 conn(0x7f7c8c038100 legacy=0x7f7c8c03a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 10 monclient: shutdown 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 >> v1:172.21.15.26:6792/0 conn(0x7f7cac132b40 legacy=0x7f7cac0987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7caad9d700 1 -- 172.21.15.26:0/2884895749 reap_dead start 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7caad9d700 1 -- 172.21.15.26:0/2884895749 reap_dead start 2024-09-15T03:47:29.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 shutdown_connections 2024-09-15T03:47:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 >> 172.21.15.26:0/2884895749 conn(0x7f7cac06e370 msgr2=0x7f7cac12eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 shutdown_connections 2024-09-15T03:47:29.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.194+0000 7f7cab59e700 1 -- 172.21.15.26:0/2884895749 wait complete. 2024-09-15T03:47:29.233 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:29.233 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:47:29.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.342+0000 7f5687fff700 10 monclient: get_monmap_and_config 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.342+0000 7f5687fff700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.342+0000 7f5687fff700 10 monclient: monmap: 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:29.347827+0000 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:29.347827+0000 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.348 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.349 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 Processor -- start 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- start start 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient: init 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient: _add_conns ranks=[2,4,8,3,7,6,0,5,1] 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient(hunting): picked mon.noname-b con 0x7f56881335e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient(hunting): picked mon.noname-c con 0x7f568809fc80 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient(hunting): picked mon.noname-e con 0x7f56880a0050 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568804ca10 con 0x7f56881335e0 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568806eab0 con 0x7f568809fc80 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880a1330 con 0x7f56880a0050 2024-09-15T03:47:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient(hunting): authenticate will time out at 993.777240s 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f56867fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f568809fc80 0x7f5688137ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42444/0 (socket says 172.21.15.26:42444) 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f56867fc700 1 -- 172.21.15.26:0/4036846314 learned_addr learned my addr 172.21.15.26:0/4036846314 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739289685 0 0) 0x7f568806eab0 con 0x7f568809fc80 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): my global_id is 4612 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5674003570 con 0x7f568809fc80 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310223804 0 0) 0x7f568804ca10 con 0x7f56881335e0 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): my global_id is 134264 2024-09-15T03:47:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f568806eab0 con 0x7f56881335e0 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559548527 0 0) 0x7f56880a1330 con 0x7f56880a0050 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient(hunting): my global_id is 134225 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f568804ca10 con 0x7f56880a0050 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1811873814 0 0) 0x7f5674003570 con 0x7f568809fc80 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 >> v1:172.21.15.26:6793/0 conn(0x7f56880a0050 legacy=0x7f568813b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 >> v1:172.21.15.26:6790/0 conn(0x7f56881335e0 legacy=0x7f5688132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 monclient: found mon.noname-c 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880a10d0 con 0x7f568809fc80 2024-09-15T03:47:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5678001a60 con 0x7f568809fc80 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: dump: 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 5 monclient: authenticate success, global_id 4612 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5678001cd0 con 0x7f568809fc80 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 4 set_mon_vals no callback set 2024-09-15T03:47:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 1 -- 172.21.15.26:0/4036846314 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5678004c70 con 0x7f568809fc80 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5685ffb700 10 monclient: dump: 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 10 monclient: shutdown 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- 172.21.15.26:0/4036846314 >> v1:172.21.15.26:6791/0 conn(0x7f568809fc80 legacy=0x7f5688137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- 172.21.15.26:0/4036846314 shutdown_connections 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- 172.21.15.26:0/4036846314 >> 172.21.15.26:0/4036846314 conn(0x7f568806e6e0 msgr2=0x7f56880a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- 172.21.15.26:0/4036846314 shutdown_connections 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.346+0000 7f5687fff700 1 -- 172.21.15.26:0/4036846314 wait complete. 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: monmap: 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 Processor -- start 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 -- start start 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: init 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: _add_conns ranks=[5,3,0,4,1,8,6,2,7] 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient(hunting): picked mon.noname-h con 0x7f568809fc80 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient(hunting): picked mon.noname-g con 0x7f56880a0050 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient(hunting): picked mon.noname-a con 0x7f56881335e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56881428f0 con 0x7f56881335e0 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568809cf10 con 0x7f56880a0050 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5688130940 con 0x7f568809fc80 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient(hunting): authenticate will time out at 993.782118s 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f56877fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56881335e0 0x7f568814a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47342/0 (socket says 172.21.15.26:47342) 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f56877fe700 1 -- 172.21.15.26:0/1020025334 learned_addr learned my addr 172.21.15.26:0/1020025334 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654038483 0 0) 0x7f56881428f0 con 0x7f56881335e0 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): my global_id is 4599 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5668003570 con 0x7f56881335e0 2024-09-15T03:47:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245841534 0 0) 0x7f568809cf10 con 0x7f56880a0050 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): my global_id is 4701 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56881428f0 con 0x7f56880a0050 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532581544 0 0) 0x7f5688130940 con 0x7f568809fc80 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): my global_id is 4739 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f568809cf10 con 0x7f568809fc80 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 630601347 0 0) 0x7f5668003570 con 0x7f56881335e0 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5688130940 con 0x7f56881335e0 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5670002af0 con 0x7f56881335e0 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): dump: 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2310156058 0 0) 0x7f5688130940 con 0x7f56881335e0 2024-09-15T03:47:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.122:6791/0 conn(0x7f568809fc80 legacy=0x7f568812fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.122:6790/0 conn(0x7f56880a0050 legacy=0x7f5688146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 monclient: found mon.a 2024-09-15T03:47:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568811ee00 con 0x7f56881335e0 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 5 monclient: authenticate success, global_id 4599 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: _renew_subs 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568811e250 con 0x7f56881335e0 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: _renew_subs 2024-09-15T03:47:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881302b0 con 0x7f56881335e0 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f56700015a0 con 0x7f56881335e0 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5670001ad0 con 0x7f56881335e0 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f567effd700 10 monclient: dump: 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.373 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.350+0000 7f56857fa700 4 set_mon_vals no callback set 2024-09-15T03:47:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f5687fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5670001cb0 con 0x7f56881335e0 2024-09-15T03:47:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f566c006590 con 0x7f566c002840 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 3911+0+0 (unknown 2439909402 0 0) 0x7f5670043f80 con 0x7f56881335e0 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808887196 0 0) 0x7f566c006590 con 0x7f566c002840 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient(hunting): my global_id is 4731 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5688130940 con 0x7f566c002840 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f56867fc700 10 monclient: get_auth_request con 0x7f56680375b0 auth_method 0 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3255738045 0 0) 0x7f5688130940 con 0x7f566c002840 2024-09-15T03:47:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f566c006590 con 0x7f566c002840 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56800018c0 con 0x7f566c002840 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient: dump: 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.380 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.380 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.380 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3648735732 0 0) 0x7f566c006590 con 0x7f566c002840 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f566c00a130 con 0x7f566c002840 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5680001eb0 con 0x7f566c002840 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.26:6792/0 conn(0x7f566c002840 legacy=0x7f566c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.354+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.26:6792/0 conn(0x7f566c002840 legacy=0x7f566c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f566c009640 con 0x7f566c0090b0 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017348199 0 0) 0x7f566c009640 con 0x7f566c0090b0 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient(hunting): my global_id is 4740 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f566c006590 con 0x7f566c0090b0 2024-09-15T03:47:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2151526279 0 0) 0x7f566c006590 con 0x7f566c0090b0 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f566c009640 con 0x7f566c0090b0 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5680002760 con 0x7f566c0090b0 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient: dump: 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 94252169 0 0) 0x7f566c009640 con 0x7f566c0090b0 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f566c0098f0 con 0x7f566c0090b0 2024-09-15T03:47:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 4178943609) 0x7f5680002b10 con 0x7f566c0090b0 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.26:6792/0 conn(0x7f566c0090b0 legacy=0x7f566c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f567effd700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.26:6792/0 conn(0x7f566c0090b0 legacy=0x7f566c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.385 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.406 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:29.407 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:29.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.122:6812/20895 conn(0x7f56680375b0 legacy=0x7f5668039a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 10 monclient: shutdown 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 >> v1:172.21.15.26:6789/0 conn(0x7f56881335e0 legacy=0x7f568814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f56877fe700 1 -- 172.21.15.26:0/1020025334 reap_dead start 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f56877fe700 1 -- 172.21.15.26:0/1020025334 reap_dead start 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 shutdown_connections 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 >> 172.21.15.26:0/1020025334 conn(0x7f568806e6e0 msgr2=0x7f56880a31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 shutdown_connections 2024-09-15T03:47:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.366+0000 7f5687fff700 1 -- 172.21.15.26:0/1020025334 wait complete. 2024-09-15T03:47:29.410 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:29.410 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:47:29.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:47:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: get_monmap_and_config 2024-09-15T03:47:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: monmap: 2024-09-15T03:47:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:29.528153+0000 2024-09-15T03:47:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:29.528153+0000 2024-09-15T03:47:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.530 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 1 Processor -- start 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 1 -- start start 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: init 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: _add_conns ranks=[7,6,2,8,0,4,5,3,1] 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient(hunting): picked mon.noname-i con 0x7f4378132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient(hunting): picked mon.noname-d con 0x7f437809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient(hunting): picked mon.noname-b con 0x7f4378138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437804ca10 con 0x7f4378138d40 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809d5d0 con 0x7f437809faa0 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4378133620 con 0x7f4378132b40 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient(hunting): authenticate will time out at 993.957700s 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437db92700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f4378138d40 0x7f437813b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36398/0 (socket says 172.21.15.26:36398) 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437db92700 1 -- 172.21.15.26:0/3222209795 learned_addr learned my addr 172.21.15.26:0/3222209795 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249944817 0 0) 0x7f437804ca10 con 0x7f4378138d40 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): my global_id is 134273 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4364003570 con 0x7f4378138d40 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051325658 0 0) 0x7f437809d5d0 con 0x7f437809faa0 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): my global_id is 4749 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437804ca10 con 0x7f437809faa0 2024-09-15T03:47:29.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948773459 0 0) 0x7f4378133620 con 0x7f4378132b40 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient(hunting): my global_id is 134197 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437809d5d0 con 0x7f4378132b40 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1739132911 0 0) 0x7f4364003570 con 0x7f4378138d40 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 >> v1:172.21.15.122:6792/0 conn(0x7f4378132b40 legacy=0x7f4378132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 >> v1:172.21.15.26:6792/0 conn(0x7f437809faa0 legacy=0x7f437809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 monclient: found mon.noname-b 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780a0cc0 con 0x7f4378138d40 2024-09-15T03:47:29.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 5 monclient: authenticate success, global_id 134273 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43700017b0 con 0x7f4378138d40 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: dump: 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43700019a0 con 0x7f4378138d40 2024-09-15T03:47:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 1 -- 172.21.15.26:0/3222209795 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43700048f0 con 0x7f4378138d40 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f436bfff700 10 monclient: dump: 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.540 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.526+0000 7f437e393700 4 set_mon_vals no callback set 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient: shutdown 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- 172.21.15.26:0/3222209795 >> v1:172.21.15.26:6790/0 conn(0x7f4378138d40 legacy=0x7f437813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- 172.21.15.26:0/3222209795 shutdown_connections 2024-09-15T03:47:29.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- 172.21.15.26:0/3222209795 >> 172.21.15.26:0/3222209795 conn(0x7f437806e370 msgr2=0x7f43780a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- 172.21.15.26:0/3222209795 shutdown_connections 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- 172.21.15.26:0/3222209795 wait complete. 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient: monmap: 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.542 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 Processor -- start 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- start start 2024-09-15T03:47:29.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient: init 2024-09-15T03:47:29.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient: _add_conns ranks=[2,8,7,1,0,6,5,4,3] 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient(hunting): picked mon.noname-b con 0x7f437809faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient(hunting): picked mon.noname-e con 0x7f4378132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient(hunting): picked mon.noname-i con 0x7f4378138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809cfa0 con 0x7f437809faa0 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437809ccc0 con 0x7f4378138d40 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4378134c50 con 0x7f4378132b40 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.530+0000 7f437e393700 10 monclient(hunting): authenticate will time out at 993.963954s 2024-09-15T03:47:29.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437d391700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f437809faa0 0x7f437809a340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36404/0 (socket says 172.21.15.26:36404) 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437d391700 1 -- 172.21.15.26:0/1712263945 learned_addr learned my addr 172.21.15.26:0/1712263945 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818008230 0 0) 0x7f4378134c50 con 0x7f4378132b40 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): my global_id is 134234 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f435c003570 con 0x7f4378132b40 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270384518 0 0) 0x7f437809cfa0 con 0x7f437809faa0 2024-09-15T03:47:29.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): my global_id is 134282 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4378134c50 con 0x7f437809faa0 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964137652 0 0) 0x7f437809ccc0 con 0x7f4378138d40 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): my global_id is 134206 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437809cfa0 con 0x7f4378138d40 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2152641498 0 0) 0x7f435c003570 con 0x7f4378132b40 2024-09-15T03:47:29.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f437809ccc0 con 0x7f4378132b40 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4374001dd0 con 0x7f4378132b40 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): dump: 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.552 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4106543139 0 0) 0x7f4378134c50 con 0x7f437809faa0 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f435c003570 con 0x7f437809faa0 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f436c001a60 con 0x7f437809faa0 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): dump: 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1427408972 0 0) 0x7f437809ccc0 con 0x7f4378132b40 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.122:6792/0 conn(0x7f4378138d40 legacy=0x7f437809b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.26:6790/0 conn(0x7f437809faa0 legacy=0x7f437809a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 monclient: found mon.e 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437809be80 con 0x7f4378132b40 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 5 monclient: authenticate success, global_id 134234 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 10 monclient: _renew_subs 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43781349f0 con 0x7f4378132b40 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 10 monclient: _renew_subs 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4378135630 con 0x7f4378132b40 2024-09-15T03:47:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4374002360 con 0x7f4378132b40 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4374002890 con 0x7f4378132b40 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f436b7fe700 4 set_mon_vals no callback set 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: dump: 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4374002a70 con 0x7f4378132b40 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4358006590 con 0x7f4358002840 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 3911+0+0 (unknown 2439909402 0 0) 0x7f4374041960 con 0x7f4378132b40 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228019300 0 0) 0x7f4358006590 con 0x7f4358002840 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient(hunting): my global_id is 134243 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437809ccc0 con 0x7f4358002840 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f437db92700 10 monclient: get_auth_request con 0x7f435c037d00 auth_method 0 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2709602194 0 0) 0x7f437809ccc0 con 0x7f4358002840 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4358006590 con 0x7f4358002840 2024-09-15T03:47:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f436c0022b0 con 0x7f4358002840 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 10 monclient: dump: 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2841101286 0 0) 0x7f4358006590 con 0x7f4358002840 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.534+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f435800a130 con 0x7f4358002840 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.538+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f436c002840 con 0x7f4358002840 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.538+0000 7f43697fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.538+0000 7f43697fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.538+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.26:6793/0 conn(0x7f4358002840 legacy=0x7f4358004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.538+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.26:6793/0 conn(0x7f4358002840 legacy=0x7f4358004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f437e393700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4358009640 con 0x7f43580090b0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508359794 0 0) 0x7f4358009640 con 0x7f43580090b0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f43697fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f43697fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f43697fa700 10 monclient(hunting): my global_id is 134252 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.542+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4358006590 con 0x7f43580090b0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1118718001 0 0) 0x7f4358006590 con 0x7f43580090b0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4358009640 con 0x7f43580090b0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f436c002ce0 con 0x7f43580090b0 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 10 monclient: dump: 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 822920711 0 0) 0x7f4358009640 con 0x7f43580090b0 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43580098f0 con 0x7f43580090b0 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1159644417) 0x7f436c007cf0 con 0x7f43580090b0 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.26:6793/0 conn(0x7f43580090b0 legacy=0x7f4358008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f43697fa700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.26:6793/0 conn(0x7f43580090b0 legacy=0x7f4358008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:29.566 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.567 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.568 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:29.569 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.570 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.571 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.572 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.575 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.576 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.577 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.578 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.579 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.580 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.582 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:29.583 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:29.584 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.586 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:29.586 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.122:6812/20895 conn(0x7f435c037d00 legacy=0x7f435c03a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 10 monclient: shutdown 2024-09-15T03:47:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 >> v1:172.21.15.26:6793/0 conn(0x7f4378132b40 legacy=0x7f437809aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437db92700 1 -- 172.21.15.26:0/1712263945 reap_dead start 2024-09-15T03:47:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437db92700 1 -- 172.21.15.26:0/1712263945 reap_dead start 2024-09-15T03:47:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 shutdown_connections 2024-09-15T03:47:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 >> 172.21.15.26:0/1712263945 conn(0x7f437806e370 msgr2=0x7f43780669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 shutdown_connections 2024-09-15T03:47:29.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.546+0000 7f437e393700 1 -- 172.21.15.26:0/1712263945 wait complete. 2024-09-15T03:47:29.588 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:29.588 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:47:29.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.698+0000 7fd454a0a700 10 monclient: get_monmap_and_config 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.698+0000 7fd454a0a700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.698+0000 7fd454a0a700 10 monclient: monmap: 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:29.703765+0000 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:29.703765+0000 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.704 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.705 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 Processor -- start 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- start start 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient: init 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient: _add_conns ranks=[1,5,8,2,4,0,6,3,7] 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient(hunting): picked mon.noname-f con 0x7fd45006e530 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient(hunting): picked mon.noname-h con 0x7fd450099240 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient(hunting): picked mon.noname-e con 0x7fd450138d70 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:29.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45004ca10 con 0x7fd45006e530 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45009e310 con 0x7fd450099240 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd450133730 con 0x7fd450138d70 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient(hunting): authenticate will time out at 994.133191s 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44f59e700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fd450138d70 0x7fd45013b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:58148/0 (socket says 172.21.15.26:58148) 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44f59e700 1 -- 172.21.15.26:0/3541844930 learned_addr learned my addr 172.21.15.26:0/3541844930 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776562082 0 0) 0x7fd45004ca10 con 0x7fd45006e530 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): my global_id is 134155 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd438003570 con 0x7fd45006e530 2024-09-15T03:47:29.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891395524 0 0) 0x7fd450133730 con 0x7fd450138d70 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): my global_id is 134261 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd45004ca10 con 0x7fd450138d70 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13817533 0 0) 0x7fd45009e310 con 0x7fd450099240 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient(hunting): my global_id is 4748 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd450133730 con 0x7fd450099240 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 194649209 0 0) 0x7fd438003570 con 0x7fd45006e530 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 >> v1:172.21.15.26:6793/0 conn(0x7fd450138d70 legacy=0x7fd45013b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 >> v1:172.21.15.122:6791/0 conn(0x7fd450099240 legacy=0x7fd450099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 monclient: found mon.noname-f 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450097b00 con 0x7fd45006e530 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd4480017b0 con 0x7fd45006e530 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: dump: 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.717 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 5 monclient: authenticate success, global_id 134155 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd448001a90 con 0x7fd45006e530 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 1 -- 172.21.15.26:0/3541844930 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd448002240 con 0x7fd45006e530 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 4 set_mon_vals no callback set 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd44dd9b700 10 monclient: dump: 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.719 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 10 monclient: shutdown 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- 172.21.15.26:0/3541844930 >> v1:172.21.15.122:6789/0 conn(0x7fd45006e530 legacy=0x7fd450133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- 172.21.15.26:0/3541844930 shutdown_connections 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- 172.21.15.26:0/3541844930 >> 172.21.15.26:0/3541844930 conn(0x7fd450070de0 msgr2=0x7fd4500711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- 172.21.15.26:0/3541844930 shutdown_connections 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.702+0000 7fd454a0a700 1 -- 172.21.15.26:0/3541844930 wait complete. 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: monmap: 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:29.721 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 Processor -- start 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 -- start start 2024-09-15T03:47:29.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: init 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: _add_conns ranks=[6,1,2,0,5,3,7,4,8] 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient(hunting): picked mon.noname-d con 0x7fd45006e530 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient(hunting): picked mon.noname-f con 0x7fd450099240 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient(hunting): picked mon.noname-b con 0x7fd450138d70 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd450138570 con 0x7fd450099240 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45009e790 con 0x7fd450138d70 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45009ea60 con 0x7fd45006e530 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient(hunting): authenticate will time out at 994.137649s 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd44ed9d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd45006e530 0x7fd45012ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41466/0 (socket says 172.21.15.26:41466) 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd44ed9d700 1 -- 172.21.15.26:0/1020212836 learned_addr learned my addr 172.21.15.26:0/1020212836 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060450219 0 0) 0x7fd45009ea60 con 0x7fd45006e530 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): my global_id is 4758 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd43c003570 con 0x7fd45006e530 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416547466 0 0) 0x7fd45009e790 con 0x7fd450138d70 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): my global_id is 134291 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd45009ea60 con 0x7fd450138d70 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911856938 0 0) 0x7fd450138570 con 0x7fd450099240 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): my global_id is 134164 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd45009e790 con 0x7fd450099240 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4183654441 0 0) 0x7fd45009ea60 con 0x7fd450138d70 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd450138570 con 0x7fd450138d70 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd440001c80 con 0x7fd450138d70 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 142461694 0 0) 0x7fd43c003570 con 0x7fd45006e530 2024-09-15T03:47:29.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd45009ea60 con 0x7fd45006e530 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd448001f40 con 0x7fd45006e530 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.727 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 770879753 0 0) 0x7fd450138570 con 0x7fd450138d70 2024-09-15T03:47:29.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.26:6792/0 conn(0x7fd45006e530 legacy=0x7fd45012ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.122:6789/0 conn(0x7fd450099240 legacy=0x7fd45012f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 monclient: found mon.b 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450130ee0 con 0x7fd450138d70 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd436ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 5 monclient: authenticate success, global_id 134291 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: _renew_subs 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4501309d0 con 0x7fd450138d70 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: _renew_subs 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.706+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4501315a0 con 0x7fd450138d70 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd440004bb0 con 0x7fd450138d70 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd440005060 con 0x7fd450138d70 2024-09-15T03:47:29.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd44d59a700 4 set_mon_vals no callback set 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: dump: 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.731 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd454a0a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd440005280 con 0x7fd450138d70 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd430006590 con 0x7fd430002840 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 3911+0+0 (unknown 2439909402 0 0) 0x7fd440043f20 con 0x7fd450138d70 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd44e59c700 10 monclient: get_auth_request con 0x7fd43c037c80 auth_method 0 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479340024 0 0) 0x7fd430006590 con 0x7fd430002840 2024-09-15T03:47:29.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient(hunting): my global_id is 134173 2024-09-15T03:47:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd450138570 con 0x7fd430002840 2024-09-15T03:47:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1371033843 0 0) 0x7fd450138570 con 0x7fd430002840 2024-09-15T03:47:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd430006590 con 0x7fd430002840 2024-09-15T03:47:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd448002850 con 0x7fd430002840 2024-09-15T03:47:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: dump: 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 230431448 0 0) 0x7fd430006590 con 0x7fd430002840 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd43000a130 con 0x7fd430002840 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd448002c30 con 0x7fd430002840 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.122:6789/0 conn(0x7fd430002840 legacy=0x7fd430004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.710+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.122:6789/0 conn(0x7fd430002840 legacy=0x7fd430004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd454a0a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd430009640 con 0x7fd4300090b0 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177836341 0 0) 0x7fd430009640 con 0x7fd4300090b0 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 10 monclient(hunting): my global_id is 134182 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd430006590 con 0x7fd4300090b0 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4268108381 0 0) 0x7fd430006590 con 0x7fd4300090b0 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd430009640 con 0x7fd4300090b0 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd448006000 con 0x7fd4300090b0 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.722+0000 7fd436ffd700 10 monclient: dump: 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4020277444 0 0) 0x7fd430009640 con 0x7fd4300090b0 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd4300098f0 con 0x7fd4300090b0 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 577597715) 0x7fd448008220 con 0x7fd4300090b0 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.122:6789/0 conn(0x7fd4300090b0 legacy=0x7fd430008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd436ffd700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.122:6789/0 conn(0x7fd4300090b0 legacy=0x7fd430008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.740 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:29.741 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.742 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.743 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:29.744 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:29.745 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:29.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.747 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:29.748 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:29.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.754 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.758 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.759 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:29.760 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:29.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.122:6812/20895 conn(0x7fd43c037c80 legacy=0x7fd43c03a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 10 monclient: shutdown 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 >> v1:172.21.15.26:6790/0 conn(0x7fd450138d70 legacy=0x7fd45012fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd44f59e700 1 -- 172.21.15.26:0/1020212836 reap_dead start 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd44f59e700 1 -- 172.21.15.26:0/1020212836 reap_dead start 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 shutdown_connections 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 >> 172.21.15.26:0/1020212836 conn(0x7fd450070de0 msgr2=0x7fd450098a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 shutdown_connections 2024-09-15T03:47:29.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.726+0000 7fd454a0a700 1 -- 172.21.15.26:0/1020212836 wait complete. 2024-09-15T03:47:29.762 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:29.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:47:29.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:47:29.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.874+0000 7ffb1f87b700 10 monclient: get_monmap_and_config 2024-09-15T03:47:29.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.874+0000 7ffb1f87b700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.874+0000 7ffb1f87b700 10 monclient: monmap: 2024-09-15T03:47:29.879 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.879 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.879 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:29.879215+0000 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:29.879215+0000 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.880 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.881 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.874+0000 7ffb1f87b700 1 Processor -- start 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- start start 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient: init 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient: _add_conns ranks=[3,7,2,1,5,4,6,8,0] 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient(hunting): picked mon.noname-g con 0x7ffb180990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient(hunting): picked mon.noname-i con 0x7ffb180a1310 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient(hunting): picked mon.noname-b con 0x7ffb181335a0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1804ca10 con 0x7ffb181335a0 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb180a1b20 con 0x7ffb180990c0 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb180a10b0 con 0x7ffb180a1310 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient(hunting): authenticate will time out at 994.308819s 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1de18700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ffb181335a0 0x7ffb1813afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36412/0 (socket says 172.21.15.26:36412) 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1de18700 1 -- 172.21.15.26:0/4018535557 learned_addr learned my addr 172.21.15.26:0/4018535557 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011768557 0 0) 0x7ffb1804ca10 con 0x7ffb181335a0 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): my global_id is 134300 2024-09-15T03:47:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb00003570 con 0x7ffb181335a0 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144212147 0 0) 0x7ffb180a1b20 con 0x7ffb180990c0 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): my global_id is 4710 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1804ca10 con 0x7ffb180990c0 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425147750 0 0) 0x7ffb180a10b0 con 0x7ffb180a1310 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient(hunting): my global_id is 134215 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb180a1b20 con 0x7ffb180a1310 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969567549 0 0) 0x7ffb00003570 con 0x7ffb181335a0 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 >> v1:172.21.15.122:6792/0 conn(0x7ffb180a1310 legacy=0x7ffb18099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 >> v1:172.21.15.122:6790/0 conn(0x7ffb180990c0 legacy=0x7ffb18132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 monclient: found mon.noname-b 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18099ba0 con 0x7ffb181335a0 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 5 monclient: authenticate success, global_id 134300 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb100017b0 con 0x7ffb181335a0 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: dump: 2024-09-15T03:47:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffb10001a90 con 0x7ffb181335a0 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 1 -- 172.21.15.26:0/4018535557 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb10001c80 con 0x7ffb181335a0 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 4 set_mon_vals no callback set 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb17fff700 10 monclient: dump: 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 10 monclient: shutdown 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- 172.21.15.26:0/4018535557 >> v1:172.21.15.26:6790/0 conn(0x7ffb181335a0 legacy=0x7ffb1813afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- 172.21.15.26:0/4018535557 shutdown_connections 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- 172.21.15.26:0/4018535557 >> 172.21.15.26:0/4018535557 conn(0x7ffb1806e370 msgr2=0x7ffb180a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- 172.21.15.26:0/4018535557 shutdown_connections 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.878+0000 7ffb1f87b700 1 -- 172.21.15.26:0/4018535557 wait complete. 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient: build_initial_monmap 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient: monmap: 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 1 Processor -- start 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 1 -- start start 2024-09-15T03:47:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient: init 2024-09-15T03:47:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient: _add_conns ranks=[2,0,5,8,4,1,6,7,3] 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient(hunting): picked mon.noname-b con 0x7ffb180990c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient(hunting): picked mon.noname-a con 0x7ffb180a1310 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient(hunting): picked mon.noname-h con 0x7ffb181335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb180a1ce0 con 0x7ffb180a1310 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1809a010 con 0x7ffb180990c0 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1d617700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ffb180990c0 0x7ffb18131ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36420/0 (socket says 172.21.15.26:36420) 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1d617700 1 -- 172.21.15.26:0/1644006931 learned_addr learned my addr 172.21.15.26:0/1644006931 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb18142c10 con 0x7ffb181335a0 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb1f87b700 10 monclient(hunting): authenticate will time out at 994.314678s 2024-09-15T03:47:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186633992 0 0) 0x7ffb1809a010 con 0x7ffb180990c0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): my global_id is 134309 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb04003570 con 0x7ffb180990c0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691614279 0 0) 0x7ffb18142c10 con 0x7ffb181335a0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): my global_id is 4757 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1809a010 con 0x7ffb181335a0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3062785740 0 0) 0x7ffb04003570 con 0x7ffb180990c0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb18142c10 con 0x7ffb180990c0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb08001ea0 con 0x7ffb180990c0 2024-09-15T03:47:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): dump: 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814223723 0 0) 0x7ffb180a1ce0 con 0x7ffb180a1310 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient(hunting): my global_id is 4608 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb04003570 con 0x7ffb180a1310 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1753429054 0 0) 0x7ffb18142c10 con 0x7ffb180990c0 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.122:6791/0 conn(0x7ffb181335a0 legacy=0x7ffb1814a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.26:6789/0 conn(0x7ffb180a1310 legacy=0x7ffb18147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.882+0000 7ffb157fa700 1 monclient: found mon.b 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18120160 con 0x7ffb180990c0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 5 monclient: authenticate success, global_id 134309 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 10 monclient: _renew_subs 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb18120860 con 0x7ffb180990c0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 10 monclient: _renew_subs 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb181341e0 con 0x7ffb180990c0 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffb080024c0 con 0x7ffb180990c0 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb080029f0 con 0x7ffb180990c0 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb177fe700 4 set_mon_vals no callback set 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: dump: 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffb08002bd0 con 0x7ffb180990c0 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffaf8006590 con 0x7ffaf8002840 2024-09-15T03:47:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(120..120 src has 1..120) v4 ==== 3911+0+0 (unknown 2439909402 0 0) 0x7ffb08044360 con 0x7ffb180990c0 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb1de18700 10 monclient: get_auth_request con 0x7ffb040375b0 auth_method 0 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565523449 0 0) 0x7ffaf8006590 con 0x7ffaf8002840 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient(hunting): my global_id is 4719 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb18142c10 con 0x7ffaf8002840 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3739748905 0 0) 0x7ffb18142c10 con 0x7ffaf8002840 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffaf8006590 con 0x7ffaf8002840 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb0c0021f0 con 0x7ffaf8002840 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: dump: 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.905 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1947539166 0 0) 0x7ffaf8006590 con 0x7ffaf8002840 2024-09-15T03:47:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffaf800a130 con 0x7ffaf8002840 2024-09-15T03:47:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ffb0c0045e0 con 0x7ffaf8002840 2024-09-15T03:47:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.122:6790/0 conn(0x7ffaf8002840 legacy=0x7ffaf8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.886+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.122:6790/0 conn(0x7ffaf8002840 legacy=0x7ffaf8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:29.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffaf8009640 con 0x7ffaf80090b0 2024-09-15T03:47:29.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234095651 0 0) 0x7ffaf8009640 con 0x7ffaf80090b0 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient(hunting): my global_id is 4728 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf8006590 con 0x7ffaf80090b0 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 431679755 0 0) 0x7ffaf8006590 con 0x7ffaf80090b0 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffaf8009640 con 0x7ffaf80090b0 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb0c002cc0 con 0x7ffaf80090b0 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient: dump: 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1788239994 0 0) 0x7ffaf8009640 con 0x7ffaf80090b0 2024-09-15T03:47:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffaf80098f0 con 0x7ffaf80090b0 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 718745180) 0x7ffb0c004160 con 0x7ffaf80090b0 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.122:6790/0 conn(0x7ffaf80090b0 legacy=0x7ffaf8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb157fa700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.122:6790/0 conn(0x7ffaf80090b0 legacy=0x7ffaf8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:29.911 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:47:29.912 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:29.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:29.914 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:29.915 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.916 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.920 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:29.921 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:29.922 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:29.925 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.928 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.929 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 6 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:29.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:29.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:29.931 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:29.931 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:29.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.122:6812/20895 conn(0x7ffb040375b0 legacy=0x7ffb04039a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 10 monclient: shutdown 2024-09-15T03:47:29.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:29.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 >> v1:172.21.15.26:6790/0 conn(0x7ffb180990c0 legacy=0x7ffb18131ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:29.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1de18700 1 -- 172.21.15.26:0/1644006931 reap_dead start 2024-09-15T03:47:29.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1de18700 1 -- 172.21.15.26:0/1644006931 reap_dead start 2024-09-15T03:47:29.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 shutdown_connections 2024-09-15T03:47:29.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 >> 172.21.15.26:0/1644006931 conn(0x7ffb1806e370 msgr2=0x7ffb180a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:29.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.898+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 shutdown_connections 2024-09-15T03:47:29.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:29.902+0000 7ffb1f87b700 1 -- 172.21.15.26:0/1644006931 wait complete. 2024-09-15T03:47:29.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:29.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:47:29.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:47:30.013 INFO:tasks.rados.rados.0.smithi122.stdout:443: done (1 left) 2024-09-15T03:47:30.013 INFO:tasks.rados.rados.0.smithi122.stdout:445: done (0 left) 2024-09-15T03:47:30.013 INFO:tasks.rados.rados.0.smithi122.stdout:446: snap_create 2024-09-15T03:47:30.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient: get_monmap_and_config 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient: build_initial_monmap 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient: monmap: 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:30.048861+0000 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:30.048861+0000 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 1 Processor -- start 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 1 -- start start 2024-09-15T03:47:30.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient: init 2024-09-15T03:47:30.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient: _add_conns ranks=[0,4,5,7,1,6,8,2,3] 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient(hunting): picked mon.noname-a con 0x7fb170132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient(hunting): picked mon.noname-c con 0x7fb17009faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient(hunting): picked mon.noname-h con 0x7fb170138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17004ca10 con 0x7fb170132b40 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17009d5d0 con 0x7fb17009faa0 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb170133620 con 0x7fb170138d40 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 10 monclient(hunting): authenticate will time out at 994.478558s 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb170132b40 0x7fb170132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47362/0 (socket says 172.21.15.26:47362) 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16ffff700 1 -- 172.21.15.26:0/2765860374 learned_addr learned my addr 172.21.15.26:0/2765860374 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:30.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16f7fe700 1 --1- 172.21.15.26:0/2765860374 >> v1:172.21.15.26:6791/0 conn(0x7fb17009faa0 0x7fb17009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42446/0 (socket says 172.21.15.26:42446) 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727980881 0 0) 0x7fb17004ca10 con 0x7fb170132b40 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): my global_id is 4617 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c003570 con 0x7fb170132b40 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362288261 0 0) 0x7fb17009d5d0 con 0x7fb17009faa0 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): my global_id is 4621 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb17004ca10 con 0x7fb17009faa0 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445384398 0 0) 0x7fb170133620 con 0x7fb170138d40 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient(hunting): my global_id is 4766 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb17009d5d0 con 0x7fb170138d40 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3748333994 0 0) 0x7fb17004ca10 con 0x7fb17009faa0 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 >> v1:172.21.15.122:6791/0 conn(0x7fb170138d40 legacy=0x7fb17013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 >> v1:172.21.15.26:6789/0 conn(0x7fb170132b40 legacy=0x7fb170132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 monclient: found mon.noname-c 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1700a0cc0 con 0x7fb17009faa0 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient: _finish_auth 0 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb158001790 con 0x7fb17009faa0 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 5 monclient: authenticate success, global_id 4621 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb17559b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.046+0000 7fb16effd700 10 monclient: dump: 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb158001a40 con 0x7fb17009faa0 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16effd700 1 -- 172.21.15.26:0/2765860374 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb158004b70 con 0x7fb17009faa0 2024-09-15T03:47:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 4 set_mon_vals no callback set 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16effd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16effd700 10 monclient: dump: 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: shutdown 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- 172.21.15.26:0/2765860374 >> v1:172.21.15.26:6791/0 conn(0x7fb17009faa0 legacy=0x7fb17009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- 172.21.15.26:0/2765860374 shutdown_connections 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- 172.21.15.26:0/2765860374 >> 172.21.15.26:0/2765860374 conn(0x7fb17006e370 msgr2=0x7fb1700a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- 172.21.15.26:0/2765860374 shutdown_connections 2024-09-15T03:47:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- 172.21.15.26:0/2765860374 wait complete. 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: build_initial_monmap 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: monmap: 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:30.067 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 Processor -- start 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- start start 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: init 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient: _add_conns ranks=[4,1,5,7,8,3,6,0,2] 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient(hunting): picked mon.noname-c con 0x7fb17009faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:30.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient(hunting): picked mon.noname-f con 0x7fb170132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient(hunting): picked mon.noname-h con 0x7fb170138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17009cfa0 con 0x7fb170132b40 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17009ccc0 con 0x7fb17009faa0 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb170098a80 con 0x7fb170138d40 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb17559b700 10 monclient(hunting): authenticate will time out at 994.483326s 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16ffff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fb17009faa0 0x7fb170130180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:42448/0 (socket says 172.21.15.26:42448) 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb16ffff700 1 -- 172.21.15.26:0/2850066709 learned_addr learned my addr 172.21.15.26:0/2850066709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:30.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664688248 0 0) 0x7fb17009ccc0 con 0x7fb17009faa0 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb167fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb167fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb167fff700 10 monclient(hunting): my global_id is 4630 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.050+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb150003570 con 0x7fb17009faa0 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437626574 0 0) 0x7fb17009cfa0 con 0x7fb170132b40 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): my global_id is 134191 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb17009ccc0 con 0x7fb170132b40 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1278759036 0 0) 0x7fb150003570 con 0x7fb17009faa0 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb17009cfa0 con 0x7fb17009faa0 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540307390 0 0) 0x7fb170098a80 con 0x7fb170138d40 2024-09-15T03:47:30.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): my global_id is 4775 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb150003570 con 0x7fb170138d40 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb160001820 con 0x7fb17009faa0 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): dump: 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.071 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3396613708 0 0) 0x7fb17009cfa0 con 0x7fb17009faa0 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6791/0 conn(0x7fb170138d40 legacy=0x7fb170130fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6789/0 conn(0x7fb170132b40 legacy=0x7fb170130890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 monclient: found mon.c 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170131b60 con 0x7fb17009faa0 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 5 monclient: authenticate success, global_id 4630 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 10 monclient: _renew_subs 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb170099040 con 0x7fb17009faa0 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 10 monclient: _renew_subs 2024-09-15T03:47:30.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17009c770 con 0x7fb17009faa0 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb160004e60 con 0x7fb17009faa0 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb160001aa0 con 0x7fb17009faa0 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb16e7fc700 4 set_mon_vals no callback set 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: dump: 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:30.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb16000fbf0 con 0x7fb17009faa0 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb154006590 con 0x7fb154002840 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(121..121 src has 1..121) v4 ==== 3911+0+0 (unknown 2772207499 0 0) 0x7fb1600441a0 con 0x7fb17009faa0 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698734183 0 0) 0x7fb154006590 con 0x7fb154002840 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient(hunting): my global_id is 4784 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb17009cfa0 con 0x7fb154002840 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb174d9a700 10 monclient: get_auth_request con 0x7fb150037710 auth_method 0 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1583151776 0 0) 0x7fb17009cfa0 con 0x7fb154002840 2024-09-15T03:47:30.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb154006590 con 0x7fb154002840 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb158004b70 con 0x7fb154002840 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 10 monclient: dump: 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4191990804 0 0) 0x7fb154006590 con 0x7fb154002840 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.054+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb15400a130 con 0x7fb154002840 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.058+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb1580017c0 con 0x7fb154002840 2024-09-15T03:47:30.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.058+0000 7fb167fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:30.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.058+0000 7fb167fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:30.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.058+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6791/0 conn(0x7fb154002840 legacy=0x7fb154004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.058+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6791/0 conn(0x7fb154002840 legacy=0x7fb154004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb17559b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb154007520 con 0x7fb1540068d0 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944831142 0 0) 0x7fb154007520 con 0x7fb1540068d0 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 10 monclient(hunting): my global_id is 4793 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb154006590 con 0x7fb1540068d0 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1806842764 0 0) 0x7fb154006590 con 0x7fb1540068d0 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb154007520 con 0x7fb1540068d0 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb1580055b0 con 0x7fb1540068d0 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 10 monclient: dump: 2024-09-15T03:47:30.081 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.082 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3635075170 0 0) 0x7fb154007520 con 0x7fb1540068d0 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.066+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb154008760 con 0x7fb1540068d0 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 4032407839) 0x7fb1580078e0 con 0x7fb1540068d0 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb167fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb167fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6791/0 conn(0x7fb1540068d0 legacy=0x7fb154006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb167fff700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6791/0 conn(0x7fb1540068d0 legacy=0x7fb154006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:30.084 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:30.085 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:30.086 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:30.087 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:30.088 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.089 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.090 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:30.092 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:30.093 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:30.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.095 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.097 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:30.098 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:30.099 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:30.100 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.101 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.102 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.103 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:30.104 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.122:6812/20895 conn(0x7fb150037710 legacy=0x7fb150039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 10 monclient: shutdown 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 >> v1:172.21.15.26:6791/0 conn(0x7fb17009faa0 legacy=0x7fb170130180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb174d9a700 1 -- 172.21.15.26:0/2850066709 reap_dead start 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb174d9a700 1 -- 172.21.15.26:0/2850066709 reap_dead start 2024-09-15T03:47:30.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 shutdown_connections 2024-09-15T03:47:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 >> 172.21.15.26:0/2850066709 conn(0x7fb17006e370 msgr2=0x7fb1700a4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 shutdown_connections 2024-09-15T03:47:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.070+0000 7fb17559b700 1 -- 172.21.15.26:0/2850066709 wait complete. 2024-09-15T03:47:30.107 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:30.107 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:47:30.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:47:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient: get_monmap_and_config 2024-09-15T03:47:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient: build_initial_monmap 2024-09-15T03:47:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient: monmap: 2024-09-15T03:47:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:30.225085+0000 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:30.225085+0000 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:30.226 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 1 Processor -- start 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 1 -- start start 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient: init 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient: _add_conns ranks=[6,0,8,5,1,4,2,3,7] 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient(hunting): picked mon.noname-d con 0x7fba30063a30 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient(hunting): picked mon.noname-a con 0x7fba300a04b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient(hunting): picked mon.noname-e con 0x7fba30138ac0 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3004ca10 con 0x7fba300a04b0 2024-09-15T03:47:30.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3009bbc0 con 0x7fba30063a30 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba30063e00 con 0x7fba30138ac0 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba36670700 10 monclient(hunting): authenticate will time out at 994.655484s 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba3566e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fba30063a30 0x7fba3009ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41478/0 (socket says 172.21.15.26:41478) 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba34e6d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba300a04b0 0x7fba3009b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47370/0 (socket says 172.21.15.26:47370) 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.222+0000 7fba3566e700 1 -- 172.21.15.26:0/1033661161 learned_addr learned my addr 172.21.15.26:0/1033661161 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537524356 0 0) 0x7fba3004ca10 con 0x7fba300a04b0 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): my global_id is 4626 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba1c003570 con 0x7fba300a04b0 2024-09-15T03:47:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840017148 0 0) 0x7fba3009bbc0 con 0x7fba30063a30 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): my global_id is 4767 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3004ca10 con 0x7fba30063a30 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858780592 0 0) 0x7fba30063e00 con 0x7fba30138ac0 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient(hunting): my global_id is 134270 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3009bbc0 con 0x7fba30138ac0 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615685055 0 0) 0x7fba3009bbc0 con 0x7fba30138ac0 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 >> v1:172.21.15.26:6792/0 conn(0x7fba30063a30 legacy=0x7fba3009ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 >> v1:172.21.15.26:6789/0 conn(0x7fba300a04b0 legacy=0x7fba3009b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 monclient: found mon.noname-e 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300687f0 con 0x7fba30138ac0 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba28001a50 con 0x7fba30138ac0 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: dump: 2024-09-15T03:47:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 5 monclient: authenticate success, global_id 134270 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba28001c70 con 0x7fba30138ac0 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 1 -- 172.21.15.26:0/1033661161 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba280021a0 con 0x7fba30138ac0 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba23fff700 10 monclient: dump: 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.236 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 4 set_mon_vals no callback set 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 10 monclient: shutdown 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 1 -- 172.21.15.26:0/1033661161 >> v1:172.21.15.26:6793/0 conn(0x7fba30138ac0 legacy=0x7fba3013aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 1 -- 172.21.15.26:0/1033661161 shutdown_connections 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 1 -- 172.21.15.26:0/1033661161 >> 172.21.15.26:0/1033661161 conn(0x7fba300a4410 msgr2=0x7fba300a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 1 -- 172.21.15.26:0/1033661161 shutdown_connections 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 1 -- 172.21.15.26:0/1033661161 wait complete. 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 10 monclient: build_initial_monmap 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 10 monclient: monmap: 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:30.238 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:30.239 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.226+0000 7fba36670700 1 Processor -- start 2024-09-15T03:47:30.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 1 -- start start 2024-09-15T03:47:30.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: init 2024-09-15T03:47:30.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:30.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: _add_conns ranks=[7,6,2,3,4,0,5,1,8] 2024-09-15T03:47:30.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient(hunting): picked mon.noname-i con 0x7fba30063a30 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:30.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient(hunting): picked mon.noname-d con 0x7fba300a04b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:30.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient(hunting): picked mon.noname-b con 0x7fba30138ac0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:30.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3009a200 con 0x7fba30138ac0 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba301339e0 con 0x7fba300a04b0 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba30098b00 con 0x7fba30063a30 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient(hunting): authenticate will time out at 994.661198s 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba34e6d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fba300a04b0 0x7fba300983f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:41480/0 (socket says 172.21.15.26:41480) 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba34e6d700 1 -- 172.21.15.26:0/2237342732 learned_addr learned my addr 172.21.15.26:0/2237342732 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227557022 0 0) 0x7fba3009a200 con 0x7fba30138ac0 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): my global_id is 134318 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba14003570 con 0x7fba30138ac0 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872302202 0 0) 0x7fba301339e0 con 0x7fba300a04b0 2024-09-15T03:47:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): my global_id is 4776 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3009a200 con 0x7fba300a04b0 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540263744 0 0) 0x7fba30098b00 con 0x7fba30063a30 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): my global_id is 134224 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba301339e0 con 0x7fba30063a30 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2522126284 0 0) 0x7fba14003570 con 0x7fba30138ac0 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba30098b00 con 0x7fba30138ac0 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba28002120 con 0x7fba30138ac0 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): dump: 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.249 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3569011552 0 0) 0x7fba3009a200 con 0x7fba300a04b0 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba14003570 con 0x7fba300a04b0 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba24001570 con 0x7fba300a04b0 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient(hunting): dump: 2024-09-15T03:47:30.250 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.251 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 981670590 0 0) 0x7fba30098b00 con 0x7fba30138ac0 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.122:6792/0 conn(0x7fba30063a30 legacy=0x7fba30097ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.26:6792/0 conn(0x7fba300a04b0 legacy=0x7fba300983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 monclient: found mon.b 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30099d10 con 0x7fba30138ac0 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:30.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 5 monclient: authenticate success, global_id 134318 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: _renew_subs 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3006e0a0 con 0x7fba30138ac0 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: _renew_subs 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fba301378a0 con 0x7fba30138ac0 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba28002680 con 0x7fba30138ac0 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba28002b30 con 0x7fba30138ac0 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba237fe700 4 set_mon_vals no callback set 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:30.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.230+0000 7fba217fa700 10 monclient: dump: 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.254 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba36670700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fba10006590 con 0x7fba10002840 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fba2800d4f0 con 0x7fba30138ac0 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510436514 0 0) 0x7fba10006590 con 0x7fba10002840 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient(hunting): my global_id is 134233 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba30098b00 con 0x7fba10002840 2024-09-15T03:47:30.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(121..121 src has 1..121) v4 ==== 3911+0+0 (unknown 2772207499 0 0) 0x7fba28041a10 con 0x7fba30138ac0 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba34e6d700 10 monclient: get_auth_request con 0x7fba14037cb0 auth_method 0 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3718540836 0 0) 0x7fba30098b00 con 0x7fba10002840 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba10006590 con 0x7fba10002840 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba2c004b70 con 0x7fba10002840 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient: dump: 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.256 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1317780198 0 0) 0x7fba10006590 con 0x7fba10002840 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba1000a130 con 0x7fba10002840 2024-09-15T03:47:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fba2c005920 con 0x7fba10002840 2024-09-15T03:47:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.122:6792/0 conn(0x7fba10002840 legacy=0x7fba10004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.234+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.122:6792/0 conn(0x7fba10002840 legacy=0x7fba10004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:30.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:30.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fba10007500 con 0x7fba100068b0 2024-09-15T03:47:30.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874883738 0 0) 0x7fba10007500 con 0x7fba100068b0 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient(hunting): my global_id is 134242 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba10006590 con 0x7fba100068b0 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2835410193 0 0) 0x7fba10006590 con 0x7fba100068b0 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba10007500 con 0x7fba100068b0 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba2c0054f0 con 0x7fba100068b0 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient: dump: 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:30.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2802061104 0 0) 0x7fba10007500 con 0x7fba100068b0 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fba10008760 con 0x7fba100068b0 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3594210434) 0x7fba2c007cf0 con 0x7fba100068b0 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.122:6792/0 conn(0x7fba100068b0 legacy=0x7fba10006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba217fa700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.122:6792/0 conn(0x7fba100068b0 legacy=0x7fba10006d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:30.262 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 34, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:30.263 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:30.264 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:30.265 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:30.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:30.267 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.268 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.269 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.271 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.272 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:30.273 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.276 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:30.278 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:30.279 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:30.280 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:30.281 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:30.282 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.122:6812/20895 conn(0x7fba14037cb0 legacy=0x7fba1403a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 10 monclient: shutdown 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 >> v1:172.21.15.26:6790/0 conn(0x7fba30138ac0 legacy=0x7fba3014a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba35e6f700 1 -- 172.21.15.26:0/2237342732 reap_dead start 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba35e6f700 1 -- 172.21.15.26:0/2237342732 reap_dead start 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 shutdown_connections 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 >> 172.21.15.26:0/2237342732 conn(0x7fba300a4410 msgr2=0x7fba300a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 shutdown_connections 2024-09-15T03:47:30.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:30.246+0000 7fba36670700 1 -- 172.21.15.26:0/2237342732 wait complete. 2024-09-15T03:47:30.285 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:47:30.285 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-15T03:47:30.285 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['f'] 2024-09-15T03:47:30.285 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T03:47:30.285 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T03:47:30.285 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T03:47:30.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:47:30.304 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T03:47:30.304 INFO:tasks.mon_thrash:Sending STOP to mon f 2024-09-15T03:47:30.305 INFO:tasks.ceph.mon.f:Sent signal 19 2024-09-15T03:47:30.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:47:45.305 INFO:tasks.mon_thrash:Sending CONT to mon f 2024-09-15T03:47:45.306 INFO:tasks.ceph.mon.f:Sent signal 18 2024-09-15T03:47:45.307 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-15T03:47:45.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:47:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.410+0000 7f8139c97700 10 monclient: get_monmap_and_config 2024-09-15T03:47:45.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.410+0000 7f8139c97700 10 monclient: build_initial_monmap 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.410+0000 7f8139c97700 10 monclient: monmap: 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:45.415481+0000 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:45.415481+0000 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:45.416 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:45.417 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 1 Processor -- start 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 1 -- start start 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient: init 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient: _add_conns ranks=[4,6,1,2,3,7,0,8,5] 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient(hunting): picked mon.noname-c con 0x7f81340a6a80 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient(hunting): picked mon.noname-d con 0x7f813409fb80 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient(hunting): picked mon.noname-f con 0x7f8134138d30 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:45.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813404ca10 con 0x7f8134138d30 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81340a0f50 con 0x7f81340a6a80 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8134133680 con 0x7f813409fb80 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8139c97700 10 monclient(hunting): authenticate will time out at 1009.844954s 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f8138c95700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f81340a6a80 legacy=0x7f8134132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f812bfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f813409fb80 0x7f813409ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:33834/0 (socket says 172.21.15.26:33834) 2024-09-15T03:47:45.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.414+0000 7f812bfff700 1 -- 172.21.15.26:0/2464616597 learned_addr learned my addr 172.21.15.26:0/2464616597 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:45.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:45.618+0000 7f8138c95700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6791/0 conn(0x7f81340a6a80 legacy=0x7f8134132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:46.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:46.022+0000 7f8138c95700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6791/0 conn(0x7f81340a6a80 legacy=0x7f8134132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:46.826+0000 7f8138c95700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6791/0 conn(0x7f81340a6a80 legacy=0x7f8134132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient(hunting): tick 2024-09-15T03:47:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 monclient(hunting): continuing hunt 2024-09-15T03:47:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:47:48.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6792/0 conn(0x7f813409fb80 legacy=0x7f813409ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6791/0 conn(0x7f81340a6a80 legacy=0x7f8134132f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.122:6789/0 conn(0x7f8134138d30 legacy=0x7f813413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient: _add_conns ranks=[5,2,8,0,7,3] 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient(hunting): picked mon.noname-h con 0x7f8120003810 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient(hunting): picked mon.noname-b con 0x7f8120007490 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient(hunting): picked mon.noname-e con 0x7f812000b210 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81340a6e50 con 0x7f8120007490 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81340a0f50 con 0x7f8120003810 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8120011020 con 0x7f812000b210 2024-09-15T03:47:48.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:48.414+0000 7f812a7fc700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060709029 0 0) 0x7f81340a0f50 con 0x7f8120003810 2024-09-15T03:47:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): my global_id is 4802 2024-09-15T03:47:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c003570 con 0x7f8120003810 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944793217 0 0) 0x7f81340a6e50 con 0x7f8120007490 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): my global_id is 134327 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81340a0f50 con 0x7f8120007490 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485479569 0 0) 0x7f8120011020 con 0x7f812000b210 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient(hunting): my global_id is 134279 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81340a6e50 con 0x7f812000b210 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401937709 0 0) 0x7f811c003570 con 0x7f8120003810 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6793/0 conn(0x7f812000b210 legacy=0x7f812000d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.26:6790/0 conn(0x7f8120007490 legacy=0x7f8120009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 monclient: found mon.noname-h 2024-09-15T03:47:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f8139496700 1 -- 172.21.15.26:0/2464616597 reap_dead start 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120012270 con 0x7f8120003810 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8130004340 con 0x7f8120003810 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f8139c97700 5 monclient: authenticate success, global_id 4802 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f8139c97700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: dump: 2024-09-15T03:47:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f8139c97700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8130004560 con 0x7f8120003810 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 1 -- 172.21.15.26:0/2464616597 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8130004b10 con 0x7f8120003810 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f8139c97700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f8139c97700 4 set_mon_vals no callback set 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.318+0000 7f812b7fe700 10 monclient: dump: 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.329 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.330 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 10 monclient: shutdown 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 -- 172.21.15.26:0/2464616597 >> v1:172.21.15.122:6791/0 conn(0x7f8120003810 legacy=0x7f8120005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 -- 172.21.15.26:0/2464616597 shutdown_connections 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 -- 172.21.15.26:0/2464616597 >> 172.21.15.26:0/2464616597 conn(0x7f813406e380 msgr2=0x7f81340a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 -- 172.21.15.26:0/2464616597 shutdown_connections 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 -- 172.21.15.26:0/2464616597 wait complete. 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 10 monclient: build_initial_monmap 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 10 monclient: monmap: 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:50.335 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 Processor -- start 2024-09-15T03:47:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 1 -- start start 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.322+0000 7f8139c97700 10 monclient: init 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient: _add_conns ranks=[6,3,1,0,4,7,8,5,2] 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient(hunting): picked mon.noname-d con 0x7f8120003810 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient(hunting): picked mon.noname-g con 0x7f813409a0f0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient(hunting): picked mon.noname-f con 0x7f81341481d0 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81340a1920 con 0x7f81341481d0 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813409cfc0 con 0x7f813409a0f0 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813409abd0 con 0x7f8120003810 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient(hunting): authenticate will time out at 1014.756740s 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8138c95700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8120003810 0x7f81340999e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44678/0 (socket says 172.21.15.26:44678) 2024-09-15T03:47:50.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8138c95700 1 -- 172.21.15.26:0/2807530231 learned_addr learned my addr 172.21.15.26:0/2807530231 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339592456 0 0) 0x7f8120012270 con 0x7f8120003810 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): my global_id is 4785 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8120010750 con 0x7f8120003810 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 429666895 0 0) 0x7f8120010750 con 0x7f8120003810 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8120012270 con 0x7f8120003810 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8130004a10 con 0x7f8120003810 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.338 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2968087987 0 0) 0x7f8120012270 con 0x7f8120003810 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 >> v1:172.21.15.122:6790/0 conn(0x7f813409a0f0 legacy=0x7f813409a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 >> v1:172.21.15.122:6789/0 conn(0x7f81341481d0 legacy=0x7f813414a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 monclient: found mon.d 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813409b2b0 con 0x7f8120003810 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 5 monclient: authenticate success, global_id 4785 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient: _renew_subs 2024-09-15T03:47:50.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813411c900 con 0x7f8120003810 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient: _renew_subs 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8134132940 con 0x7f8120003810 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8130003d80 con 0x7f8120003810 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81300053b0 con 0x7f8120003810 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f812a7fc700 4 set_mon_vals no callback set 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 10 monclient: dump: 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.341 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.342 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.326+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f813000fdf0 con 0x7f8120003810 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.330+0000 7f8139c97700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.330+0000 7f8139c97700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.330+0000 7f8139c97700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.330+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81140050b0 con 0x7f8120003810 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.330+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(121..121 src has 1..121) v4 ==== 3911+0+0 (unknown 2772207499 0 0) 0x7f81300440e0 con 0x7f8120003810 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.330+0000 7f812bfff700 10 monclient: get_auth_request con 0x7f8120038730 auth_method 0 2024-09-15T03:47:50.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.334+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f8130015030 con 0x7f8120003810 2024-09-15T03:47:50.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.334+0000 7f8128ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.334+0000 7f8128ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:50.392 INFO:tasks.rados.rados.0.smithi122.stdout:446: done (0 left) 2024-09-15T03:47:50.392 INFO:tasks.rados.rados.0.smithi122.stdout:447: read oid 44 snap -1 2024-09-15T03:47:50.392 INFO:tasks.rados.rados.0.smithi122.stdout:447: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:50.392 INFO:tasks.rados.rados.0.smithi122.stdout:448: copy_from oid 24 from oid 33 current snap is 52 2024-09-15T03:47:50.392 INFO:tasks.rados.rados.0.smithi122.stdout:449: snap_create 2024-09-15T03:47:50.403 INFO:tasks.rados.rados.0.smithi122.stdout:447: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:50.493 INFO:tasks.rados.rados.0.smithi122.stdout:448: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:47:50.493 INFO:tasks.rados.rados.0.smithi122.stdout:448: finishing copy_from to smithi12223906-24 2024-09-15T03:47:50.493 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 119 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:47:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8139c97700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:47:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8139c97700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:47:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8139c97700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f81140058e0 con 0x7f8120003810 2024-09-15T03:47:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8128ff9700 1 -- 172.21.15.26:0/2807530231 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2694 (unknown 254746657 0 1761116805) 0x7f8130014a30 con 0x7f8120003810 2024-09-15T03:47:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8128ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:47:50.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.578+0000 7f8128ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:50.582 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":40,"quorum":[0,1,2,3,5,6,7,8],"quorum_names":["a","f","b","g","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:47:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 >> v1:172.21.15.122:6812/20895 conn(0x7f8120038730 legacy=0x7f812003abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 10 monclient: shutdown 2024-09-15T03:47:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 >> v1:172.21.15.26:6792/0 conn(0x7f8120003810 legacy=0x7f81340999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 shutdown_connections 2024-09-15T03:47:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 >> 172.21.15.26:0/2807530231 conn(0x7f813406e380 msgr2=0x7f81340a4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 shutdown_connections 2024-09-15T03:47:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.582+0000 7f8139c97700 1 -- 172.21.15.26:0/2807530231 wait complete. 2024-09-15T03:47:50.596 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-15T03:47:50.596 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-15T03:47:50.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:47:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient: get_monmap_and_config 2024-09-15T03:47:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient: build_initial_monmap 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient: monmap: 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:50.702469+0000 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:50.702469+0000 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:50.703 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:50.704 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:50.704 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 1 Processor -- start 2024-09-15T03:47:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 1 -- start start 2024-09-15T03:47:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient: init 2024-09-15T03:47:50.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient: _add_conns ranks=[5,0,4,7,3,8,6,2,1] 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient(hunting): picked mon.noname-h con 0x7f78ac0a6880 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient(hunting): picked mon.noname-a con 0x7f78ac09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient(hunting): picked mon.noname-c con 0x7f78ac138d30 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac04ca10 con 0x7f78ac09faa0 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0a0580 con 0x7f78ac138d30 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac133780 con 0x7f78ac0a6880 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b18d9700 10 monclient(hunting): authenticate will time out at 1015.131975s 2024-09-15T03:47:50.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.698+0000 7f78b10d8700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f78ac138d30 legacy=0x7f78ac13b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78abfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f78ac09faa0 0x7f78ac09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51928/0 (socket says 172.21.15.26:51928) 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78abfff700 1 -- 172.21.15.26:0/4009402786 learned_addr learned my addr 172.21.15.26:0/4009402786 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695105079 0 0) 0x7f78ac04ca10 con 0x7f78ac09faa0 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient(hunting): my global_id is 4635 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f788c003570 con 0x7f78ac09faa0 2024-09-15T03:47:50.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655924268 0 0) 0x7f78ac133780 con 0x7f78ac0a6880 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient(hunting): my global_id is 4811 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac04ca10 con 0x7f78ac0a6880 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1219994717 0 0) 0x7f788c003570 con 0x7f78ac09faa0 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 >> v1:172.21.15.122:6791/0 conn(0x7f78ac0a6880 legacy=0x7f78ac133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 >> v1:172.21.15.26:6791/0 conn(0x7f78ac138d30 legacy=0x7f78ac13b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 monclient: found mon.noname-a 2024-09-15T03:47:50.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0a6c50 con 0x7f78ac09faa0 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f78940017b0 con 0x7f78ac09faa0 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 5 monclient: authenticate success, global_id 4635 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:50.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: dump: 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.712 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7894001a90 con 0x7f78ac09faa0 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:50.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 4 set_mon_vals no callback set 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 1 -- 172.21.15.26:0/4009402786 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7894004c30 con 0x7f78ac09faa0 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78ab7fe700 10 monclient: dump: 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.714 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 10 monclient: shutdown 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 1 -- 172.21.15.26:0/4009402786 >> v1:172.21.15.26:6789/0 conn(0x7f78ac09faa0 legacy=0x7f78ac09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 1 -- 172.21.15.26:0/4009402786 shutdown_connections 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 1 -- 172.21.15.26:0/4009402786 >> 172.21.15.26:0/4009402786 conn(0x7f78ac06e2a0 msgr2=0x7f78ac0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:50.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 1 -- 172.21.15.26:0/4009402786 shutdown_connections 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 1 -- 172.21.15.26:0/4009402786 wait complete. 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 10 monclient: build_initial_monmap 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.702+0000 7f78b18d9700 10 monclient: monmap: 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:50.716 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:50.717 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:50.717 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 Processor -- start 2024-09-15T03:47:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 -- start start 2024-09-15T03:47:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: init 2024-09-15T03:47:50.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: _add_conns ranks=[8,4,6,7,0,1,3,5,2] 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient(hunting): picked mon.noname-e con 0x7f78ac09faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient(hunting): picked mon.noname-c con 0x7f78ac0a6880 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient(hunting): picked mon.noname-d con 0x7f78ac138d30 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0a18c0 con 0x7f78ac0a6880 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac09cf10 con 0x7f78ac138d30 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac09b3d0 con 0x7f78ac09faa0 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78abfff700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f78ac0a6880 legacy=0x7f78ac099a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient(hunting): authenticate will time out at 1015.137265s 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b08d7700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f78ac09faa0 0x7f78ac099310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38846/0 (socket says 172.21.15.26:38846) 2024-09-15T03:47:50.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b08d7700 1 -- 172.21.15.26:0/4248508950 learned_addr learned my addr 172.21.15.26:0/4248508950 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575643060 0 0) 0x7f78ac09b3d0 con 0x7f78ac09faa0 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): my global_id is 134288 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7890003570 con 0x7f78ac09faa0 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292496338 0 0) 0x7f78ac09cf10 con 0x7f78ac138d30 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): my global_id is 4794 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac09b3d0 con 0x7f78ac138d30 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1814021078 0 0) 0x7f78ac09b3d0 con 0x7f78ac138d30 2024-09-15T03:47:50.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78ac09cf10 con 0x7f78ac138d30 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f78a4004530 con 0x7f78ac138d30 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.726 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3968953542 0 0) 0x7f7890003570 con 0x7f78ac09faa0 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78ac09b3d0 con 0x7f78ac09faa0 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f789c002a70 con 0x7f78ac09faa0 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 815445823 0 0) 0x7f78ac09cf10 con 0x7f78ac138d30 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6793/0 conn(0x7f78ac09faa0 legacy=0x7f78ac099310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6791/0 conn(0x7f78ac0a6880 legacy=0x7f78ac099a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 monclient: found mon.d 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac09a130 con 0x7f78ac138d30 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 5 monclient: authenticate success, global_id 4794 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: _renew_subs 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac11ca60 con 0x7f78ac138d30 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: _renew_subs 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac09b780 con 0x7f78ac138d30 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f78a4004750 con 0x7f78ac138d30 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f78a4004c80 con 0x7f78ac138d30 2024-09-15T03:47:50.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78aaffd700 4 set_mon_vals no callback set 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 10 monclient: dump: 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.731 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f78a4004de0 con 0x7f78ac138d30 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.706+0000 7f78b18d9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 3911+0+0 (unknown 4190846776 0 0) 0x7f78a4044000 con 0x7f78ac138d30 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f78ac04bbf0 con 0x7f78ac02ae80 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78b08d7700 10 monclient: get_auth_request con 0x7f7890037d30 auth_method 0 2024-09-15T03:47:50.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744399408 0 0) 0x7f78ac04bbf0 con 0x7f78ac02ae80 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient(hunting): my global_id is 4644 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac09cf10 con 0x7f78ac02ae80 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1935800632 0 0) 0x7f78ac09cf10 con 0x7f78ac02ae80 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78ac04bbf0 con 0x7f78ac02ae80 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7894004c30 con 0x7f78ac02ae80 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient: dump: 2024-09-15T03:47:50.733 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.734 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2849243739 0 0) 0x7f78ac04bbf0 con 0x7f78ac02ae80 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78ac066de0 con 0x7f78ac02ae80 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7894001630 con 0x7f78ac02ae80 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6789/0 conn(0x7f78ac02ae80 legacy=0x7f78ac05e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.710+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6789/0 conn(0x7f78ac02ae80 legacy=0x7f78ac05e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f78ac02b250 con 0x7f78ac1315f0 2024-09-15T03:47:50.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193216837 0 0) 0x7f78ac02b250 con 0x7f78ac1315f0 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient(hunting): my global_id is 4653 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ac04bbf0 con 0x7f78ac1315f0 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1649671442 0 0) 0x7f78ac04bbf0 con 0x7f78ac1315f0 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78ac02b250 con 0x7f78ac1315f0 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7894005530 con 0x7f78ac1315f0 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient: dump: 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.736 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1330957819 0 0) 0x7f78ac02b250 con 0x7f78ac1315f0 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f78ac137040 con 0x7f78ac1315f0 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 2812865989) 0x7f7894007770 con 0x7f78ac1315f0 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6789/0 conn(0x7f78ac1315f0 legacy=0x7f78ac136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78a8ff9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6789/0 conn(0x7f78ac1315f0 legacy=0x7f78ac136930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:50.738 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:50.739 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.740 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:50.742 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:50.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:50.744 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:50.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.746 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:50.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:50.750 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.751 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:50.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:50.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:47:50.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:50.758 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.122:6812/20895 conn(0x7f7890037d30 legacy=0x7f789003a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 10 monclient: shutdown 2024-09-15T03:47:50.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 >> v1:172.21.15.26:6792/0 conn(0x7f78ac138d30 legacy=0x7f78ac14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b10d8700 1 -- 172.21.15.26:0/4248508950 reap_dead start 2024-09-15T03:47:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 shutdown_connections 2024-09-15T03:47:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 >> 172.21.15.26:0/4248508950 conn(0x7f78ac06e2a0 msgr2=0x7f78ac0a4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 shutdown_connections 2024-09-15T03:47:50.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.718+0000 7f78b18d9700 1 -- 172.21.15.26:0/4248508950 wait complete. 2024-09-15T03:47:50.760 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:50.760 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-15T03:47:50.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:47:50.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:50.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient: build_initial_monmap 2024-09-15T03:47:50.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient: monmap: 2024-09-15T03:47:50.877 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:50.877 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.878 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:50.876353+0000 2024-09-15T03:47:50.878 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:50.876353+0000 2024-09-15T03:47:50.878 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:50.878 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:50.879 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:50.880 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:50.881 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 1 Processor -- start 2024-09-15T03:47:50.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 1 -- start start 2024-09-15T03:47:50.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient: init 2024-09-15T03:47:50.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:50.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient: _add_conns ranks=[0,4,5,7,6,1,8,3,2] 2024-09-15T03:47:50.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient(hunting): picked mon.noname-a con 0x7f61480a48e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient(hunting): picked mon.noname-c con 0x7f614809aca0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient(hunting): picked mon.noname-h con 0x7f614809a5e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614804ca10 con 0x7f61480a48e0 2024-09-15T03:47:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61480696a0 con 0x7f614809aca0 2024-09-15T03:47:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6148068d20 con 0x7f614809a5e0 2024-09-15T03:47:50.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 10 monclient(hunting): authenticate will time out at 1015.306084s 2024-09-15T03:47:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f6145d9b700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f614809aca0 legacy=0x7f6148137c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614659c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61480a48e0 0x7f6148099ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51954/0 (socket says 172.21.15.26:51954) 2024-09-15T03:47:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614659c700 1 -- 172.21.15.26:0/2822567504 learned_addr learned my addr 172.21.15.26:0/2822567504 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134539385 0 0) 0x7f614804ca10 con 0x7f61480a48e0 2024-09-15T03:47:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient(hunting): my global_id is 4662 2024-09-15T03:47:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6134003570 con 0x7f61480a48e0 2024-09-15T03:47:50.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480210690 0 0) 0x7f6148068d20 con 0x7f614809a5e0 2024-09-15T03:47:50.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient(hunting): my global_id is 4820 2024-09-15T03:47:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614804ca10 con 0x7f614809a5e0 2024-09-15T03:47:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3018925408 0 0) 0x7f6134003570 con 0x7f61480a48e0 2024-09-15T03:47:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 >> v1:172.21.15.122:6791/0 conn(0x7f614809a5e0 legacy=0x7f614813b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 >> v1:172.21.15.26:6791/0 conn(0x7f614809aca0 legacy=0x7f6148137c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:47:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient: _finish_hunting 0 2024-09-15T03:47:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 monclient: found mon.noname-a 2024-09-15T03:47:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:47:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148067640 con 0x7f61480a48e0 2024-09-15T03:47:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient: _finish_auth 0 2024-09-15T03:47:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61380017b0 con 0x7f61480a48e0 2024-09-15T03:47:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 5 monclient: authenticate success, global_id 4662 2024-09-15T03:47:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.874+0000 7f614559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 10 monclient: dump: 2024-09-15T03:47:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.919 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6138001a20 con 0x7f61480a48e0 2024-09-15T03:47:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 1 -- 172.21.15.26:0/2822567504 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6138004c30 con 0x7f61480a48e0 2024-09-15T03:47:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 4 set_mon_vals no callback set 2024-09-15T03:47:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614559a700 10 monclient: dump: 2024-09-15T03:47:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.925 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 10 monclient: shutdown 2024-09-15T03:47:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 -- 172.21.15.26:0/2822567504 >> v1:172.21.15.26:6789/0 conn(0x7f61480a48e0 legacy=0x7f6148099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 -- 172.21.15.26:0/2822567504 shutdown_connections 2024-09-15T03:47:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 -- 172.21.15.26:0/2822567504 >> 172.21.15.26:0/2822567504 conn(0x7f61480638b0 msgr2=0x7f61480a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 -- 172.21.15.26:0/2822567504 shutdown_connections 2024-09-15T03:47:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 -- 172.21.15.26:0/2822567504 wait complete. 2024-09-15T03:47:50.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 10 monclient: build_initial_monmap 2024-09-15T03:47:50.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 10 monclient: monmap: 2024-09-15T03:47:50.927 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:50.927 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:50.927 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:50.929 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:50.930 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:50.930 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:50.930 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 Processor -- start 2024-09-15T03:47:50.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 1 -- start start 2024-09-15T03:47:50.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.878+0000 7f614759e700 10 monclient: init 2024-09-15T03:47:50.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:50.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient: _add_conns ranks=[7,6,5,3,8,4,1,0,2] 2024-09-15T03:47:50.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient(hunting): picked mon.noname-i con 0x7f614809a5e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:50.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient(hunting): picked mon.noname-d con 0x7f614809aca0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:50.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient(hunting): picked mon.noname-h con 0x7f61480a48e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:50.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6148122bf0 con 0x7f61480a48e0 2024-09-15T03:47:50.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614813df80 con 0x7f614809aca0 2024-09-15T03:47:50.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614813ec40 con 0x7f614809a5e0 2024-09-15T03:47:50.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient(hunting): authenticate will time out at 1015.313004s 2024-09-15T03:47:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614659c700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f614809a5e0 0x7f614813d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:50378/0 (socket says 172.21.15.26:50378) 2024-09-15T03:47:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f6145d9b700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f614809aca0 0x7f6148146ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44700/0 (socket says 172.21.15.26:44700) 2024-09-15T03:47:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614659c700 1 -- 172.21.15.26:0/2073378702 learned_addr learned my addr 172.21.15.26:0/2073378702 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:50.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472952947 0 0) 0x7f614813df80 con 0x7f614809aca0 2024-09-15T03:47:50.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): my global_id is 4803 2024-09-15T03:47:50.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6128003570 con 0x7f614809aca0 2024-09-15T03:47:50.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187223715 0 0) 0x7f6148122bf0 con 0x7f61480a48e0 2024-09-15T03:47:50.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): my global_id is 4829 2024-09-15T03:47:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614813df80 con 0x7f61480a48e0 2024-09-15T03:47:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339398448 0 0) 0x7f614813ec40 con 0x7f614809a5e0 2024-09-15T03:47:50.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): my global_id is 134251 2024-09-15T03:47:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6148122bf0 con 0x7f614809a5e0 2024-09-15T03:47:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2611093921 0 0) 0x7f6128003570 con 0x7f614809aca0 2024-09-15T03:47:50.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f614813ec40 con 0x7f614809aca0 2024-09-15T03:47:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61300042c0 con 0x7f614809aca0 2024-09-15T03:47:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:50.937 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.938 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.938 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.938 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.938 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.938 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.938 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.939 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.940 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 419196052 0 0) 0x7f614813df80 con 0x7f61480a48e0 2024-09-15T03:47:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6128003570 con 0x7f61480a48e0 2024-09-15T03:47:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61400019c0 con 0x7f61480a48e0 2024-09-15T03:47:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:50.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient(hunting): dump: 2024-09-15T03:47:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.942 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.943 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.945 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3057682469 0 0) 0x7f614813ec40 con 0x7f614809aca0 2024-09-15T03:47:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.122:6792/0 conn(0x7f614809a5e0 legacy=0x7f614813d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.122:6791/0 conn(0x7f61480a48e0 legacy=0x7f614814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 monclient: found mon.d 2024-09-15T03:47:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148122910 con 0x7f614809aca0 2024-09-15T03:47:50.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 5 monclient: authenticate success, global_id 4803 2024-09-15T03:47:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient: _renew_subs 2024-09-15T03:47:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6148122ed0 con 0x7f614809aca0 2024-09-15T03:47:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient: _renew_subs 2024-09-15T03:47:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f614813e390 con 0x7f614809aca0 2024-09-15T03:47:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f61300045a0 con 0x7f614809aca0 2024-09-15T03:47:50.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6130004ad0 con 0x7f614809aca0 2024-09-15T03:47:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f6144d99700 4 set_mon_vals no callback set 2024-09-15T03:47:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f613e7fc700 10 monclient: dump: 2024-09-15T03:47:50.949 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.950 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.951 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.953 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.882+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f612c006590 con 0x7f612c002840 2024-09-15T03:47:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6130004cb0 con 0x7f614809aca0 2024-09-15T03:47:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025403619 0 0) 0x7f612c006590 con 0x7f612c002840 2024-09-15T03:47:50.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient(hunting): my global_id is 134336 2024-09-15T03:47:50.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f614813ec40 con 0x7f612c002840 2024-09-15T03:47:50.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 3911+0+0 (unknown 4190846776 0 0) 0x7f6130043dd0 con 0x7f614809aca0 2024-09-15T03:47:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2128265920 0 0) 0x7f614813ec40 con 0x7f612c002840 2024-09-15T03:47:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f6146d9d700 10 monclient: get_auth_request con 0x7f6128037d20 auth_method 0 2024-09-15T03:47:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f612c006590 con 0x7f612c002840 2024-09-15T03:47:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6138001770 con 0x7f612c002840 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient: dump: 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 613007311 0 0) 0x7f612c006590 con 0x7f612c002840 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f612c00a130 con 0x7f612c002840 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f61380068d0 con 0x7f612c002840 2024-09-15T03:47:50.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.26:6790/0 conn(0x7f612c002840 legacy=0x7f612c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.886+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.26:6790/0 conn(0x7f612c002840 legacy=0x7f612c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f612c007520 con 0x7f612c0068d0 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27137182 0 0) 0x7f612c007520 con 0x7f612c0068d0 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:50.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient(hunting): my global_id is 134345 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f612c006590 con 0x7f612c0068d0 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2418436730 0 0) 0x7f612c006590 con 0x7f612c0068d0 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f612c007520 con 0x7f612c0068d0 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6138005090 con 0x7f612c0068d0 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient: dump: 2024-09-15T03:47:50.959 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:50.960 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:50.961 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:50.961 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:50.961 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:50.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3204658711 0 0) 0x7f612c007520 con 0x7f612c0068d0 2024-09-15T03:47:50.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f612c008760 con 0x7f612c0068d0 2024-09-15T03:47:50.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1567729687) 0x7f6138006540 con 0x7f612c0068d0 2024-09-15T03:47:50.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:50.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:50.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.26:6790/0 conn(0x7f612c0068d0 legacy=0x7f612c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f613e7fc700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.26:6790/0 conn(0x7f612c0068d0 legacy=0x7f612c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:50.963 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:50.985 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.122:6812/20895 conn(0x7f6128037d20 legacy=0x7f612803a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 10 monclient: shutdown 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 >> v1:172.21.15.26:6792/0 conn(0x7f614809aca0 legacy=0x7f6148146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f6146d9d700 1 -- 172.21.15.26:0/2073378702 reap_dead start 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f6146d9d700 1 -- 172.21.15.26:0/2073378702 reap_dead start 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 shutdown_connections 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.898+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 >> 172.21.15.26:0/2073378702 conn(0x7f61480638b0 msgr2=0x7f6148099df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.902+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 shutdown_connections 2024-09-15T03:47:50.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:50.902+0000 7f614759e700 1 -- 172.21.15.26:0/2073378702 wait complete. 2024-09-15T03:47:50.988 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:50.988 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-15T03:47:50.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:47:51.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient: get_monmap_and_config 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient: monmap: 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:51.101842+0000 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:51.101842+0000 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.113 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 1 Processor -- start 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 1 -- start start 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient: init 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient: _add_conns ranks=[3,7,0,4,2,5,6,1,8] 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient(hunting): picked mon.noname-g con 0x7f1f700990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient(hunting): picked mon.noname-i con 0x7f1f700a1310 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient(hunting): picked mon.noname-a con 0x7f1f701335a0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7004ca10 con 0x7f1f701335a0 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f700a1b20 con 0x7f1f700990c0 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f700a10b0 con 0x7f1f700a1310 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.098+0000 7f1f77836700 10 monclient(hunting): authenticate will time out at 1015.531984s 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f75dd3700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f701335a0 0x7f1f7013afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51964/0 (socket says 172.21.15.26:51964) 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f75dd3700 1 -- 172.21.15.26:0/1850510636 learned_addr learned my addr 172.21.15.26:0/1850510636 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505536535 0 0) 0x7f1f7004ca10 con 0x7f1f701335a0 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient(hunting): my global_id is 4671 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f5c003570 con 0x7f1f701335a0 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583694237 0 0) 0x7f1f5c003570 con 0x7f1f701335a0 2024-09-15T03:47:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 >> v1:172.21.15.122:6792/0 conn(0x7f1f700a1310 legacy=0x7f1f70099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 >> v1:172.21.15.122:6790/0 conn(0x7f1f700990c0 legacy=0x7f1f70132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 monclient: found mon.noname-a 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70099ba0 con 0x7f1f701335a0 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1f68001b30 con 0x7f1f701335a0 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 5 monclient: authenticate success, global_id 4671 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: dump: 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1f68001e50 con 0x7f1f701335a0 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 1 -- 172.21.15.26:0/1850510636 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1f68004fb0 con 0x7f1f701335a0 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 4 set_mon_vals no callback set 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f67fff700 10 monclient: dump: 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: shutdown 2024-09-15T03:47:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- 172.21.15.26:0/1850510636 >> v1:172.21.15.26:6789/0 conn(0x7f1f701335a0 legacy=0x7f1f7013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- 172.21.15.26:0/1850510636 shutdown_connections 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- 172.21.15.26:0/1850510636 >> 172.21.15.26:0/1850510636 conn(0x7f1f7006e370 msgr2=0x7f1f700a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- 172.21.15.26:0/1850510636 shutdown_connections 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- 172.21.15.26:0/1850510636 wait complete. 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: monmap: 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 Processor -- start 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- start start 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: init 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient: _add_conns ranks=[0,1,5,8,6,2,7,3,4] 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient(hunting): picked mon.noname-a con 0x7f1f700990c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient(hunting): picked mon.noname-f con 0x7f1f700a1310 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient(hunting): picked mon.noname-h con 0x7f1f701335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f700a1ce0 con 0x7f1f700990c0 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7009a010 con 0x7f1f700a1310 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7009f680 con 0x7f1f701335a0 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f77836700 10 monclient(hunting): authenticate will time out at 1015.535686s 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f755d2700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f700990c0 0x7f1f7009e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51974/0 (socket says 172.21.15.26:51974) 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.102+0000 7f1f755d2700 1 -- 172.21.15.26:0/1900125617 learned_addr learned my addr 172.21.15.26:0/1900125617 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823247671 0 0) 0x7f1f700a1ce0 con 0x7f1f700990c0 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): my global_id is 4680 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f54003570 con 0x7f1f700990c0 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245350429 0 0) 0x7f1f7009f680 con 0x7f1f701335a0 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): my global_id is 4838 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f700a1ce0 con 0x7f1f701335a0 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839115737 0 0) 0x7f1f7009a010 con 0x7f1f700a1310 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): my global_id is 134200 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7009f680 con 0x7f1f700a1310 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1565520426 0 0) 0x7f1f54003570 con 0x7f1f700990c0 2024-09-15T03:47:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f7009a010 con 0x7f1f700990c0 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1f6c001a50 con 0x7f1f700990c0 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): dump: 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2480695674 0 0) 0x7f1f7009a010 con 0x7f1f700990c0 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.122:6791/0 conn(0x7f1f701335a0 legacy=0x7f1f7014a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.122:6789/0 conn(0x7f1f700a1310 legacy=0x7f1f7009ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 monclient: found mon.a 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700a03c0 con 0x7f1f700990c0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 5 monclient: authenticate success, global_id 4680 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 10 monclient: _renew_subs 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7012d4c0 con 0x7f1f700990c0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 10 monclient: _renew_subs 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f70132960 con 0x7f1f700990c0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1f6c002270 con 0x7f1f700990c0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1f6c0027a0 con 0x7f1f700990c0 2024-09-15T03:47:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f677fe700 4 set_mon_vals no callback set 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: dump: 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1f6c002900 con 0x7f1f700990c0 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1f7004bbf0 con 0x7f1f7012d740 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 3911+0+0 (unknown 4190846776 0 0) 0x7f1f6c043eb0 con 0x7f1f700990c0 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199489223 0 0) 0x7f1f7004bbf0 con 0x7f1f7012d740 2024-09-15T03:47:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient(hunting): my global_id is 4812 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7009a010 con 0x7f1f7012d740 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f75dd3700 10 monclient: get_auth_request con 0x7f1f54037740 auth_method 0 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 878888521 0 0) 0x7f1f7009a010 con 0x7f1f7012d740 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f7004bbf0 con 0x7f1f7012d740 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1f60001970 con 0x7f1f7012d740 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.106+0000 7f1f657fa700 10 monclient: dump: 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 940748307 0 0) 0x7f1f7004bbf0 con 0x7f1f7012d740 2024-09-15T03:47:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f7012f3d0 con 0x7f1f7012d740 2024-09-15T03:47:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1f60001f10 con 0x7f1f7012d740 2024-09-15T03:47:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.26:6792/0 conn(0x7f1f7012d740 legacy=0x7f1f7009a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.110+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.26:6792/0 conn(0x7f1f7012d740 legacy=0x7f1f7009a2d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f77836700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1f70131b60 con 0x7f1f70131790 2024-09-15T03:47:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956652130 0 0) 0x7f1f70131b60 con 0x7f1f70131790 2024-09-15T03:47:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient(hunting): my global_id is 4821 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7004bbf0 con 0x7f1f70131790 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 436073202 0 0) 0x7f1f7004bbf0 con 0x7f1f70131790 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1f70131b60 con 0x7f1f70131790 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1f60002720 con 0x7f1f70131790 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient: dump: 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3478989567 0 0) 0x7f1f70131b60 con 0x7f1f70131790 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1f7012ecf0 con 0x7f1f70131790 2024-09-15T03:47:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7559 (unknown 0 0 664065467) 0x7f1f600028d0 con 0x7f1f70131790 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.26:6792/0 conn(0x7f1f70131790 legacy=0x7f1f7002ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.122+0000 7f1f657fa700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.26:6792/0 conn(0x7f1f70131790 legacy=0x7f1f7002ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:51.154 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.122:6812/20895 conn(0x7f1f54037740 legacy=0x7f1f54039bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 10 monclient: shutdown 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 >> v1:172.21.15.26:6789/0 conn(0x7f1f700990c0 legacy=0x7f1f7009e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f75dd3700 1 -- 172.21.15.26:0/1900125617 reap_dead start 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 shutdown_connections 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 >> 172.21.15.26:0/1900125617 conn(0x7f1f7006e370 msgr2=0x7f1f7012bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 shutdown_connections 2024-09-15T03:47:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.126+0000 7f1f77836700 1 -- 172.21.15.26:0/1900125617 wait complete. 2024-09-15T03:47:51.157 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:51.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-15T03:47:51.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:47:51.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient: get_monmap_and_config 2024-09-15T03:47:51.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient: monmap: 2024-09-15T03:47:51.273 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:51.273421+0000 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:51.273421+0000 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.274 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 1 Processor -- start 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 1 -- start start 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient: init 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient: _add_conns ranks=[0,6,8,7,5,1,4,3,2] 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient(hunting): picked mon.noname-a con 0x7f75e80a6a70 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient(hunting): picked mon.noname-d con 0x7f75e8099020 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient(hunting): picked mon.noname-e con 0x7f75e8138cc0 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e804ca10 con 0x7f75e80a6a70 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e8098c20 con 0x7f75e8099020 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e80a2280 con 0x7f75e8138cc0 2024-09-15T03:47:51.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75ecd2a700 10 monclient(hunting): authenticate will time out at 1015.703685s 2024-09-15T03:47:51.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75e7fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f75e8138cc0 0x7f75e813b0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38856/0 (socket says 172.21.15.26:38856) 2024-09-15T03:47:51.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.270+0000 7f75e7fff700 1 -- 172.21.15.26:0/3344031034 learned_addr learned my addr 172.21.15.26:0/3344031034 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027948675 0 0) 0x7f75e80a2280 con 0x7f75e8138cc0 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): my global_id is 134297 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d0003570 con 0x7f75e8138cc0 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972484998 0 0) 0x7f75e804ca10 con 0x7f75e80a6a70 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): my global_id is 4689 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e80a2280 con 0x7f75e80a6a70 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954838115 0 0) 0x7f75e8098c20 con 0x7f75e8099020 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient(hunting): my global_id is 4830 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e804ca10 con 0x7f75e8099020 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1097410261 0 0) 0x7f75d0003570 con 0x7f75e8138cc0 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 >> v1:172.21.15.26:6792/0 conn(0x7f75e8099020 legacy=0x7f75e80993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 >> v1:172.21.15.26:6789/0 conn(0x7f75e80a6a70 legacy=0x7f75e8098510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 monclient: found mon.noname-e 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80a2840 con 0x7f75e8138cc0 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75e0001b30 con 0x7f75e8138cc0 2024-09-15T03:47:51.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: dump: 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.282 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 5 monclient: authenticate success, global_id 134297 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75e0001d90 con 0x7f75e8138cc0 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 1 -- 172.21.15.26:0/3344031034 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75e0004fb0 con 0x7f75e8138cc0 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 4 set_mon_vals no callback set 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75e67fc700 10 monclient: dump: 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.284 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 10 monclient: shutdown 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3344031034 >> v1:172.21.15.26:6793/0 conn(0x7f75e8138cc0 legacy=0x7f75e813b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3344031034 shutdown_connections 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3344031034 >> 172.21.15.26:0/3344031034 conn(0x7f75e806e1d0 msgr2=0x7f75e80a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3344031034 shutdown_connections 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3344031034 wait complete. 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.274+0000 7f75ecd2a700 10 monclient: monmap: 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.285 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 Processor -- start 2024-09-15T03:47:51.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 -- start start 2024-09-15T03:47:51.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: init 2024-09-15T03:47:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: _add_conns ranks=[6,1,0,3,7,2,4,8,5] 2024-09-15T03:47:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient(hunting): picked mon.noname-d con 0x7f75e8099020 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:51.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient(hunting): picked mon.noname-f con 0x7f75e80a6a70 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient(hunting): picked mon.noname-a con 0x7f75e8138cc0 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e8131a60 con 0x7f75e8138cc0 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e811ca00 con 0x7f75e80a6a70 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75e77fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f75e8099020 0x7f75e809d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44742/0 (socket says 172.21.15.26:44742) 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75e77fe700 1 -- 172.21.15.26:0/3141183095 learned_addr learned my addr 172.21.15.26:0/3141183095 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e809e3c0 con 0x7f75e8099020 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient(hunting): authenticate will time out at 1015.709281s 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737859213 0 0) 0x7f75e809e3c0 con 0x7f75e8099020 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): my global_id is 4839 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d4003570 con 0x7f75e8099020 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345662913 0 0) 0x7f75e8131a60 con 0x7f75e8138cc0 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): my global_id is 4698 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e809e3c0 con 0x7f75e8138cc0 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284950029 0 0) 0x7f75e811ca00 con 0x7f75e80a6a70 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): my global_id is 134209 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e8131a60 con 0x7f75e80a6a70 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4229347932 0 0) 0x7f75d4003570 con 0x7f75e8099020 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75e811ca00 con 0x7f75e8099020 2024-09-15T03:47:51.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75d8001aa0 con 0x7f75e8099020 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): dump: 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.294 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.296 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 319691628 0 0) 0x7f75e809e3c0 con 0x7f75e8138cc0 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75d4003570 con 0x7f75e8138cc0 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75e0001af0 con 0x7f75e8138cc0 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient(hunting): dump: 2024-09-15T03:47:51.297 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.298 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3588429965 0 0) 0x7f75d4003570 con 0x7f75e8138cc0 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.26:6792/0 conn(0x7f75e8099020 legacy=0x7f75e809d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.122:6789/0 conn(0x7f75e80a6a70 legacy=0x7f75e809dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 monclient: found mon.a 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e809ed10 con 0x7f75e8138cc0 2024-09-15T03:47:51.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 5 monclient: authenticate success, global_id 4698 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: _renew_subs 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e809e800 con 0x7f75e8138cc0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: _renew_subs 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e809f1f0 con 0x7f75e8138cc0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75e0004f50 con 0x7f75e8138cc0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75e0005520 con 0x7f75e8138cc0 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75e5ffb700 4 set_mon_vals no callback set 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.278+0000 7f75cf7fe700 10 monclient: dump: 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.301 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75ecd2a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f75e000fee0 con 0x7f75e8138cc0 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f75c8006590 con 0x7f75c8002840 2024-09-15T03:47:51.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(122..122 src has 1..122) v4 ==== 3911+0+0 (unknown 4190846776 0 0) 0x7f75e0044150 con 0x7f75e8138cc0 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75e6ffd700 10 monclient: get_auth_request con 0x7f75d4037cb0 auth_method 0 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522376679 0 0) 0x7f75c8006590 con 0x7f75c8002840 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient(hunting): my global_id is 134306 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d4003570 con 0x7f75c8002840 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3149791749 0 0) 0x7f75d4003570 con 0x7f75c8002840 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75c8006590 con 0x7f75c8002840 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75d8002300 con 0x7f75c8002840 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient: dump: 2024-09-15T03:47:51.303 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.304 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4263862204 0 0) 0x7f75c8006590 con 0x7f75c8002840 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75c800a130 con 0x7f75c8002840 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f75d8002800 con 0x7f75c8002840 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.26:6793/0 conn(0x7f75c8002840 legacy=0x7f75c8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.282+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.26:6793/0 conn(0x7f75c8002840 legacy=0x7f75c8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f75c8009640 con 0x7f75c80090b0 2024-09-15T03:47:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252566674 0 0) 0x7f75c8009640 con 0x7f75c80090b0 2024-09-15T03:47:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient(hunting): my global_id is 134315 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c8006590 con 0x7f75c80090b0 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3284852909 0 0) 0x7f75c8006590 con 0x7f75c80090b0 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75c8009640 con 0x7f75c80090b0 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75d8002ca0 con 0x7f75c80090b0 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient: dump: 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.308 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1853798187 0 0) 0x7f75c8009640 con 0x7f75c80090b0 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f75c80098f0 con 0x7f75c80090b0 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3376181209) 0x7f75d8005dc0 con 0x7f75c80090b0 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.26:6793/0 conn(0x7f75c80090b0 legacy=0x7f75c8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75cf7fe700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.26:6793/0 conn(0x7f75c80090b0 legacy=0x7f75c8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:51.331 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.122:6812/20895 conn(0x7f75d4037cb0 legacy=0x7f75d403a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 10 monclient: shutdown 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 >> v1:172.21.15.26:6789/0 conn(0x7f75e8138cc0 legacy=0x7f75e814a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75e7fff700 1 -- 172.21.15.26:0/3141183095 reap_dead start 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75e7fff700 1 -- 172.21.15.26:0/3141183095 reap_dead start 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 shutdown_connections 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 >> 172.21.15.26:0/3141183095 conn(0x7f75e806e1d0 msgr2=0x7f75e80a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 shutdown_connections 2024-09-15T03:47:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.294+0000 7f75ecd2a700 1 -- 172.21.15.26:0/3141183095 wait complete. 2024-09-15T03:47:51.335 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:51.335 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-15T03:47:51.335 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:47:51.397 INFO:tasks.rados.rados.0.smithi122.stdout:447: done (2 left) 2024-09-15T03:47:51.397 INFO:tasks.rados.rados.0.smithi122.stdout:448: done (1 left) 2024-09-15T03:47:51.398 INFO:tasks.rados.rados.0.smithi122.stdout:449: done (0 left) 2024-09-15T03:47:51.398 INFO:tasks.rados.rados.0.smithi122.stdout:450: write oid 40 current snap is 53 2024-09-15T03:47:51.398 INFO:tasks.rados.rados.0.smithi122.stdout:450: seq_num 190 ranges {793330=689782,2035461=617731,3101545=91785} 2024-09-15T03:47:51.413 INFO:tasks.rados.rados.0.smithi122.stdout:450: writing smithi12223906-40 from 793330 to 1483112 tid 1 2024-09-15T03:47:51.421 INFO:tasks.rados.rados.0.smithi122.stdout:450: writing smithi12223906-40 from 2035461 to 2653192 tid 2 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:450: writing smithi12223906-40 from 3101545 to 3193330 tid 3 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:451: read oid 28 snap -1 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:451: expect (ObjNum 173 snap 45 seq_num 173) 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:452: write oid 5 current snap is 53 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:450: finishing write tid 1 to smithi12223906-40 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:450: finishing write tid 2 to smithi12223906-40 2024-09-15T03:47:51.425 INFO:tasks.rados.rados.0.smithi122.stdout:452: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2024-09-15T03:47:51.430 INFO:tasks.rados.rados.0.smithi122.stdout:452: writing smithi12223906-5 from 410137 to 891521 tid 1 2024-09-15T03:47:51.437 INFO:tasks.rados.rados.0.smithi122.stdout:452: writing smithi12223906-5 from 1410296 to 2171685 tid 2 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:452: writing smithi12223906-5 from 2728968 to 3210137 tid 3 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:453: read oid 46 snap -1 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:453: expect deleted 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:454: write (excl) oid 22 current snap is 53 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:450: finishing write tid 3 to smithi12223906-40 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:450: finishing write tid 4 to smithi12223906-40 2024-09-15T03:47:51.444 INFO:tasks.rados.rados.0.smithi122.stdout:450: finishing write tid 5 to smithi12223906-40 2024-09-15T03:47:51.445 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 83 (ObjNum 190 snap 53 seq_num 190) dirty exists 2024-09-15T03:47:51.445 INFO:tasks.rados.rados.0.smithi122.stdout:450: left oid 40 (ObjNum 190 snap 53 seq_num 190) 2024-09-15T03:47:51.445 INFO:tasks.rados.rados.0.smithi122.stdout:454: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2024-09-15T03:47:51.449 INFO:tasks.rados.rados.0.smithi122.stdout:454: writing smithi12223906-22 from 426944 to 983577 tid 1 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.450+0000 7fe471e5e700 10 monclient: get_monmap_and_config 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.450+0000 7fe471e5e700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.450+0000 7fe471e5e700 10 monclient: monmap: 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:51.455208+0000 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:51.455208+0000 2024-09-15T03:47:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.456 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.457 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.458 INFO:tasks.rados.rados.0.smithi122.stdout:454: writing smithi12223906-22 from 1468778 to 2090178 tid 2 2024-09-15T03:47:51.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 Processor -- start 2024-09-15T03:47:51.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- start start 2024-09-15T03:47:51.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient: init 2024-09-15T03:47:51.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient: _add_conns ranks=[4,7,3,6,5,0,8,2,1] 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient(hunting): picked mon.noname-c con 0x7fe46c070de0 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient(hunting): picked mon.noname-i con 0x7fe46c09bdd0 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient(hunting): picked mon.noname-g con 0x7fe46c136b20 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe470e5c700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fe46c070de0 legacy=0x7fe46c130d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c04ca10 con 0x7fe46c136b20 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c0999f0 con 0x7fe46c070de0 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c09c8b0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient(hunting): authenticate will time out at 1015.885509s 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe47165d700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fe46c136b20 0x7fe46c138f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:56020/0 (socket says 172.21.15.26:56020) 2024-09-15T03:47:51.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46bfff700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fe46c09bdd0 0x7fe46c09c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:50392/0 (socket says 172.21.15.26:50392) 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe47165d700 1 -- 172.21.15.26:0/182275124 learned_addr learned my addr 172.21.15.26:0/182275124 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272521235 0 0) 0x7fe46c04ca10 con 0x7fe46c136b20 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient(hunting): my global_id is 4746 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe45c003570 con 0x7fe46c136b20 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292493438 0 0) 0x7fe46c09c8b0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient(hunting): my global_id is 134269 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46c04ca10 con 0x7fe46c09bdd0 2024-09-15T03:47:51.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4099279598 0 0) 0x7fe45c003570 con 0x7fe46c136b20 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 >> v1:172.21.15.122:6792/0 conn(0x7fe46c09bdd0 legacy=0x7fe46c09c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 >> v1:172.21.15.26:6791/0 conn(0x7fe46c070de0 legacy=0x7fe46c130d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 monclient: found mon.noname-g 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c098d10 con 0x7fe46c136b20 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe464001b30 con 0x7fe46c136b20 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:47:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: dump: 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 5 monclient: authenticate success, global_id 4746 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe464001d20 con 0x7fe46c136b20 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 1 -- 172.21.15.26:0/182275124 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe464004c70 con 0x7fe46c136b20 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe46b7fe700 10 monclient: dump: 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.467 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 4 set_mon_vals no callback set 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 10 monclient: shutdown 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- 172.21.15.26:0/182275124 >> v1:172.21.15.122:6790/0 conn(0x7fe46c136b20 legacy=0x7fe46c138f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- 172.21.15.26:0/182275124 shutdown_connections 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.454+0000 7fe471e5e700 1 -- 172.21.15.26:0/182275124 >> 172.21.15.26:0/182275124 conn(0x7fe46c0a2ea0 msgr2=0x7fe46c09eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- 172.21.15.26:0/182275124 shutdown_connections 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- 172.21.15.26:0/182275124 wait complete. 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: monmap: 2024-09-15T03:47:51.469 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.470 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 Processor -- start 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- start start 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: init 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: _add_conns ranks=[5,6,0,1,4,2,8,7,3] 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient(hunting): picked mon.noname-h con 0x7fe46c070de0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient(hunting): picked mon.noname-d con 0x7fe46c09bdd0 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient(hunting): picked mon.noname-a con 0x7fe46c136b20 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c06bd70 con 0x7fe46c136b20 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c09f910 con 0x7fe46c070de0 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46c06abe0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient(hunting): authenticate will time out at 1015.890105s 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe46bfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fe46c09bdd0 0x7fe46c06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44746/0 (socket says 172.21.15.26:44746) 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe46bfff700 1 -- 172.21.15.26:0/2667403210 learned_addr learned my addr 172.21.15.26:0/2667403210 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474710760 0 0) 0x7fe46c06abe0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): my global_id is 4848 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe450003570 con 0x7fe46c09bdd0 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732718495 0 0) 0x7fe46c06bd70 con 0x7fe46c136b20 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): my global_id is 4707 2024-09-15T03:47:51.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46c06abe0 con 0x7fe46c136b20 2024-09-15T03:47:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423185688 0 0) 0x7fe46c09f910 con 0x7fe46c070de0 2024-09-15T03:47:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): my global_id is 4847 2024-09-15T03:47:51.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46c06bd70 con 0x7fe46c070de0 2024-09-15T03:47:51.474 INFO:tasks.rados.rados.0.smithi122.stdout:454: writing smithi12223906-22 from 2872744 to 3226944 tid 3 2024-09-15T03:47:51.474 INFO:tasks.rados.rados.0.smithi122.stdout:450: done (4 left) 2024-09-15T03:47:51.474 INFO:tasks.rados.rados.0.smithi122.stdout:455: copy_from oid 36 from oid 41 current snap is 53 2024-09-15T03:47:51.474 INFO:tasks.rados.rados.0.smithi122.stdout:452: finishing write tid 1 to smithi12223906-5 2024-09-15T03:47:51.474 INFO:tasks.rados.rados.0.smithi122.stdout:451: expect (ObjNum 173 snap 45 seq_num 173) 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4165048664 0 0) 0x7fe450003570 con 0x7fe46c09bdd0 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe46c09f910 con 0x7fe46c09bdd0 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe4600013a0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3477204743 0 0) 0x7fe46c06abe0 con 0x7fe46c136b20 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe450003570 con 0x7fe46c136b20 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe464001af0 con 0x7fe46c136b20 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3278762626 0 0) 0x7fe46c06bd70 con 0x7fe46c070de0 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe46c06abe0 con 0x7fe46c070de0 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe4580042c0 con 0x7fe46c070de0 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient(hunting): dump: 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 896468370 0 0) 0x7fe46c09f910 con 0x7fe46c09bdd0 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.122:6791/0 conn(0x7fe46c070de0 legacy=0x7fe46c069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.26:6789/0 conn(0x7fe46c136b20 legacy=0x7fe46c147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 monclient: found mon.d 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c06b5b0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe468ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 5 monclient: authenticate success, global_id 4848 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: _renew_subs 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.458+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46c06d0e0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe471e5e700 10 monclient: _renew_subs 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe471e5e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46c06c900 con 0x7fe46c09bdd0 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe460001640 con 0x7fe46c09bdd0 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe460004e20 con 0x7fe46c09bdd0 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe46a7fc700 4 set_mon_vals no callback set 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 10 monclient: dump: 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe46000f7e0 con 0x7fe46c09bdd0 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(123..123 src has 1..123) v4 ==== 3911+0+0 (unknown 3449399465 0 0) 0x7fe460043e00 con 0x7fe46c09bdd0 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe470e5c700 10 monclient: get_auth_request con 0x7fe450038070 auth_method 0 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe471e5e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe44c006590 con 0x7fe44c002840 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.462+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546750838 0 0) 0x7fe44c006590 con 0x7fe44c002840 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient(hunting): my global_id is 134218 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46c09f910 con 0x7fe44c002840 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 355723192 0 0) 0x7fe46c09f910 con 0x7fe44c002840 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe44c006590 con 0x7fe44c002840 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe464005270 con 0x7fe44c002840 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient: dump: 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2869079157 0 0) 0x7fe44c006590 con 0x7fe44c002840 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe44c00a130 con 0x7fe44c002840 2024-09-15T03:47:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe464005660 con 0x7fe44c002840 2024-09-15T03:47:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.122:6789/0 conn(0x7fe44c002840 legacy=0x7fe44c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.466+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.122:6789/0 conn(0x7fe44c002840 legacy=0x7fe44c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe471e5e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe44c009640 con 0x7fe44c0090b0 2024-09-15T03:47:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760369666 0 0) 0x7fe44c009640 con 0x7fe44c0090b0 2024-09-15T03:47:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 10 monclient(hunting): my global_id is 134227 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe44c006590 con 0x7fe44c0090b0 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 7242279 0 0) 0x7fe44c006590 con 0x7fe44c0090b0 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe44c009640 con 0x7fe44c0090b0 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe464005e20 con 0x7fe44c0090b0 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 10 monclient: dump: 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.478+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2901881098 0 0) 0x7fe44c009640 con 0x7fe44c0090b0 2024-09-15T03:47:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe44c0098f0 con 0x7fe44c0090b0 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 2207080400) 0x7fe464006120 con 0x7fe44c0090b0 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe468ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe468ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.122:6789/0 conn(0x7fe44c0090b0 legacy=0x7fe44c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe468ff9700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.122:6789/0 conn(0x7fe44c0090b0 legacy=0x7fe44c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:51.512 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.122:6812/20895 conn(0x7fe450038070 legacy=0x7fe45003a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 10 monclient: shutdown 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 >> v1:172.21.15.26:6792/0 conn(0x7fe46c09bdd0 legacy=0x7fe46c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe47165d700 1 -- 172.21.15.26:0/2667403210 reap_dead start 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe47165d700 1 -- 172.21.15.26:0/2667403210 reap_dead start 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 shutdown_connections 2024-09-15T03:47:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 >> 172.21.15.26:0/2667403210 conn(0x7fe46c0a2ea0 msgr2=0x7fe46c1360e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 shutdown_connections 2024-09-15T03:47:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.482+0000 7fe471e5e700 1 -- 172.21.15.26:0/2667403210 wait complete. 2024-09-15T03:47:51.515 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:51.515 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-15T03:47:51.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:47:51.567 INFO:tasks.rados.rados.0.smithi122.stdout:452: finishing write tid 2 to smithi12223906-5 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:452: finishing write tid 3 to smithi12223906-5 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:452: finishing write tid 4 to smithi12223906-5 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:452: finishing write tid 5 to smithi12223906-5 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 66 (ObjNum 191 snap 53 seq_num 191) dirty exists 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:452: left oid 5 (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:454: finishing write tid 1 to smithi12223906-22 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:454: finishing write tid 2 to smithi12223906-22 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:454: finishing write tid 3 to smithi12223906-22 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:454: finishing write tid 4 to smithi12223906-22 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:454: finishing write tid 5 to smithi12223906-22 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 222 (ObjNum 192 snap 53 seq_num 192) dirty exists 2024-09-15T03:47:51.568 INFO:tasks.rados.rados.0.smithi122.stdout:454: left oid 22 (ObjNum 192 snap 53 seq_num 192) 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:455: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:455: finishing copy_from to smithi12223906-36 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 44 (ObjNum 156 snap 37 seq_num 156) dirty exists 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:451: done (4 left) 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:452: done (3 left) 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:453: done (2 left) 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:454: done (1 left) 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:455: done (0 left) 2024-09-15T03:47:51.569 INFO:tasks.rados.rados.0.smithi122.stdout:456: delete oid 15 current snap is 53 2024-09-15T03:47:51.572 INFO:tasks.rados.rados.0.smithi122.stdout:456: done (0 left) 2024-09-15T03:47:51.573 INFO:tasks.rados.rados.0.smithi122.stdout:457: snap_remove snap 49 2024-09-15T03:47:51.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.626+0000 7f141bfff700 10 monclient: get_monmap_and_config 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.626+0000 7f141bfff700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.626+0000 7f141bfff700 10 monclient: monmap: 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:51.631430+0000 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:51.631430+0000 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 Processor -- start 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- start start 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: init 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: _add_conns ranks=[4,8,5,2,6,0,7,1,3] 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f141c0a6a70 addr v1:172.21.15.26:6791/0 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient(hunting): picked mon.noname-e con 0x7f141c099020 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient(hunting): picked mon.noname-h con 0x7f141c138cc0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c04ca10 con 0x7f141c0a6a70 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c098c20 con 0x7f141c138cc0 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0a2280 con 0x7f141c099020 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient(hunting): authenticate will time out at 1016.060786s 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141affd700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f141c0a6a70 legacy=0x7f141c098510 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f1413fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f141c099020 0x7f141c0993f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38892/0 (socket says 172.21.15.26:38892) 2024-09-15T03:47:51.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f1413fff700 1 -- 172.21.15.26:0/2977097270 learned_addr learned my addr 172.21.15.26:0/2977097270 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929592036 0 0) 0x7f141c0a2280 con 0x7f141c099020 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient(hunting): my global_id is 134324 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1408003570 con 0x7f141c099020 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049729647 0 0) 0x7f141c098c20 con 0x7f141c138cc0 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient(hunting): my global_id is 4856 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c0a2280 con 0x7f141c138cc0 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554351301 0 0) 0x7f1408003570 con 0x7f141c099020 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 >> v1:172.21.15.122:6791/0 conn(0x7f141c138cc0 legacy=0x7f141c13b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 >> v1:172.21.15.26:6791/0 conn(0x7f141c0a6a70 legacy=0x7f141c098510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 monclient: found mon.noname-e 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0a2840 con 0x7f141c099020 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f14040017b0 con 0x7f141c099020 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 5 monclient: authenticate success, global_id 134324 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: dump: 2024-09-15T03:47:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.643 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1404001a90 con 0x7f141c099020 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 1 -- 172.21.15.26:0/2977097270 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1404004c30 con 0x7f141c099020 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141a7fc700 10 monclient: dump: 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 4 set_mon_vals no callback set 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: shutdown 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- 172.21.15.26:0/2977097270 >> v1:172.21.15.26:6793/0 conn(0x7f141c099020 legacy=0x7f141c0993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- 172.21.15.26:0/2977097270 shutdown_connections 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- 172.21.15.26:0/2977097270 >> 172.21.15.26:0/2977097270 conn(0x7f141c06e1d0 msgr2=0x7f141c0a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- 172.21.15.26:0/2977097270 shutdown_connections 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 1 -- 172.21.15.26:0/2977097270 wait complete. 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.630+0000 7f141bfff700 10 monclient: monmap: 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 Processor -- start 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 -- start start 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: init 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: _add_conns ranks=[7,5,1,3,4,6,8,2,0] 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient(hunting): picked mon.noname-i con 0x7f141c099020 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient(hunting): picked mon.noname-h con 0x7f141c0a6a70 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient(hunting): picked mon.noname-f con 0x7f141c138cc0 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c09de00 con 0x7f141c138cc0 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0a0000 con 0x7f141c0a6a70 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c099d60 con 0x7f141c099020 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient(hunting): authenticate will time out at 1016.065563s 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1413fff700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f141c0a6a70 0x7f141c137ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:53506/0 (socket says 172.21.15.26:53506) 2024-09-15T03:47:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1413fff700 1 -- 172.21.15.26:0/4251322652 learned_addr learned my addr 172.21.15.26:0/4251322652 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414262296 0 0) 0x7f141c0a0000 con 0x7f141c0a6a70 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): my global_id is 4865 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13fc003570 con 0x7f141c0a6a70 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438660035 0 0) 0x7f141c099d60 con 0x7f141c099020 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): my global_id is 134278 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c0a0000 con 0x7f141c099020 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717796112 0 0) 0x7f141c09de00 con 0x7f141c138cc0 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): my global_id is 134236 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c099d60 con 0x7f141c138cc0 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1676058256 0 0) 0x7f141c099d60 con 0x7f141c138cc0 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f141c09de00 con 0x7f141c138cc0 2024-09-15T03:47:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 828480288 0 0) 0x7f13fc003570 con 0x7f141c0a6a70 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f141c099d60 con 0x7f141c0a6a70 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1414001c80 con 0x7f141c138cc0 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 971029455 0 0) 0x7f141c0a0000 con 0x7f141c099020 2024-09-15T03:47:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13fc003570 con 0x7f141c099020 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1404004b70 con 0x7f141c0a6a70 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f140c004240 con 0x7f141c099020 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient(hunting): dump: 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4084005307 0 0) 0x7f141c09de00 con 0x7f141c138cc0 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6792/0 conn(0x7f141c099020 legacy=0x7f141c1373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6791/0 conn(0x7f141c0a6a70 legacy=0x7f141c137ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 monclient: found mon.f 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c1381e0 con 0x7f141c138cc0 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 5 monclient: authenticate success, global_id 134236 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: _renew_subs 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0a1770 con 0x7f141c138cc0 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: _renew_subs 2024-09-15T03:47:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c09d880 con 0x7f141c138cc0 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.634+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1414001f60 con 0x7f141c138cc0 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1419ffb700 4 set_mon_vals no callback set 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1414002410 con 0x7f141c138cc0 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: dump: 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f141bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1414002630 con 0x7f141c138cc0 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1400006590 con 0x7f1400002840 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(123..123 src has 1..123) v4 ==== 3911+0+0 (unknown 3449399465 0 0) 0x7f1414041800 con 0x7f141c138cc0 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1413fff700 10 monclient: get_auth_request con 0x7f13fc037fe0 auth_method 0 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485989905 0 0) 0x7f1400006590 con 0x7f1400002840 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient(hunting): my global_id is 4755 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c09de00 con 0x7f1400002840 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2786078923 0 0) 0x7f141c09de00 con 0x7f1400002840 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1400006590 con 0x7f1400002840 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f140c004990 con 0x7f1400002840 2024-09-15T03:47:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 10 monclient: dump: 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3615207850 0 0) 0x7f1400006590 con 0x7f1400002840 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.638+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f140000a130 con 0x7f1400002840 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.642+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f140c004fd0 con 0x7f1400002840 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.642+0000 7f1412ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.642+0000 7f1412ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.642+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6790/0 conn(0x7f1400002840 legacy=0x7f1400004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.642+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6790/0 conn(0x7f1400002840 legacy=0x7f1400004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f141bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1400007520 con 0x7f14000068d0 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062678026 0 0) 0x7f1400007520 con 0x7f14000068d0 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 10 monclient(hunting): my global_id is 4764 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1400006590 con 0x7f14000068d0 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3993533576 0 0) 0x7f1400006590 con 0x7f14000068d0 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1400007520 con 0x7f14000068d0 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f140c005880 con 0x7f14000068d0 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 10 monclient: dump: 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.672 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4252116648 0 0) 0x7f1400007520 con 0x7f14000068d0 2024-09-15T03:47:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.650+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1400008760 con 0x7f14000068d0 2024-09-15T03:47:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1584650627) 0x7f140c005b70 con 0x7f14000068d0 2024-09-15T03:47:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f1412ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f1412ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6790/0 conn(0x7f14000068d0 legacy=0x7f1400006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f1412ffd700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6790/0 conn(0x7f14000068d0 legacy=0x7f1400006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 6 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:51.704 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:51.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6812/20895 conn(0x7f13fc037fe0 legacy=0x7f13fc03a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 10 monclient: shutdown 2024-09-15T03:47:51.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 >> v1:172.21.15.122:6789/0 conn(0x7f141c138cc0 legacy=0x7f141c14a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141b7fe700 1 -- 172.21.15.26:0/4251322652 reap_dead start 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141b7fe700 1 -- 172.21.15.26:0/4251322652 reap_dead start 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 shutdown_connections 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 >> 172.21.15.26:0/4251322652 conn(0x7f141c06e1d0 msgr2=0x7f141c06f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 shutdown_connections 2024-09-15T03:47:51.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.654+0000 7f141bfff700 1 -- 172.21.15.26:0/4251322652 wait complete. 2024-09-15T03:47:51.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:51.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-15T03:47:51.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:47:51.726 INFO:tasks.rados.rados.0.smithi122.stdout:457: done (0 left) 2024-09-15T03:47:51.726 INFO:tasks.rados.rados.0.smithi122.stdout:458: read oid 23 snap -1 2024-09-15T03:47:51.726 INFO:tasks.rados.rados.0.smithi122.stdout:458: expect (ObjNum 161 snap 42 seq_num 161) 2024-09-15T03:47:51.727 INFO:tasks.rados.rados.0.smithi122.stdout:459: delete oid 45 current snap is 53 2024-09-15T03:47:51.741 INFO:tasks.rados.rados.0.smithi122.stdout:458: expect (ObjNum 161 snap 42 seq_num 161) 2024-09-15T03:47:51.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.814+0000 7f164947d700 10 monclient: get_monmap_and_config 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.814+0000 7f164947d700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.814+0000 7f164947d700 10 monclient: monmap: 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:51.819815+0000 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:51.819815+0000 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.820 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.821 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 Processor -- start 2024-09-15T03:47:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- start start 2024-09-15T03:47:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient: init 2024-09-15T03:47:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient: _add_conns ranks=[8,1,7,3,0,4,2,5,6] 2024-09-15T03:47:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient(hunting): picked mon.noname-e con 0x7f164406e530 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient(hunting): picked mon.noname-f con 0x7f1644099240 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient(hunting): picked mon.noname-i con 0x7f1644138d70 addr v1:172.21.15.122:6792/0 2024-09-15T03:47:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164404ca10 con 0x7f1644099240 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164409e310 con 0x7f1644138d70 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1644133730 con 0x7f164406e530 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient(hunting): authenticate will time out at 1016.249309s 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1643fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f164406e530 0x7f1644133020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38908/0 (socket says 172.21.15.26:38908) 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1643fff700 1 -- 172.21.15.26:0/1887626621 learned_addr learned my addr 172.21.15.26:0/1887626621 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960567267 0 0) 0x7f1644133730 con 0x7f164406e530 2024-09-15T03:47:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): my global_id is 134333 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f162c003570 con 0x7f164406e530 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189222209 0 0) 0x7f164409e310 con 0x7f1644138d70 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): my global_id is 134287 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1644133730 con 0x7f1644138d70 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214673567 0 0) 0x7f164404ca10 con 0x7f1644099240 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient(hunting): my global_id is 134245 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164409e310 con 0x7f1644099240 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623601179 0 0) 0x7f162c003570 con 0x7f164406e530 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 >> v1:172.21.15.122:6792/0 conn(0x7f1644138d70 legacy=0x7f164413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 >> v1:172.21.15.122:6789/0 conn(0x7f1644099240 legacy=0x7f1644099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 monclient: found mon.noname-e 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644097b00 con 0x7f164406e530 2024-09-15T03:47:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 5 monclient: authenticate success, global_id 134333 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f16340017b0 con 0x7f164406e530 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: dump: 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f16340019d0 con 0x7f164406e530 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 4 set_mon_vals no callback set 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 1 -- 172.21.15.26:0/1887626621 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1634004c30 con 0x7f164406e530 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f1642ffd700 10 monclient: dump: 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.833 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 10 monclient: shutdown 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- 172.21.15.26:0/1887626621 >> v1:172.21.15.26:6793/0 conn(0x7f164406e530 legacy=0x7f1644133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- 172.21.15.26:0/1887626621 shutdown_connections 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- 172.21.15.26:0/1887626621 >> 172.21.15.26:0/1887626621 conn(0x7f1644070de0 msgr2=0x7f16440711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- 172.21.15.26:0/1887626621 shutdown_connections 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.818+0000 7f164947d700 1 -- 172.21.15.26:0/1887626621 wait complete. 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient: build_initial_monmap 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient: monmap: 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 1 Processor -- start 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 1 -- start start 2024-09-15T03:47:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient: init 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient: _add_conns ranks=[0,2,6,3,7,8,1,4,5] 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient(hunting): picked mon.noname-a con 0x7f164406e530 addr v1:172.21.15.26:6789/0 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient(hunting): picked mon.noname-b con 0x7f1644099240 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient(hunting): picked mon.noname-d con 0x7f1644138d70 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164409e790 con 0x7f164406e530 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164409ea60 con 0x7f1644099240 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1644130f20 con 0x7f1644138d70 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:51.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f164947d700 10 monclient(hunting): authenticate will time out at 1016.254981s 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f1643fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f164406e530 0x7f164412ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52004/0 (socket says 172.21.15.26:52004) 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f1643fff700 1 -- 172.21.15.26:0/150975186 learned_addr learned my addr 172.21.15.26:0/150975186 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f1648c7c700 1 --1- 172.21.15.26:0/150975186 >> v1:172.21.15.26:6792/0 conn(0x7f1644138d70 0x7f164412fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44758/0 (socket says 172.21.15.26:44758) 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166097304 0 0) 0x7f1644130f20 con 0x7f1644138d70 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): my global_id is 4857 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1630003570 con 0x7f1644138d70 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224907786 0 0) 0x7f164409ea60 con 0x7f1644099240 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): my global_id is 134354 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1644130f20 con 0x7f1644099240 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420109376 0 0) 0x7f164409e790 con 0x7f164406e530 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 10 monclient(hunting): my global_id is 4716 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164409ea60 con 0x7f164406e530 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.822+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1981125543 0 0) 0x7f1630003570 con 0x7f1644138d70 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f164409e790 con 0x7f1644138d70 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f163c001d00 con 0x7f1644138d70 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): dump: 2024-09-15T03:47:51.843 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.844 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1460983519 0 0) 0x7f164409e790 con 0x7f1644138d70 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.26:6790/0 conn(0x7f1644099240 legacy=0x7f164412f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.26:6789/0 conn(0x7f164406e530 legacy=0x7f164412ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 monclient: found mon.d 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644130a10 con 0x7f1644138d70 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: _finish_auth 0 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:51.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 5 monclient: authenticate success, global_id 4857 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 10 monclient: _renew_subs 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16441324d0 con 0x7f1644138d70 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f163c0030f0 con 0x7f1644138d70 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 10 monclient: _renew_subs 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f16441315e0 con 0x7f1644138d70 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f163c0025e0 con 0x7f1644138d70 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f16427fc700 4 set_mon_vals no callback set 2024-09-15T03:47:51.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: dump: 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.847 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1644131830 con 0x7f164409a310 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f163c0027c0 con 0x7f1644138d70 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 3967+0+0 (unknown 2295217823 0 0) 0x7f163c044140 con 0x7f1644138d70 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400893947 0 0) 0x7f164409ea60 con 0x7f164409a310 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient(hunting): my global_id is 4874 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164409e790 con 0x7f164409a310 2024-09-15T03:47:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f16437fe700 10 monclient: get_auth_request con 0x7f1630037630 auth_method 0 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2747402164 0 0) 0x7f164409e790 con 0x7f164409a310 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f164409ea60 con 0x7f164409a310 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1634004b70 con 0x7f164409a310 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 10 monclient: dump: 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.849 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2263501566 0 0) 0x7f164409ea60 con 0x7f164409a310 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.826+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1644133ab0 con 0x7f164409a310 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.830+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1634001490 con 0x7f164409a310 2024-09-15T03:47:51.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.830+0000 7f162bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:51.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.830+0000 7f162bfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:51.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.830+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.122:6791/0 conn(0x7f164409a310 legacy=0x7f164409a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.830+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.122:6791/0 conn(0x7f164409a310 legacy=0x7f164409a6e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f164947d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:51.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f164404bbf0 con 0x7f164409adf0 2024-09-15T03:47:51.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029729923 0 0) 0x7f164404bbf0 con 0x7f164409adf0 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient(hunting): my global_id is 4883 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f164409ea60 con 0x7f164409adf0 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3555964459 0 0) 0x7f164409ea60 con 0x7f164409adf0 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f164404bbf0 con 0x7f164409adf0 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1634001770 con 0x7f164409adf0 2024-09-15T03:47:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient: dump: 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4148177386 0 0) 0x7f164404bbf0 con 0x7f164409adf0 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1644066de0 con 0x7f164409adf0 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 4266842232) 0x7f16340076c0 con 0x7f164409adf0 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.122:6791/0 conn(0x7f164409adf0 legacy=0x7f164402ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.842+0000 7f162bfff700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.122:6791/0 conn(0x7f164409adf0 legacy=0x7f164402ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:51.876 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.122:6812/20895 conn(0x7f1630037630 legacy=0x7f1630039ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 10 monclient: shutdown 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 >> v1:172.21.15.26:6792/0 conn(0x7f1644138d70 legacy=0x7f164412fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f1648c7c700 1 -- 172.21.15.26:0/150975186 reap_dead start 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f1648c7c700 1 -- 172.21.15.26:0/150975186 reap_dead start 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 shutdown_connections 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 >> 172.21.15.26:0/150975186 conn(0x7f1644070de0 msgr2=0x7f164412c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 shutdown_connections 2024-09-15T03:47:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.846+0000 7f164947d700 1 -- 172.21.15.26:0/150975186 wait complete. 2024-09-15T03:47:51.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:51.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-15T03:47:51.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:47:51.903 INFO:tasks.rados.rados.0.smithi122.stdout:458: done (1 left) 2024-09-15T03:47:51.903 INFO:tasks.rados.rados.0.smithi122.stdout:459: done (0 left) 2024-09-15T03:47:51.903 INFO:tasks.rados.rados.0.smithi122.stdout:460: delete oid 14 current snap is 53 2024-09-15T03:47:51.912 INFO:tasks.rados.rados.0.smithi122.stdout:460: done (0 left) 2024-09-15T03:47:51.912 INFO:tasks.rados.rados.0.smithi122.stdout:461: rollback oid 15 current snap is 53 2024-09-15T03:47:51.912 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 48 2024-09-15T03:47:51.912 INFO:tasks.rados.rados.0.smithi122.stdout:462: snap_create 2024-09-15T03:47:51.924 INFO:tasks.rados.rados.0.smithi122.stdout:461: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:47:51.925 INFO:tasks.rados.rados.0.smithi122.stdout:461: finishing rollback tid 2 to smithi12223906-15 2024-09-15T03:47:51.925 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 126 (ObjNum 179 snap 48 seq_num 179) dirty exists 2024-09-15T03:47:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: get_monmap_and_config 2024-09-15T03:47:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: build_initial_monmap 2024-09-15T03:47:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: monmap: 2024-09-15T03:47:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:52.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:47:52.000199+0000 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:47:52.000199+0000 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:52.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 1 Processor -- start 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 1 -- start start 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: init 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: _add_conns ranks=[1,2,6,7,0,5,3,4,8] 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient(hunting): picked mon.noname-f con 0x7f754c132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient(hunting): picked mon.noname-b con 0x7f754c09faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient(hunting): picked mon.noname-d con 0x7f754c138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c04ca10 con 0x7f754c132b40 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c09d5d0 con 0x7f754c09faa0 2024-09-15T03:47:52.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c133620 con 0x7f754c138d40 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient(hunting): authenticate will time out at 1016.429966s 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f7551819700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f754c138d40 0x7f754c13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44764/0 (socket says 172.21.15.26:44764) 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f7551819700 1 -- 172.21.15.26:0/2446717388 learned_addr learned my addr 172.21.15.26:0/2446717388 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800955826 0 0) 0x7f754c09d5d0 con 0x7f754c09faa0 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): my global_id is 134363 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7534003570 con 0x7f754c09faa0 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175628837 0 0) 0x7f754c133620 con 0x7f754c138d40 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): my global_id is 4866 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c09d5d0 con 0x7f754c138d40 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800621674 0 0) 0x7f754c04ca10 con 0x7f754c132b40 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient(hunting): my global_id is 134254 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c133620 con 0x7f754c132b40 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1943015865 0 0) 0x7f7534003570 con 0x7f754c09faa0 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 >> v1:172.21.15.26:6792/0 conn(0x7f754c138d40 legacy=0x7f754c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 >> v1:172.21.15.122:6789/0 conn(0x7f754c132b40 legacy=0x7f754c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 monclient: found mon.noname-b 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:47:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0a0cc0 con 0x7f754c09faa0 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: _finish_auth 0 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75400017b0 con 0x7f754c09faa0 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: dump: 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.008 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.009 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.010 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 5 monclient: authenticate success, global_id 134363 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7540001a20 con 0x7f754c09faa0 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:52.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 1 -- 172.21.15.26:0/2446717388 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7540001c10 con 0x7f754c09faa0 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 10 monclient: get_monmap_and_config success 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f755201a700 4 set_mon_vals no callback set 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:51.998+0000 7f754bfff700 10 monclient: dump: 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.012 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient: shutdown 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- 172.21.15.26:0/2446717388 >> v1:172.21.15.26:6790/0 conn(0x7f754c09faa0 legacy=0x7f754c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- 172.21.15.26:0/2446717388 shutdown_connections 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- 172.21.15.26:0/2446717388 >> 172.21.15.26:0/2446717388 conn(0x7f754c06e370 msgr2=0x7f754c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- 172.21.15.26:0/2446717388 shutdown_connections 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- 172.21.15.26:0/2446717388 wait complete. 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient: build_initial_monmap 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient: monmap: 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:47:52.014 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:47:52.015 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 Processor -- start 2024-09-15T03:47:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- start start 2024-09-15T03:47:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient: init 2024-09-15T03:47:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient: _reopen_session rank -1 2024-09-15T03:47:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient: _add_conns ranks=[5,8,3,7,4,1,6,2,0] 2024-09-15T03:47:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient(hunting): picked mon.noname-h con 0x7f754c09faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient(hunting): picked mon.noname-e con 0x7f754c132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient(hunting): picked mon.noname-g con 0x7f754c138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c09ccc0 con 0x7f754c138d40 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c09b6e0 con 0x7f754c09faa0 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c09cfa0 con 0x7f754c132b40 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient(hunting): _renew_subs 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f755201a700 10 monclient(hunting): authenticate will time out at 1016.435237s 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f7550817700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f754c132b40 0x7f754c09a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38924/0 (socket says 172.21.15.26:38924) 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f7550817700 1 -- 172.21.15.26:0/4226831038 learned_addr learned my addr 172.21.15.26:0/4226831038 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12343397 0 0) 0x7f754c09cfa0 con 0x7f754c132b40 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 10 monclient(hunting): my global_id is 134342 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7538003570 con 0x7f754c132b40 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589858257 0 0) 0x7f754c09b6e0 con 0x7f754c09faa0 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 10 monclient(hunting): my global_id is 4892 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.002+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c09cfa0 con 0x7f754c09faa0 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256929661 0 0) 0x7f754c09ccc0 con 0x7f754c138d40 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): my global_id is 4773 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c09b6e0 con 0x7f754c138d40 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1038296101 0 0) 0x7f7538003570 con 0x7f754c132b40 2024-09-15T03:47:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f754c09ccc0 con 0x7f754c132b40 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7540002220 con 0x7f754c132b40 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): dump: 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4000831415 0 0) 0x7f754c09b6e0 con 0x7f754c138d40 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7538003570 con 0x7f754c138d40 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75440017b0 con 0x7f754c138d40 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:47:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): dump: 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 864558570 0 0) 0x7f754c09cfa0 con 0x7f754c09faa0 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f754c09b6e0 con 0x7f754c09faa0 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 877844538 0 0) 0x7f754c09ccc0 con 0x7f754c132b40 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6791/0 conn(0x7f754c09faa0 legacy=0x7f754c099970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6790/0 conn(0x7f754c138d40 legacy=0x7f754c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: _finish_hunting 0 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 monclient: found mon.e 2024-09-15T03:47:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c11cd30 con 0x7f754c132b40 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: _finish_auth 0 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 5 monclient: authenticate success, global_id 134342 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 10 monclient: _renew_subs 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c09b920 con 0x7f754c132b40 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 10 monclient: _renew_subs 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:47:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c09b1d0 con 0x7f754c132b40 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7540001c10 con 0x7f754c132b40 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7540002a80 con 0x7f754c132b40 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f754b7fe700 4 set_mon_vals no callback set 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: dump: 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.029 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f7540010030 con 0x7f754c132b40 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f752c006590 con 0x7f752c002840 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(124..124 src has 1..124) v4 ==== 3967+0+0 (unknown 2295217823 0 0) 0x7f7540044250 con 0x7f754c132b40 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f7551819700 10 monclient: get_auth_request con 0x7f7538037fb0 auth_method 0 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236466693 0 0) 0x7f752c006590 con 0x7f752c002840 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient(hunting): my global_id is 134296 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f754c09ccc0 con 0x7f752c002840 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 979108942 0 0) 0x7f754c09ccc0 con 0x7f752c002840 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f752c006590 con 0x7f752c002840 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f753c0022c0 con 0x7f752c002840 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.006+0000 7f75497fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 10 monclient: dump: 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4180947972 0 0) 0x7f752c006590 con 0x7f752c002840 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f752c00a130 con 0x7f752c002840 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f753c002860 con 0x7f752c002840 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6792/0 conn(0x7f752c002840 legacy=0x7f752c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.010+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6792/0 conn(0x7f752c002840 legacy=0x7f752c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.018+0000 7f755201a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:47:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.018+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f752c009640 con 0x7f752c0090b0 2024-09-15T03:47:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520906916 0 0) 0x7f752c009640 con 0x7f752c0090b0 2024-09-15T03:47:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:47:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:47:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient(hunting): my global_id is 134305 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c006590 con 0x7f752c0090b0 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1615800956 0 0) 0x7f752c006590 con 0x7f752c0090b0 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f752c009640 con 0x7f752c0090b0 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f753c0035f0 con 0x7f752c0090b0 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient: dump: 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:47:52.036 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2997267724 0 0) 0x7f752c009640 con 0x7f752c0090b0 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f752c0098f0 con 0x7f752c0090b0 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1734089864) 0x7f753c0062a0 con 0x7f752c0090b0 2024-09-15T03:47:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6792/0 conn(0x7f752c0090b0 legacy=0x7f752c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f75497fa700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6792/0 conn(0x7f752c0090b0 legacy=0x7f752c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 40, 2024-09-15T03:47:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:47:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:47:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:47:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:47:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:47:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:47:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:47:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:47:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:47:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:47:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:47:52.058 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:47:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.122:6812/20895 conn(0x7f7538037fb0 legacy=0x7f753803a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 10 monclient: shutdown 2024-09-15T03:47:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:47:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 >> v1:172.21.15.26:6793/0 conn(0x7f754c132b40 legacy=0x7f754c09a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:47:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f7551819700 1 -- 172.21.15.26:0/4226831038 reap_dead start 2024-09-15T03:47:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f7551819700 1 -- 172.21.15.26:0/4226831038 reap_dead start 2024-09-15T03:47:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 shutdown_connections 2024-09-15T03:47:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 >> 172.21.15.26:0/4226831038 conn(0x7f754c06e370 msgr2=0x7f754c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:47:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 shutdown_connections 2024-09-15T03:47:52.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:47:52.022+0000 7f755201a700 1 -- 172.21.15.26:0/4226831038 wait complete. 2024-09-15T03:47:52.060 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:47:52.060 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:47:52.728 INFO:tasks.rados.rados.0.smithi122.stdout:461: done (1 left) 2024-09-15T03:47:52.728 INFO:tasks.rados.rados.0.smithi122.stdout:462: done (0 left) 2024-09-15T03:47:52.728 INFO:tasks.rados.rados.0.smithi122.stdout:463: snap_create 2024-09-15T03:47:53.731 INFO:tasks.rados.rados.0.smithi122.stdout:463: done (0 left) 2024-09-15T03:47:53.731 INFO:tasks.rados.rados.0.smithi122.stdout:464: snap_remove snap 54 2024-09-15T03:47:54.745 INFO:tasks.rados.rados.0.smithi122.stdout:464: done (0 left) 2024-09-15T03:47:54.746 INFO:tasks.rados.rados.0.smithi122.stdout:465: write oid 1 current snap is 55 2024-09-15T03:47:54.746 INFO:tasks.rados.rados.0.smithi122.stdout:465: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2024-09-15T03:47:54.751 INFO:tasks.rados.rados.0.smithi122.stdout:465: writing smithi12223906-1 from 494172 to 951801 tid 1 2024-09-15T03:47:54.757 INFO:tasks.rados.rados.0.smithi122.stdout:465: writing smithi12223906-1 from 1586353 to 2098738 tid 2 2024-09-15T03:47:54.764 INFO:tasks.rados.rados.0.smithi122.stdout:465: writing smithi12223906-1 from 2749730 to 3294172 tid 3 2024-09-15T03:47:54.765 INFO:tasks.rados.rados.0.smithi122.stdout:466: write oid 23 current snap is 55 2024-09-15T03:47:54.765 INFO:tasks.rados.rados.0.smithi122.stdout:466: seq_num 197 ranges {510979=532878,1528482=772396,3060800=250179} 2024-09-15T03:47:54.771 INFO:tasks.rados.rados.0.smithi122.stdout:466: writing smithi12223906-23 from 510979 to 1043857 tid 1 2024-09-15T03:47:54.778 INFO:tasks.rados.rados.0.smithi122.stdout:466: writing smithi12223906-23 from 1528482 to 2300878 tid 2 2024-09-15T03:47:54.783 INFO:tasks.rados.rados.0.smithi122.stdout:466: writing smithi12223906-23 from 3060800 to 3310979 tid 3 2024-09-15T03:47:54.784 INFO:tasks.rados.rados.0.smithi122.stdout:467: write (excl) oid 47 current snap is 55 2024-09-15T03:47:54.784 INFO:tasks.rados.rados.0.smithi122.stdout:467: seq_num 198 ranges {527786=724480,1703317=516054,2804576=435035} 2024-09-15T03:47:54.790 INFO:tasks.rados.rados.0.smithi122.stdout:467: writing smithi12223906-47 from 527786 to 1252266 tid 1 2024-09-15T03:47:54.796 INFO:tasks.rados.rados.0.smithi122.stdout:467: writing smithi12223906-47 from 1703317 to 2219371 tid 2 2024-09-15T03:47:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:467: writing smithi12223906-47 from 2804576 to 3239611 tid 3 2024-09-15T03:47:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:468: read oid 30 snap -1 2024-09-15T03:47:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:468: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:47:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:469: read oid 18 snap -1 2024-09-15T03:47:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:469: expect (ObjNum 172 snap 45 seq_num 172) 2024-09-15T03:47:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:470: snap_remove snap 53 2024-09-15T03:47:55.761 INFO:tasks.rados.rados.0.smithi122.stdout:470: done (5 left) 2024-09-15T03:47:55.761 INFO:tasks.rados.rados.0.smithi122.stdout:471: read oid 20 snap -1 2024-09-15T03:47:55.761 INFO:tasks.rados.rados.0.smithi122.stdout:471: expect (ObjNum 176 snap 46 seq_num 176) 2024-09-15T03:47:55.761 INFO:tasks.rados.rados.0.smithi122.stdout:472: snap_remove snap 52 2024-09-15T03:47:56.728 INFO:tasks.rados.rados.0.smithi122.stdout:472: done (6 left) 2024-09-15T03:47:56.728 INFO:tasks.rados.rados.0.smithi122.stdout:473: read oid 33 snap -1 2024-09-15T03:47:56.728 INFO:tasks.rados.rados.0.smithi122.stdout:473: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:474: delete oid 22 current snap is 55 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:465: finishing write tid 1 to smithi12223906-1 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:465: finishing write tid 2 to smithi12223906-1 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:465: finishing write tid 3 to smithi12223906-1 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:465: finishing write tid 4 to smithi12223906-1 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:465: finishing write tid 5 to smithi12223906-1 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 130 (ObjNum 196 snap 55 seq_num 196) dirty exists 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:465: left oid 1 (ObjNum 196 snap 55 seq_num 196) 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:466: finishing write tid 1 to smithi12223906-23 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:466: finishing write tid 2 to smithi12223906-23 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:466: finishing write tid 3 to smithi12223906-23 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:466: finishing write tid 4 to smithi12223906-23 2024-09-15T03:47:56.729 INFO:tasks.rados.rados.0.smithi122.stdout:466: finishing write tid 5 to smithi12223906-23 2024-09-15T03:47:56.730 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 227 (ObjNum 197 snap 55 seq_num 197) dirty exists 2024-09-15T03:47:56.730 INFO:tasks.rados.rados.0.smithi122.stdout:466: left oid 23 (ObjNum 197 snap 55 seq_num 197) 2024-09-15T03:47:56.730 INFO:tasks.rados.rados.0.smithi122.stdout:467: finishing write tid 1 to smithi12223906-47 2024-09-15T03:47:56.730 INFO:tasks.rados.rados.0.smithi122.stdout:467: finishing write tid 2 to smithi12223906-47 2024-09-15T03:47:56.730 INFO:tasks.rados.rados.0.smithi122.stdout:468: expect (ObjNum 10 snap 0 seq_num 10) 2024-09-15T03:47:56.800 INFO:tasks.rados.rados.0.smithi122.stdout:467: finishing write tid 3 to smithi12223906-47 2024-09-15T03:47:56.800 INFO:tasks.rados.rados.0.smithi122.stdout:467: finishing write tid 4 to smithi12223906-47 2024-09-15T03:47:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:467: finishing write tid 5 to smithi12223906-47 2024-09-15T03:47:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 72 (ObjNum 198 snap 55 seq_num 198) dirty exists 2024-09-15T03:47:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:467: left oid 47 (ObjNum 198 snap 55 seq_num 198) 2024-09-15T03:47:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:469: expect (ObjNum 172 snap 45 seq_num 172) 2024-09-15T03:47:56.893 INFO:tasks.rados.rados.0.smithi122.stdout:471: expect (ObjNum 176 snap 46 seq_num 176) 2024-09-15T03:47:57.071 INFO:tasks.rados.rados.0.smithi122.stdout:473: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:465: done (7 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:466: done (6 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:467: done (5 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:468: done (4 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:469: done (3 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:471: done (2 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:473: done (1 left) 2024-09-15T03:47:57.154 INFO:tasks.rados.rados.0.smithi122.stdout:474: done (0 left) 2024-09-15T03:47:57.155 INFO:tasks.rados.rados.0.smithi122.stdout:475: read oid 50 snap -1 2024-09-15T03:47:57.155 INFO:tasks.rados.rados.0.smithi122.stdout:475: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:47:57.155 INFO:tasks.rados.rados.0.smithi122.stdout:476: snap_create 2024-09-15T03:47:57.168 INFO:tasks.rados.rados.0.smithi122.stdout:475: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:47:57.772 INFO:tasks.rados.rados.0.smithi122.stdout:475: done (1 left) 2024-09-15T03:47:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:476: done (0 left) 2024-09-15T03:47:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:477: read oid 44 snap -1 2024-09-15T03:47:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:477: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:478: read oid 38 snap -1 2024-09-15T03:47:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:478: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:47:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:479: delete oid 36 current snap is 56 2024-09-15T03:47:57.789 INFO:tasks.rados.rados.0.smithi122.stdout:477: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:57.878 INFO:tasks.rados.rados.0.smithi122.stdout:478: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:47:57.991 INFO:tasks.rados.rados.0.smithi122.stdout:477: done (2 left) 2024-09-15T03:47:57.991 INFO:tasks.rados.rados.0.smithi122.stdout:478: done (1 left) 2024-09-15T03:47:57.992 INFO:tasks.rados.rados.0.smithi122.stdout:479: done (0 left) 2024-09-15T03:47:57.992 INFO:tasks.rados.rados.0.smithi122.stdout:480: write (excl) oid 20 current snap is 56 2024-09-15T03:47:57.992 INFO:tasks.rados.rados.0.smithi122.stdout:480: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2024-09-15T03:47:58.006 INFO:tasks.rados.rados.0.smithi122.stdout:480: writing smithi12223906-20 from 578207 to 1128434 tid 1 2024-09-15T03:47:58.015 INFO:tasks.rados.rados.0.smithi122.stdout:480: writing smithi12223906-20 from 1762410 to 2542144 tid 2 2024-09-15T03:47:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:480: writing smithi12223906-20 from 3286845 to 3378207 tid 3 2024-09-15T03:47:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:481: read oid 41 snap -1 2024-09-15T03:47:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:481: expect (ObjNum 156 snap 37 seq_num 156) 2024-09-15T03:47:58.020 INFO:tasks.rados.rados.0.smithi122.stdout:482: snap_remove snap 55 2024-09-15T03:47:58.778 INFO:tasks.rados.rados.0.smithi122.stdout:482: done (2 left) 2024-09-15T03:47:58.778 INFO:tasks.rados.rados.0.smithi122.stdout:483: write (excl) oid 16 current snap is 56 2024-09-15T03:47:58.779 INFO:tasks.rados.rados.0.smithi122.stdout:480: finishing write tid 1 to smithi12223906-20 2024-09-15T03:47:58.779 INFO:tasks.rados.rados.0.smithi122.stdout:483: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2024-09-15T03:47:58.786 INFO:tasks.rados.rados.0.smithi122.stdout:483: writing smithi12223906-16 from 595014 to 1220490 tid 1 2024-09-15T03:47:58.793 INFO:tasks.rados.rados.0.smithi122.stdout:483: writing smithi12223906-16 from 1820892 to 2344284 tid 2 2024-09-15T03:47:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:483: writing smithi12223906-16 from 2797915 to 3394391 tid 3 2024-09-15T03:47:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:484: copy_from oid 14 from oid 2 current snap is 56 2024-09-15T03:47:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:480: finishing write tid 2 to smithi12223906-20 2024-09-15T03:47:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:480: finishing write tid 3 to smithi12223906-20 2024-09-15T03:47:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:480: finishing write tid 4 to smithi12223906-20 2024-09-15T03:47:58.800 INFO:tasks.rados.rados.0.smithi122.stdout:480: finishing write tid 5 to smithi12223906-20 2024-09-15T03:47:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 23 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:47:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:480: left oid 20 (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:47:58.801 INFO:tasks.rados.rados.0.smithi122.stdout:481: expect (ObjNum 156 snap 37 seq_num 156) 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:483: finishing write tid 1 to smithi12223906-16 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:483: finishing write tid 2 to smithi12223906-16 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:483: finishing write tid 3 to smithi12223906-16 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:483: finishing write tid 4 to smithi12223906-16 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:483: finishing write tid 5 to smithi12223906-16 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 134 (ObjNum 202 snap 56 seq_num 202) dirty exists 2024-09-15T03:47:59.001 INFO:tasks.rados.rados.0.smithi122.stdout:483: left oid 16 (ObjNum 202 snap 56 seq_num 202) 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:484: finishing copy_from racing read to smithi12223906-14 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:484: finishing copy_from to smithi12223906-14 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 95 (ObjNum 165 snap 44 seq_num 165) dirty exists 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:480: done (3 left) 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:481: done (2 left) 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:483: done (1 left) 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:484: done (0 left) 2024-09-15T03:47:59.002 INFO:tasks.rados.rados.0.smithi122.stdout:485: delete oid 11 current snap is 56 2024-09-15T03:47:59.008 INFO:tasks.rados.rados.0.smithi122.stdout:485: done (0 left) 2024-09-15T03:47:59.009 INFO:tasks.rados.rados.0.smithi122.stdout:486: read oid 34 snap -1 2024-09-15T03:47:59.009 INFO:tasks.rados.rados.0.smithi122.stdout:486: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:47:59.009 INFO:tasks.rados.rados.0.smithi122.stdout:487: write (excl) oid 36 current snap is 56 2024-09-15T03:47:59.009 INFO:tasks.rados.rados.0.smithi122.stdout:487: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2024-09-15T03:47:59.019 INFO:tasks.rados.rados.0.smithi122.stdout:487: writing smithi12223906-36 from 628628 to 1404602 tid 1 2024-09-15T03:47:59.025 INFO:tasks.rados.rados.0.smithi122.stdout:487: writing smithi12223906-36 from 1937856 to 2464917 tid 2 2024-09-15T03:47:59.031 INFO:tasks.rados.rados.0.smithi122.stdout:487: writing smithi12223906-36 from 3252761 to 3428628 tid 3 2024-09-15T03:47:59.031 INFO:tasks.rados.rados.0.smithi122.stdout:488: read oid 13 snap -1 2024-09-15T03:47:59.031 INFO:tasks.rados.rados.0.smithi122.stdout:488: expect (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:47:59.031 INFO:tasks.rados.rados.0.smithi122.stdout:489: copy_from oid 15 from oid 21 current snap is 56 2024-09-15T03:47:59.031 INFO:tasks.rados.rados.0.smithi122.stdout:486: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:487: finishing write tid 1 to smithi12223906-36 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:487: finishing write tid 2 to smithi12223906-36 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:486: done (3 left) 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:490: write (excl) oid 2 current snap is 56 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:489: finishing copy_from to smithi12223906-15 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:489: got expected ENOENT (src dne) 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:489: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:487: finishing write tid 3 to smithi12223906-36 2024-09-15T03:47:59.154 INFO:tasks.rados.rados.0.smithi122.stdout:487: finishing write tid 4 to smithi12223906-36 2024-09-15T03:47:59.155 INFO:tasks.rados.rados.0.smithi122.stdout:487: finishing write tid 5 to smithi12223906-36 2024-09-15T03:47:59.155 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 278 (ObjNum 204 snap 56 seq_num 204) dirty exists 2024-09-15T03:47:59.155 INFO:tasks.rados.rados.0.smithi122.stdout:487: left oid 36 (ObjNum 204 snap 56 seq_num 204) 2024-09-15T03:47:59.155 INFO:tasks.rados.rados.0.smithi122.stdout:490: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2024-09-15T03:47:59.160 INFO:tasks.rados.rados.0.smithi122.stdout:490: writing smithi12223906-2 from 645435 to 1096658 tid 1 2024-09-15T03:47:59.169 INFO:tasks.rados.rados.0.smithi122.stdout:490: writing smithi12223906-2 from 1879985 to 2667057 tid 2 2024-09-15T03:47:59.174 INFO:tasks.rados.rados.0.smithi122.stdout:490: writing smithi12223906-2 from 3280184 to 3445435 tid 3 2024-09-15T03:47:59.175 INFO:tasks.rados.rados.0.smithi122.stdout:487: done (3 left) 2024-09-15T03:47:59.175 INFO:tasks.rados.rados.0.smithi122.stdout:489: done (2 left) 2024-09-15T03:47:59.175 INFO:tasks.rados.rados.0.smithi122.stdout:491: read oid 44 snap -1 2024-09-15T03:47:59.175 INFO:tasks.rados.rados.0.smithi122.stdout:491: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:47:59.175 INFO:tasks.rados.rados.0.smithi122.stdout:492: snap_remove snap 51 2024-09-15T03:47:59.797 INFO:tasks.rados.rados.0.smithi122.stdout:492: done (3 left) 2024-09-15T03:47:59.797 INFO:tasks.rados.rados.0.smithi122.stdout:493: read oid 32 snap 48 2024-09-15T03:47:59.797 INFO:tasks.rados.rados.0.smithi122.stdout:493: expect deleted 2024-09-15T03:47:59.797 INFO:tasks.rados.rados.0.smithi122.stdout:494: copy_from oid 28 from oid 1 current snap is 56 2024-09-15T03:47:59.797 INFO:tasks.rados.rados.0.smithi122.stdout:488: expect (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:490: finishing write tid 1 to smithi12223906-2 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:490: finishing write tid 2 to smithi12223906-2 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:490: finishing write tid 3 to smithi12223906-2 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:490: finishing write tid 4 to smithi12223906-2 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:490: finishing write tid 5 to smithi12223906-2 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 133 (ObjNum 205 snap 56 seq_num 205) dirty exists 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:490: left oid 2 (ObjNum 205 snap 56 seq_num 205) 2024-09-15T03:47:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:491: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:494: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:494: finishing copy_from to smithi12223906-28 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 130 (ObjNum 196 snap 55 seq_num 196) dirty exists 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:488: done (4 left) 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:490: done (3 left) 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:491: done (2 left) 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:493: done (1 left) 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:494: done (0 left) 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:495: write oid 11 current snap is 56 2024-09-15T03:48:00.010 INFO:tasks.rados.rados.0.smithi122.stdout:495: seq_num 206 ranges {662242=642825,2054820=530730,3307607=154635} 2024-09-15T03:48:00.025 INFO:tasks.rados.rados.0.smithi122.stdout:495: writing smithi12223906-11 from 662242 to 1305067 tid 1 2024-09-15T03:48:00.033 INFO:tasks.rados.rados.0.smithi122.stdout:495: writing smithi12223906-11 from 2054820 to 2585550 tid 2 2024-09-15T03:48:00.037 INFO:tasks.rados.rados.0.smithi122.stdout:495: writing smithi12223906-11 from 3307607 to 3462242 tid 3 2024-09-15T03:48:00.037 INFO:tasks.rados.rados.0.smithi122.stdout:496: snap_remove snap 48 2024-09-15T03:48:00.809 INFO:tasks.rados.rados.0.smithi122.stdout:496: done (1 left) 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:497: rollback oid 48 current snap is 56 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 48 to 50 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: finishing write tid 1 to smithi12223906-11 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: finishing write tid 2 to smithi12223906-11 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: finishing write tid 3 to smithi12223906-11 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: finishing write tid 4 to smithi12223906-11 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: finishing write tid 5 to smithi12223906-11 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 139 (ObjNum 206 snap 56 seq_num 206) dirty exists 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: left oid 11 (ObjNum 206 snap 56 seq_num 206) 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:495: done (1 left) 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:498: write (excl) oid 22 current snap is 56 2024-09-15T03:48:00.810 INFO:tasks.rados.rados.0.smithi122.stdout:498: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2024-09-15T03:48:00.819 INFO:tasks.rados.rados.0.smithi122.stdout:498: writing smithi12223906-22 from 679049 to 1397123 tid 1 2024-09-15T03:48:00.827 INFO:tasks.rados.rados.0.smithi122.stdout:498: writing smithi12223906-22 from 2113302 to 2787690 tid 2 2024-09-15T03:48:00.831 INFO:tasks.rados.rados.0.smithi122.stdout:498: writing smithi12223906-22 from 3335030 to 3479049 tid 3 2024-09-15T03:48:00.831 INFO:tasks.rados.rados.0.smithi122.stdout:499: copy_from oid 35 from oid 12 current snap is 56 2024-09-15T03:48:00.831 INFO:tasks.rados.rados.0.smithi122.stdout:497: finishing rollback tid 0 to smithi12223906-48 2024-09-15T03:48:00.831 INFO:tasks.rados.rados.0.smithi122.stdout:497: finishing rollback tid 1 to smithi12223906-48 2024-09-15T03:48:00.832 INFO:tasks.rados.rados.0.smithi122.stdout:497: finishing rollback tid 2 to smithi12223906-48 2024-09-15T03:48:00.832 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 129 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-15T03:48:00.832 INFO:tasks.rados.rados.0.smithi122.stdout:498: finishing write tid 1 to smithi12223906-22 2024-09-15T03:48:00.832 INFO:tasks.rados.rados.0.smithi122.stdout:498: finishing write tid 2 to smithi12223906-22 2024-09-15T03:48:00.832 INFO:tasks.rados.rados.0.smithi122.stdout:497: done (2 left) 2024-09-15T03:48:00.832 INFO:tasks.rados.rados.0.smithi122.stdout:500: delete oid 17 current snap is 56 2024-09-15T03:48:00.833 INFO:tasks.rados.rados.0.smithi122.stdout:498: finishing write tid 3 to smithi12223906-22 2024-09-15T03:48:00.833 INFO:tasks.rados.rados.0.smithi122.stdout:498: finishing write tid 4 to smithi12223906-22 2024-09-15T03:48:00.834 INFO:tasks.rados.rados.0.smithi122.stdout:498: finishing write tid 5 to smithi12223906-22 2024-09-15T03:48:00.834 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 242 (ObjNum 207 snap 56 seq_num 207) dirty exists 2024-09-15T03:48:00.834 INFO:tasks.rados.rados.0.smithi122.stdout:498: left oid 22 (ObjNum 207 snap 56 seq_num 207) 2024-09-15T03:48:00.838 INFO:tasks.rados.rados.0.smithi122.stdout:498: done (2 left) 2024-09-15T03:48:00.838 INFO:tasks.rados.rados.0.smithi122.stdout:500: done (1 left) 2024-09-15T03:48:00.838 INFO:tasks.rados.rados.0.smithi122.stdout:501: write oid 34 current snap is 56 2024-09-15T03:48:00.838 INFO:tasks.rados.rados.0.smithi122.stdout:501: seq_num 209 ranges {712663=468572,1713913=678057,2873523=591733} 2024-09-15T03:48:00.848 INFO:tasks.rados.rados.0.smithi122.stdout:501: writing smithi12223906-34 from 712663 to 1181235 tid 1 2024-09-15T03:48:00.856 INFO:tasks.rados.rados.0.smithi122.stdout:501: writing smithi12223906-34 from 1713913 to 2391970 tid 2 2024-09-15T03:48:00.863 INFO:tasks.rados.rados.0.smithi122.stdout:501: writing smithi12223906-34 from 2873523 to 3465256 tid 3 2024-09-15T03:48:00.864 INFO:tasks.rados.rados.0.smithi122.stdout:502: copy_from oid 6 from oid 43 current snap is 56 2024-09-15T03:48:00.864 INFO:tasks.rados.rados.0.smithi122.stdout:499: finishing copy_from racing read to smithi12223906-35 2024-09-15T03:48:00.864 INFO:tasks.rados.rados.0.smithi122.stdout:501: finishing write tid 1 to smithi12223906-34 2024-09-15T03:48:00.865 INFO:tasks.rados.rados.0.smithi122.stdout:499: finishing copy_from to smithi12223906-35 2024-09-15T03:48:00.865 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 25 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:48:00.865 INFO:tasks.rados.rados.0.smithi122.stdout:501: finishing write tid 2 to smithi12223906-34 2024-09-15T03:48:00.865 INFO:tasks.rados.rados.0.smithi122.stdout:499: done (2 left) 2024-09-15T03:48:00.865 INFO:tasks.rados.rados.0.smithi122.stdout:503: snap_create 2024-09-15T03:48:00.867 INFO:tasks.rados.rados.0.smithi122.stdout:501: finishing write tid 3 to smithi12223906-34 2024-09-15T03:48:00.869 INFO:tasks.rados.rados.0.smithi122.stdout:501: finishing write tid 4 to smithi12223906-34 2024-09-15T03:48:00.869 INFO:tasks.rados.rados.0.smithi122.stdout:501: finishing write tid 5 to smithi12223906-34 2024-09-15T03:48:00.870 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 69 (ObjNum 209 snap 56 seq_num 209) dirty exists 2024-09-15T03:48:00.870 INFO:tasks.rados.rados.0.smithi122.stdout:501: left oid 34 (ObjNum 209 snap 56 seq_num 209) 2024-09-15T03:48:00.880 INFO:tasks.rados.rados.0.smithi122.stdout:502: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:48:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:502: finishing copy_from to smithi12223906-6 2024-09-15T03:48:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 217 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:48:01.730 INFO:tasks.rados.rados.0.smithi122.stdout:501: done (2 left) 2024-09-15T03:48:01.730 INFO:tasks.rados.rados.0.smithi122.stdout:502: done (1 left) 2024-09-15T03:48:01.730 INFO:tasks.rados.rados.0.smithi122.stdout:503: done (0 left) 2024-09-15T03:48:01.731 INFO:tasks.rados.rados.0.smithi122.stdout:504: snap_remove snap 50 2024-09-15T03:48:02.736 INFO:tasks.rados.rados.0.smithi122.stdout:504: done (0 left) 2024-09-15T03:48:02.736 INFO:tasks.rados.rados.0.smithi122.stdout:505: delete oid 44 current snap is 57 2024-09-15T03:48:02.742 INFO:tasks.rados.rados.0.smithi122.stdout:505: done (0 left) 2024-09-15T03:48:02.742 INFO:tasks.rados.rados.0.smithi122.stdout:506: write oid 33 current snap is 57 2024-09-15T03:48:02.742 INFO:tasks.rados.rados.0.smithi122.stdout:506: seq_num 211 ranges {746277=619070,1830877=681726,2928369=617908} 2024-09-15T03:48:02.756 INFO:tasks.rados.rados.0.smithi122.stdout:506: writing smithi12223906-33 from 746277 to 1365347 tid 1 2024-09-15T03:48:02.764 INFO:tasks.rados.rados.0.smithi122.stdout:506: writing smithi12223906-33 from 1830877 to 2512603 tid 2 2024-09-15T03:48:02.770 INFO:tasks.rados.rados.0.smithi122.stdout:506: writing smithi12223906-33 from 2928369 to 3546277 tid 3 2024-09-15T03:48:02.770 INFO:tasks.rados.rados.0.smithi122.stdout:507: snap_create 2024-09-15T03:48:02.770 INFO:tasks.rados.rados.0.smithi122.stdout:506: finishing write tid 1 to smithi12223906-33 2024-09-15T03:48:02.770 INFO:tasks.rados.rados.0.smithi122.stdout:506: finishing write tid 2 to smithi12223906-33 2024-09-15T03:48:02.775 INFO:tasks.rados.rados.0.smithi122.stdout:506: finishing write tid 3 to smithi12223906-33 2024-09-15T03:48:02.775 INFO:tasks.rados.rados.0.smithi122.stdout:506: finishing write tid 4 to smithi12223906-33 2024-09-15T03:48:02.775 INFO:tasks.rados.rados.0.smithi122.stdout:506: finishing write tid 5 to smithi12223906-33 2024-09-15T03:48:02.776 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 251 (ObjNum 211 snap 57 seq_num 211) dirty exists 2024-09-15T03:48:02.776 INFO:tasks.rados.rados.0.smithi122.stdout:506: left oid 33 (ObjNum 211 snap 57 seq_num 211) 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:506: done (1 left) 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:507: done (0 left) 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:508: copy_from oid 19 from oid 20 current snap is 58 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:509: read oid 44 snap 57 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:509: expect deleted 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:510: write (excl) oid 4 current snap is 58 2024-09-15T03:48:03.740 INFO:tasks.rados.rados.0.smithi122.stdout:510: seq_num 212 ranges {763084=694319,1889359=541737,2955792=541902} 2024-09-15T03:48:03.755 INFO:tasks.rados.rados.0.smithi122.stdout:510: writing smithi12223906-4 from 763084 to 1457403 tid 1 2024-09-15T03:48:03.760 INFO:tasks.rados.rados.0.smithi122.stdout:510: writing smithi12223906-4 from 1889359 to 2431096 tid 2 2024-09-15T03:48:03.766 INFO:tasks.rados.rados.0.smithi122.stdout:510: writing smithi12223906-4 from 2955792 to 3497694 tid 3 2024-09-15T03:48:03.766 INFO:tasks.rados.rados.0.smithi122.stdout:511: read oid 23 snap -1 2024-09-15T03:48:03.766 INFO:tasks.rados.rados.0.smithi122.stdout:511: expect (ObjNum 197 snap 55 seq_num 197) 2024-09-15T03:48:03.767 INFO:tasks.rados.rados.0.smithi122.stdout:512: rollback oid 34 current snap is 58 2024-09-15T03:48:03.767 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 57 2024-09-15T03:48:03.767 INFO:tasks.rados.rados.0.smithi122.stdout:508: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:48:03.767 INFO:tasks.rados.rados.0.smithi122.stdout:508: finishing copy_from to smithi12223906-19 2024-09-15T03:48:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 23 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:48:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:508: done (4 left) 2024-09-15T03:48:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:509: done (3 left) 2024-09-15T03:48:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:513: copy_from oid 31 from oid 27 current snap is 58 2024-09-15T03:48:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:514: write oid 24 current snap is 58 2024-09-15T03:48:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:514: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2024-09-15T03:48:03.774 INFO:tasks.rados.rados.0.smithi122.stdout:514: writing smithi12223906-24 from 779891 to 1265812 tid 1 2024-09-15T03:48:03.782 INFO:tasks.rados.rados.0.smithi122.stdout:514: writing smithi12223906-24 from 1947841 to 2633236 tid 2 2024-09-15T03:48:03.788 INFO:tasks.rados.rados.0.smithi122.stdout:514: writing smithi12223906-24 from 3383215 to 3579891 tid 3 2024-09-15T03:48:03.788 INFO:tasks.rados.rados.0.smithi122.stdout:515: rollback oid 17 current snap is 58 2024-09-15T03:48:03.788 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 57 2024-09-15T03:48:03.788 INFO:tasks.rados.rados.0.smithi122.stdout:510: finishing write tid 1 to smithi12223906-4 2024-09-15T03:48:03.788 INFO:tasks.rados.rados.0.smithi122.stdout:510: finishing write tid 2 to smithi12223906-4 2024-09-15T03:48:03.788 INFO:tasks.rados.rados.0.smithi122.stdout:510: finishing write tid 3 to smithi12223906-4 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:510: finishing write tid 4 to smithi12223906-4 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:510: finishing write tid 5 to smithi12223906-4 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 58 (ObjNum 212 snap 58 seq_num 212) dirty exists 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:510: left oid 4 (ObjNum 212 snap 58 seq_num 212) 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:512: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:510: done (5 left) 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:516: write (excl) oid 18 current snap is 58 2024-09-15T03:48:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:511: expect (ObjNum 197 snap 55 seq_num 197) 2024-09-15T03:48:03.979 INFO:tasks.rados.rados.0.smithi122.stdout:514: finishing write tid 1 to smithi12223906-24 2024-09-15T03:48:03.979 INFO:tasks.rados.rados.0.smithi122.stdout:512: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:48:03.979 INFO:tasks.rados.rados.0.smithi122.stdout:512: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:48:03.979 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 75 (ObjNum 209 snap 56 seq_num 209) dirty exists 2024-09-15T03:48:03.980 INFO:tasks.rados.rados.0.smithi122.stdout:514: finishing write tid 2 to smithi12223906-24 2024-09-15T03:48:03.980 INFO:tasks.rados.rados.0.smithi122.stdout:516: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2024-09-15T03:48:03.987 INFO:tasks.rados.rados.0.smithi122.stdout:516: writing smithi12223906-18 from 796698 to 1357868 tid 1 2024-09-15T03:48:03.995 INFO:tasks.rados.rados.0.smithi122.stdout:516: writing smithi12223906-18 from 2006323 to 2551729 tid 2 2024-09-15T03:48:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:516: writing smithi12223906-18 from 3126991 to 3596698 tid 3 2024-09-15T03:48:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:511: done (5 left) 2024-09-15T03:48:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:512: done (4 left) 2024-09-15T03:48:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:517: write oid 48 current snap is 58 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:515: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 0 (ObjNum 33 snap 0 seq_num 1817658064) dirty dne 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:514: finishing write tid 3 to smithi12223906-24 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:514: finishing write tid 4 to smithi12223906-24 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:514: finishing write tid 5 to smithi12223906-24 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 301 (ObjNum 213 snap 58 seq_num 213) dirty exists 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:514: left oid 24 (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:513: finishing copy_from racing read to smithi12223906-31 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:513: finishing copy_from to smithi12223906-31 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 170 (ObjNum 115 snap 23 seq_num 115) dirty exists 2024-09-15T03:48:04.002 INFO:tasks.rados.rados.0.smithi122.stdout:517: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2024-09-15T03:48:04.007 INFO:tasks.rados.rados.0.smithi122.stdout:517: writing smithi12223906-48 from 413505 to 1049924 tid 1 2024-09-15T03:48:04.015 INFO:tasks.rados.rados.0.smithi122.stdout:517: writing smithi12223906-48 from 1664805 to 2353869 tid 2 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:517: writing smithi12223906-48 from 3038061 to 3530132 tid 3 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:513: done (4 left) 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:514: done (3 left) 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:515: done (2 left) 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:518: rollback oid 41 current snap is 58 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 56 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:516: finishing write tid 1 to smithi12223906-18 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:516: finishing write tid 2 to smithi12223906-18 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:516: finishing write tid 3 to smithi12223906-18 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:516: finishing write tid 4 to smithi12223906-18 2024-09-15T03:48:04.022 INFO:tasks.rados.rados.0.smithi122.stdout:516: finishing write tid 5 to smithi12223906-18 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 256 (ObjNum 214 snap 58 seq_num 214) dirty exists 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:516: left oid 18 (ObjNum 214 snap 58 seq_num 214) 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:517: finishing write tid 1 to smithi12223906-48 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:517: finishing write tid 2 to smithi12223906-48 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:516: done (2 left) 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:519: rollback oid 38 current snap is 58 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 57 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:520: rollback oid 35 current snap is 58 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 56 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:521: read oid 42 snap -1 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:521: expect deleted 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:522: write (excl) oid 36 current snap is 58 2024-09-15T03:48:04.023 INFO:tasks.rados.rados.0.smithi122.stdout:522: seq_num 216 ranges {430312=711668,1723287=549075,2781837=703343} 2024-09-15T03:48:04.028 INFO:tasks.rados.rados.0.smithi122.stdout:522: writing smithi12223906-36 from 430312 to 1141980 tid 1 2024-09-15T03:48:04.034 INFO:tasks.rados.rados.0.smithi122.stdout:522: writing smithi12223906-36 from 1723287 to 2272362 tid 2 2024-09-15T03:48:04.042 INFO:tasks.rados.rados.0.smithi122.stdout:522: writing smithi12223906-36 from 2781837 to 3485180 tid 3 2024-09-15T03:48:04.042 INFO:tasks.rados.rados.0.smithi122.stdout:523: copy_from oid 32 from oid 31 current snap is 58 2024-09-15T03:48:04.042 INFO:tasks.rados.rados.0.smithi122.stdout:517: finishing write tid 3 to smithi12223906-48 2024-09-15T03:48:04.042 INFO:tasks.rados.rados.0.smithi122.stdout:517: finishing write tid 4 to smithi12223906-48 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:517: finishing write tid 5 to smithi12223906-48 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 133 (ObjNum 215 snap 58 seq_num 215) dirty exists 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:517: left oid 48 (ObjNum 215 snap 58 seq_num 215) 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:518: finishing rollback tid 0 to smithi12223906-41 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:520: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:518: finishing rollback tid 1 to smithi12223906-41 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:519: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:519: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:519: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 116 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:48:04.043 INFO:tasks.rados.rados.0.smithi122.stdout:517: done (6 left) 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:519: done (5 left) 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:521: done (4 left) 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:524: rollback oid 44 current snap is 58 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 57 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:518: finishing rollback tid 2 to smithi12223906-41 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 87 (ObjNum 156 snap 37 seq_num 156) dirty exists 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:520: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:520: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 76 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:518: done (4 left) 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:520: done (3 left) 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:525: read oid 40 snap 57 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:525: expect (ObjNum 190 snap 53 seq_num 190) 2024-09-15T03:48:04.044 INFO:tasks.rados.rados.0.smithi122.stdout:526: read oid 34 snap -1 2024-09-15T03:48:04.045 INFO:tasks.rados.rados.0.smithi122.stdout:526: expect (ObjNum 209 snap 56 seq_num 209) 2024-09-15T03:48:04.045 INFO:tasks.rados.rados.0.smithi122.stdout:527: snap_create 2024-09-15T03:48:04.045 INFO:tasks.rados.rados.0.smithi122.stdout:522: finishing write tid 1 to smithi12223906-36 2024-09-15T03:48:04.045 INFO:tasks.rados.rados.0.smithi122.stdout:522: finishing write tid 2 to smithi12223906-36 2024-09-15T03:48:04.046 INFO:tasks.rados.rados.0.smithi122.stdout:524: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:48:04.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 246 (ObjNum 593 snap 0 seq_num 1842304488) dirty dne 2024-09-15T03:48:04.049 INFO:tasks.rados.rados.0.smithi122.stdout:522: finishing write tid 3 to smithi12223906-36 2024-09-15T03:48:04.049 INFO:tasks.rados.rados.0.smithi122.stdout:522: finishing write tid 4 to smithi12223906-36 2024-09-15T03:48:04.049 INFO:tasks.rados.rados.0.smithi122.stdout:522: finishing write tid 5 to smithi12223906-36 2024-09-15T03:48:04.050 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 306 (ObjNum 216 snap 58 seq_num 216) dirty exists 2024-09-15T03:48:04.050 INFO:tasks.rados.rados.0.smithi122.stdout:522: left oid 36 (ObjNum 216 snap 58 seq_num 216) 2024-09-15T03:48:04.050 INFO:tasks.rados.rados.0.smithi122.stdout:525: expect (ObjNum 190 snap 53 seq_num 190) 2024-09-15T03:48:04.159 INFO:tasks.rados.rados.0.smithi122.stdout:526: expect (ObjNum 209 snap 56 seq_num 209) 2024-09-15T03:48:04.314 INFO:tasks.rados.rados.0.smithi122.stdout:523: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:48:04.314 INFO:tasks.rados.rados.0.smithi122.stdout:523: finishing copy_from to smithi12223906-32 2024-09-15T03:48:04.314 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 170 (ObjNum 115 snap 23 seq_num 115) dirty exists 2024-09-15T03:48:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:522: done (5 left) 2024-09-15T03:48:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:523: done (4 left) 2024-09-15T03:48:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:524: done (3 left) 2024-09-15T03:48:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:525: done (2 left) 2024-09-15T03:48:04.831 INFO:tasks.rados.rados.0.smithi122.stdout:526: done (1 left) 2024-09-15T03:48:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:527: done (0 left) 2024-09-15T03:48:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:528: write (excl) oid 30 current snap is 59 2024-09-15T03:48:04.832 INFO:tasks.rados.rados.0.smithi122.stdout:528: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2024-09-15T03:48:04.840 INFO:tasks.rados.rados.0.smithi122.stdout:528: writing smithi12223906-30 from 447119 to 1234036 tid 1 2024-09-15T03:48:04.848 INFO:tasks.rados.rados.0.smithi122.stdout:528: writing smithi12223906-30 from 1665416 to 2358149 tid 2 2024-09-15T03:48:04.854 INFO:tasks.rados.rados.0.smithi122.stdout:528: writing smithi12223906-30 from 2976554 to 3491169 tid 3 2024-09-15T03:48:04.854 INFO:tasks.rados.rados.0.smithi122.stdout:529: copy_from oid 41 from oid 5 current snap is 59 2024-09-15T03:48:04.854 INFO:tasks.rados.rados.0.smithi122.stdout:530: read oid 27 snap -1 2024-09-15T03:48:04.854 INFO:tasks.rados.rados.0.smithi122.stdout:530: expect (ObjNum 115 snap 23 seq_num 115) 2024-09-15T03:48:04.854 INFO:tasks.rados.rados.0.smithi122.stdout:531: write oid 17 current snap is 59 2024-09-15T03:48:04.855 INFO:tasks.rados.rados.0.smithi122.stdout:531: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2024-09-15T03:48:04.859 INFO:tasks.rados.rados.0.smithi122.stdout:531: writing smithi12223906-17 from 463926 to 926092 tid 1 2024-09-15T03:48:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:531: writing smithi12223906-17 from 1723898 to 2160289 tid 2 2024-09-15T03:48:04.874 INFO:tasks.rados.rados.0.smithi122.stdout:531: writing smithi12223906-17 from 2603977 to 3329864 tid 3 2024-09-15T03:48:04.874 INFO:tasks.rados.rados.0.smithi122.stdout:532: snap_remove snap 58 2024-09-15T03:48:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:532: done (4 left) 2024-09-15T03:48:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:533: read oid 6 snap -1 2024-09-15T03:48:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:533: expect (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:48:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:534: write (excl) oid 26 current snap is 59 2024-09-15T03:48:05.840 INFO:tasks.rados.rados.0.smithi122.stdout:534: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2024-09-15T03:48:05.847 INFO:tasks.rados.rados.0.smithi122.stdout:534: writing smithi12223906-26 from 480733 to 1018148 tid 1 2024-09-15T03:48:05.856 INFO:tasks.rados.rados.0.smithi122.stdout:534: writing smithi12223906-26 from 1782380 to 2478782 tid 2 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:534: writing smithi12223906-26 from 3031400 to 3568559 tid 3 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:535: rollback oid 8 current snap is 59 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 57 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:528: finishing write tid 1 to smithi12223906-30 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:528: finishing write tid 2 to smithi12223906-30 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:528: finishing write tid 3 to smithi12223906-30 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:528: finishing write tid 4 to smithi12223906-30 2024-09-15T03:48:05.863 INFO:tasks.rados.rados.0.smithi122.stdout:528: finishing write tid 5 to smithi12223906-30 2024-09-15T03:48:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 92 (ObjNum 217 snap 59 seq_num 217) dirty exists 2024-09-15T03:48:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:528: left oid 30 (ObjNum 217 snap 59 seq_num 217) 2024-09-15T03:48:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:531: finishing write tid 1 to smithi12223906-17 2024-09-15T03:48:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:530: expect (ObjNum 115 snap 23 seq_num 115) 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:531: finishing write tid 2 to smithi12223906-17 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:529: finishing copy_from racing read to smithi12223906-41 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:528: done (6 left) 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:530: done (5 left) 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:536: rollback oid 21 current snap is 59 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 57 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:531: finishing write tid 3 to smithi12223906-17 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:531: finishing write tid 4 to smithi12223906-17 2024-09-15T03:48:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:531: finishing write tid 5 to smithi12223906-17 2024-09-15T03:48:06.055 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 138 (ObjNum 218 snap 59 seq_num 218) dirty exists 2024-09-15T03:48:06.055 INFO:tasks.rados.rados.0.smithi122.stdout:531: left oid 17 (ObjNum 218 snap 59 seq_num 218) 2024-09-15T03:48:06.055 INFO:tasks.rados.rados.0.smithi122.stdout:529: finishing copy_from to smithi12223906-41 2024-09-15T03:48:06.055 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 66 (ObjNum 191 snap 53 seq_num 191) dirty exists 2024-09-15T03:48:06.055 INFO:tasks.rados.rados.0.smithi122.stdout:533: expect (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:48:06.174 INFO:tasks.rados.rados.0.smithi122.stdout:534: finishing write tid 1 to smithi12223906-26 2024-09-15T03:48:06.174 INFO:tasks.rados.rados.0.smithi122.stdout:534: finishing write tid 2 to smithi12223906-26 2024-09-15T03:48:06.174 INFO:tasks.rados.rados.0.smithi122.stdout:534: finishing write tid 3 to smithi12223906-26 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:534: finishing write tid 4 to smithi12223906-26 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:534: finishing write tid 5 to smithi12223906-26 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 121 (ObjNum 219 snap 59 seq_num 219) dirty exists 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:534: left oid 26 (ObjNum 219 snap 59 seq_num 219) 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:535: finishing rollback tid 0 to smithi12223906-8 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:535: finishing rollback tid 1 to smithi12223906-8 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:535: finishing rollback tid 2 to smithi12223906-8 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 125 (ObjNum 186 snap 48 seq_num 186) dirty exists 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:536: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 0 (ObjNum 33 snap 0 seq_num 1842679056) dirty dne 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:529: done (5 left) 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:531: done (4 left) 2024-09-15T03:48:06.175 INFO:tasks.rados.rados.0.smithi122.stdout:533: done (3 left) 2024-09-15T03:48:06.176 INFO:tasks.rados.rados.0.smithi122.stdout:534: done (2 left) 2024-09-15T03:48:06.176 INFO:tasks.rados.rados.0.smithi122.stdout:535: done (1 left) 2024-09-15T03:48:06.176 INFO:tasks.rados.rados.0.smithi122.stdout:536: done (0 left) 2024-09-15T03:48:06.176 INFO:tasks.rados.rados.0.smithi122.stdout:537: write (excl) oid 7 current snap is 59 2024-09-15T03:48:06.176 INFO:tasks.rados.rados.0.smithi122.stdout:537: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2024-09-15T03:48:06.189 INFO:tasks.rados.rados.0.smithi122.stdout:537: writing smithi12223906-7 from 497540 to 1110204 tid 1 2024-09-15T03:48:06.196 INFO:tasks.rados.rados.0.smithi122.stdout:537: writing smithi12223906-7 from 1840862 to 2280922 tid 2 2024-09-15T03:48:06.202 INFO:tasks.rados.rados.0.smithi122.stdout:537: writing smithi12223906-7 from 3058823 to 3523607 tid 3 2024-09-15T03:48:06.202 INFO:tasks.rados.rados.0.smithi122.stdout:538: snap_create 2024-09-15T03:48:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:537: finishing write tid 1 to smithi12223906-7 2024-09-15T03:48:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:537: finishing write tid 2 to smithi12223906-7 2024-09-15T03:48:06.207 INFO:tasks.rados.rados.0.smithi122.stdout:537: finishing write tid 3 to smithi12223906-7 2024-09-15T03:48:06.207 INFO:tasks.rados.rados.0.smithi122.stdout:537: finishing write tid 4 to smithi12223906-7 2024-09-15T03:48:06.207 INFO:tasks.rados.rados.0.smithi122.stdout:537: finishing write tid 5 to smithi12223906-7 2024-09-15T03:48:06.207 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 134 (ObjNum 220 snap 59 seq_num 220) dirty exists 2024-09-15T03:48:06.207 INFO:tasks.rados.rados.0.smithi122.stdout:537: left oid 7 (ObjNum 220 snap 59 seq_num 220) 2024-09-15T03:48:06.737 INFO:tasks.rados.rados.0.smithi122.stdout:537: done (1 left) 2024-09-15T03:48:06.737 INFO:tasks.rados.rados.0.smithi122.stdout:538: done (0 left) 2024-09-15T03:48:06.737 INFO:tasks.rados.rados.0.smithi122.stdout:539: snap_remove snap 57 2024-09-15T03:48:07.753 INFO:tasks.rados.rados.0.smithi122.stdout:539: done (0 left) 2024-09-15T03:48:07.753 INFO:tasks.rados.rados.0.smithi122.stdout:540: rollback oid 45 current snap is 60 2024-09-15T03:48:07.753 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 59 2024-09-15T03:48:07.754 INFO:tasks.rados.rados.0.smithi122.stdout:541: snap_remove snap 56 2024-09-15T03:48:08.755 INFO:tasks.rados.rados.0.smithi122.stdout:541: done (1 left) 2024-09-15T03:48:08.755 INFO:tasks.rados.rados.0.smithi122.stdout:542: delete oid 13 current snap is 60 2024-09-15T03:48:08.755 INFO:tasks.rados.rados.0.smithi122.stdout:540: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:48:08.755 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 0 (ObjNum 129 snap 0 seq_num 1843226760) dirty dne 2024-09-15T03:48:08.764 INFO:tasks.rados.rados.0.smithi122.stdout:540: done (1 left) 2024-09-15T03:48:08.765 INFO:tasks.rados.rados.0.smithi122.stdout:542: done (0 left) 2024-09-15T03:48:08.765 INFO:tasks.rados.rados.0.smithi122.stdout:543: copy_from oid 49 from oid 46 current snap is 60 2024-09-15T03:48:08.765 INFO:tasks.rados.rados.0.smithi122.stdout:544: read oid 35 snap -1 2024-09-15T03:48:08.765 INFO:tasks.rados.rados.0.smithi122.stdout:544: expect (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:48:08.765 INFO:tasks.rados.rados.0.smithi122.stdout:545: snap_create 2024-09-15T03:48:08.770 INFO:tasks.rados.rados.0.smithi122.stdout:543: finishing copy_from to smithi12223906-49 2024-09-15T03:48:08.770 INFO:tasks.rados.rados.0.smithi122.stdout:543: got expected ENOENT (src dne) 2024-09-15T03:48:08.770 INFO:tasks.rados.rados.0.smithi122.stdout:543: finishing copy_from racing read to smithi12223906-49 2024-09-15T03:48:08.784 INFO:tasks.rados.rados.0.smithi122.stdout:544: expect (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:48:09.772 INFO:tasks.rados.rados.0.smithi122.stdout:543: done (2 left) 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:544: done (1 left) 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:545: done (0 left) 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:546: rollback oid 45 current snap is 61 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 60 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:547: read oid 11 snap -1 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:547: expect (ObjNum 206 snap 56 seq_num 206) 2024-09-15T03:48:09.773 INFO:tasks.rados.rados.0.smithi122.stdout:548: delete oid 33 current snap is 61 2024-09-15T03:48:09.776 INFO:tasks.rados.rados.0.smithi122.stdout:546: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:48:09.776 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 0 (ObjNum 97 snap 0 seq_num 1842618944) dirty dne 2024-09-15T03:48:09.779 INFO:tasks.rados.rados.0.smithi122.stdout:546: done (2 left) 2024-09-15T03:48:09.779 INFO:tasks.rados.rados.0.smithi122.stdout:548: done (1 left) 2024-09-15T03:48:09.779 INFO:tasks.rados.rados.0.smithi122.stdout:549: copy_from oid 10 from oid 26 current snap is 61 2024-09-15T03:48:09.779 INFO:tasks.rados.rados.0.smithi122.stdout:550: snap_remove snap 60 2024-09-15T03:48:10.789 INFO:tasks.rados.rados.0.smithi122.stdout:550: done (2 left) 2024-09-15T03:48:10.790 INFO:tasks.rados.rados.0.smithi122.stdout:551: snap_remove snap 59 2024-09-15T03:48:11.804 INFO:tasks.rados.rados.0.smithi122.stdout:551: done (2 left) 2024-09-15T03:48:11.804 INFO:tasks.rados.rados.0.smithi122.stdout:552: write oid 2 current snap is 61 2024-09-15T03:48:11.804 INFO:tasks.rados.rados.0.smithi122.stdout:552: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2024-09-15T03:48:11.811 INFO:tasks.rados.rados.0.smithi122.stdout:552: writing smithi12223906-2 from 547961 to 1102725 tid 1 2024-09-15T03:48:11.818 INFO:tasks.rados.rados.0.smithi122.stdout:552: writing smithi12223906-2 from 1616308 to 2320048 tid 2 2024-09-15T03:48:11.825 INFO:tasks.rados.rados.0.smithi122.stdout:552: writing smithi12223906-2 from 2741092 to 3439692 tid 3 2024-09-15T03:48:11.825 INFO:tasks.rados.rados.0.smithi122.stdout:553: snap_create 2024-09-15T03:48:11.825 INFO:tasks.rados.rados.0.smithi122.stdout:547: expect (ObjNum 206 snap 56 seq_num 206) 2024-09-15T03:48:11.942 INFO:tasks.rados.rados.0.smithi122.stdout:549: finishing copy_from racing read to smithi12223906-10 2024-09-15T03:48:11.942 INFO:tasks.rados.rados.0.smithi122.stdout:549: finishing copy_from to smithi12223906-10 2024-09-15T03:48:11.942 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 121 (ObjNum 219 snap 59 seq_num 219) dirty exists 2024-09-15T03:48:11.942 INFO:tasks.rados.rados.0.smithi122.stdout:552: finishing write tid 1 to smithi12223906-2 2024-09-15T03:48:11.942 INFO:tasks.rados.rados.0.smithi122.stdout:552: finishing write tid 2 to smithi12223906-2 2024-09-15T03:48:11.943 INFO:tasks.rados.rados.0.smithi122.stdout:552: finishing write tid 3 to smithi12223906-2 2024-09-15T03:48:11.943 INFO:tasks.rados.rados.0.smithi122.stdout:552: finishing write tid 4 to smithi12223906-2 2024-09-15T03:48:11.943 INFO:tasks.rados.rados.0.smithi122.stdout:552: finishing write tid 5 to smithi12223906-2 2024-09-15T03:48:11.943 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 142 (ObjNum 223 snap 61 seq_num 223) dirty exists 2024-09-15T03:48:11.943 INFO:tasks.rados.rados.0.smithi122.stdout:552: left oid 2 (ObjNum 223 snap 61 seq_num 223) 2024-09-15T03:48:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:547: done (3 left) 2024-09-15T03:48:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:549: done (2 left) 2024-09-15T03:48:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:552: done (1 left) 2024-09-15T03:48:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:553: done (0 left) 2024-09-15T03:48:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:554: write oid 19 current snap is 62 2024-09-15T03:48:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:554: seq_num 224 ranges {564768=630013,1674790=447398,2768515=626225} 2024-09-15T03:48:12.825 INFO:tasks.rados.rados.0.smithi122.stdout:554: writing smithi12223906-19 from 564768 to 1194781 tid 1 2024-09-15T03:48:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:554: writing smithi12223906-19 from 1674790 to 2122188 tid 2 2024-09-15T03:48:12.840 INFO:tasks.rados.rados.0.smithi122.stdout:554: writing smithi12223906-19 from 2768515 to 3394740 tid 3 2024-09-15T03:48:12.840 INFO:tasks.rados.rados.0.smithi122.stdout:555: write (excl) oid 49 current snap is 62 2024-09-15T03:48:12.840 INFO:tasks.rados.rados.0.smithi122.stdout:555: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2024-09-15T03:48:12.847 INFO:tasks.rados.rados.0.smithi122.stdout:555: writing smithi12223906-49 from 581575 to 1286837 tid 1 2024-09-15T03:48:12.854 INFO:tasks.rados.rados.0.smithi122.stdout:555: writing smithi12223906-49 from 1733272 to 2440681 tid 2 2024-09-15T03:48:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:555: writing smithi12223906-49 from 3195938 to 3633435 tid 3 2024-09-15T03:48:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:556: write (excl) oid 14 current snap is 62 2024-09-15T03:48:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:554: finishing write tid 1 to smithi12223906-19 2024-09-15T03:48:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:556: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2024-09-15T03:48:12.869 INFO:tasks.rados.rados.0.smithi122.stdout:556: writing smithi12223906-14 from 598382 to 1378893 tid 1 2024-09-15T03:48:12.875 INFO:tasks.rados.rados.0.smithi122.stdout:556: writing smithi12223906-14 from 2075401 to 2526468 tid 2 2024-09-15T03:48:12.882 INFO:tasks.rados.rados.0.smithi122.stdout:556: writing smithi12223906-14 from 3107008 to 3755777 tid 3 2024-09-15T03:48:12.882 INFO:tasks.rados.rados.0.smithi122.stdout:557: write oid 4 current snap is 62 2024-09-15T03:48:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:554: finishing write tid 2 to smithi12223906-19 2024-09-15T03:48:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:557: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2024-09-15T03:48:12.888 INFO:tasks.rados.rados.0.smithi122.stdout:557: writing smithi12223906-4 from 615189 to 1070949 tid 1 2024-09-15T03:48:12.896 INFO:tasks.rados.rados.0.smithi122.stdout:557: writing smithi12223906-4 from 1733883 to 2444961 tid 2 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:557: writing smithi12223906-4 from 3134431 to 3594472 tid 3 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:554: finishing write tid 3 to smithi12223906-19 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:554: finishing write tid 4 to smithi12223906-19 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:554: finishing write tid 5 to smithi12223906-19 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 223 (ObjNum 224 snap 62 seq_num 224) dirty exists 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:554: left oid 19 (ObjNum 224 snap 62 seq_num 224) 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:555: finishing write tid 1 to smithi12223906-49 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:555: finishing write tid 2 to smithi12223906-49 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:555: finishing write tid 3 to smithi12223906-49 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:555: finishing write tid 4 to smithi12223906-49 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:555: finishing write tid 5 to smithi12223906-49 2024-09-15T03:48:12.904 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 146 (ObjNum 225 snap 62 seq_num 225) dirty exists 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:555: left oid 49 (ObjNum 225 snap 62 seq_num 225) 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:554: done (3 left) 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:555: done (2 left) 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:558: delete oid 15 current snap is 62 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:556: finishing write tid 1 to smithi12223906-14 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:556: finishing write tid 2 to smithi12223906-14 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:556: finishing write tid 3 to smithi12223906-14 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:556: finishing write tid 4 to smithi12223906-14 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:556: finishing write tid 5 to smithi12223906-14 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 319 (ObjNum 226 snap 62 seq_num 226) dirty exists 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:556: left oid 14 (ObjNum 226 snap 62 seq_num 226) 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:557: finishing write tid 1 to smithi12223906-4 2024-09-15T03:48:12.905 INFO:tasks.rados.rados.0.smithi122.stdout:557: finishing write tid 2 to smithi12223906-4 2024-09-15T03:48:12.907 INFO:tasks.rados.rados.0.smithi122.stdout:557: finishing write tid 3 to smithi12223906-4 2024-09-15T03:48:12.907 INFO:tasks.rados.rados.0.smithi122.stdout:557: finishing write tid 4 to smithi12223906-4 2024-09-15T03:48:12.909 INFO:tasks.rados.rados.0.smithi122.stdout:557: finishing write tid 5 to smithi12223906-4 2024-09-15T03:48:12.909 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 174 (ObjNum 227 snap 62 seq_num 227) dirty exists 2024-09-15T03:48:12.909 INFO:tasks.rados.rados.0.smithi122.stdout:557: left oid 4 (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:48:12.911 INFO:tasks.rados.rados.0.smithi122.stdout:556: done (2 left) 2024-09-15T03:48:12.911 INFO:tasks.rados.rados.0.smithi122.stdout:557: done (1 left) 2024-09-15T03:48:12.911 INFO:tasks.rados.rados.0.smithi122.stdout:558: done (0 left) 2024-09-15T03:48:12.911 INFO:tasks.rados.rados.0.smithi122.stdout:559: snap_create 2024-09-15T03:48:13.825 INFO:tasks.rados.rados.0.smithi122.stdout:559: done (0 left) 2024-09-15T03:48:13.825 INFO:tasks.rados.rados.0.smithi122.stdout:560: snap_create 2024-09-15T03:48:14.831 INFO:tasks.rados.rados.0.smithi122.stdout:560: done (0 left) 2024-09-15T03:48:14.831 INFO:tasks.rados.rados.0.smithi122.stdout:561: read oid 10 snap -1 2024-09-15T03:48:14.832 INFO:tasks.rados.rados.0.smithi122.stdout:561: expect (ObjNum 219 snap 59 seq_num 219) 2024-09-15T03:48:14.832 INFO:tasks.rados.rados.0.smithi122.stdout:562: delete oid 16 current snap is 64 2024-09-15T03:48:14.838 INFO:tasks.rados.rados.0.smithi122.stdout:562: done (1 left) 2024-09-15T03:48:14.839 INFO:tasks.rados.rados.0.smithi122.stdout:563: read oid 29 snap -1 2024-09-15T03:48:14.839 INFO:tasks.rados.rados.0.smithi122.stdout:563: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:48:14.839 INFO:tasks.rados.rados.0.smithi122.stdout:564: read oid 18 snap -1 2024-09-15T03:48:14.839 INFO:tasks.rados.rados.0.smithi122.stdout:564: expect (ObjNum 214 snap 58 seq_num 214) 2024-09-15T03:48:14.839 INFO:tasks.rados.rados.0.smithi122.stdout:565: snap_remove snap 62 2024-09-15T03:48:15.841 INFO:tasks.rados.rados.0.smithi122.stdout:565: done (3 left) 2024-09-15T03:48:15.841 INFO:tasks.rados.rados.0.smithi122.stdout:566: write oid 22 current snap is 64 2024-09-15T03:48:15.841 INFO:tasks.rados.rados.0.smithi122.stdout:566: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2024-09-15T03:48:15.852 INFO:tasks.rados.rados.0.smithi122.stdout:566: writing smithi12223906-22 from 665610 to 1463470 tid 1 2024-09-15T03:48:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:566: writing smithi12223906-22 from 1909329 to 2367734 tid 2 2024-09-15T03:48:15.862 INFO:tasks.rados.rados.0.smithi122.stdout:566: writing smithi12223906-22 from 2816700 to 3226910 tid 3 2024-09-15T03:48:15.862 INFO:tasks.rados.rados.0.smithi122.stdout:567: snap_remove snap 61 2024-09-15T03:48:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:567: done (4 left) 2024-09-15T03:48:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:568: rollback oid 47 current snap is 64 2024-09-15T03:48:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 47 to 63 2024-09-15T03:48:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:561: expect (ObjNum 219 snap 59 seq_num 219) 2024-09-15T03:48:16.915 INFO:tasks.rados.rados.0.smithi122.stdout:563: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:48:17.038 INFO:tasks.rados.rados.0.smithi122.stdout:564: expect (ObjNum 214 snap 58 seq_num 214) 2024-09-15T03:48:17.183 INFO:tasks.rados.rados.0.smithi122.stdout:566: finishing write tid 1 to smithi12223906-22 2024-09-15T03:48:17.183 INFO:tasks.rados.rados.0.smithi122.stdout:566: finishing write tid 2 to smithi12223906-22 2024-09-15T03:48:17.183 INFO:tasks.rados.rados.0.smithi122.stdout:566: finishing write tid 3 to smithi12223906-22 2024-09-15T03:48:17.183 INFO:tasks.rados.rados.0.smithi122.stdout:566: finishing write tid 4 to smithi12223906-22 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:566: finishing write tid 5 to smithi12223906-22 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 282 (ObjNum 230 snap 64 seq_num 230) dirty exists 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:566: left oid 22 (ObjNum 230 snap 64 seq_num 230) 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:561: done (4 left) 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:563: done (3 left) 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:564: done (2 left) 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:566: done (1 left) 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:569: write (excl) oid 22 current snap is 64 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:568: finishing rollback tid 0 to smithi12223906-47 2024-09-15T03:48:17.184 INFO:tasks.rados.rados.0.smithi122.stdout:569: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2024-09-15T03:48:17.196 INFO:tasks.rados.rados.0.smithi122.stdout:569: writing smithi12223906-22 from 682417 to 1155526 tid 1 2024-09-15T03:48:17.201 INFO:tasks.rados.rados.0.smithi122.stdout:569: writing smithi12223906-22 from 1567811 to 2169874 tid 2 2024-09-15T03:48:17.209 INFO:tasks.rados.rados.0.smithi122.stdout:569: writing smithi12223906-22 from 2844123 to 3465605 tid 3 2024-09-15T03:48:17.209 INFO:tasks.rados.rados.0.smithi122.stdout:570: read oid 30 snap -1 2024-09-15T03:48:17.209 INFO:tasks.rados.rados.0.smithi122.stdout:570: expect (ObjNum 217 snap 59 seq_num 217) 2024-09-15T03:48:17.209 INFO:tasks.rados.rados.0.smithi122.stdout:571: read oid 15 snap -1 2024-09-15T03:48:17.209 INFO:tasks.rados.rados.0.smithi122.stdout:571: expect deleted 2024-09-15T03:48:17.209 INFO:tasks.rados.rados.0.smithi122.stdout:572: snap_create 2024-09-15T03:48:17.210 INFO:tasks.rados.rados.0.smithi122.stdout:568: finishing rollback tid 1 to smithi12223906-47 2024-09-15T03:48:17.210 INFO:tasks.rados.rados.0.smithi122.stdout:568: finishing rollback tid 2 to smithi12223906-47 2024-09-15T03:48:17.210 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 84 (ObjNum 198 snap 55 seq_num 198) dirty exists 2024-09-15T03:48:17.210 INFO:tasks.rados.rados.0.smithi122.stdout:569: finishing write tid 1 to smithi12223906-22 2024-09-15T03:48:17.210 INFO:tasks.rados.rados.0.smithi122.stdout:569: finishing write tid 2 to smithi12223906-22 2024-09-15T03:48:17.215 INFO:tasks.rados.rados.0.smithi122.stdout:569: finishing write tid 3 to smithi12223906-22 2024-09-15T03:48:17.215 INFO:tasks.rados.rados.0.smithi122.stdout:569: finishing write tid 4 to smithi12223906-22 2024-09-15T03:48:17.216 INFO:tasks.rados.rados.0.smithi122.stdout:569: finishing write tid 5 to smithi12223906-22 2024-09-15T03:48:17.216 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 288 (ObjNum 231 snap 64 seq_num 231) dirty exists 2024-09-15T03:48:17.216 INFO:tasks.rados.rados.0.smithi122.stdout:569: left oid 22 (ObjNum 231 snap 64 seq_num 231) 2024-09-15T03:48:17.221 INFO:tasks.rados.rados.0.smithi122.stdout:570: expect (ObjNum 217 snap 59 seq_num 217) 2024-09-15T03:48:17.853 INFO:tasks.rados.rados.0.smithi122.stdout:568: done (4 left) 2024-09-15T03:48:17.854 INFO:tasks.rados.rados.0.smithi122.stdout:569: done (3 left) 2024-09-15T03:48:17.854 INFO:tasks.rados.rados.0.smithi122.stdout:570: done (2 left) 2024-09-15T03:48:17.854 INFO:tasks.rados.rados.0.smithi122.stdout:571: done (1 left) 2024-09-15T03:48:17.854 INFO:tasks.rados.rados.0.smithi122.stdout:572: done (0 left) 2024-09-15T03:48:17.854 INFO:tasks.rados.rados.0.smithi122.stdout:573: write oid 19 current snap is 65 2024-09-15T03:48:17.854 INFO:tasks.rados.rados.0.smithi122.stdout:573: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2024-09-15T03:48:17.862 INFO:tasks.rados.rados.0.smithi122.stdout:573: writing smithi12223906-19 from 699224 to 1247582 tid 1 2024-09-15T03:48:17.870 INFO:tasks.rados.rados.0.smithi122.stdout:573: writing smithi12223906-19 from 2026293 to 2488367 tid 2 2024-09-15T03:48:17.877 INFO:tasks.rados.rados.0.smithi122.stdout:573: writing smithi12223906-19 from 3271546 to 3704300 tid 3 2024-09-15T03:48:17.877 INFO:tasks.rados.rados.0.smithi122.stdout:574: read oid 10 snap -1 2024-09-15T03:48:17.877 INFO:tasks.rados.rados.0.smithi122.stdout:574: expect (ObjNum 219 snap 59 seq_num 219) 2024-09-15T03:48:17.877 INFO:tasks.rados.rados.0.smithi122.stdout:575: write (excl) oid 1 current snap is 65 2024-09-15T03:48:17.877 INFO:tasks.rados.rados.0.smithi122.stdout:573: finishing write tid 1 to smithi12223906-19 2024-09-15T03:48:17.877 INFO:tasks.rados.rados.0.smithi122.stdout:575: seq_num 233 ranges {716031=623607,2084775=605732,3298969=617062} 2024-09-15T03:48:17.884 INFO:tasks.rados.rados.0.smithi122.stdout:575: writing smithi12223906-1 from 716031 to 1339638 tid 1 2024-09-15T03:48:17.892 INFO:tasks.rados.rados.0.smithi122.stdout:575: writing smithi12223906-1 from 2084775 to 2690507 tid 2 2024-09-15T03:48:17.899 INFO:tasks.rados.rados.0.smithi122.stdout:575: writing smithi12223906-1 from 3298969 to 3916031 tid 3 2024-09-15T03:48:17.899 INFO:tasks.rados.rados.0.smithi122.stdout:576: snap_remove snap 63 2024-09-15T03:48:18.866 INFO:tasks.rados.rados.0.smithi122.stdout:576: done (3 left) 2024-09-15T03:48:18.866 INFO:tasks.rados.rados.0.smithi122.stdout:577: write oid 38 current snap is 65 2024-09-15T03:48:18.866 INFO:tasks.rados.rados.0.smithi122.stdout:577: seq_num 234 ranges {732838=698856,2026904=465743,3210039=571651} 2024-09-15T03:48:18.876 INFO:tasks.rados.rados.0.smithi122.stdout:577: writing smithi12223906-38 from 732838 to 1431694 tid 1 2024-09-15T03:48:18.881 INFO:tasks.rados.rados.0.smithi122.stdout:577: writing smithi12223906-38 from 2026904 to 2492647 tid 2 2024-09-15T03:48:18.889 INFO:tasks.rados.rados.0.smithi122.stdout:577: writing smithi12223906-38 from 3210039 to 3781690 tid 3 2024-09-15T03:48:18.889 INFO:tasks.rados.rados.0.smithi122.stdout:578: delete oid 23 current snap is 65 2024-09-15T03:48:18.889 INFO:tasks.rados.rados.0.smithi122.stdout:573: finishing write tid 2 to smithi12223906-19 2024-09-15T03:48:18.890 INFO:tasks.rados.rados.0.smithi122.stdout:573: finishing write tid 3 to smithi12223906-19 2024-09-15T03:48:18.890 INFO:tasks.rados.rados.0.smithi122.stdout:573: finishing write tid 4 to smithi12223906-19 2024-09-15T03:48:18.890 INFO:tasks.rados.rados.0.smithi122.stdout:573: finishing write tid 5 to smithi12223906-19 2024-09-15T03:48:18.890 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 227 (ObjNum 232 snap 65 seq_num 232) dirty exists 2024-09-15T03:48:18.890 INFO:tasks.rados.rados.0.smithi122.stdout:573: left oid 19 (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:48:18.890 INFO:tasks.rados.rados.0.smithi122.stdout:574: expect (ObjNum 219 snap 59 seq_num 219) 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:575: finishing write tid 1 to smithi12223906-1 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:575: finishing write tid 2 to smithi12223906-1 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:575: finishing write tid 3 to smithi12223906-1 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:575: finishing write tid 4 to smithi12223906-1 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:575: finishing write tid 5 to smithi12223906-1 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 164 (ObjNum 233 snap 65 seq_num 233) dirty exists 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:575: left oid 1 (ObjNum 233 snap 65 seq_num 233) 2024-09-15T03:48:19.064 INFO:tasks.rados.rados.0.smithi122.stdout:577: finishing write tid 1 to smithi12223906-38 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:573: done (4 left) 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:574: done (3 left) 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:575: done (2 left) 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:578: done (1 left) 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:579: write (excl) oid 2 current snap is 65 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:577: finishing write tid 2 to smithi12223906-38 2024-09-15T03:48:19.065 INFO:tasks.rados.rados.0.smithi122.stdout:579: seq_num 236 ranges {766452=565707,1860221=469412,2981238=594195} 2024-09-15T03:48:19.078 INFO:tasks.rados.rados.0.smithi122.stdout:579: writing smithi12223906-2 from 766452 to 1332159 tid 1 2024-09-15T03:48:19.085 INFO:tasks.rados.rados.0.smithi122.stdout:579: writing smithi12223906-2 from 1860221 to 2329633 tid 2 2024-09-15T03:48:19.092 INFO:tasks.rados.rados.0.smithi122.stdout:579: writing smithi12223906-2 from 2981238 to 3575433 tid 3 2024-09-15T03:48:19.092 INFO:tasks.rados.rados.0.smithi122.stdout:580: delete oid 33 current snap is 65 2024-09-15T03:48:19.092 INFO:tasks.rados.rados.0.smithi122.stdout:577: finishing write tid 3 to smithi12223906-38 2024-09-15T03:48:19.093 INFO:tasks.rados.rados.0.smithi122.stdout:577: finishing write tid 4 to smithi12223906-38 2024-09-15T03:48:19.093 INFO:tasks.rados.rados.0.smithi122.stdout:577: finishing write tid 5 to smithi12223906-38 2024-09-15T03:48:19.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 142 (ObjNum 234 snap 65 seq_num 234) dirty exists 2024-09-15T03:48:19.093 INFO:tasks.rados.rados.0.smithi122.stdout:577: left oid 38 (ObjNum 234 snap 65 seq_num 234) 2024-09-15T03:48:19.093 INFO:tasks.rados.rados.0.smithi122.stdout:579: finishing write tid 1 to smithi12223906-2 2024-09-15T03:48:19.093 INFO:tasks.rados.rados.0.smithi122.stdout:579: finishing write tid 2 to smithi12223906-2 2024-09-15T03:48:19.095 INFO:tasks.rados.rados.0.smithi122.stdout:577: done (2 left) 2024-09-15T03:48:19.095 INFO:tasks.rados.rados.0.smithi122.stdout:580: done (1 left) 2024-09-15T03:48:19.095 INFO:tasks.rados.rados.0.smithi122.stdout:581: write oid 47 current snap is 65 2024-09-15T03:48:19.095 INFO:tasks.rados.rados.0.smithi122.stdout:581: seq_num 238 ranges {400066=716205,1860832=139234} 2024-09-15T03:48:19.101 INFO:tasks.rados.rados.0.smithi122.stdout:581: writing smithi12223906-47 from 400066 to 1116271 tid 1 2024-09-15T03:48:19.106 INFO:tasks.rados.rados.0.smithi122.stdout:581: writing smithi12223906-47 from 1860832 to 2000066 tid 2 2024-09-15T03:48:19.106 INFO:tasks.rados.rados.0.smithi122.stdout:582: delete oid 39 current snap is 65 2024-09-15T03:48:19.106 INFO:tasks.rados.rados.0.smithi122.stdout:579: finishing write tid 3 to smithi12223906-2 2024-09-15T03:48:19.106 INFO:tasks.rados.rados.0.smithi122.stdout:579: finishing write tid 4 to smithi12223906-2 2024-09-15T03:48:19.106 INFO:tasks.rados.rados.0.smithi122.stdout:579: finishing write tid 5 to smithi12223906-2 2024-09-15T03:48:19.107 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 150 (ObjNum 236 snap 65 seq_num 236) dirty exists 2024-09-15T03:48:19.107 INFO:tasks.rados.rados.0.smithi122.stdout:579: left oid 2 (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:48:19.111 INFO:tasks.rados.rados.0.smithi122.stdout:579: done (2 left) 2024-09-15T03:48:19.111 INFO:tasks.rados.rados.0.smithi122.stdout:582: done (1 left) 2024-09-15T03:48:19.111 INFO:tasks.rados.rados.0.smithi122.stdout:583: rollback oid 11 current snap is 65 2024-09-15T03:48:19.111 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 11 to 64 2024-09-15T03:48:19.112 INFO:tasks.rados.rados.0.smithi122.stdout:584: snap_create 2024-09-15T03:48:19.114 INFO:tasks.rados.rados.0.smithi122.stdout:581: finishing write tid 1 to smithi12223906-47 2024-09-15T03:48:19.114 INFO:tasks.rados.rados.0.smithi122.stdout:581: finishing write tid 2 to smithi12223906-47 2024-09-15T03:48:19.114 INFO:tasks.rados.rados.0.smithi122.stdout:581: finishing write tid 3 to smithi12223906-47 2024-09-15T03:48:19.115 INFO:tasks.rados.rados.0.smithi122.stdout:581: finishing write tid 4 to smithi12223906-47 2024-09-15T03:48:19.115 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 90 (ObjNum 238 snap 65 seq_num 238) dirty exists 2024-09-15T03:48:19.115 INFO:tasks.rados.rados.0.smithi122.stdout:581: left oid 47 (ObjNum 238 snap 65 seq_num 238) 2024-09-15T03:48:19.117 INFO:tasks.rados.rados.0.smithi122.stdout:583: finishing rollback tid 0 to smithi12223906-11 2024-09-15T03:48:19.118 INFO:tasks.rados.rados.0.smithi122.stdout:583: finishing rollback tid 1 to smithi12223906-11 2024-09-15T03:48:19.118 INFO:tasks.rados.rados.0.smithi122.stdout:583: finishing rollback tid 2 to smithi12223906-11 2024-09-15T03:48:19.118 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 172 (ObjNum 206 snap 56 seq_num 206) dirty exists 2024-09-15T03:48:19.875 INFO:tasks.rados.rados.0.smithi122.stdout:581: done (2 left) 2024-09-15T03:48:19.875 INFO:tasks.rados.rados.0.smithi122.stdout:583: done (1 left) 2024-09-15T03:48:19.875 INFO:tasks.rados.rados.0.smithi122.stdout:584: done (0 left) 2024-09-15T03:48:19.875 INFO:tasks.rados.rados.0.smithi122.stdout:585: copy_from oid 27 from oid 9 current snap is 66 2024-09-15T03:48:19.876 INFO:tasks.rados.rados.0.smithi122.stdout:586: snap_create 2024-09-15T03:48:19.878 INFO:tasks.rados.rados.0.smithi122.stdout:585: finishing copy_from to smithi12223906-27 2024-09-15T03:48:19.878 INFO:tasks.rados.rados.0.smithi122.stdout:585: got expected ENOENT (src dne) 2024-09-15T03:48:19.878 INFO:tasks.rados.rados.0.smithi122.stdout:585: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:585: done (1 left) 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:586: done (0 left) 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:587: rollback oid 42 current snap is 67 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 65 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:588: rollback oid 41 current snap is 67 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 66 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:589: copy_from oid 34 from oid 3 current snap is 67 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:590: write oid 17 current snap is 67 2024-09-15T03:48:20.895 INFO:tasks.rados.rados.0.smithi122.stdout:590: seq_num 240 ranges {433680=466703,1577796=455884} 2024-09-15T03:48:20.902 INFO:tasks.rados.rados.0.smithi122.stdout:590: writing smithi12223906-17 from 433680 to 900383 tid 1 2024-09-15T03:48:20.909 INFO:tasks.rados.rados.0.smithi122.stdout:590: writing smithi12223906-17 from 1577796 to 2033680 tid 2 2024-09-15T03:48:20.909 INFO:tasks.rados.rados.0.smithi122.stdout:591: snap_remove snap 64 2024-09-15T03:48:21.897 INFO:tasks.rados.rados.0.smithi122.stdout:591: done (4 left) 2024-09-15T03:48:21.897 INFO:tasks.rados.rados.0.smithi122.stdout:592: snap_create 2024-09-15T03:48:21.897 INFO:tasks.rados.rados.0.smithi122.stdout:587: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:48:21.897 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:588: finishing rollback tid 0 to smithi12223906-41 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:588: finishing rollback tid 1 to smithi12223906-41 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:588: finishing rollback tid 2 to smithi12223906-41 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 110 (ObjNum 191 snap 53 seq_num 191) dirty exists 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:590: finishing write tid 1 to smithi12223906-17 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:589: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:590: finishing write tid 2 to smithi12223906-17 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:589: finishing copy_from to smithi12223906-34 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 186 (ObjNum 142 snap 32 seq_num 142) dirty exists 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:590: finishing write tid 3 to smithi12223906-17 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:590: finishing write tid 4 to smithi12223906-17 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 153 (ObjNum 240 snap 67 seq_num 240) dirty exists 2024-09-15T03:48:21.898 INFO:tasks.rados.rados.0.smithi122.stdout:590: left oid 17 (ObjNum 240 snap 67 seq_num 240) 2024-09-15T03:48:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:587: done (4 left) 2024-09-15T03:48:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:588: done (3 left) 2024-09-15T03:48:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:589: done (2 left) 2024-09-15T03:48:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:590: done (1 left) 2024-09-15T03:48:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:592: done (0 left) 2024-09-15T03:48:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:593: delete oid 39 current snap is 68 2024-09-15T03:48:22.907 INFO:tasks.rados.rados.0.smithi122.stdout:593: done (0 left) 2024-09-15T03:48:22.907 INFO:tasks.rados.rados.0.smithi122.stdout:594: snap_remove snap 67 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:594: done (0 left) 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:595: rollback oid 1 current snap is 68 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 1 to 65 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:596: read oid 19 snap -1 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:596: expect (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:597: copy_from oid 17 from oid 40 current snap is 68 2024-09-15T03:48:23.920 INFO:tasks.rados.rados.0.smithi122.stdout:598: snap_remove snap 66 2024-09-15T03:48:24.922 INFO:tasks.rados.rados.0.smithi122.stdout:598: done (3 left) 2024-09-15T03:48:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:599: write oid 15 current snap is 68 2024-09-15T03:48:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:595: finishing rollback tid 0 to smithi12223906-1 2024-09-15T03:48:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:599: seq_num 242 ranges {467294=617201,1578407=488887} 2024-09-15T03:48:24.932 INFO:tasks.rados.rados.0.smithi122.stdout:599: writing smithi12223906-15 from 467294 to 1084495 tid 1 2024-09-15T03:48:24.938 INFO:tasks.rados.rados.0.smithi122.stdout:599: writing smithi12223906-15 from 1578407 to 2067294 tid 2 2024-09-15T03:48:24.938 INFO:tasks.rados.rados.0.smithi122.stdout:600: snap_create 2024-09-15T03:48:24.938 INFO:tasks.rados.rados.0.smithi122.stdout:595: finishing rollback tid 1 to smithi12223906-1 2024-09-15T03:48:24.939 INFO:tasks.rados.rados.0.smithi122.stdout:595: finishing rollback tid 2 to smithi12223906-1 2024-09-15T03:48:24.939 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 180 (ObjNum 233 snap 65 seq_num 233) dirty exists 2024-09-15T03:48:24.939 INFO:tasks.rados.rados.0.smithi122.stdout:597: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:48:24.939 INFO:tasks.rados.rados.0.smithi122.stdout:596: expect (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:597: finishing copy_from to smithi12223906-17 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 83 (ObjNum 190 snap 53 seq_num 190) dirty exists 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:599: finishing write tid 1 to smithi12223906-15 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:599: finishing write tid 2 to smithi12223906-15 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:599: finishing write tid 3 to smithi12223906-15 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:599: finishing write tid 4 to smithi12223906-15 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 185 (ObjNum 242 snap 68 seq_num 242) dirty exists 2024-09-15T03:48:25.246 INFO:tasks.rados.rados.0.smithi122.stdout:599: left oid 15 (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:48:25.941 INFO:tasks.rados.rados.0.smithi122.stdout:595: done (4 left) 2024-09-15T03:48:25.941 INFO:tasks.rados.rados.0.smithi122.stdout:596: done (3 left) 2024-09-15T03:48:25.941 INFO:tasks.rados.rados.0.smithi122.stdout:597: done (2 left) 2024-09-15T03:48:25.941 INFO:tasks.rados.rados.0.smithi122.stdout:599: done (1 left) 2024-09-15T03:48:25.941 INFO:tasks.rados.rados.0.smithi122.stdout:600: done (0 left) 2024-09-15T03:48:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:601: read oid 23 snap -1 2024-09-15T03:48:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:601: expect deleted 2024-09-15T03:48:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:602: delete oid 16 current snap is 69 2024-09-15T03:48:25.948 INFO:tasks.rados.rados.0.smithi122.stdout:602: done (1 left) 2024-09-15T03:48:25.948 INFO:tasks.rados.rados.0.smithi122.stdout:603: snap_create 2024-09-15T03:48:26.742 INFO:tasks.rados.rados.0.smithi122.stdout:601: done (1 left) 2024-09-15T03:48:26.742 INFO:tasks.rados.rados.0.smithi122.stdout:603: done (0 left) 2024-09-15T03:48:26.742 INFO:tasks.rados.rados.0.smithi122.stdout:604: delete oid 5 current snap is 70 2024-09-15T03:48:26.754 INFO:tasks.rados.rados.0.smithi122.stdout:604: done (0 left) 2024-09-15T03:48:26.754 INFO:tasks.rados.rados.0.smithi122.stdout:605: snap_create 2024-09-15T03:48:27.753 INFO:tasks.rados.rados.0.smithi122.stdout:605: done (0 left) 2024-09-15T03:48:27.754 INFO:tasks.rados.rados.0.smithi122.stdout:606: write oid 49 current snap is 71 2024-09-15T03:48:27.754 INFO:tasks.rados.rados.0.smithi122.stdout:606: seq_num 245 ranges {517715=559301,1870206=247509} 2024-09-15T03:48:27.762 INFO:tasks.rados.rados.0.smithi122.stdout:606: writing smithi12223906-49 from 517715 to 1077016 tid 1 2024-09-15T03:48:27.769 INFO:tasks.rados.rados.0.smithi122.stdout:606: writing smithi12223906-49 from 1870206 to 2117715 tid 2 2024-09-15T03:48:27.769 INFO:tasks.rados.rados.0.smithi122.stdout:607: read oid 35 snap -1 2024-09-15T03:48:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:607: expect (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:48:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:608: read oid 18 snap -1 2024-09-15T03:48:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:608: expect (ObjNum 214 snap 58 seq_num 214) 2024-09-15T03:48:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:609: read oid 47 snap -1 2024-09-15T03:48:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:609: expect (ObjNum 238 snap 65 seq_num 238) 2024-09-15T03:48:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:610: snap_remove snap 70 2024-09-15T03:48:28.760 INFO:tasks.rados.rados.0.smithi122.stdout:610: done (4 left) 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:611: rollback oid 13 current snap is 71 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 65 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:606: finishing write tid 1 to smithi12223906-49 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:606: finishing write tid 2 to smithi12223906-49 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:606: finishing write tid 3 to smithi12223906-49 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:606: finishing write tid 4 to smithi12223906-49 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 164 (ObjNum 245 snap 71 seq_num 245) dirty exists 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:606: left oid 49 (ObjNum 245 snap 71 seq_num 245) 2024-09-15T03:48:28.761 INFO:tasks.rados.rados.0.smithi122.stdout:608: expect (ObjNum 214 snap 58 seq_num 214) 2024-09-15T03:48:28.917 INFO:tasks.rados.rados.0.smithi122.stdout:609: expect (ObjNum 238 snap 65 seq_num 238) 2024-09-15T03:48:29.011 INFO:tasks.rados.rados.0.smithi122.stdout:607: expect (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:48:29.195 INFO:tasks.rados.rados.0.smithi122.stdout:611: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:48:29.195 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:48:29.195 INFO:tasks.rados.rados.0.smithi122.stdout:606: done (4 left) 2024-09-15T03:48:29.196 INFO:tasks.rados.rados.0.smithi122.stdout:607: done (3 left) 2024-09-15T03:48:29.196 INFO:tasks.rados.rados.0.smithi122.stdout:608: done (2 left) 2024-09-15T03:48:29.196 INFO:tasks.rados.rados.0.smithi122.stdout:609: done (1 left) 2024-09-15T03:48:29.196 INFO:tasks.rados.rados.0.smithi122.stdout:611: done (0 left) 2024-09-15T03:48:29.196 INFO:tasks.rados.rados.0.smithi122.stdout:612: delete oid 8 current snap is 71 2024-09-15T03:48:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:612: done (0 left) 2024-09-15T03:48:29.202 INFO:tasks.rados.rados.0.smithi122.stdout:613: copy_from oid 32 from oid 3 current snap is 71 2024-09-15T03:48:29.202 INFO:tasks.rados.rados.0.smithi122.stdout:614: write oid 39 current snap is 71 2024-09-15T03:48:29.202 INFO:tasks.rados.rados.0.smithi122.stdout:614: seq_num 247 ranges {551329=709799,1870817=280512} 2024-09-15T03:48:29.215 INFO:tasks.rados.rados.0.smithi122.stdout:614: writing smithi12223906-39 from 551329 to 1261128 tid 1 2024-09-15T03:48:29.222 INFO:tasks.rados.rados.0.smithi122.stdout:614: writing smithi12223906-39 from 1870817 to 2151329 tid 2 2024-09-15T03:48:29.222 INFO:tasks.rados.rados.0.smithi122.stdout:615: delete oid 33 current snap is 71 2024-09-15T03:48:29.222 INFO:tasks.rados.rados.0.smithi122.stdout:613: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:48:29.222 INFO:tasks.rados.rados.0.smithi122.stdout:613: finishing copy_from to smithi12223906-32 2024-09-15T03:48:29.222 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 186 (ObjNum 142 snap 32 seq_num 142) dirty exists 2024-09-15T03:48:29.223 INFO:tasks.rados.rados.0.smithi122.stdout:614: finishing write tid 1 to smithi12223906-39 2024-09-15T03:48:29.224 INFO:tasks.rados.rados.0.smithi122.stdout:613: done (2 left) 2024-09-15T03:48:29.224 INFO:tasks.rados.rados.0.smithi122.stdout:615: done (1 left) 2024-09-15T03:48:29.224 INFO:tasks.rados.rados.0.smithi122.stdout:616: write (excl) oid 13 current snap is 71 2024-09-15T03:48:29.225 INFO:tasks.rados.rados.0.smithi122.stdout:616: seq_num 249 ranges {584943=460297,1587781=597162} 2024-09-15T03:48:29.230 INFO:tasks.rados.rados.0.smithi122.stdout:616: writing smithi12223906-13 from 584943 to 1045240 tid 1 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:616: writing smithi12223906-13 from 1587781 to 2184943 tid 2 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:617: copy_from oid 47 from oid 4 current snap is 71 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:614: finishing write tid 2 to smithi12223906-39 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:614: finishing write tid 3 to smithi12223906-39 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:614: finishing write tid 4 to smithi12223906-39 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 327 (ObjNum 247 snap 71 seq_num 247) dirty exists 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:614: left oid 39 (ObjNum 247 snap 71 seq_num 247) 2024-09-15T03:48:29.237 INFO:tasks.rados.rados.0.smithi122.stdout:616: finishing write tid 1 to smithi12223906-13 2024-09-15T03:48:29.238 INFO:tasks.rados.rados.0.smithi122.stdout:614: done (2 left) 2024-09-15T03:48:29.238 INFO:tasks.rados.rados.0.smithi122.stdout:618: rollback oid 24 current snap is 71 2024-09-15T03:48:29.238 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 69 2024-09-15T03:48:29.238 INFO:tasks.rados.rados.0.smithi122.stdout:619: read oid 23 snap -1 2024-09-15T03:48:29.238 INFO:tasks.rados.rados.0.smithi122.stdout:619: expect deleted 2024-09-15T03:48:29.238 INFO:tasks.rados.rados.0.smithi122.stdout:620: snap_create 2024-09-15T03:48:29.242 INFO:tasks.rados.rados.0.smithi122.stdout:616: finishing write tid 2 to smithi12223906-13 2024-09-15T03:48:29.243 INFO:tasks.rados.rados.0.smithi122.stdout:616: finishing write tid 3 to smithi12223906-13 2024-09-15T03:48:29.243 INFO:tasks.rados.rados.0.smithi122.stdout:616: finishing write tid 4 to smithi12223906-13 2024-09-15T03:48:29.243 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 230 (ObjNum 249 snap 71 seq_num 249) dirty exists 2024-09-15T03:48:29.243 INFO:tasks.rados.rados.0.smithi122.stdout:616: left oid 13 (ObjNum 249 snap 71 seq_num 249) 2024-09-15T03:48:29.247 INFO:tasks.rados.rados.0.smithi122.stdout:618: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:48:29.247 INFO:tasks.rados.rados.0.smithi122.stdout:618: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:48:29.247 INFO:tasks.rados.rados.0.smithi122.stdout:618: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:48:29.247 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 331 (ObjNum 213 snap 58 seq_num 213) dirty exists 2024-09-15T03:48:29.255 INFO:tasks.rados.rados.0.smithi122.stdout:617: finishing copy_from racing read to smithi12223906-47 2024-09-15T03:48:29.256 INFO:tasks.rados.rados.0.smithi122.stdout:617: finishing copy_from to smithi12223906-47 2024-09-15T03:48:29.256 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 174 (ObjNum 227 snap 62 seq_num 227) dirty exists 2024-09-15T03:48:29.772 INFO:tasks.rados.rados.0.smithi122.stdout:616: done (4 left) 2024-09-15T03:48:29.772 INFO:tasks.rados.rados.0.smithi122.stdout:617: done (3 left) 2024-09-15T03:48:29.772 INFO:tasks.rados.rados.0.smithi122.stdout:618: done (2 left) 2024-09-15T03:48:29.772 INFO:tasks.rados.rados.0.smithi122.stdout:619: done (1 left) 2024-09-15T03:48:29.773 INFO:tasks.rados.rados.0.smithi122.stdout:620: done (0 left) 2024-09-15T03:48:29.773 INFO:tasks.rados.rados.0.smithi122.stdout:621: write (excl) oid 35 current snap is 72 2024-09-15T03:48:29.773 INFO:tasks.rados.rados.0.smithi122.stdout:621: seq_num 250 ranges {601750=535546,1929910=271840} 2024-09-15T03:48:29.779 INFO:tasks.rados.rados.0.smithi122.stdout:621: writing smithi12223906-35 from 601750 to 1137296 tid 1 2024-09-15T03:48:29.786 INFO:tasks.rados.rados.0.smithi122.stdout:621: writing smithi12223906-35 from 1929910 to 2201750 tid 2 2024-09-15T03:48:29.786 INFO:tasks.rados.rados.0.smithi122.stdout:622: write oid 31 current snap is 72 2024-09-15T03:48:29.786 INFO:tasks.rados.rados.0.smithi122.stdout:622: seq_num 251 ranges {618557=727148,2104745=113812} 2024-09-15T03:48:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:622: writing smithi12223906-31 from 618557 to 1345705 tid 1 2024-09-15T03:48:29.798 INFO:tasks.rados.rados.0.smithi122.stdout:622: writing smithi12223906-31 from 2104745 to 2218557 tid 2 2024-09-15T03:48:29.798 INFO:tasks.rados.rados.0.smithi122.stdout:623: write oid 30 current snap is 72 2024-09-15T03:48:29.798 INFO:tasks.rados.rados.0.smithi122.stdout:621: finishing write tid 1 to smithi12223906-35 2024-09-15T03:48:29.799 INFO:tasks.rados.rados.0.smithi122.stdout:623: seq_num 252 ranges {635364=402397,1763227=472137} 2024-09-15T03:48:29.804 INFO:tasks.rados.rados.0.smithi122.stdout:623: writing smithi12223906-30 from 635364 to 1037761 tid 1 2024-09-15T03:48:29.811 INFO:tasks.rados.rados.0.smithi122.stdout:623: writing smithi12223906-30 from 1763227 to 2235364 tid 2 2024-09-15T03:48:29.811 INFO:tasks.rados.rados.0.smithi122.stdout:624: delete oid 34 current snap is 72 2024-09-15T03:48:29.811 INFO:tasks.rados.rados.0.smithi122.stdout:621: finishing write tid 2 to smithi12223906-35 2024-09-15T03:48:29.811 INFO:tasks.rados.rados.0.smithi122.stdout:621: finishing write tid 3 to smithi12223906-35 2024-09-15T03:48:29.811 INFO:tasks.rados.rados.0.smithi122.stdout:621: finishing write tid 4 to smithi12223906-35 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 79 (ObjNum 250 snap 72 seq_num 250) dirty exists 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:621: left oid 35 (ObjNum 250 snap 72 seq_num 250) 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:622: finishing write tid 1 to smithi12223906-31 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:622: finishing write tid 2 to smithi12223906-31 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:622: finishing write tid 3 to smithi12223906-31 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:622: finishing write tid 4 to smithi12223906-31 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 177 (ObjNum 251 snap 72 seq_num 251) dirty exists 2024-09-15T03:48:29.812 INFO:tasks.rados.rados.0.smithi122.stdout:622: left oid 31 (ObjNum 251 snap 72 seq_num 251) 2024-09-15T03:48:29.814 INFO:tasks.rados.rados.0.smithi122.stdout:623: finishing write tid 1 to smithi12223906-30 2024-09-15T03:48:29.816 INFO:tasks.rados.rados.0.smithi122.stdout:623: finishing write tid 2 to smithi12223906-30 2024-09-15T03:48:29.818 INFO:tasks.rados.rados.0.smithi122.stdout:621: done (3 left) 2024-09-15T03:48:29.818 INFO:tasks.rados.rados.0.smithi122.stdout:622: done (2 left) 2024-09-15T03:48:29.818 INFO:tasks.rados.rados.0.smithi122.stdout:624: done (1 left) 2024-09-15T03:48:29.818 INFO:tasks.rados.rados.0.smithi122.stdout:625: read oid 13 snap 65 2024-09-15T03:48:29.818 INFO:tasks.rados.rados.0.smithi122.stdout:625: expect deleted 2024-09-15T03:48:29.818 INFO:tasks.rados.rados.0.smithi122.stdout:626: delete oid 8 current snap is 72 2024-09-15T03:48:29.819 INFO:tasks.rados.rados.0.smithi122.stdout:623: finishing write tid 3 to smithi12223906-30 2024-09-15T03:48:29.820 INFO:tasks.rados.rados.0.smithi122.stdout:623: finishing write tid 4 to smithi12223906-30 2024-09-15T03:48:29.820 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 118 (ObjNum 252 snap 72 seq_num 252) dirty exists 2024-09-15T03:48:29.820 INFO:tasks.rados.rados.0.smithi122.stdout:623: left oid 30 (ObjNum 252 snap 72 seq_num 252) 2024-09-15T03:48:29.821 INFO:tasks.rados.rados.0.smithi122.stdout:623: done (2 left) 2024-09-15T03:48:29.821 INFO:tasks.rados.rados.0.smithi122.stdout:625: done (1 left) 2024-09-15T03:48:29.821 INFO:tasks.rados.rados.0.smithi122.stdout:626: done (0 left) 2024-09-15T03:48:29.821 INFO:tasks.rados.rados.0.smithi122.stdout:627: write oid 26 current snap is 72 2024-09-15T03:48:29.821 INFO:tasks.rados.rados.0.smithi122.stdout:627: seq_num 255 ranges {685785=628144,1822320=463465} 2024-09-15T03:48:29.828 INFO:tasks.rados.rados.0.smithi122.stdout:627: writing smithi12223906-26 from 685785 to 1313929 tid 1 2024-09-15T03:48:29.835 INFO:tasks.rados.rados.0.smithi122.stdout:627: writing smithi12223906-26 from 1822320 to 2285785 tid 2 2024-09-15T03:48:29.835 INFO:tasks.rados.rados.0.smithi122.stdout:628: copy_from oid 38 from oid 40 current snap is 72 2024-09-15T03:48:29.835 INFO:tasks.rados.rados.0.smithi122.stdout:629: read oid 50 snap -1 2024-09-15T03:48:29.836 INFO:tasks.rados.rados.0.smithi122.stdout:629: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:48:29.836 INFO:tasks.rados.rados.0.smithi122.stdout:630: read oid 41 snap -1 2024-09-15T03:48:29.836 INFO:tasks.rados.rados.0.smithi122.stdout:630: expect (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:48:29.836 INFO:tasks.rados.rados.0.smithi122.stdout:631: snap_remove snap 68 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:631: done (4 left) 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:632: copy_from oid 17 from oid 18 current snap is 72 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:627: finishing write tid 1 to smithi12223906-26 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:627: finishing write tid 2 to smithi12223906-26 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:627: finishing write tid 3 to smithi12223906-26 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:627: finishing write tid 4 to smithi12223906-26 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 151 (ObjNum 255 snap 72 seq_num 255) dirty exists 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:627: left oid 26 (ObjNum 255 snap 72 seq_num 255) 2024-09-15T03:48:30.789 INFO:tasks.rados.rados.0.smithi122.stdout:630: expect (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:48:30.967 INFO:tasks.rados.rados.0.smithi122.stdout:629: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-15T03:48:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:628: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:628: finishing copy_from to smithi12223906-38 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 83 (ObjNum 190 snap 53 seq_num 190) dirty exists 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:632: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:632: finishing copy_from to smithi12223906-17 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 256 (ObjNum 214 snap 58 seq_num 214) dirty exists 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:627: done (4 left) 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:628: done (3 left) 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:629: done (2 left) 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:630: done (1 left) 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:632: done (0 left) 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:633: write oid 10 current snap is 72 2024-09-15T03:48:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:633: seq_num 256 ranges {702592=703393,1880802=421790} 2024-09-15T03:48:31.071 INFO:tasks.rados.rados.0.smithi122.stdout:633: writing smithi12223906-10 from 702592 to 1405985 tid 1 2024-09-15T03:48:31.075 INFO:tasks.rados.rados.0.smithi122.stdout:633: writing smithi12223906-10 from 1880802 to 2302592 tid 2 2024-09-15T03:48:31.075 INFO:tasks.rados.rados.0.smithi122.stdout:634: read oid 24 snap 65 2024-09-15T03:48:31.075 INFO:tasks.rados.rados.0.smithi122.stdout:634: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:31.075 INFO:tasks.rados.rados.0.smithi122.stdout:635: snap_remove snap 69 2024-09-15T03:48:31.798 INFO:tasks.rados.rados.0.smithi122.stdout:633: finishing write tid 1 to smithi12223906-10 2024-09-15T03:48:31.798 INFO:tasks.rados.rados.0.smithi122.stdout:633: finishing write tid 2 to smithi12223906-10 2024-09-15T03:48:31.798 INFO:tasks.rados.rados.0.smithi122.stdout:633: finishing write tid 3 to smithi12223906-10 2024-09-15T03:48:31.799 INFO:tasks.rados.rados.0.smithi122.stdout:633: finishing write tid 4 to smithi12223906-10 2024-09-15T03:48:31.799 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 136 (ObjNum 256 snap 72 seq_num 256) dirty exists 2024-09-15T03:48:31.799 INFO:tasks.rados.rados.0.smithi122.stdout:633: left oid 10 (ObjNum 256 snap 72 seq_num 256) 2024-09-15T03:48:31.799 INFO:tasks.rados.rados.0.smithi122.stdout:634: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:633: done (2 left) 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:634: done (1 left) 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:635: done (0 left) 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:636: read oid 25 snap -1 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:636: expect deleted 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:637: read oid 3 snap 65 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:637: expect (ObjNum 142 snap 32 seq_num 142) 2024-09-15T03:48:31.954 INFO:tasks.rados.rados.0.smithi122.stdout:638: snap_create 2024-09-15T03:48:31.965 INFO:tasks.rados.rados.0.smithi122.stdout:637: expect (ObjNum 142 snap 32 seq_num 142) 2024-09-15T03:48:32.805 INFO:tasks.rados.rados.0.smithi122.stdout:636: done (2 left) 2024-09-15T03:48:32.806 INFO:tasks.rados.rados.0.smithi122.stdout:637: done (1 left) 2024-09-15T03:48:32.806 INFO:tasks.rados.rados.0.smithi122.stdout:638: done (0 left) 2024-09-15T03:48:32.806 INFO:tasks.rados.rados.0.smithi122.stdout:639: snap_remove snap 72 2024-09-15T03:48:33.816 INFO:tasks.rados.rados.0.smithi122.stdout:639: done (0 left) 2024-09-15T03:48:33.816 INFO:tasks.rados.rados.0.smithi122.stdout:640: write oid 28 current snap is 73 2024-09-15T03:48:33.816 INFO:tasks.rados.rados.0.smithi122.stdout:640: seq_num 257 ranges {719399=778642,1939284=380115} 2024-09-15T03:48:33.826 INFO:tasks.rados.rados.0.smithi122.stdout:640: writing smithi12223906-28 from 719399 to 1498041 tid 1 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:640: writing smithi12223906-28 from 1939284 to 2319399 tid 2 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:641: rollback oid 6 current snap is 73 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 6 to 71 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:642: read oid 14 snap 71 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:642: expect (ObjNum 226 snap 62 seq_num 226) 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:643: write oid 1 current snap is 73 2024-09-15T03:48:33.832 INFO:tasks.rados.rados.0.smithi122.stdout:643: seq_num 258 ranges {736206=453891,1881413=454793} 2024-09-15T03:48:33.839 INFO:tasks.rados.rados.0.smithi122.stdout:643: writing smithi12223906-1 from 736206 to 1190097 tid 1 2024-09-15T03:48:33.847 INFO:tasks.rados.rados.0.smithi122.stdout:643: writing smithi12223906-1 from 1881413 to 2336206 tid 2 2024-09-15T03:48:33.847 INFO:tasks.rados.rados.0.smithi122.stdout:644: read oid 20 snap -1 2024-09-15T03:48:33.847 INFO:tasks.rados.rados.0.smithi122.stdout:644: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:48:33.847 INFO:tasks.rados.rados.0.smithi122.stdout:645: snap_remove snap 65 2024-09-15T03:48:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:645: done (5 left) 2024-09-15T03:48:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:646: write oid 23 current snap is 73 2024-09-15T03:48:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:646: seq_num 259 ranges {753013=645493,2056248=296765} 2024-09-15T03:48:34.837 INFO:tasks.rados.rados.0.smithi122.stdout:646: writing smithi12223906-23 from 753013 to 1398506 tid 1 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:646: writing smithi12223906-23 from 2056248 to 2353013 tid 2 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:647: read oid 31 snap -1 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:647: expect (ObjNum 251 snap 72 seq_num 251) 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:648: delete oid 33 current snap is 73 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:641: finishing rollback tid 0 to smithi12223906-6 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:641: finishing rollback tid 1 to smithi12223906-6 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:641: finishing rollback tid 2 to smithi12223906-6 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 220 (ObjNum 106 snap 20 seq_num 106) dirty exists 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:640: finishing write tid 1 to smithi12223906-28 2024-09-15T03:48:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:642: expect (ObjNum 226 snap 62 seq_num 226) 2024-09-15T03:48:35.061 INFO:tasks.rados.rados.0.smithi122.stdout:644: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:643: finishing write tid 1 to smithi12223906-1 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:640: finishing write tid 2 to smithi12223906-28 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:640: finishing write tid 3 to smithi12223906-28 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:643: finishing write tid 2 to smithi12223906-1 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:643: finishing write tid 3 to smithi12223906-1 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:640: finishing write tid 4 to smithi12223906-28 2024-09-15T03:48:35.297 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 137 (ObjNum 257 snap 73 seq_num 257) dirty exists 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:640: left oid 28 (ObjNum 257 snap 73 seq_num 257) 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:643: finishing write tid 4 to smithi12223906-1 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 190 (ObjNum 258 snap 73 seq_num 258) dirty exists 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:643: left oid 1 (ObjNum 258 snap 73 seq_num 258) 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:646: finishing write tid 1 to smithi12223906-23 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:646: finishing write tid 2 to smithi12223906-23 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:646: finishing write tid 3 to smithi12223906-23 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:646: finishing write tid 4 to smithi12223906-23 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 307 (ObjNum 259 snap 73 seq_num 259) dirty exists 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:646: left oid 23 (ObjNum 259 snap 73 seq_num 259) 2024-09-15T03:48:35.298 INFO:tasks.rados.rados.0.smithi122.stdout:647: expect (ObjNum 251 snap 72 seq_num 251) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:640: done (7 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:641: done (6 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:642: done (5 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:643: done (4 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:644: done (3 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:646: done (2 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:647: done (1 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:648: done (0 left) 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:649: write oid 4 current snap is 73 2024-09-15T03:48:35.418 INFO:tasks.rados.rados.0.smithi122.stdout:649: seq_num 261 ranges {786627=795991,2173212=213415} 2024-09-15T03:48:35.435 INFO:tasks.rados.rados.0.smithi122.stdout:649: writing smithi12223906-4 from 786627 to 1582618 tid 1 2024-09-15T03:48:35.439 INFO:tasks.rados.rados.0.smithi122.stdout:649: writing smithi12223906-4 from 2173212 to 2386627 tid 2 2024-09-15T03:48:35.439 INFO:tasks.rados.rados.0.smithi122.stdout:650: snap_create 2024-09-15T03:48:35.447 INFO:tasks.rados.rados.0.smithi122.stdout:649: finishing write tid 1 to smithi12223906-4 2024-09-15T03:48:35.447 INFO:tasks.rados.rados.0.smithi122.stdout:649: finishing write tid 2 to smithi12223906-4 2024-09-15T03:48:35.448 INFO:tasks.rados.rados.0.smithi122.stdout:649: finishing write tid 3 to smithi12223906-4 2024-09-15T03:48:35.448 INFO:tasks.rados.rados.0.smithi122.stdout:649: finishing write tid 4 to smithi12223906-4 2024-09-15T03:48:35.448 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 180 (ObjNum 261 snap 73 seq_num 261) dirty exists 2024-09-15T03:48:35.448 INFO:tasks.rados.rados.0.smithi122.stdout:649: left oid 4 (ObjNum 261 snap 73 seq_num 261) 2024-09-15T03:48:35.842 INFO:tasks.rados.rados.0.smithi122.stdout:649: done (1 left) 2024-09-15T03:48:35.843 INFO:tasks.rados.rados.0.smithi122.stdout:650: done (0 left) 2024-09-15T03:48:35.843 INFO:tasks.rados.rados.0.smithi122.stdout:651: write (excl) oid 22 current snap is 74 2024-09-15T03:48:35.843 INFO:tasks.rados.rados.0.smithi122.stdout:651: seq_num 262 ranges {403434=471240,1315341=400756} 2024-09-15T03:48:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:651: writing smithi12223906-22 from 403434 to 874674 tid 1 2024-09-15T03:48:35.858 INFO:tasks.rados.rados.0.smithi122.stdout:651: writing smithi12223906-22 from 1315341 to 1716097 tid 2 2024-09-15T03:48:35.858 INFO:tasks.rados.rados.0.smithi122.stdout:652: read oid 16 snap 71 2024-09-15T03:48:35.858 INFO:tasks.rados.rados.0.smithi122.stdout:652: expect deleted 2024-09-15T03:48:35.859 INFO:tasks.rados.rados.0.smithi122.stdout:653: snap_remove snap 73 2024-09-15T03:48:36.748 INFO:tasks.rados.rados.0.smithi122.stdout:653: done (2 left) 2024-09-15T03:48:36.748 INFO:tasks.rados.rados.0.smithi122.stdout:654: read oid 15 snap -1 2024-09-15T03:48:36.748 INFO:tasks.rados.rados.0.smithi122.stdout:654: expect (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:48:36.748 INFO:tasks.rados.rados.0.smithi122.stdout:651: finishing write tid 1 to smithi12223906-22 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:651: finishing write tid 2 to smithi12223906-22 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:651: finishing write tid 3 to smithi12223906-22 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:651: finishing write tid 4 to smithi12223906-22 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 312 (ObjNum 262 snap 74 seq_num 262) dirty exists 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:651: left oid 22 (ObjNum 262 snap 74 seq_num 262) 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:651: done (2 left) 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:652: done (1 left) 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:655: write oid 18 current snap is 74 2024-09-15T03:48:36.749 INFO:tasks.rados.rados.0.smithi122.stdout:655: seq_num 263 ranges {420241=546489,1373823=544414} 2024-09-15T03:48:36.755 INFO:tasks.rados.rados.0.smithi122.stdout:655: writing smithi12223906-18 from 420241 to 966730 tid 1 2024-09-15T03:48:36.761 INFO:tasks.rados.rados.0.smithi122.stdout:655: writing smithi12223906-18 from 1373823 to 1918237 tid 2 2024-09-15T03:48:36.761 INFO:tasks.rados.rados.0.smithi122.stdout:656: read oid 47 snap -1 2024-09-15T03:48:36.761 INFO:tasks.rados.rados.0.smithi122.stdout:656: expect (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:48:36.762 INFO:tasks.rados.rados.0.smithi122.stdout:657: delete oid 13 current snap is 74 2024-09-15T03:48:36.762 INFO:tasks.rados.rados.0.smithi122.stdout:654: expect (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:655: finishing write tid 1 to smithi12223906-18 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:655: finishing write tid 2 to smithi12223906-18 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:655: finishing write tid 3 to smithi12223906-18 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:655: finishing write tid 4 to smithi12223906-18 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 316 (ObjNum 263 snap 74 seq_num 263) dirty exists 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:655: left oid 18 (ObjNum 263 snap 74 seq_num 263) 2024-09-15T03:48:36.829 INFO:tasks.rados.rados.0.smithi122.stdout:656: expect (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:48:37.008 INFO:tasks.rados.rados.0.smithi122.stdout:654: done (3 left) 2024-09-15T03:48:37.008 INFO:tasks.rados.rados.0.smithi122.stdout:655: done (2 left) 2024-09-15T03:48:37.008 INFO:tasks.rados.rados.0.smithi122.stdout:656: done (1 left) 2024-09-15T03:48:37.008 INFO:tasks.rados.rados.0.smithi122.stdout:657: done (0 left) 2024-09-15T03:48:37.008 INFO:tasks.rados.rados.0.smithi122.stdout:658: write oid 38 current snap is 74 2024-09-15T03:48:37.009 INFO:tasks.rados.rados.0.smithi122.stdout:658: seq_num 265 ranges {453855=696987,1890787=548083} 2024-09-15T03:48:37.021 INFO:tasks.rados.rados.0.smithi122.stdout:658: writing smithi12223906-38 from 453855 to 1150842 tid 1 2024-09-15T03:48:37.029 INFO:tasks.rados.rados.0.smithi122.stdout:658: writing smithi12223906-38 from 1890787 to 2438870 tid 2 2024-09-15T03:48:37.029 INFO:tasks.rados.rados.0.smithi122.stdout:659: read oid 22 snap -1 2024-09-15T03:48:37.029 INFO:tasks.rados.rados.0.smithi122.stdout:659: expect (ObjNum 262 snap 74 seq_num 262) 2024-09-15T03:48:37.029 INFO:tasks.rados.rados.0.smithi122.stdout:660: snap_create 2024-09-15T03:48:37.030 INFO:tasks.rados.rados.0.smithi122.stdout:658: finishing write tid 1 to smithi12223906-38 2024-09-15T03:48:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:658: finishing write tid 2 to smithi12223906-38 2024-09-15T03:48:37.037 INFO:tasks.rados.rados.0.smithi122.stdout:658: finishing write tid 3 to smithi12223906-38 2024-09-15T03:48:37.037 INFO:tasks.rados.rados.0.smithi122.stdout:658: finishing write tid 4 to smithi12223906-38 2024-09-15T03:48:37.038 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 174 (ObjNum 265 snap 74 seq_num 265) dirty exists 2024-09-15T03:48:37.038 INFO:tasks.rados.rados.0.smithi122.stdout:658: left oid 38 (ObjNum 265 snap 74 seq_num 265) 2024-09-15T03:48:37.042 INFO:tasks.rados.rados.0.smithi122.stdout:659: expect (ObjNum 262 snap 74 seq_num 262) 2024-09-15T03:48:37.850 INFO:tasks.rados.rados.0.smithi122.stdout:658: done (2 left) 2024-09-15T03:48:37.850 INFO:tasks.rados.rados.0.smithi122.stdout:659: done (1 left) 2024-09-15T03:48:37.850 INFO:tasks.rados.rados.0.smithi122.stdout:660: done (0 left) 2024-09-15T03:48:37.850 INFO:tasks.rados.rados.0.smithi122.stdout:661: snap_remove snap 74 2024-09-15T03:48:38.866 INFO:tasks.rados.rados.0.smithi122.stdout:661: done (0 left) 2024-09-15T03:48:38.866 INFO:tasks.rados.rados.0.smithi122.stdout:662: read oid 43 snap 71 2024-09-15T03:48:38.866 INFO:tasks.rados.rados.0.smithi122.stdout:662: expect (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:48:38.866 INFO:tasks.rados.rados.0.smithi122.stdout:663: read oid 24 snap 71 2024-09-15T03:48:38.867 INFO:tasks.rados.rados.0.smithi122.stdout:663: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:38.867 INFO:tasks.rados.rados.0.smithi122.stdout:664: rollback oid 49 current snap is 75 2024-09-15T03:48:38.867 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 71 2024-09-15T03:48:38.867 INFO:tasks.rados.rados.0.smithi122.stdout:665: copy_from oid 26 from oid 25 current snap is 75 2024-09-15T03:48:38.867 INFO:tasks.rados.rados.0.smithi122.stdout:666: snap_create 2024-09-15T03:48:38.878 INFO:tasks.rados.rados.0.smithi122.stdout:664: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:48:38.879 INFO:tasks.rados.rados.0.smithi122.stdout:662: expect (ObjNum 106 snap 20 seq_num 106) 2024-09-15T03:48:39.003 INFO:tasks.rados.rados.0.smithi122.stdout:664: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:48:39.004 INFO:tasks.rados.rados.0.smithi122.stdout:664: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:48:39.004 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 259 (ObjNum 245 snap 71 seq_num 245) dirty exists 2024-09-15T03:48:39.004 INFO:tasks.rados.rados.0.smithi122.stdout:663: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:665: finishing copy_from to smithi12223906-26 2024-09-15T03:48:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:665: got expected ENOENT (src dne) 2024-09-15T03:48:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:665: finishing copy_from racing read to smithi12223906-26 2024-09-15T03:48:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:662: done (4 left) 2024-09-15T03:48:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:663: done (3 left) 2024-09-15T03:48:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:664: done (2 left) 2024-09-15T03:48:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:665: done (1 left) 2024-09-15T03:48:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:666: done (0 left) 2024-09-15T03:48:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:667: snap_remove snap 71 2024-09-15T03:48:40.882 INFO:tasks.rados.rados.0.smithi122.stdout:667: done (0 left) 2024-09-15T03:48:40.883 INFO:tasks.rados.rados.0.smithi122.stdout:668: delete oid 40 current snap is 76 2024-09-15T03:48:40.900 INFO:tasks.rados.rados.0.smithi122.stdout:668: done (0 left) 2024-09-15T03:48:40.900 INFO:tasks.rados.rados.0.smithi122.stdout:669: read oid 47 snap -1 2024-09-15T03:48:40.900 INFO:tasks.rados.rados.0.smithi122.stdout:669: expect (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:48:40.900 INFO:tasks.rados.rados.0.smithi122.stdout:670: read oid 12 snap -1 2024-09-15T03:48:40.900 INFO:tasks.rados.rados.0.smithi122.stdout:670: expect (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:48:40.900 INFO:tasks.rados.rados.0.smithi122.stdout:671: snap_create 2024-09-15T03:48:40.906 INFO:tasks.rados.rados.0.smithi122.stdout:670: expect (ObjNum 111 snap 23 seq_num 111) 2024-09-15T03:48:41.090 INFO:tasks.rados.rados.0.smithi122.stdout:669: expect (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:669: done (2 left) 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:670: done (1 left) 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:671: done (0 left) 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:672: rollback oid 13 current snap is 77 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 76 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:673: snap_create 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:672: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:48:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 0 (ObjNum 289 snap 0 seq_num 1842317280) dirty dne 2024-09-15T03:48:42.912 INFO:tasks.rados.rados.0.smithi122.stdout:672: done (1 left) 2024-09-15T03:48:42.912 INFO:tasks.rados.rados.0.smithi122.stdout:673: done (0 left) 2024-09-15T03:48:42.912 INFO:tasks.rados.rados.0.smithi122.stdout:674: write (excl) oid 5 current snap is 78 2024-09-15T03:48:42.912 INFO:tasks.rados.rados.0.smithi122.stdout:674: seq_num 267 ranges {487469=563838,1607751=551752} 2024-09-15T03:48:42.920 INFO:tasks.rados.rados.0.smithi122.stdout:674: writing smithi12223906-5 from 487469 to 1051307 tid 1 2024-09-15T03:48:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:674: writing smithi12223906-5 from 1607751 to 2159503 tid 2 2024-09-15T03:48:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:675: write oid 16 current snap is 78 2024-09-15T03:48:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:674: finishing write tid 1 to smithi12223906-5 2024-09-15T03:48:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:675: seq_num 268 ranges {504276=639087,1666233=695410} 2024-09-15T03:48:42.934 INFO:tasks.rados.rados.0.smithi122.stdout:675: writing smithi12223906-16 from 504276 to 1143363 tid 1 2024-09-15T03:48:42.941 INFO:tasks.rados.rados.0.smithi122.stdout:675: writing smithi12223906-16 from 1666233 to 2361643 tid 2 2024-09-15T03:48:42.941 INFO:tasks.rados.rados.0.smithi122.stdout:676: read oid 32 snap -1 2024-09-15T03:48:42.941 INFO:tasks.rados.rados.0.smithi122.stdout:676: expect (ObjNum 142 snap 32 seq_num 142) 2024-09-15T03:48:42.941 INFO:tasks.rados.rados.0.smithi122.stdout:677: write oid 27 current snap is 78 2024-09-15T03:48:42.941 INFO:tasks.rados.rados.0.smithi122.stdout:674: finishing write tid 2 to smithi12223906-5 2024-09-15T03:48:42.941 INFO:tasks.rados.rados.0.smithi122.stdout:677: seq_num 269 ranges {521083=714336,1724715=555421} 2024-09-15T03:48:42.948 INFO:tasks.rados.rados.0.smithi122.stdout:677: writing smithi12223906-27 from 521083 to 1235419 tid 1 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:677: writing smithi12223906-27 from 1724715 to 2280136 tid 2 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:678: delete oid 25 current snap is 78 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:674: finishing write tid 3 to smithi12223906-5 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:674: finishing write tid 4 to smithi12223906-5 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 177 (ObjNum 267 snap 78 seq_num 267) dirty exists 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:674: left oid 5 (ObjNum 267 snap 78 seq_num 267) 2024-09-15T03:48:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:675: finishing write tid 1 to smithi12223906-16 2024-09-15T03:48:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:675: finishing write tid 2 to smithi12223906-16 2024-09-15T03:48:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:675: finishing write tid 3 to smithi12223906-16 2024-09-15T03:48:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:675: finishing write tid 4 to smithi12223906-16 2024-09-15T03:48:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 197 (ObjNum 268 snap 78 seq_num 268) dirty exists 2024-09-15T03:48:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:675: left oid 16 (ObjNum 268 snap 78 seq_num 268) 2024-09-15T03:48:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:676: expect (ObjNum 142 snap 32 seq_num 142) 2024-09-15T03:48:43.085 INFO:tasks.rados.rados.0.smithi122.stdout:677: finishing write tid 1 to smithi12223906-27 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:677: finishing write tid 2 to smithi12223906-27 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:677: finishing write tid 3 to smithi12223906-27 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:677: finishing write tid 4 to smithi12223906-27 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 326 (ObjNum 269 snap 78 seq_num 269) dirty exists 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:677: left oid 27 (ObjNum 269 snap 78 seq_num 269) 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:674: done (4 left) 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:675: done (3 left) 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:676: done (2 left) 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:677: done (1 left) 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:678: done (0 left) 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:679: rollback oid 17 current snap is 78 2024-09-15T03:48:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 76 2024-09-15T03:48:43.087 INFO:tasks.rados.rados.0.smithi122.stdout:680: snap_remove snap 77 2024-09-15T03:48:43.920 INFO:tasks.rados.rados.0.smithi122.stdout:680: done (1 left) 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:681: read oid 4 snap -1 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:681: expect (ObjNum 261 snap 73 seq_num 261) 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:682: delete oid 1 current snap is 78 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:679: finishing rollback tid 0 to smithi12223906-17 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:679: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:679: finishing rollback tid 2 to smithi12223906-17 2024-09-15T03:48:43.921 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 262 (ObjNum 214 snap 58 seq_num 214) dirty exists 2024-09-15T03:48:43.925 INFO:tasks.rados.rados.0.smithi122.stdout:679: done (2 left) 2024-09-15T03:48:43.925 INFO:tasks.rados.rados.0.smithi122.stdout:682: done (1 left) 2024-09-15T03:48:43.926 INFO:tasks.rados.rados.0.smithi122.stdout:683: snap_create 2024-09-15T03:48:43.930 INFO:tasks.rados.rados.0.smithi122.stdout:681: expect (ObjNum 261 snap 73 seq_num 261) 2024-09-15T03:48:44.932 INFO:tasks.rados.rados.0.smithi122.stdout:681: done (1 left) 2024-09-15T03:48:44.933 INFO:tasks.rados.rados.0.smithi122.stdout:683: done (0 left) 2024-09-15T03:48:44.933 INFO:tasks.rados.rados.0.smithi122.stdout:684: read oid 15 snap -1 2024-09-15T03:48:44.933 INFO:tasks.rados.rados.0.smithi122.stdout:684: expect (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:48:44.933 INFO:tasks.rados.rados.0.smithi122.stdout:685: snap_remove snap 78 2024-09-15T03:48:45.940 INFO:tasks.rados.rados.0.smithi122.stdout:685: done (1 left) 2024-09-15T03:48:45.940 INFO:tasks.rados.rados.0.smithi122.stdout:686: snap_remove snap 75 2024-09-15T03:48:46.750 INFO:tasks.rados.rados.0.smithi122.stdout:686: done (1 left) 2024-09-15T03:48:46.750 INFO:tasks.rados.rados.0.smithi122.stdout:687: copy_from oid 4 from oid 2 current snap is 79 2024-09-15T03:48:46.750 INFO:tasks.rados.rados.0.smithi122.stdout:684: expect (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:48:46.825 INFO:tasks.rados.rados.0.smithi122.stdout:687: finishing copy_from racing read to smithi12223906-4 2024-09-15T03:48:46.825 INFO:tasks.rados.rados.0.smithi122.stdout:687: finishing copy_from to smithi12223906-4 2024-09-15T03:48:46.825 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 150 (ObjNum 236 snap 65 seq_num 236) dirty exists 2024-09-15T03:48:46.825 INFO:tasks.rados.rados.0.smithi122.stdout:684: done (1 left) 2024-09-15T03:48:46.825 INFO:tasks.rados.rados.0.smithi122.stdout:687: done (0 left) 2024-09-15T03:48:46.825 INFO:tasks.rados.rados.0.smithi122.stdout:688: snap_create 2024-09-15T03:48:47.763 INFO:tasks.rados.rados.0.smithi122.stdout:688: done (0 left) 2024-09-15T03:48:47.763 INFO:tasks.rados.rados.0.smithi122.stdout:689: read oid 35 snap -1 2024-09-15T03:48:47.764 INFO:tasks.rados.rados.0.smithi122.stdout:689: expect (ObjNum 250 snap 72 seq_num 250) 2024-09-15T03:48:47.764 INFO:tasks.rados.rados.0.smithi122.stdout:690: rollback oid 12 current snap is 80 2024-09-15T03:48:47.764 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 76 2024-09-15T03:48:47.764 INFO:tasks.rados.rados.0.smithi122.stdout:691: write (excl) oid 1 current snap is 80 2024-09-15T03:48:47.764 INFO:tasks.rados.rados.0.smithi122.stdout:691: seq_num 272 ranges {571504=540083,1783808=702748} 2024-09-15T03:48:47.772 INFO:tasks.rados.rados.0.smithi122.stdout:691: writing smithi12223906-1 from 571504 to 1111587 tid 1 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:691: writing smithi12223906-1 from 1783808 to 2486556 tid 2 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:692: delete oid 6 current snap is 80 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:690: finishing rollback tid 0 to smithi12223906-12 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:690: finishing rollback tid 1 to smithi12223906-12 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:690: finishing rollback tid 2 to smithi12223906-12 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 190 (ObjNum 111 snap 23 seq_num 111) dirty exists 2024-09-15T03:48:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:689: expect (ObjNum 250 snap 72 seq_num 250) 2024-09-15T03:48:47.880 INFO:tasks.rados.rados.0.smithi122.stdout:691: finishing write tid 1 to smithi12223906-1 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:691: finishing write tid 2 to smithi12223906-1 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:691: finishing write tid 3 to smithi12223906-1 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:691: finishing write tid 4 to smithi12223906-1 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 204 (ObjNum 272 snap 80 seq_num 272) dirty exists 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:691: left oid 1 (ObjNum 272 snap 80 seq_num 272) 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:689: done (3 left) 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:690: done (2 left) 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:691: done (1 left) 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:692: done (0 left) 2024-09-15T03:48:47.881 INFO:tasks.rados.rados.0.smithi122.stdout:693: write (excl) oid 26 current snap is 80 2024-09-15T03:48:47.882 INFO:tasks.rados.rados.0.smithi122.stdout:693: seq_num 274 ranges {605118=406934,1617125=706417} 2024-09-15T03:48:47.895 INFO:tasks.rados.rados.0.smithi122.stdout:693: writing smithi12223906-26 from 605118 to 1012052 tid 1 2024-09-15T03:48:47.903 INFO:tasks.rados.rados.0.smithi122.stdout:693: writing smithi12223906-26 from 1617125 to 2323542 tid 2 2024-09-15T03:48:47.903 INFO:tasks.rados.rados.0.smithi122.stdout:694: copy_from oid 36 from oid 6 current snap is 80 2024-09-15T03:48:47.903 INFO:tasks.rados.rados.0.smithi122.stdout:695: delete oid 39 current snap is 80 2024-09-15T03:48:47.908 INFO:tasks.rados.rados.0.smithi122.stdout:694: finishing copy_from to smithi12223906-36 2024-09-15T03:48:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:694: got expected ENOENT (src dne) 2024-09-15T03:48:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:694: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:48:47.911 INFO:tasks.rados.rados.0.smithi122.stdout:694: done (2 left) 2024-09-15T03:48:47.911 INFO:tasks.rados.rados.0.smithi122.stdout:695: done (1 left) 2024-09-15T03:48:47.911 INFO:tasks.rados.rados.0.smithi122.stdout:696: write (excl) oid 17 current snap is 80 2024-09-15T03:48:47.911 INFO:tasks.rados.rados.0.smithi122.stdout:696: seq_num 276 ranges {638732=557432,1617736=710086} 2024-09-15T03:48:47.917 INFO:tasks.rados.rados.0.smithi122.stdout:696: writing smithi12223906-17 from 638732 to 1196164 tid 1 2024-09-15T03:48:47.925 INFO:tasks.rados.rados.0.smithi122.stdout:696: writing smithi12223906-17 from 1617736 to 2327822 tid 2 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:697: copy_from oid 24 from oid 44 current snap is 80 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:693: finishing write tid 1 to smithi12223906-26 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:693: finishing write tid 2 to smithi12223906-26 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:693: finishing write tid 3 to smithi12223906-26 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:693: finishing write tid 4 to smithi12223906-26 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 182 (ObjNum 274 snap 80 seq_num 274) dirty exists 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:693: left oid 26 (ObjNum 274 snap 80 seq_num 274) 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:693: done (2 left) 2024-09-15T03:48:47.926 INFO:tasks.rados.rados.0.smithi122.stdout:698: snap_create 2024-09-15T03:48:47.927 INFO:tasks.rados.rados.0.smithi122.stdout:696: finishing write tid 1 to smithi12223906-17 2024-09-15T03:48:47.927 INFO:tasks.rados.rados.0.smithi122.stdout:697: finishing copy_from to smithi12223906-24 2024-09-15T03:48:47.927 INFO:tasks.rados.rados.0.smithi122.stdout:697: got expected ENOENT (src dne) 2024-09-15T03:48:47.927 INFO:tasks.rados.rados.0.smithi122.stdout:697: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:48:47.936 INFO:tasks.rados.rados.0.smithi122.stdout:696: finishing write tid 2 to smithi12223906-17 2024-09-15T03:48:47.936 INFO:tasks.rados.rados.0.smithi122.stdout:696: finishing write tid 3 to smithi12223906-17 2024-09-15T03:48:47.936 INFO:tasks.rados.rados.0.smithi122.stdout:696: finishing write tid 4 to smithi12223906-17 2024-09-15T03:48:47.936 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 265 (ObjNum 276 snap 80 seq_num 276) dirty exists 2024-09-15T03:48:47.936 INFO:tasks.rados.rados.0.smithi122.stdout:696: left oid 17 (ObjNum 276 snap 80 seq_num 276) 2024-09-15T03:48:48.765 INFO:tasks.rados.rados.0.smithi122.stdout:696: done (2 left) 2024-09-15T03:48:48.765 INFO:tasks.rados.rados.0.smithi122.stdout:697: done (1 left) 2024-09-15T03:48:48.765 INFO:tasks.rados.rados.0.smithi122.stdout:698: done (0 left) 2024-09-15T03:48:48.765 INFO:tasks.rados.rados.0.smithi122.stdout:699: snap_remove snap 80 2024-09-15T03:48:49.785 INFO:tasks.rados.rados.0.smithi122.stdout:699: done (0 left) 2024-09-15T03:48:49.785 INFO:tasks.rados.rados.0.smithi122.stdout:700: snap_remove snap 76 2024-09-15T03:48:50.800 INFO:tasks.rados.rados.0.smithi122.stdout:700: done (0 left) 2024-09-15T03:48:50.800 INFO:tasks.rados.rados.0.smithi122.stdout:701: snap_remove snap 79 2024-09-15T03:48:51.814 INFO:tasks.rados.rados.0.smithi122.stdout:701: done (0 left) 2024-09-15T03:48:51.814 INFO:tasks.rados.rados.0.smithi122.stdout:702: rollback oid 17 current snap is 81 2024-09-15T03:48:51.814 INFO:tasks.rados.rados.0.smithi122.stdout:702: done (0 left) 2024-09-15T03:48:51.814 INFO:tasks.rados.rados.0.smithi122.stdout:703: read oid 45 snap -1 2024-09-15T03:48:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:703: expect deleted 2024-09-15T03:48:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:704: write oid 32 current snap is 81 2024-09-15T03:48:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:704: seq_num 277 ranges {655539=632681,2076218=570097} 2024-09-15T03:48:51.818 INFO:tasks.rados.rados.0.smithi122.stdout:704: writing smithi12223906-32 from 655539 to 1288220 tid 1 2024-09-15T03:48:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:704: writing smithi12223906-32 from 2076218 to 2646315 tid 2 2024-09-15T03:48:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:705: delete oid 13 current snap is 81 2024-09-15T03:48:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:704: finishing write tid 1 to smithi12223906-32 2024-09-15T03:48:51.831 INFO:tasks.rados.rados.0.smithi122.stdout:703: done (2 left) 2024-09-15T03:48:51.831 INFO:tasks.rados.rados.0.smithi122.stdout:705: done (1 left) 2024-09-15T03:48:51.831 INFO:tasks.rados.rados.0.smithi122.stdout:706: snap_create 2024-09-15T03:48:51.833 INFO:tasks.rados.rados.0.smithi122.stdout:704: finishing write tid 2 to smithi12223906-32 2024-09-15T03:48:51.833 INFO:tasks.rados.rados.0.smithi122.stdout:704: finishing write tid 3 to smithi12223906-32 2024-09-15T03:48:51.833 INFO:tasks.rados.rados.0.smithi122.stdout:704: finishing write tid 4 to smithi12223906-32 2024-09-15T03:48:51.833 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 335 (ObjNum 277 snap 81 seq_num 277) dirty exists 2024-09-15T03:48:51.834 INFO:tasks.rados.rados.0.smithi122.stdout:704: left oid 32 (ObjNum 277 snap 81 seq_num 277) 2024-09-15T03:48:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:704: done (1 left) 2024-09-15T03:48:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:706: done (0 left) 2024-09-15T03:48:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:707: write (excl) oid 30 current snap is 82 2024-09-15T03:48:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:707: seq_num 279 ranges {689153=783179,2076829=457413} 2024-09-15T03:48:52.824 INFO:tasks.rados.rados.0.smithi122.stdout:707: writing smithi12223906-30 from 689153 to 1472332 tid 1 2024-09-15T03:48:52.830 INFO:tasks.rados.rados.0.smithi122.stdout:707: writing smithi12223906-30 from 2076829 to 2534242 tid 2 2024-09-15T03:48:52.830 INFO:tasks.rados.rados.0.smithi122.stdout:708: rollback oid 23 current snap is 82 2024-09-15T03:48:52.831 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 81 2024-09-15T03:48:52.831 INFO:tasks.rados.rados.0.smithi122.stdout:709: write (excl) oid 26 current snap is 82 2024-09-15T03:48:52.831 INFO:tasks.rados.rados.0.smithi122.stdout:709: seq_num 280 ranges {705960=458428,1735311=717424} 2024-09-15T03:48:52.837 INFO:tasks.rados.rados.0.smithi122.stdout:709: writing smithi12223906-26 from 705960 to 1164388 tid 1 2024-09-15T03:48:52.845 INFO:tasks.rados.rados.0.smithi122.stdout:709: writing smithi12223906-26 from 1735311 to 2452735 tid 2 2024-09-15T03:48:52.845 INFO:tasks.rados.rados.0.smithi122.stdout:710: copy_from oid 17 from oid 40 current snap is 82 2024-09-15T03:48:52.845 INFO:tasks.rados.rados.0.smithi122.stdout:707: finishing write tid 1 to smithi12223906-30 2024-09-15T03:48:52.845 INFO:tasks.rados.rados.0.smithi122.stdout:707: finishing write tid 2 to smithi12223906-30 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:708: finishing rollback tid 0 to smithi12223906-23 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:707: finishing write tid 3 to smithi12223906-30 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:708: finishing rollback tid 1 to smithi12223906-23 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:707: finishing write tid 4 to smithi12223906-30 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 136 (ObjNum 279 snap 82 seq_num 279) dirty exists 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:707: left oid 30 (ObjNum 279 snap 82 seq_num 279) 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:708: finishing rollback tid 2 to smithi12223906-23 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 339 (ObjNum 259 snap 73 seq_num 259) dirty exists 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:709: finishing write tid 1 to smithi12223906-26 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:707: done (3 left) 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:708: done (2 left) 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:711: copy_from oid 38 from oid 29 current snap is 82 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:712: write oid 36 current snap is 82 2024-09-15T03:48:52.846 INFO:tasks.rados.rados.0.smithi122.stdout:712: seq_num 281 ranges {722767=533677,1793793=461082} 2024-09-15T03:48:52.852 INFO:tasks.rados.rados.0.smithi122.stdout:712: writing smithi12223906-36 from 722767 to 1256444 tid 1 2024-09-15T03:48:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:712: writing smithi12223906-36 from 1793793 to 2254875 tid 2 2024-09-15T03:48:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:713: rollback oid 25 current snap is 82 2024-09-15T03:48:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 81 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:710: finishing copy_from to smithi12223906-17 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:710: got expected ENOENT (src dne) 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:714: snap_create 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:710: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:709: finishing write tid 2 to smithi12223906-26 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:709: finishing write tid 3 to smithi12223906-26 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:709: finishing write tid 4 to smithi12223906-26 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 190 (ObjNum 280 snap 82 seq_num 280) dirty exists 2024-09-15T03:48:52.859 INFO:tasks.rados.rados.0.smithi122.stdout:709: left oid 26 (ObjNum 280 snap 82 seq_num 280) 2024-09-15T03:48:52.864 INFO:tasks.rados.rados.0.smithi122.stdout:712: finishing write tid 1 to smithi12223906-36 2024-09-15T03:48:52.865 INFO:tasks.rados.rados.0.smithi122.stdout:711: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:48:52.866 INFO:tasks.rados.rados.0.smithi122.stdout:712: finishing write tid 2 to smithi12223906-36 2024-09-15T03:48:52.866 INFO:tasks.rados.rados.0.smithi122.stdout:711: finishing copy_from to smithi12223906-38 2024-09-15T03:48:52.866 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 46 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-15T03:48:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:712: finishing write tid 3 to smithi12223906-36 2024-09-15T03:48:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:712: finishing write tid 4 to smithi12223906-36 2024-09-15T03:48:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 348 (ObjNum 281 snap 82 seq_num 281) dirty exists 2024-09-15T03:48:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:712: left oid 36 (ObjNum 281 snap 82 seq_num 281) 2024-09-15T03:48:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:713: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:48:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 0 (ObjNum 129 snap 0 seq_num 1842683088) dirty dne 2024-09-15T03:48:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:709: done (5 left) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:710: done (4 left) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:711: done (3 left) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:712: done (2 left) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:713: done (1 left) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:714: done (0 left) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:715: read oid 35 snap -1 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:715: expect (ObjNum 250 snap 72 seq_num 250) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:716: read oid 24 snap -1 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:716: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:717: read oid 39 snap -1 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:717: expect deleted 2024-09-15T03:48:53.830 INFO:tasks.rados.rados.0.smithi122.stdout:718: write oid 8 current snap is 83 2024-09-15T03:48:53.831 INFO:tasks.rados.rados.0.smithi122.stdout:718: seq_num 282 ranges {739574=725279,1968628=721093} 2024-09-15T03:48:53.841 INFO:tasks.rados.rados.0.smithi122.stdout:718: writing smithi12223906-8 from 739574 to 1464853 tid 1 2024-09-15T03:48:53.847 INFO:tasks.rados.rados.0.smithi122.stdout:718: writing smithi12223906-8 from 1968628 to 2689721 tid 2 2024-09-15T03:48:53.848 INFO:tasks.rados.rados.0.smithi122.stdout:719: write oid 26 current snap is 83 2024-09-15T03:48:53.848 INFO:tasks.rados.rados.0.smithi122.stdout:719: seq_num 283 ranges {756381=400528,1910757=464751} 2024-09-15T03:48:53.854 INFO:tasks.rados.rados.0.smithi122.stdout:719: writing smithi12223906-26 from 756381 to 1156909 tid 1 2024-09-15T03:48:53.862 INFO:tasks.rados.rados.0.smithi122.stdout:719: writing smithi12223906-26 from 1910757 to 2375508 tid 2 2024-09-15T03:48:53.862 INFO:tasks.rados.rados.0.smithi122.stdout:720: read oid 37 snap -1 2024-09-15T03:48:53.862 INFO:tasks.rados.rados.0.smithi122.stdout:720: expect (ObjNum 133 snap 29 seq_num 133) 2024-09-15T03:48:53.863 INFO:tasks.rados.rados.0.smithi122.stdout:721: delete oid 3 current snap is 83 2024-09-15T03:48:53.863 INFO:tasks.rados.rados.0.smithi122.stdout:715: expect (ObjNum 250 snap 72 seq_num 250) 2024-09-15T03:48:53.960 INFO:tasks.rados.rados.0.smithi122.stdout:716: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:54.118 INFO:tasks.rados.rados.0.smithi122.stdout:718: finishing write tid 1 to smithi12223906-8 2024-09-15T03:48:54.118 INFO:tasks.rados.rados.0.smithi122.stdout:718: finishing write tid 2 to smithi12223906-8 2024-09-15T03:48:54.118 INFO:tasks.rados.rados.0.smithi122.stdout:718: finishing write tid 3 to smithi12223906-8 2024-09-15T03:48:54.118 INFO:tasks.rados.rados.0.smithi122.stdout:718: finishing write tid 4 to smithi12223906-8 2024-09-15T03:48:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 195 (ObjNum 282 snap 83 seq_num 282) dirty exists 2024-09-15T03:48:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:718: left oid 8 (ObjNum 282 snap 83 seq_num 282) 2024-09-15T03:48:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:719: finishing write tid 1 to smithi12223906-26 2024-09-15T03:48:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:719: finishing write tid 2 to smithi12223906-26 2024-09-15T03:48:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:719: finishing write tid 3 to smithi12223906-26 2024-09-15T03:48:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:720: expect (ObjNum 133 snap 29 seq_num 133) 2024-09-15T03:48:54.229 INFO:tasks.rados.rados.0.smithi122.stdout:719: finishing write tid 4 to smithi12223906-26 2024-09-15T03:48:54.229 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 199 (ObjNum 283 snap 83 seq_num 283) dirty exists 2024-09-15T03:48:54.229 INFO:tasks.rados.rados.0.smithi122.stdout:719: left oid 26 (ObjNum 283 snap 83 seq_num 283) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:715: done (6 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:716: done (5 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:717: done (4 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:718: done (3 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:719: done (2 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:720: done (1 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:721: done (0 left) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:722: read oid 24 snap -1 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:722: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:723: read oid 41 snap -1 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:723: expect (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:724: read oid 1 snap -1 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:724: expect (ObjNum 272 snap 80 seq_num 272) 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:725: read oid 16 snap -1 2024-09-15T03:48:54.230 INFO:tasks.rados.rados.0.smithi122.stdout:725: expect (ObjNum 268 snap 78 seq_num 268) 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:726: rollback oid 20 current snap is 83 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 82 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:727: read oid 30 snap 82 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:727: expect (ObjNum 279 snap 82 seq_num 279) 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:728: read oid 38 snap 82 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:728: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:48:54.231 INFO:tasks.rados.rados.0.smithi122.stdout:729: snap_create 2024-09-15T03:48:54.238 INFO:tasks.rados.rados.0.smithi122.stdout:726: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:48:54.239 INFO:tasks.rados.rados.0.smithi122.stdout:722: expect (ObjNum 213 snap 58 seq_num 213) 2024-09-15T03:48:54.389 INFO:tasks.rados.rados.0.smithi122.stdout:723: expect (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:48:54.550 INFO:tasks.rados.rados.0.smithi122.stdout:724: expect (ObjNum 272 snap 80 seq_num 272) 2024-09-15T03:48:54.626 INFO:tasks.rados.rados.0.smithi122.stdout:726: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:48:54.626 INFO:tasks.rados.rados.0.smithi122.stdout:726: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:48:54.626 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 35 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:48:54.626 INFO:tasks.rados.rados.0.smithi122.stdout:727: expect (ObjNum 279 snap 82 seq_num 279) 2024-09-15T03:48:54.790 INFO:tasks.rados.rados.0.smithi122.stdout:728: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-15T03:48:54.905 INFO:tasks.rados.rados.0.smithi122.stdout:725: expect (ObjNum 268 snap 78 seq_num 268) 2024-09-15T03:48:54.978 INFO:tasks.rados.rados.0.smithi122.stdout:722: done (7 left) 2024-09-15T03:48:54.978 INFO:tasks.rados.rados.0.smithi122.stdout:723: done (6 left) 2024-09-15T03:48:54.978 INFO:tasks.rados.rados.0.smithi122.stdout:724: done (5 left) 2024-09-15T03:48:54.978 INFO:tasks.rados.rados.0.smithi122.stdout:725: done (4 left) 2024-09-15T03:48:54.978 INFO:tasks.rados.rados.0.smithi122.stdout:726: done (3 left) 2024-09-15T03:48:54.978 INFO:tasks.rados.rados.0.smithi122.stdout:727: done (2 left) 2024-09-15T03:48:54.979 INFO:tasks.rados.rados.0.smithi122.stdout:728: done (1 left) 2024-09-15T03:48:54.979 INFO:tasks.rados.rados.0.smithi122.stdout:729: done (0 left) 2024-09-15T03:48:54.979 INFO:tasks.rados.rados.0.smithi122.stdout:730: snap_remove snap 81 2024-09-15T03:48:55.852 INFO:tasks.rados.rados.0.smithi122.stdout:730: done (0 left) 2024-09-15T03:48:55.852 INFO:tasks.rados.rados.0.smithi122.stdout:731: read oid 21 snap 82 2024-09-15T03:48:55.852 INFO:tasks.rados.rados.0.smithi122.stdout:731: expect deleted 2024-09-15T03:48:55.853 INFO:tasks.rados.rados.0.smithi122.stdout:732: rollback oid 50 current snap is 84 2024-09-15T03:48:55.853 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 82 2024-09-15T03:48:55.853 INFO:tasks.rados.rados.0.smithi122.stdout:733: write (excl) oid 43 current snap is 84 2024-09-15T03:48:55.853 INFO:tasks.rados.rados.0.smithi122.stdout:733: seq_num 285 ranges {789995=551026,2027721=468420} 2024-09-15T03:48:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:733: writing smithi12223906-43 from 789995 to 1341021 tid 1 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:733: writing smithi12223906-43 from 2027721 to 2496141 tid 2 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:734: read oid 2 snap -1 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:734: expect (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:735: write (excl) oid 16 current snap is 84 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:732: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:732: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:48:55.870 INFO:tasks.rados.rados.0.smithi122.stdout:732: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:48:55.871 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 343 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-15T03:48:55.871 INFO:tasks.rados.rados.0.smithi122.stdout:735: seq_num 286 ranges {406802=626275,1686203=728431} 2024-09-15T03:48:55.876 INFO:tasks.rados.rados.0.smithi122.stdout:735: writing smithi12223906-16 from 406802 to 1033077 tid 1 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:735: writing smithi12223906-16 from 1686203 to 2414634 tid 2 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:731: done (4 left) 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:732: done (3 left) 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:736: read oid 40 snap 82 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:736: expect deleted 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:737: rollback oid 15 current snap is 84 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 82 2024-09-15T03:48:55.885 INFO:tasks.rados.rados.0.smithi122.stdout:733: finishing write tid 1 to smithi12223906-43 2024-09-15T03:48:55.886 INFO:tasks.rados.rados.0.smithi122.stdout:733: finishing write tid 2 to smithi12223906-43 2024-09-15T03:48:55.886 INFO:tasks.rados.rados.0.smithi122.stdout:733: finishing write tid 3 to smithi12223906-43 2024-09-15T03:48:55.886 INFO:tasks.rados.rados.0.smithi122.stdout:733: finishing write tid 4 to smithi12223906-43 2024-09-15T03:48:55.886 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 359 (ObjNum 285 snap 84 seq_num 285) dirty exists 2024-09-15T03:48:55.886 INFO:tasks.rados.rados.0.smithi122.stdout:733: left oid 43 (ObjNum 285 snap 84 seq_num 285) 2024-09-15T03:48:55.886 INFO:tasks.rados.rados.0.smithi122.stdout:734: expect (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:48:56.177 INFO:tasks.rados.rados.0.smithi122.stdout:735: finishing write tid 1 to smithi12223906-16 2024-09-15T03:48:56.177 INFO:tasks.rados.rados.0.smithi122.stdout:735: finishing write tid 2 to smithi12223906-16 2024-09-15T03:48:56.177 INFO:tasks.rados.rados.0.smithi122.stdout:735: finishing write tid 3 to smithi12223906-16 2024-09-15T03:48:56.177 INFO:tasks.rados.rados.0.smithi122.stdout:735: finishing write tid 4 to smithi12223906-16 2024-09-15T03:48:56.177 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 210 (ObjNum 286 snap 84 seq_num 286) dirty exists 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:735: left oid 16 (ObjNum 286 snap 84 seq_num 286) 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:737: finishing rollback tid 0 to smithi12223906-15 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:737: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:737: finishing rollback tid 2 to smithi12223906-15 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 214 (ObjNum 242 snap 68 seq_num 242) dirty exists 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:733: done (4 left) 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:734: done (3 left) 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:735: done (2 left) 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:736: done (1 left) 2024-09-15T03:48:56.178 INFO:tasks.rados.rados.0.smithi122.stdout:737: done (0 left) 2024-09-15T03:48:56.179 INFO:tasks.rados.rados.0.smithi122.stdout:738: snap_create 2024-09-15T03:48:56.754 INFO:tasks.rados.rados.0.smithi122.stdout:738: done (0 left) 2024-09-15T03:48:56.754 INFO:tasks.rados.rados.0.smithi122.stdout:739: write (excl) oid 39 current snap is 85 2024-09-15T03:48:56.754 INFO:tasks.rados.rados.0.smithi122.stdout:739: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2024-09-15T03:48:56.765 INFO:tasks.rados.rados.0.smithi122.stdout:739: writing smithi12223906-39 from 423609 to 1125133 tid 1 2024-09-15T03:48:56.771 INFO:tasks.rados.rados.0.smithi122.stdout:739: writing smithi12223906-39 from 1628332 to 2100421 tid 2 2024-09-15T03:48:56.775 INFO:tasks.rados.rados.0.smithi122.stdout:739: writing smithi12223906-39 from 2648987 to 2823609 tid 3 2024-09-15T03:48:56.775 INFO:tasks.rados.rados.0.smithi122.stdout:740: write oid 14 current snap is 85 2024-09-15T03:48:56.775 INFO:tasks.rados.rados.0.smithi122.stdout:740: seq_num 288 ranges {440416=776773,1686814=732100} 2024-09-15T03:48:56.782 INFO:tasks.rados.rados.0.smithi122.stdout:740: writing smithi12223906-14 from 440416 to 1217189 tid 1 2024-09-15T03:48:56.789 INFO:tasks.rados.rados.0.smithi122.stdout:740: writing smithi12223906-14 from 1686814 to 2418914 tid 2 2024-09-15T03:48:56.789 INFO:tasks.rados.rados.0.smithi122.stdout:741: snap_create 2024-09-15T03:48:56.789 INFO:tasks.rados.rados.0.smithi122.stdout:739: finishing write tid 1 to smithi12223906-39 2024-09-15T03:48:56.789 INFO:tasks.rados.rados.0.smithi122.stdout:739: finishing write tid 2 to smithi12223906-39 2024-09-15T03:48:56.789 INFO:tasks.rados.rados.0.smithi122.stdout:739: finishing write tid 3 to smithi12223906-39 2024-09-15T03:48:56.789 INFO:tasks.rados.rados.0.smithi122.stdout:739: finishing write tid 4 to smithi12223906-39 2024-09-15T03:48:56.790 INFO:tasks.rados.rados.0.smithi122.stdout:739: finishing write tid 5 to smithi12223906-39 2024-09-15T03:48:56.790 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 363 (ObjNum 287 snap 85 seq_num 287) dirty exists 2024-09-15T03:48:56.790 INFO:tasks.rados.rados.0.smithi122.stdout:739: left oid 39 (ObjNum 287 snap 85 seq_num 287) 2024-09-15T03:48:56.790 INFO:tasks.rados.rados.0.smithi122.stdout:740: finishing write tid 1 to smithi12223906-14 2024-09-15T03:48:56.794 INFO:tasks.rados.rados.0.smithi122.stdout:740: finishing write tid 2 to smithi12223906-14 2024-09-15T03:48:56.795 INFO:tasks.rados.rados.0.smithi122.stdout:740: finishing write tid 3 to smithi12223906-14 2024-09-15T03:48:56.795 INFO:tasks.rados.rados.0.smithi122.stdout:740: finishing write tid 4 to smithi12223906-14 2024-09-15T03:48:56.795 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 367 (ObjNum 288 snap 85 seq_num 288) dirty exists 2024-09-15T03:48:56.795 INFO:tasks.rados.rados.0.smithi122.stdout:740: left oid 14 (ObjNum 288 snap 85 seq_num 288) 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:739: done (2 left) 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:740: done (1 left) 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:741: done (0 left) 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:742: copy_from oid 9 from oid 28 current snap is 86 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:743: read oid 33 snap -1 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:743: expect deleted 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:744: read oid 7 snap -1 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:744: expect (ObjNum 220 snap 59 seq_num 220) 2024-09-15T03:48:57.768 INFO:tasks.rados.rados.0.smithi122.stdout:745: snap_create 2024-09-15T03:48:57.782 INFO:tasks.rados.rados.0.smithi122.stdout:744: expect (ObjNum 220 snap 59 seq_num 220) 2024-09-15T03:48:58.003 INFO:tasks.rados.rados.0.smithi122.stdout:742: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:48:58.003 INFO:tasks.rados.rados.0.smithi122.stdout:742: finishing copy_from to smithi12223906-9 2024-09-15T03:48:58.003 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 137 (ObjNum 257 snap 73 seq_num 257) dirty exists 2024-09-15T03:48:58.773 INFO:tasks.rados.rados.0.smithi122.stdout:742: done (3 left) 2024-09-15T03:48:58.773 INFO:tasks.rados.rados.0.smithi122.stdout:743: done (2 left) 2024-09-15T03:48:58.773 INFO:tasks.rados.rados.0.smithi122.stdout:744: done (1 left) 2024-09-15T03:48:58.773 INFO:tasks.rados.rados.0.smithi122.stdout:745: done (0 left) 2024-09-15T03:48:58.773 INFO:tasks.rados.rados.0.smithi122.stdout:746: write (excl) oid 2 current snap is 87 2024-09-15T03:48:58.773 INFO:tasks.rados.rados.0.smithi122.stdout:746: seq_num 289 ranges {457223=568375,1461649=475758,2420186=437037} 2024-09-15T03:48:58.780 INFO:tasks.rados.rados.0.smithi122.stdout:746: writing smithi12223906-2 from 457223 to 1025598 tid 1 2024-09-15T03:48:58.786 INFO:tasks.rados.rados.0.smithi122.stdout:746: writing smithi12223906-2 from 1461649 to 1937407 tid 2 2024-09-15T03:48:58.792 INFO:tasks.rados.rados.0.smithi122.stdout:746: writing smithi12223906-2 from 2420186 to 2857223 tid 3 2024-09-15T03:48:58.793 INFO:tasks.rados.rados.0.smithi122.stdout:747: read oid 17 snap -1 2024-09-15T03:48:58.793 INFO:tasks.rados.rados.0.smithi122.stdout:747: expect (ObjNum 276 snap 80 seq_num 276) 2024-09-15T03:48:58.793 INFO:tasks.rados.rados.0.smithi122.stdout:748: snap_create 2024-09-15T03:48:58.793 INFO:tasks.rados.rados.0.smithi122.stdout:746: finishing write tid 1 to smithi12223906-2 2024-09-15T03:48:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:746: finishing write tid 2 to smithi12223906-2 2024-09-15T03:48:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:746: finishing write tid 3 to smithi12223906-2 2024-09-15T03:48:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:746: finishing write tid 4 to smithi12223906-2 2024-09-15T03:48:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:746: finishing write tid 5 to smithi12223906-2 2024-09-15T03:48:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 269 (ObjNum 289 snap 87 seq_num 289) dirty exists 2024-09-15T03:48:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:746: left oid 2 (ObjNum 289 snap 87 seq_num 289) 2024-09-15T03:48:58.804 INFO:tasks.rados.rados.0.smithi122.stdout:747: expect (ObjNum 276 snap 80 seq_num 276) 2024-09-15T03:48:59.789 INFO:tasks.rados.rados.0.smithi122.stdout:746: done (2 left) 2024-09-15T03:48:59.790 INFO:tasks.rados.rados.0.smithi122.stdout:747: done (1 left) 2024-09-15T03:48:59.790 INFO:tasks.rados.rados.0.smithi122.stdout:748: done (0 left) 2024-09-15T03:48:59.790 INFO:tasks.rados.rados.0.smithi122.stdout:749: write (excl) oid 12 current snap is 88 2024-09-15T03:48:59.790 INFO:tasks.rados.rados.0.smithi122.stdout:749: seq_num 290 ranges {474030=643624,1520131=735769} 2024-09-15T03:48:59.798 INFO:tasks.rados.rados.0.smithi122.stdout:749: writing smithi12223906-12 from 474030 to 1117654 tid 1 2024-09-15T03:48:59.807 INFO:tasks.rados.rados.0.smithi122.stdout:749: writing smithi12223906-12 from 1520131 to 2255900 tid 2 2024-09-15T03:48:59.807 INFO:tasks.rados.rados.0.smithi122.stdout:750: write (excl) oid 11 current snap is 88 2024-09-15T03:48:59.807 INFO:tasks.rados.rados.0.smithi122.stdout:749: finishing write tid 1 to smithi12223906-12 2024-09-15T03:48:59.807 INFO:tasks.rados.rados.0.smithi122.stdout:750: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2024-09-15T03:48:59.813 INFO:tasks.rados.rados.0.smithi122.stdout:750: writing smithi12223906-11 from 490837 to 1209710 tid 1 2024-09-15T03:48:59.820 INFO:tasks.rados.rados.0.smithi122.stdout:750: writing smithi12223906-11 from 1862260 to 2341687 tid 2 2024-09-15T03:48:59.823 INFO:tasks.rados.rados.0.smithi122.stdout:750: writing smithi12223906-11 from 2875032 to 2890837 tid 3 2024-09-15T03:48:59.823 INFO:tasks.rados.rados.0.smithi122.stdout:751: snap_create 2024-09-15T03:48:59.823 INFO:tasks.rados.rados.0.smithi122.stdout:749: finishing write tid 2 to smithi12223906-12 2024-09-15T03:48:59.823 INFO:tasks.rados.rados.0.smithi122.stdout:749: finishing write tid 3 to smithi12223906-12 2024-09-15T03:48:59.823 INFO:tasks.rados.rados.0.smithi122.stdout:749: finishing write tid 4 to smithi12223906-12 2024-09-15T03:48:59.823 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 193 (ObjNum 290 snap 88 seq_num 290) dirty exists 2024-09-15T03:48:59.824 INFO:tasks.rados.rados.0.smithi122.stdout:749: left oid 12 (ObjNum 290 snap 88 seq_num 290) 2024-09-15T03:48:59.825 INFO:tasks.rados.rados.0.smithi122.stdout:750: finishing write tid 1 to smithi12223906-11 2024-09-15T03:48:59.826 INFO:tasks.rados.rados.0.smithi122.stdout:750: finishing write tid 2 to smithi12223906-11 2024-09-15T03:48:59.827 INFO:tasks.rados.rados.0.smithi122.stdout:750: finishing write tid 3 to smithi12223906-11 2024-09-15T03:48:59.827 INFO:tasks.rados.rados.0.smithi122.stdout:750: finishing write tid 4 to smithi12223906-11 2024-09-15T03:48:59.828 INFO:tasks.rados.rados.0.smithi122.stdout:750: finishing write tid 5 to smithi12223906-11 2024-09-15T03:48:59.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 219 (ObjNum 291 snap 88 seq_num 291) dirty exists 2024-09-15T03:48:59.828 INFO:tasks.rados.rados.0.smithi122.stdout:750: left oid 11 (ObjNum 291 snap 88 seq_num 291) 2024-09-15T03:49:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:749: done (2 left) 2024-09-15T03:49:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:750: done (1 left) 2024-09-15T03:49:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:751: done (0 left) 2024-09-15T03:49:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:752: read oid 15 snap -1 2024-09-15T03:49:00.800 INFO:tasks.rados.rados.0.smithi122.stdout:752: expect (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:49:00.800 INFO:tasks.rados.rados.0.smithi122.stdout:753: write (excl) oid 23 current snap is 89 2024-09-15T03:49:00.800 INFO:tasks.rados.rados.0.smithi122.stdout:753: seq_num 292 ranges {507644=794122,1920742=623085} 2024-09-15T03:49:00.808 INFO:tasks.rados.rados.0.smithi122.stdout:753: writing smithi12223906-23 from 507644 to 1301766 tid 1 2024-09-15T03:49:00.816 INFO:tasks.rados.rados.0.smithi122.stdout:753: writing smithi12223906-23 from 1920742 to 2543827 tid 2 2024-09-15T03:49:00.816 INFO:tasks.rados.rados.0.smithi122.stdout:754: delete oid 50 current snap is 89 2024-09-15T03:49:00.816 INFO:tasks.rados.rados.0.smithi122.stdout:752: expect (ObjNum 242 snap 68 seq_num 242) 2024-09-15T03:49:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:753: finishing write tid 1 to smithi12223906-23 2024-09-15T03:49:01.359 INFO:tasks.rados.rados.0.smithi122.stdout:753: finishing write tid 2 to smithi12223906-23 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:753: finishing write tid 3 to smithi12223906-23 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:753: finishing write tid 4 to smithi12223906-23 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 349 (ObjNum 292 snap 89 seq_num 292) dirty exists 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:753: left oid 23 (ObjNum 292 snap 89 seq_num 292) 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:752: done (2 left) 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:753: done (1 left) 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:754: done (0 left) 2024-09-15T03:49:01.360 INFO:tasks.rados.rados.0.smithi122.stdout:755: snap_remove snap 86 2024-09-15T03:49:01.814 INFO:tasks.rados.rados.0.smithi122.stdout:755: done (0 left) 2024-09-15T03:49:01.814 INFO:tasks.rados.rados.0.smithi122.stdout:756: read oid 17 snap 84 2024-09-15T03:49:01.814 INFO:tasks.rados.rados.0.smithi122.stdout:756: expect (ObjNum 276 snap 80 seq_num 276) 2024-09-15T03:49:01.814 INFO:tasks.rados.rados.0.smithi122.stdout:757: write (excl) oid 11 current snap is 89 2024-09-15T03:49:01.815 INFO:tasks.rados.rados.0.smithi122.stdout:757: seq_num 294 ranges {541258=544620,1637706=626754,2840948=100310} 2024-09-15T03:49:01.821 INFO:tasks.rados.rados.0.smithi122.stdout:757: writing smithi12223906-11 from 541258 to 1085878 tid 1 2024-09-15T03:49:01.828 INFO:tasks.rados.rados.0.smithi122.stdout:757: writing smithi12223906-11 from 1637706 to 2264460 tid 2 2024-09-15T03:49:01.832 INFO:tasks.rados.rados.0.smithi122.stdout:757: writing smithi12223906-11 from 2840948 to 2941258 tid 3 2024-09-15T03:49:01.832 INFO:tasks.rados.rados.0.smithi122.stdout:758: snap_remove snap 88 2024-09-15T03:49:02.826 INFO:tasks.rados.rados.0.smithi122.stdout:758: done (2 left) 2024-09-15T03:49:02.826 INFO:tasks.rados.rados.0.smithi122.stdout:759: snap_remove snap 83 2024-09-15T03:49:03.839 INFO:tasks.rados.rados.0.smithi122.stdout:759: done (2 left) 2024-09-15T03:49:03.839 INFO:tasks.rados.rados.0.smithi122.stdout:760: write oid 40 current snap is 89 2024-09-15T03:49:03.839 INFO:tasks.rados.rados.0.smithi122.stdout:760: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2024-09-15T03:49:03.848 INFO:tasks.rados.rados.0.smithi122.stdout:760: writing smithi12223906-40 from 558065 to 1177934 tid 1 2024-09-15T03:49:03.853 INFO:tasks.rados.rados.0.smithi122.stdout:760: writing smithi12223906-40 from 1579835 to 2066600 tid 2 2024-09-15T03:49:03.858 INFO:tasks.rados.rados.0.smithi122.stdout:760: writing smithi12223906-40 from 2468371 to 2958065 tid 3 2024-09-15T03:49:03.858 INFO:tasks.rados.rados.0.smithi122.stdout:761: snap_create 2024-09-15T03:49:03.858 INFO:tasks.rados.rados.0.smithi122.stdout:756: expect (ObjNum 276 snap 80 seq_num 276) 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:757: finishing write tid 1 to smithi12223906-11 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:757: finishing write tid 2 to smithi12223906-11 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:757: finishing write tid 3 to smithi12223906-11 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:757: finishing write tid 4 to smithi12223906-11 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:757: finishing write tid 5 to smithi12223906-11 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 226 (ObjNum 294 snap 89 seq_num 294) dirty exists 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:757: left oid 11 (ObjNum 294 snap 89 seq_num 294) 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:760: finishing write tid 1 to smithi12223906-40 2024-09-15T03:49:04.013 INFO:tasks.rados.rados.0.smithi122.stdout:760: finishing write tid 2 to smithi12223906-40 2024-09-15T03:49:04.014 INFO:tasks.rados.rados.0.smithi122.stdout:760: finishing write tid 3 to smithi12223906-40 2024-09-15T03:49:04.014 INFO:tasks.rados.rados.0.smithi122.stdout:760: finishing write tid 4 to smithi12223906-40 2024-09-15T03:49:04.014 INFO:tasks.rados.rados.0.smithi122.stdout:760: finishing write tid 5 to smithi12223906-40 2024-09-15T03:49:04.014 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 142 (ObjNum 295 snap 89 seq_num 295) dirty exists 2024-09-15T03:49:04.014 INFO:tasks.rados.rados.0.smithi122.stdout:760: left oid 40 (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:49:04.850 INFO:tasks.rados.rados.0.smithi122.stdout:756: done (3 left) 2024-09-15T03:49:04.850 INFO:tasks.rados.rados.0.smithi122.stdout:757: done (2 left) 2024-09-15T03:49:04.850 INFO:tasks.rados.rados.0.smithi122.stdout:760: done (1 left) 2024-09-15T03:49:04.850 INFO:tasks.rados.rados.0.smithi122.stdout:761: done (0 left) 2024-09-15T03:49:04.850 INFO:tasks.rados.rados.0.smithi122.stdout:762: write (excl) oid 11 current snap is 90 2024-09-15T03:49:04.850 INFO:tasks.rados.rados.0.smithi122.stdout:762: seq_num 296 ranges {574872=695118,2038317=630423} 2024-09-15T03:49:04.860 INFO:tasks.rados.rados.0.smithi122.stdout:762: writing smithi12223906-11 from 574872 to 1269990 tid 1 2024-09-15T03:49:04.870 INFO:tasks.rados.rados.0.smithi122.stdout:762: writing smithi12223906-11 from 2038317 to 2668740 tid 2 2024-09-15T03:49:04.870 INFO:tasks.rados.rados.0.smithi122.stdout:763: rollback oid 26 current snap is 90 2024-09-15T03:49:04.870 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 89 2024-09-15T03:49:04.870 INFO:tasks.rados.rados.0.smithi122.stdout:762: finishing write tid 1 to smithi12223906-11 2024-09-15T03:49:04.871 INFO:tasks.rados.rados.0.smithi122.stdout:764: read oid 21 snap -1 2024-09-15T03:49:04.871 INFO:tasks.rados.rados.0.smithi122.stdout:764: expect deleted 2024-09-15T03:49:04.871 INFO:tasks.rados.rados.0.smithi122.stdout:765: rollback oid 25 current snap is 90 2024-09-15T03:49:04.871 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 84 2024-09-15T03:49:04.871 INFO:tasks.rados.rados.0.smithi122.stdout:766: snap_create 2024-09-15T03:49:04.874 INFO:tasks.rados.rados.0.smithi122.stdout:762: finishing write tid 2 to smithi12223906-11 2024-09-15T03:49:04.878 INFO:tasks.rados.rados.0.smithi122.stdout:765: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:49:04.878 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 0 (ObjNum 49 snap 0 seq_num 1843158048) dirty dne 2024-09-15T03:49:04.878 INFO:tasks.rados.rados.0.smithi122.stdout:762: finishing write tid 3 to smithi12223906-11 2024-09-15T03:49:04.878 INFO:tasks.rados.rados.0.smithi122.stdout:762: finishing write tid 4 to smithi12223906-11 2024-09-15T03:49:04.878 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 238 (ObjNum 296 snap 90 seq_num 296) dirty exists 2024-09-15T03:49:04.878 INFO:tasks.rados.rados.0.smithi122.stdout:762: left oid 11 (ObjNum 296 snap 90 seq_num 296) 2024-09-15T03:49:04.879 INFO:tasks.rados.rados.0.smithi122.stdout:763: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:49:04.880 INFO:tasks.rados.rados.0.smithi122.stdout:763: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:49:04.881 INFO:tasks.rados.rados.0.smithi122.stdout:763: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:49:04.881 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 207 (ObjNum 283 snap 83 seq_num 283) dirty exists 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:762: done (4 left) 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:763: done (3 left) 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:764: done (2 left) 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:765: done (1 left) 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:766: done (0 left) 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:767: write (excl) oid 13 current snap is 91 2024-09-15T03:49:05.864 INFO:tasks.rados.rados.0.smithi122.stdout:767: seq_num 297 ranges {591679=486720,1813152=490434} 2024-09-15T03:49:05.871 INFO:tasks.rados.rados.0.smithi122.stdout:767: writing smithi12223906-13 from 591679 to 1078399 tid 1 2024-09-15T03:49:05.880 INFO:tasks.rados.rados.0.smithi122.stdout:767: writing smithi12223906-13 from 1813152 to 2303586 tid 2 2024-09-15T03:49:05.880 INFO:tasks.rados.rados.0.smithi122.stdout:768: snap_create 2024-09-15T03:49:05.880 INFO:tasks.rados.rados.0.smithi122.stdout:767: finishing write tid 1 to smithi12223906-13 2024-09-15T03:49:05.883 INFO:tasks.rados.rados.0.smithi122.stdout:767: finishing write tid 2 to smithi12223906-13 2024-09-15T03:49:05.884 INFO:tasks.rados.rados.0.smithi122.stdout:767: finishing write tid 3 to smithi12223906-13 2024-09-15T03:49:05.884 INFO:tasks.rados.rados.0.smithi122.stdout:767: finishing write tid 4 to smithi12223906-13 2024-09-15T03:49:05.884 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 234 (ObjNum 297 snap 91 seq_num 297) dirty exists 2024-09-15T03:49:05.884 INFO:tasks.rados.rados.0.smithi122.stdout:767: left oid 13 (ObjNum 297 snap 91 seq_num 297) 2024-09-15T03:49:06.760 INFO:tasks.rados.rados.0.smithi122.stdout:767: done (1 left) 2024-09-15T03:49:06.760 INFO:tasks.rados.rados.0.smithi122.stdout:768: done (0 left) 2024-09-15T03:49:06.760 INFO:tasks.rados.rados.0.smithi122.stdout:769: rollback oid 34 current snap is 92 2024-09-15T03:49:06.760 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 82 2024-09-15T03:49:06.760 INFO:tasks.rados.rados.0.smithi122.stdout:770: delete oid 2 current snap is 92 2024-09-15T03:49:06.775 INFO:tasks.rados.rados.0.smithi122.stdout:769: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:49:06.775 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 0 (ObjNum 0 snap 0 seq_num 304) dirty dne 2024-09-15T03:49:06.776 INFO:tasks.rados.rados.0.smithi122.stdout:769: done (1 left) 2024-09-15T03:49:06.776 INFO:tasks.rados.rados.0.smithi122.stdout:770: done (0 left) 2024-09-15T03:49:06.776 INFO:tasks.rados.rados.0.smithi122.stdout:771: snap_create 2024-09-15T03:49:07.771 INFO:tasks.rados.rados.0.smithi122.stdout:771: done (0 left) 2024-09-15T03:49:07.771 INFO:tasks.rados.rados.0.smithi122.stdout:772: read oid 21 snap -1 2024-09-15T03:49:07.771 INFO:tasks.rados.rados.0.smithi122.stdout:772: expect deleted 2024-09-15T03:49:07.771 INFO:tasks.rados.rados.0.smithi122.stdout:773: copy_from oid 9 from oid 6 current snap is 93 2024-09-15T03:49:07.771 INFO:tasks.rados.rados.0.smithi122.stdout:774: snap_create 2024-09-15T03:49:07.774 INFO:tasks.rados.rados.0.smithi122.stdout:773: finishing copy_from to smithi12223906-9 2024-09-15T03:49:07.774 INFO:tasks.rados.rados.0.smithi122.stdout:773: got expected ENOENT (src dne) 2024-09-15T03:49:07.774 INFO:tasks.rados.rados.0.smithi122.stdout:773: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:49:08.775 INFO:tasks.rados.rados.0.smithi122.stdout:772: done (2 left) 2024-09-15T03:49:08.775 INFO:tasks.rados.rados.0.smithi122.stdout:773: done (1 left) 2024-09-15T03:49:08.775 INFO:tasks.rados.rados.0.smithi122.stdout:774: done (0 left) 2024-09-15T03:49:08.775 INFO:tasks.rados.rados.0.smithi122.stdout:775: read oid 40 snap -1 2024-09-15T03:49:08.775 INFO:tasks.rados.rados.0.smithi122.stdout:775: expect (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:49:08.776 INFO:tasks.rados.rados.0.smithi122.stdout:776: delete oid 31 current snap is 94 2024-09-15T03:49:08.787 INFO:tasks.rados.rados.0.smithi122.stdout:776: done (1 left) 2024-09-15T03:49:08.787 INFO:tasks.rados.rados.0.smithi122.stdout:777: snap_create 2024-09-15T03:49:08.787 INFO:tasks.rados.rados.0.smithi122.stdout:775: expect (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:49:09.794 INFO:tasks.rados.rados.0.smithi122.stdout:775: done (1 left) 2024-09-15T03:49:09.795 INFO:tasks.rados.rados.0.smithi122.stdout:777: done (0 left) 2024-09-15T03:49:09.795 INFO:tasks.rados.rados.0.smithi122.stdout:778: write oid 1 current snap is 95 2024-09-15T03:49:09.795 INFO:tasks.rados.rados.0.smithi122.stdout:778: seq_num 300 ranges {642100=712467,1872245=637761} 2024-09-15T03:49:09.804 INFO:tasks.rados.rados.0.smithi122.stdout:778: writing smithi12223906-1 from 642100 to 1354567 tid 1 2024-09-15T03:49:09.811 INFO:tasks.rados.rados.0.smithi122.stdout:778: writing smithi12223906-1 from 1872245 to 2510006 tid 2 2024-09-15T03:49:09.811 INFO:tasks.rados.rados.0.smithi122.stdout:779: rollback oid 49 current snap is 95 2024-09-15T03:49:09.812 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 94 2024-09-15T03:49:09.812 INFO:tasks.rados.rados.0.smithi122.stdout:778: finishing write tid 1 to smithi12223906-1 2024-09-15T03:49:09.812 INFO:tasks.rados.rados.0.smithi122.stdout:780: write oid 13 current snap is 95 2024-09-15T03:49:09.812 INFO:tasks.rados.rados.0.smithi122.stdout:780: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2024-09-15T03:49:09.821 INFO:tasks.rados.rados.0.smithi122.stdout:780: writing smithi12223906-13 from 658907 to 1446623 tid 1 2024-09-15T03:49:09.827 INFO:tasks.rados.rados.0.smithi122.stdout:780: writing smithi12223906-13 from 1930727 to 2428499 tid 2 2024-09-15T03:49:09.830 INFO:tasks.rados.rados.0.smithi122.stdout:780: writing smithi12223906-13 from 3032909 to 3058907 tid 3 2024-09-15T03:49:09.830 INFO:tasks.rados.rados.0.smithi122.stdout:781: snap_remove snap 93 2024-09-15T03:49:10.799 INFO:tasks.rados.rados.0.smithi122.stdout:781: done (3 left) 2024-09-15T03:49:10.799 INFO:tasks.rados.rados.0.smithi122.stdout:782: snap_create 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:778: finishing write tid 2 to smithi12223906-1 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:778: finishing write tid 3 to smithi12223906-1 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:778: finishing write tid 4 to smithi12223906-1 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 242 (ObjNum 300 snap 95 seq_num 300) dirty exists 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:778: left oid 1 (ObjNum 300 snap 95 seq_num 300) 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:779: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:779: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:779: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 273 (ObjNum 245 snap 71 seq_num 245) dirty exists 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:780: finishing write tid 1 to smithi12223906-13 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:780: finishing write tid 2 to smithi12223906-13 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:780: finishing write tid 3 to smithi12223906-13 2024-09-15T03:49:10.800 INFO:tasks.rados.rados.0.smithi122.stdout:780: finishing write tid 4 to smithi12223906-13 2024-09-15T03:49:10.801 INFO:tasks.rados.rados.0.smithi122.stdout:780: finishing write tid 5 to smithi12223906-13 2024-09-15T03:49:10.801 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 238 (ObjNum 301 snap 95 seq_num 301) dirty exists 2024-09-15T03:49:10.801 INFO:tasks.rados.rados.0.smithi122.stdout:780: left oid 13 (ObjNum 301 snap 95 seq_num 301) 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:778: done (3 left) 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:779: done (2 left) 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:780: done (1 left) 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:782: done (0 left) 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:783: rollback oid 20 current snap is 96 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 85 2024-09-15T03:49:11.820 INFO:tasks.rados.rados.0.smithi122.stdout:784: write oid 45 current snap is 96 2024-09-15T03:49:11.821 INFO:tasks.rados.rados.0.smithi122.stdout:784: seq_num 302 ranges {675714=462965,1589209=641430,2943979=131735} 2024-09-15T03:49:11.828 INFO:tasks.rados.rados.0.smithi122.stdout:784: writing smithi12223906-45 from 675714 to 1138679 tid 1 2024-09-15T03:49:11.834 INFO:tasks.rados.rados.0.smithi122.stdout:784: writing smithi12223906-45 from 1589209 to 2230639 tid 2 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:784: writing smithi12223906-45 from 2943979 to 3075714 tid 3 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:785: read oid 44 snap 87 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:785: expect deleted 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:783: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:786: read oid 3 snap -1 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:786: expect deleted 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:787: write oid 22 current snap is 96 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:783: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:783: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:49:11.840 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 41 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:49:11.841 INFO:tasks.rados.rados.0.smithi122.stdout:784: finishing write tid 1 to smithi12223906-45 2024-09-15T03:49:11.841 INFO:tasks.rados.rados.0.smithi122.stdout:784: finishing write tid 2 to smithi12223906-45 2024-09-15T03:49:11.841 INFO:tasks.rados.rados.0.smithi122.stdout:787: seq_num 303 ranges {692521=538214,1931338=785088} 2024-09-15T03:49:11.846 INFO:tasks.rados.rados.0.smithi122.stdout:787: writing smithi12223906-22 from 692521 to 1230735 tid 1 2024-09-15T03:49:11.856 INFO:tasks.rados.rados.0.smithi122.stdout:787: writing smithi12223906-22 from 1931338 to 2716426 tid 2 2024-09-15T03:49:11.856 INFO:tasks.rados.rados.0.smithi122.stdout:783: done (4 left) 2024-09-15T03:49:11.856 INFO:tasks.rados.rados.0.smithi122.stdout:788: rollback oid 46 current snap is 96 2024-09-15T03:49:11.856 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 46 to 94 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:784: finishing write tid 3 to smithi12223906-45 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:784: finishing write tid 4 to smithi12223906-45 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:784: finishing write tid 5 to smithi12223906-45 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 242 (ObjNum 302 snap 96 seq_num 302) dirty exists 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:784: left oid 45 (ObjNum 302 snap 96 seq_num 302) 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:787: finishing write tid 1 to smithi12223906-22 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:784: done (4 left) 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:785: done (3 left) 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:786: done (2 left) 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:789: write (excl) oid 5 current snap is 96 2024-09-15T03:49:11.857 INFO:tasks.rados.rados.0.smithi122.stdout:789: seq_num 304 ranges {709328=613463,1989820=645099} 2024-09-15T03:49:11.862 INFO:tasks.rados.rados.0.smithi122.stdout:789: writing smithi12223906-5 from 709328 to 1322791 tid 1 2024-09-15T03:49:11.870 INFO:tasks.rados.rados.0.smithi122.stdout:789: writing smithi12223906-5 from 1989820 to 2634919 tid 2 2024-09-15T03:49:11.870 INFO:tasks.rados.rados.0.smithi122.stdout:790: rollback oid 44 current snap is 96 2024-09-15T03:49:11.870 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 84 2024-09-15T03:49:11.870 INFO:tasks.rados.rados.0.smithi122.stdout:788: finishing rollback tid 1 to smithi12223906-46 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 0 (ObjNum 33 snap 0 seq_num 1842153424) dirty dne 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:788: done (3 left) 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:791: write oid 38 current snap is 96 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:787: finishing write tid 2 to smithi12223906-22 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:787: finishing write tid 3 to smithi12223906-22 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:787: finishing write tid 4 to smithi12223906-22 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 367 (ObjNum 303 snap 96 seq_num 303) dirty exists 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:787: left oid 22 (ObjNum 303 snap 96 seq_num 303) 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:789: finishing write tid 1 to smithi12223906-5 2024-09-15T03:49:11.871 INFO:tasks.rados.rados.0.smithi122.stdout:791: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2024-09-15T03:49:11.876 INFO:tasks.rados.rados.0.smithi122.stdout:791: writing smithi12223906-38 from 726135 to 1131200 tid 1 2024-09-15T03:49:11.884 INFO:tasks.rados.rados.0.smithi122.stdout:791: writing smithi12223906-38 from 1764655 to 2553412 tid 2 2024-09-15T03:49:11.888 INFO:tasks.rados.rados.0.smithi122.stdout:791: writing smithi12223906-38 from 3026248 to 3126135 tid 3 2024-09-15T03:49:11.888 INFO:tasks.rados.rados.0.smithi122.stdout:787: done (3 left) 2024-09-15T03:49:11.888 INFO:tasks.rados.rados.0.smithi122.stdout:792: read oid 14 snap 82 2024-09-15T03:49:11.888 INFO:tasks.rados.rados.0.smithi122.stdout:792: expect (ObjNum 226 snap 62 seq_num 226) 2024-09-15T03:49:11.888 INFO:tasks.rados.rados.0.smithi122.stdout:793: snap_remove snap 90 2024-09-15T03:49:12.830 INFO:tasks.rados.rados.0.smithi122.stdout:793: done (4 left) 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:794: delete oid 15 current snap is 96 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:790: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:789: finishing write tid 2 to smithi12223906-5 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:789: finishing write tid 3 to smithi12223906-5 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:789: finishing write tid 4 to smithi12223906-5 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 180 (ObjNum 304 snap 96 seq_num 304) dirty exists 2024-09-15T03:49:12.831 INFO:tasks.rados.rados.0.smithi122.stdout:789: left oid 5 (ObjNum 304 snap 96 seq_num 304) 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:791: finishing write tid 1 to smithi12223906-38 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:791: finishing write tid 2 to smithi12223906-38 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:791: finishing write tid 3 to smithi12223906-38 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:791: finishing write tid 4 to smithi12223906-38 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:791: finishing write tid 5 to smithi12223906-38 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 212 (ObjNum 305 snap 96 seq_num 305) dirty exists 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:791: left oid 38 (ObjNum 305 snap 96 seq_num 305) 2024-09-15T03:49:12.832 INFO:tasks.rados.rados.0.smithi122.stdout:792: expect (ObjNum 226 snap 62 seq_num 226) 2024-09-15T03:49:13.047 INFO:tasks.rados.rados.0.smithi122.stdout:789: done (4 left) 2024-09-15T03:49:13.047 INFO:tasks.rados.rados.0.smithi122.stdout:790: done (3 left) 2024-09-15T03:49:13.047 INFO:tasks.rados.rados.0.smithi122.stdout:791: done (2 left) 2024-09-15T03:49:13.047 INFO:tasks.rados.rados.0.smithi122.stdout:792: done (1 left) 2024-09-15T03:49:13.048 INFO:tasks.rados.rados.0.smithi122.stdout:794: done (0 left) 2024-09-15T03:49:13.048 INFO:tasks.rados.rados.0.smithi122.stdout:795: write oid 29 current snap is 96 2024-09-15T03:49:13.048 INFO:tasks.rados.rados.0.smithi122.stdout:795: seq_num 307 ranges {759749=555563,1765266=792426,2964741=195008} 2024-09-15T03:49:13.060 INFO:tasks.rados.rados.0.smithi122.stdout:795: writing smithi12223906-29 from 759749 to 1315312 tid 1 2024-09-15T03:49:13.069 INFO:tasks.rados.rados.0.smithi122.stdout:795: writing smithi12223906-29 from 1765266 to 2557692 tid 2 2024-09-15T03:49:13.073 INFO:tasks.rados.rados.0.smithi122.stdout:795: writing smithi12223906-29 from 2964741 to 3159749 tid 3 2024-09-15T03:49:13.073 INFO:tasks.rados.rados.0.smithi122.stdout:796: write (excl) oid 28 current snap is 96 2024-09-15T03:49:13.073 INFO:tasks.rados.rados.0.smithi122.stdout:796: seq_num 308 ranges {776556=630812,1823748=652437,3108517=68039} 2024-09-15T03:49:13.081 INFO:tasks.rados.rados.0.smithi122.stdout:796: writing smithi12223906-28 from 776556 to 1407368 tid 1 2024-09-15T03:49:13.087 INFO:tasks.rados.rados.0.smithi122.stdout:796: writing smithi12223906-28 from 1823748 to 2476185 tid 2 2024-09-15T03:49:13.091 INFO:tasks.rados.rados.0.smithi122.stdout:796: writing smithi12223906-28 from 3108517 to 3176556 tid 3 2024-09-15T03:49:13.091 INFO:tasks.rados.rados.0.smithi122.stdout:797: read oid 13 snap -1 2024-09-15T03:49:13.091 INFO:tasks.rados.rados.0.smithi122.stdout:797: expect (ObjNum 301 snap 95 seq_num 301) 2024-09-15T03:49:13.092 INFO:tasks.rados.rados.0.smithi122.stdout:798: snap_remove snap 92 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:798: done (3 left) 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:799: read oid 18 snap -1 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:799: expect (ObjNum 263 snap 74 seq_num 263) 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:795: finishing write tid 1 to smithi12223906-29 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:795: finishing write tid 2 to smithi12223906-29 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:795: finishing write tid 3 to smithi12223906-29 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:795: finishing write tid 4 to smithi12223906-29 2024-09-15T03:49:13.845 INFO:tasks.rados.rados.0.smithi122.stdout:795: finishing write tid 5 to smithi12223906-29 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 141 (ObjNum 307 snap 96 seq_num 307) dirty exists 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:795: left oid 29 (ObjNum 307 snap 96 seq_num 307) 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:796: finishing write tid 1 to smithi12223906-28 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:796: finishing write tid 2 to smithi12223906-28 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:796: finishing write tid 3 to smithi12223906-28 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:796: finishing write tid 4 to smithi12223906-28 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:796: finishing write tid 5 to smithi12223906-28 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 145 (ObjNum 308 snap 96 seq_num 308) dirty exists 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:796: left oid 28 (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:13.846 INFO:tasks.rados.rados.0.smithi122.stdout:797: expect (ObjNum 301 snap 95 seq_num 301) 2024-09-15T03:49:13.996 INFO:tasks.rados.rados.0.smithi122.stdout:799: expect (ObjNum 263 snap 74 seq_num 263) 2024-09-15T03:49:14.136 INFO:tasks.rados.rados.0.smithi122.stdout:795: done (3 left) 2024-09-15T03:49:14.136 INFO:tasks.rados.rados.0.smithi122.stdout:796: done (2 left) 2024-09-15T03:49:14.136 INFO:tasks.rados.rados.0.smithi122.stdout:797: done (1 left) 2024-09-15T03:49:14.136 INFO:tasks.rados.rados.0.smithi122.stdout:799: done (0 left) 2024-09-15T03:49:14.136 INFO:tasks.rados.rados.0.smithi122.stdout:800: copy_from oid 5 from oid 34 current snap is 96 2024-09-15T03:49:14.136 INFO:tasks.rados.rados.0.smithi122.stdout:801: snap_remove snap 91 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:801: done (1 left) 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:802: read oid 37 snap -1 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:802: expect (ObjNum 133 snap 29 seq_num 133) 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:803: write (excl) oid 27 current snap is 96 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:800: finishing copy_from to smithi12223906-5 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:800: got expected ENOENT (src dne) 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:800: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:49:14.854 INFO:tasks.rados.rados.0.smithi122.stdout:803: seq_num 309 ranges {793363=706061,2282230=796095} 2024-09-15T03:49:14.864 INFO:tasks.rados.rados.0.smithi122.stdout:803: writing smithi12223906-27 from 793363 to 1499424 tid 1 2024-09-15T03:49:14.874 INFO:tasks.rados.rados.0.smithi122.stdout:803: writing smithi12223906-27 from 2282230 to 3078325 tid 2 2024-09-15T03:49:14.874 INFO:tasks.rados.rados.0.smithi122.stdout:800: done (2 left) 2024-09-15T03:49:14.874 INFO:tasks.rados.rados.0.smithi122.stdout:804: copy_from oid 33 from oid 35 current snap is 96 2024-09-15T03:49:14.875 INFO:tasks.rados.rados.0.smithi122.stdout:802: expect (ObjNum 133 snap 29 seq_num 133) 2024-09-15T03:49:14.985 INFO:tasks.rados.rados.0.smithi122.stdout:803: finishing write tid 1 to smithi12223906-27 2024-09-15T03:49:14.986 INFO:tasks.rados.rados.0.smithi122.stdout:802: done (2 left) 2024-09-15T03:49:14.986 INFO:tasks.rados.rados.0.smithi122.stdout:805: read oid 29 snap -1 2024-09-15T03:49:14.986 INFO:tasks.rados.rados.0.smithi122.stdout:805: expect (ObjNum 307 snap 96 seq_num 307) 2024-09-15T03:49:14.986 INFO:tasks.rados.rados.0.smithi122.stdout:806: write (excl) oid 18 current snap is 96 2024-09-15T03:49:14.986 INFO:tasks.rados.rados.0.smithi122.stdout:803: finishing write tid 2 to smithi12223906-27 2024-09-15T03:49:14.986 INFO:tasks.rados.rados.0.smithi122.stdout:806: seq_num 310 ranges {410170=781310,1940712=656106,3163363=46807} 2024-09-15T03:49:14.993 INFO:tasks.rados.rados.0.smithi122.stdout:806: writing smithi12223906-18 from 410170 to 1191480 tid 1 2024-09-15T03:49:15.001 INFO:tasks.rados.rados.0.smithi122.stdout:806: writing smithi12223906-18 from 1940712 to 2596818 tid 2 2024-09-15T03:49:15.005 INFO:tasks.rados.rados.0.smithi122.stdout:806: writing smithi12223906-18 from 3163363 to 3210170 tid 3 2024-09-15T03:49:15.005 INFO:tasks.rados.rados.0.smithi122.stdout:807: write (excl) oid 10 current snap is 96 2024-09-15T03:49:15.005 INFO:tasks.rados.rados.0.smithi122.stdout:807: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2024-09-15T03:49:15.010 INFO:tasks.rados.rados.0.smithi122.stdout:807: writing smithi12223906-10 from 426977 to 883536 tid 1 2024-09-15T03:49:15.017 INFO:tasks.rados.rados.0.smithi122.stdout:807: writing smithi12223906-10 from 1482841 to 2282605 tid 2 2024-09-15T03:49:15.023 INFO:tasks.rados.rados.0.smithi122.stdout:807: writing smithi12223906-10 from 2958080 to 3226977 tid 3 2024-09-15T03:49:15.023 INFO:tasks.rados.rados.0.smithi122.stdout:808: write (excl) oid 17 current snap is 96 2024-09-15T03:49:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:808: seq_num 312 ranges {443784=648161,1657676=659775,2818209=425575} 2024-09-15T03:49:15.029 INFO:tasks.rados.rados.0.smithi122.stdout:808: writing smithi12223906-17 from 443784 to 1091945 tid 1 2024-09-15T03:49:15.037 INFO:tasks.rados.rados.0.smithi122.stdout:808: writing smithi12223906-17 from 1657676 to 2317451 tid 2 2024-09-15T03:49:15.042 INFO:tasks.rados.rados.0.smithi122.stdout:808: writing smithi12223906-17 from 2818209 to 3243784 tid 3 2024-09-15T03:49:15.042 INFO:tasks.rados.rados.0.smithi122.stdout:809: snap_remove snap 82 2024-09-15T03:49:15.862 INFO:tasks.rados.rados.0.smithi122.stdout:809: done (6 left) 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:810: delete oid 1 current snap is 96 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:803: finishing write tid 3 to smithi12223906-27 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:803: finishing write tid 4 to smithi12223906-27 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 377 (ObjNum 309 snap 96 seq_num 309) dirty exists 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:803: left oid 27 (ObjNum 309 snap 96 seq_num 309) 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:804: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:804: finishing copy_from to smithi12223906-33 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 79 (ObjNum 250 snap 72 seq_num 250) dirty exists 2024-09-15T03:49:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:805: expect (ObjNum 307 snap 96 seq_num 307) 2024-09-15T03:49:16.003 INFO:tasks.rados.rados.0.smithi122.stdout:806: finishing write tid 1 to smithi12223906-18 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:806: finishing write tid 2 to smithi12223906-18 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:806: finishing write tid 3 to smithi12223906-18 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:806: finishing write tid 4 to smithi12223906-18 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:806: finishing write tid 5 to smithi12223906-18 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 383 (ObjNum 310 snap 96 seq_num 310) dirty exists 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:806: left oid 18 (ObjNum 310 snap 96 seq_num 310) 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:807: finishing write tid 1 to smithi12223906-10 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:807: finishing write tid 2 to smithi12223906-10 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:807: finishing write tid 3 to smithi12223906-10 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:807: finishing write tid 4 to smithi12223906-10 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:807: finishing write tid 5 to smithi12223906-10 2024-09-15T03:49:16.004 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 146 (ObjNum 311 snap 96 seq_num 311) dirty exists 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:807: left oid 10 (ObjNum 311 snap 96 seq_num 311) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:808: finishing write tid 1 to smithi12223906-17 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:808: finishing write tid 2 to smithi12223906-17 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:808: finishing write tid 3 to smithi12223906-17 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:803: done (6 left) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:804: done (5 left) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:805: done (4 left) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:806: done (3 left) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:807: done (2 left) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:810: done (1 left) 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:811: write oid 6 current snap is 96 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:808: finishing write tid 4 to smithi12223906-17 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:808: finishing write tid 5 to smithi12223906-17 2024-09-15T03:49:16.005 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 277 (ObjNum 312 snap 96 seq_num 312) dirty exists 2024-09-15T03:49:16.006 INFO:tasks.rados.rados.0.smithi122.stdout:808: left oid 17 (ObjNum 312 snap 96 seq_num 312) 2024-09-15T03:49:16.006 INFO:tasks.rados.rados.0.smithi122.stdout:811: seq_num 314 ranges {477398=798659,1774640=663444,2873055=404343} 2024-09-15T03:49:16.016 INFO:tasks.rados.rados.0.smithi122.stdout:811: writing smithi12223906-6 from 477398 to 1276057 tid 1 2024-09-15T03:49:16.024 INFO:tasks.rados.rados.0.smithi122.stdout:811: writing smithi12223906-6 from 1774640 to 2438084 tid 2 2024-09-15T03:49:16.029 INFO:tasks.rados.rados.0.smithi122.stdout:811: writing smithi12223906-6 from 2873055 to 3277398 tid 3 2024-09-15T03:49:16.029 INFO:tasks.rados.rados.0.smithi122.stdout:808: done (1 left) 2024-09-15T03:49:16.029 INFO:tasks.rados.rados.0.smithi122.stdout:812: write (excl) oid 35 current snap is 96 2024-09-15T03:49:16.029 INFO:tasks.rados.rados.0.smithi122.stdout:811: finishing write tid 1 to smithi12223906-6 2024-09-15T03:49:16.030 INFO:tasks.rados.rados.0.smithi122.stdout:812: seq_num 315 ranges {494205=473908,1433122=407102,2384125=462684} 2024-09-15T03:49:16.035 INFO:tasks.rados.rados.0.smithi122.stdout:812: writing smithi12223906-35 from 494205 to 968113 tid 1 2024-09-15T03:49:16.040 INFO:tasks.rados.rados.0.smithi122.stdout:812: writing smithi12223906-35 from 1433122 to 1840224 tid 2 2024-09-15T03:49:16.045 INFO:tasks.rados.rados.0.smithi122.stdout:812: writing smithi12223906-35 from 2384125 to 2846809 tid 3 2024-09-15T03:49:16.045 INFO:tasks.rados.rados.0.smithi122.stdout:813: snap_remove snap 89 2024-09-15T03:49:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:813: done (2 left) 2024-09-15T03:49:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:814: write oid 10 current snap is 96 2024-09-15T03:49:16.765 INFO:tasks.rados.rados.0.smithi122.stdout:811: finishing write tid 2 to smithi12223906-6 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:811: finishing write tid 3 to smithi12223906-6 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:811: finishing write tid 4 to smithi12223906-6 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:811: finishing write tid 5 to smithi12223906-6 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 225 (ObjNum 314 snap 96 seq_num 314) dirty exists 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:811: left oid 6 (ObjNum 314 snap 96 seq_num 314) 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:812: finishing write tid 1 to smithi12223906-35 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:812: finishing write tid 2 to smithi12223906-35 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:812: finishing write tid 3 to smithi12223906-35 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:812: finishing write tid 4 to smithi12223906-35 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:812: finishing write tid 5 to smithi12223906-35 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 83 (ObjNum 315 snap 96 seq_num 315) dirty exists 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:812: left oid 35 (ObjNum 315 snap 96 seq_num 315) 2024-09-15T03:49:16.766 INFO:tasks.rados.rados.0.smithi122.stdout:814: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2024-09-15T03:49:16.770 INFO:tasks.rados.rados.0.smithi122.stdout:814: writing smithi12223906-10 from 511012 to 1060169 tid 1 2024-09-15T03:49:16.779 INFO:tasks.rados.rados.0.smithi122.stdout:814: writing smithi12223906-10 from 1775251 to 2326011 tid 2 2024-09-15T03:49:16.786 INFO:tasks.rados.rados.0.smithi122.stdout:814: writing smithi12223906-10 from 3095195 to 3311012 tid 3 2024-09-15T03:49:16.786 INFO:tasks.rados.rados.0.smithi122.stdout:811: done (2 left) 2024-09-15T03:49:16.786 INFO:tasks.rados.rados.0.smithi122.stdout:812: done (1 left) 2024-09-15T03:49:16.786 INFO:tasks.rados.rados.0.smithi122.stdout:815: write (excl) oid 46 current snap is 96 2024-09-15T03:49:16.786 INFO:tasks.rados.rados.0.smithi122.stdout:815: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2024-09-15T03:49:16.792 INFO:tasks.rados.rados.0.smithi122.stdout:815: writing smithi12223906-46 from 527819 to 1152225 tid 1 2024-09-15T03:49:16.799 INFO:tasks.rados.rados.0.smithi122.stdout:815: writing smithi12223906-46 from 1833733 to 2244504 tid 2 2024-09-15T03:49:16.804 INFO:tasks.rados.rados.0.smithi122.stdout:815: writing smithi12223906-46 from 2722618 to 3324199 tid 3 2024-09-15T03:49:16.805 INFO:tasks.rados.rados.0.smithi122.stdout:816: write (excl) oid 38 current snap is 96 2024-09-15T03:49:16.805 INFO:tasks.rados.rados.0.smithi122.stdout:816: seq_num 318 ranges {544626=699655,1892215=554429,3150041=194585} 2024-09-15T03:49:16.812 INFO:tasks.rados.rados.0.smithi122.stdout:816: writing smithi12223906-38 from 544626 to 1244281 tid 1 2024-09-15T03:49:16.818 INFO:tasks.rados.rados.0.smithi122.stdout:816: writing smithi12223906-38 from 1892215 to 2446644 tid 2 2024-09-15T03:49:16.824 INFO:tasks.rados.rados.0.smithi122.stdout:816: writing smithi12223906-38 from 3150041 to 3344626 tid 3 2024-09-15T03:49:16.824 INFO:tasks.rados.rados.0.smithi122.stdout:817: write oid 34 current snap is 96 2024-09-15T03:49:16.824 INFO:tasks.rados.rados.0.smithi122.stdout:814: finishing write tid 1 to smithi12223906-10 2024-09-15T03:49:16.824 INFO:tasks.rados.rados.0.smithi122.stdout:817: seq_num 319 ranges {561433=774904,1950697=414440,2777464=583969} 2024-09-15T03:49:16.830 INFO:tasks.rados.rados.0.smithi122.stdout:817: writing smithi12223906-34 from 561433 to 1336337 tid 1 2024-09-15T03:49:16.836 INFO:tasks.rados.rados.0.smithi122.stdout:817: writing smithi12223906-34 from 1950697 to 2365137 tid 2 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:817: writing smithi12223906-34 from 2777464 to 3361433 tid 3 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:818: snap_create 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:814: finishing write tid 2 to smithi12223906-10 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:814: finishing write tid 3 to smithi12223906-10 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:814: finishing write tid 4 to smithi12223906-10 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:814: finishing write tid 5 to smithi12223906-10 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 154 (ObjNum 316 snap 96 seq_num 316) dirty exists 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:814: left oid 10 (ObjNum 316 snap 96 seq_num 316) 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:815: finishing write tid 1 to smithi12223906-46 2024-09-15T03:49:16.843 INFO:tasks.rados.rados.0.smithi122.stdout:815: finishing write tid 2 to smithi12223906-46 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:815: finishing write tid 3 to smithi12223906-46 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:815: finishing write tid 4 to smithi12223906-46 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:815: finishing write tid 5 to smithi12223906-46 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 246 (ObjNum 317 snap 96 seq_num 317) dirty exists 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:815: left oid 46 (ObjNum 317 snap 96 seq_num 317) 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:816: finishing write tid 1 to smithi12223906-38 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:816: finishing write tid 2 to smithi12223906-38 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:816: finishing write tid 3 to smithi12223906-38 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:816: finishing write tid 4 to smithi12223906-38 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:816: finishing write tid 5 to smithi12223906-38 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 230 (ObjNum 318 snap 96 seq_num 318) dirty exists 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:816: left oid 38 (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:49:16.844 INFO:tasks.rados.rados.0.smithi122.stdout:817: finishing write tid 1 to smithi12223906-34 2024-09-15T03:49:16.845 INFO:tasks.rados.rados.0.smithi122.stdout:817: finishing write tid 2 to smithi12223906-34 2024-09-15T03:49:16.847 INFO:tasks.rados.rados.0.smithi122.stdout:817: finishing write tid 3 to smithi12223906-34 2024-09-15T03:49:16.847 INFO:tasks.rados.rados.0.smithi122.stdout:817: finishing write tid 4 to smithi12223906-34 2024-09-15T03:49:16.847 INFO:tasks.rados.rados.0.smithi122.stdout:817: finishing write tid 5 to smithi12223906-34 2024-09-15T03:49:16.847 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 197 (ObjNum 319 snap 96 seq_num 319) dirty exists 2024-09-15T03:49:16.848 INFO:tasks.rados.rados.0.smithi122.stdout:817: left oid 34 (ObjNum 319 snap 96 seq_num 319) 2024-09-15T03:49:17.775 INFO:tasks.rados.rados.0.smithi122.stdout:814: done (4 left) 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:815: done (3 left) 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:816: done (2 left) 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:817: done (1 left) 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:818: done (0 left) 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:819: copy_from oid 5 from oid 15 current snap is 97 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:820: copy_from oid 28 from oid 42 current snap is 97 2024-09-15T03:49:17.776 INFO:tasks.rados.rados.0.smithi122.stdout:821: delete oid 7 current snap is 97 2024-09-15T03:49:17.779 INFO:tasks.rados.rados.0.smithi122.stdout:820: finishing copy_from to smithi12223906-28 2024-09-15T03:49:17.779 INFO:tasks.rados.rados.0.smithi122.stdout:820: got expected ENOENT (src dne) 2024-09-15T03:49:17.779 INFO:tasks.rados.rados.0.smithi122.stdout:819: finishing copy_from to smithi12223906-5 2024-09-15T03:49:17.779 INFO:tasks.rados.rados.0.smithi122.stdout:819: got expected ENOENT (src dne) 2024-09-15T03:49:17.780 INFO:tasks.rados.rados.0.smithi122.stdout:820: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:49:17.780 INFO:tasks.rados.rados.0.smithi122.stdout:819: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:49:17.781 INFO:tasks.rados.rados.0.smithi122.stdout:819: done (2 left) 2024-09-15T03:49:17.781 INFO:tasks.rados.rados.0.smithi122.stdout:820: done (1 left) 2024-09-15T03:49:17.781 INFO:tasks.rados.rados.0.smithi122.stdout:821: done (0 left) 2024-09-15T03:49:17.781 INFO:tasks.rados.rados.0.smithi122.stdout:822: rollback oid 3 current snap is 97 2024-09-15T03:49:17.781 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 94 2024-09-15T03:49:17.782 INFO:tasks.rados.rados.0.smithi122.stdout:823: copy_from oid 46 from oid 24 current snap is 97 2024-09-15T03:49:17.782 INFO:tasks.rados.rados.0.smithi122.stdout:824: write (excl) oid 14 current snap is 97 2024-09-15T03:49:17.782 INFO:tasks.rados.rados.0.smithi122.stdout:824: seq_num 321 ranges {595047=641755,1667661=418109,2548663=763022} 2024-09-15T03:49:17.789 INFO:tasks.rados.rados.0.smithi122.stdout:824: writing smithi12223906-14 from 595047 to 1236802 tid 1 2024-09-15T03:49:17.794 INFO:tasks.rados.rados.0.smithi122.stdout:824: writing smithi12223906-14 from 1667661 to 2085770 tid 2 2024-09-15T03:49:17.801 INFO:tasks.rados.rados.0.smithi122.stdout:824: writing smithi12223906-14 from 2548663 to 3311685 tid 3 2024-09-15T03:49:17.801 INFO:tasks.rados.rados.0.smithi122.stdout:825: snap_remove snap 84 2024-09-15T03:49:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:825: done (3 left) 2024-09-15T03:49:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:826: write (excl) oid 6 current snap is 97 2024-09-15T03:49:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:822: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:49:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:49:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:826: seq_num 322 ranges {611854=717004,2126143=561767,3259733=152121} 2024-09-15T03:49:18.788 INFO:tasks.rados.rados.0.smithi122.stdout:826: writing smithi12223906-6 from 611854 to 1328858 tid 1 2024-09-15T03:49:18.797 INFO:tasks.rados.rados.0.smithi122.stdout:826: writing smithi12223906-6 from 2126143 to 2687910 tid 2 2024-09-15T03:49:18.802 INFO:tasks.rados.rados.0.smithi122.stdout:826: writing smithi12223906-6 from 3259733 to 3411854 tid 3 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:822: done (3 left) 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:827: copy_from oid 18 from oid 45 current snap is 97 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:828: copy_from oid 9 from oid 17 current snap is 97 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:824: finishing write tid 1 to smithi12223906-14 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:823: finishing copy_from racing read to smithi12223906-46 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:824: finishing write tid 2 to smithi12223906-14 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:823: finishing copy_from to smithi12223906-46 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 331 (ObjNum 213 snap 58 seq_num 213) dirty exists 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:824: finishing write tid 3 to smithi12223906-14 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:824: finishing write tid 4 to smithi12223906-14 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:824: finishing write tid 5 to smithi12223906-14 2024-09-15T03:49:18.803 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 383 (ObjNum 321 snap 97 seq_num 321) dirty exists 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:824: left oid 14 (ObjNum 321 snap 97 seq_num 321) 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:823: done (4 left) 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:824: done (3 left) 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:829: rollback oid 46 current snap is 97 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 46 to 96 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:826: finishing write tid 1 to smithi12223906-6 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:826: finishing write tid 2 to smithi12223906-6 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:830: rollback oid 31 current snap is 97 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 31 to 94 2024-09-15T03:49:18.804 INFO:tasks.rados.rados.0.smithi122.stdout:831: snap_create 2024-09-15T03:49:18.805 INFO:tasks.rados.rados.0.smithi122.stdout:826: finishing write tid 3 to smithi12223906-6 2024-09-15T03:49:18.805 INFO:tasks.rados.rados.0.smithi122.stdout:830: finishing rollback tid 1 to smithi12223906-31 2024-09-15T03:49:18.805 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 181 (ObjNum 129 snap 0 seq_num 1842317104) dirty dne 2024-09-15T03:49:18.805 INFO:tasks.rados.rados.0.smithi122.stdout:826: finishing write tid 4 to smithi12223906-6 2024-09-15T03:49:18.805 INFO:tasks.rados.rados.0.smithi122.stdout:826: finishing write tid 5 to smithi12223906-6 2024-09-15T03:49:18.806 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 229 (ObjNum 322 snap 97 seq_num 322) dirty exists 2024-09-15T03:49:18.806 INFO:tasks.rados.rados.0.smithi122.stdout:826: left oid 6 (ObjNum 322 snap 97 seq_num 322) 2024-09-15T03:49:18.810 INFO:tasks.rados.rados.0.smithi122.stdout:829: finishing rollback tid 0 to smithi12223906-46 2024-09-15T03:49:18.810 INFO:tasks.rados.rados.0.smithi122.stdout:829: finishing rollback tid 1 to smithi12223906-46 2024-09-15T03:49:18.810 INFO:tasks.rados.rados.0.smithi122.stdout:829: finishing rollback tid 2 to smithi12223906-46 2024-09-15T03:49:18.810 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 334 (ObjNum 317 snap 96 seq_num 317) dirty exists 2024-09-15T03:49:18.818 INFO:tasks.rados.rados.0.smithi122.stdout:828: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:49:18.821 INFO:tasks.rados.rados.0.smithi122.stdout:827: finishing copy_from racing read to smithi12223906-18 2024-09-15T03:49:18.821 INFO:tasks.rados.rados.0.smithi122.stdout:828: finishing copy_from to smithi12223906-9 2024-09-15T03:49:18.821 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 277 (ObjNum 312 snap 96 seq_num 312) dirty exists 2024-09-15T03:49:18.822 INFO:tasks.rados.rados.0.smithi122.stdout:827: finishing copy_from to smithi12223906-18 2024-09-15T03:49:18.822 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 242 (ObjNum 302 snap 96 seq_num 302) dirty exists 2024-09-15T03:49:19.796 INFO:tasks.rados.rados.0.smithi122.stdout:826: done (5 left) 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:827: done (4 left) 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:828: done (3 left) 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:829: done (2 left) 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:830: done (1 left) 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:831: done (0 left) 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:832: copy_from oid 24 from oid 22 current snap is 98 2024-09-15T03:49:19.797 INFO:tasks.rados.rados.0.smithi122.stdout:833: delete oid 45 current snap is 98 2024-09-15T03:49:19.800 INFO:tasks.rados.rados.0.smithi122.stdout:833: done (1 left) 2024-09-15T03:49:19.800 INFO:tasks.rados.rados.0.smithi122.stdout:834: snap_remove snap 87 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:834: done (1 left) 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:835: read oid 29 snap -1 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:835: expect (ObjNum 307 snap 96 seq_num 307) 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:836: delete oid 23 current snap is 98 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:832: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:832: finishing copy_from to smithi12223906-24 2024-09-15T03:49:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 367 (ObjNum 303 snap 96 seq_num 303) dirty exists 2024-09-15T03:49:20.815 INFO:tasks.rados.rados.0.smithi122.stdout:835: expect (ObjNum 307 snap 96 seq_num 307) 2024-09-15T03:49:20.951 INFO:tasks.rados.rados.0.smithi122.stdout:832: done (2 left) 2024-09-15T03:49:20.951 INFO:tasks.rados.rados.0.smithi122.stdout:835: done (1 left) 2024-09-15T03:49:20.951 INFO:tasks.rados.rados.0.smithi122.stdout:836: done (0 left) 2024-09-15T03:49:20.951 INFO:tasks.rados.rados.0.smithi122.stdout:837: write (excl) oid 33 current snap is 98 2024-09-15T03:49:20.951 INFO:tasks.rados.rados.0.smithi122.stdout:837: seq_num 325 ranges {662275=542751,1785236=425447,2942002=520273} 2024-09-15T03:49:20.963 INFO:tasks.rados.rados.0.smithi122.stdout:837: writing smithi12223906-33 from 662275 to 1205026 tid 1 2024-09-15T03:49:20.970 INFO:tasks.rados.rados.0.smithi122.stdout:837: writing smithi12223906-33 from 1785236 to 2210683 tid 2 2024-09-15T03:49:20.977 INFO:tasks.rados.rados.0.smithi122.stdout:837: writing smithi12223906-33 from 2942002 to 3462275 tid 3 2024-09-15T03:49:20.977 INFO:tasks.rados.rados.0.smithi122.stdout:838: write oid 49 current snap is 98 2024-09-15T03:49:20.977 INFO:tasks.rados.rados.0.smithi122.stdout:838: seq_num 326 ranges {679082=618000,1843718=569105,2853072=626010} 2024-09-15T03:49:20.984 INFO:tasks.rados.rados.0.smithi122.stdout:838: writing smithi12223906-49 from 679082 to 1297082 tid 1 2024-09-15T03:49:20.991 INFO:tasks.rados.rados.0.smithi122.stdout:838: writing smithi12223906-49 from 1843718 to 2412823 tid 2 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:838: writing smithi12223906-49 from 2853072 to 3479082 tid 3 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:839: read oid 23 snap -1 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:839: expect deleted 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:840: write (excl) oid 1 current snap is 98 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:837: finishing write tid 1 to smithi12223906-33 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:837: finishing write tid 2 to smithi12223906-33 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:837: finishing write tid 3 to smithi12223906-33 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:837: finishing write tid 4 to smithi12223906-33 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:837: finishing write tid 5 to smithi12223906-33 2024-09-15T03:49:20.998 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 428 (ObjNum 325 snap 98 seq_num 325) dirty exists 2024-09-15T03:49:20.999 INFO:tasks.rados.rados.0.smithi122.stdout:837: left oid 33 (ObjNum 325 snap 98 seq_num 325) 2024-09-15T03:49:20.999 INFO:tasks.rados.rados.0.smithi122.stdout:838: finishing write tid 1 to smithi12223906-49 2024-09-15T03:49:20.999 INFO:tasks.rados.rados.0.smithi122.stdout:840: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2024-09-15T03:49:21.003 INFO:tasks.rados.rados.0.smithi122.stdout:840: writing smithi12223906-1 from 695889 to 1105491 tid 1 2024-09-15T03:49:21.010 INFO:tasks.rados.rados.0.smithi122.stdout:840: writing smithi12223906-1 from 1618553 to 2331316 tid 2 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:840: writing smithi12223906-1 from 2996848 to 3495889 tid 3 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:837: done (3 left) 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:841: read oid 28 snap -1 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:841: expect (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:842: read oid 35 snap -1 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:842: expect (ObjNum 315 snap 96 seq_num 315) 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:843: write oid 22 current snap is 98 2024-09-15T03:49:21.018 INFO:tasks.rados.rados.0.smithi122.stdout:838: finishing write tid 2 to smithi12223906-49 2024-09-15T03:49:21.019 INFO:tasks.rados.rados.0.smithi122.stdout:843: seq_num 328 ranges {712696=484851,1960682=572774,3307918=204778} 2024-09-15T03:49:21.024 INFO:tasks.rados.rados.0.smithi122.stdout:843: writing smithi12223906-22 from 712696 to 1197547 tid 1 2024-09-15T03:49:21.033 INFO:tasks.rados.rados.0.smithi122.stdout:843: writing smithi12223906-22 from 1960682 to 2533456 tid 2 2024-09-15T03:49:21.039 INFO:tasks.rados.rados.0.smithi122.stdout:843: writing smithi12223906-22 from 3307918 to 3512696 tid 3 2024-09-15T03:49:21.039 INFO:tasks.rados.rados.0.smithi122.stdout:844: snap_remove snap 95 2024-09-15T03:49:21.767 INFO:tasks.rados.rados.0.smithi122.stdout:844: done (6 left) 2024-09-15T03:49:21.767 INFO:tasks.rados.rados.0.smithi122.stdout:845: write (excl) oid 33 current snap is 98 2024-09-15T03:49:21.767 INFO:tasks.rados.rados.0.smithi122.stdout:845: seq_num 329 ranges {729503=560100,2019164=716432,3335341=194162} 2024-09-15T03:49:21.775 INFO:tasks.rados.rados.0.smithi122.stdout:845: writing smithi12223906-33 from 729503 to 1289603 tid 1 2024-09-15T03:49:21.784 INFO:tasks.rados.rados.0.smithi122.stdout:845: writing smithi12223906-33 from 2019164 to 2735596 tid 2 2024-09-15T03:49:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:845: writing smithi12223906-33 from 3335341 to 3529503 tid 3 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:846: snap_create 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:838: finishing write tid 3 to smithi12223906-49 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:838: finishing write tid 4 to smithi12223906-49 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:838: finishing write tid 5 to smithi12223906-49 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 281 (ObjNum 326 snap 98 seq_num 326) dirty exists 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:838: left oid 49 (ObjNum 326 snap 98 seq_num 326) 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:840: finishing write tid 1 to smithi12223906-1 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:840: finishing write tid 2 to smithi12223906-1 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:840: finishing write tid 3 to smithi12223906-1 2024-09-15T03:49:21.789 INFO:tasks.rados.rados.0.smithi122.stdout:841: expect (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:21.976 INFO:tasks.rados.rados.0.smithi122.stdout:840: finishing write tid 4 to smithi12223906-1 2024-09-15T03:49:21.977 INFO:tasks.rados.rados.0.smithi122.stdout:840: finishing write tid 5 to smithi12223906-1 2024-09-15T03:49:21.977 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 288 (ObjNum 327 snap 98 seq_num 327) dirty exists 2024-09-15T03:49:21.977 INFO:tasks.rados.rados.0.smithi122.stdout:840: left oid 1 (ObjNum 327 snap 98 seq_num 327) 2024-09-15T03:49:21.977 INFO:tasks.rados.rados.0.smithi122.stdout:843: finishing write tid 1 to smithi12223906-22 2024-09-15T03:49:21.977 INFO:tasks.rados.rados.0.smithi122.stdout:842: expect (ObjNum 315 snap 96 seq_num 315) 2024-09-15T03:49:22.062 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T03:49:22.062 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T03:49:22.062 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T03:49:22.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T03:49:22.065 INFO:tasks.ceph.mon.c:Started 2024-09-15T03:49:22.065 INFO:tasks.mon_thrash:Sending STOP to mon f 2024-09-15T03:49:22.065 INFO:tasks.ceph.mon.f:Sent signal 19 2024-09-15T03:49:22.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:49:22.144 INFO:tasks.rados.rados.0.smithi122.stdout:843: finishing write tid 2 to smithi12223906-22 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:843: finishing write tid 3 to smithi12223906-22 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:843: finishing write tid 4 to smithi12223906-22 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:843: finishing write tid 5 to smithi12223906-22 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 433 (ObjNum 328 snap 98 seq_num 328) dirty exists 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:843: left oid 22 (ObjNum 328 snap 98 seq_num 328) 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:845: finishing write tid 1 to smithi12223906-33 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:845: finishing write tid 2 to smithi12223906-33 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:845: finishing write tid 3 to smithi12223906-33 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:845: finishing write tid 4 to smithi12223906-33 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:845: finishing write tid 5 to smithi12223906-33 2024-09-15T03:49:22.145 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 445 (ObjNum 329 snap 98 seq_num 329) dirty exists 2024-09-15T03:49:22.146 INFO:tasks.rados.rados.0.smithi122.stdout:845: left oid 33 (ObjNum 329 snap 98 seq_num 329) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:838: done (7 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:839: done (6 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:840: done (5 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:841: done (4 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:842: done (3 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:843: done (2 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:845: done (1 left) 2024-09-15T03:49:27.284 INFO:tasks.rados.rados.0.smithi122.stdout:846: done (0 left) 2024-09-15T03:49:27.285 INFO:tasks.rados.rados.0.smithi122.stdout:847: rollback oid 26 current snap is 99 2024-09-15T03:49:27.285 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 85 2024-09-15T03:49:27.285 INFO:tasks.rados.rados.0.smithi122.stdout:848: delete oid 29 current snap is 99 2024-09-15T03:49:27.295 INFO:tasks.rados.rados.0.smithi122.stdout:847: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:49:27.297 INFO:tasks.rados.rados.0.smithi122.stdout:847: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:49:27.300 INFO:tasks.rados.rados.0.smithi122.stdout:848: done (1 left) 2024-09-15T03:49:27.300 INFO:tasks.rados.rados.0.smithi122.stdout:849: write oid 16 current snap is 99 2024-09-15T03:49:27.300 INFO:tasks.rados.rados.0.smithi122.stdout:849: seq_num 331 ranges {763117=710598,2136128=720101,3390187=172930} 2024-09-15T03:49:27.312 INFO:tasks.rados.rados.0.smithi122.stdout:849: writing smithi12223906-16 from 763117 to 1473715 tid 1 2024-09-15T03:49:27.322 INFO:tasks.rados.rados.0.smithi122.stdout:849: writing smithi12223906-16 from 2136128 to 2856229 tid 2 2024-09-15T03:49:27.326 INFO:tasks.rados.rados.0.smithi122.stdout:849: writing smithi12223906-16 from 3390187 to 3563117 tid 3 2024-09-15T03:49:27.326 INFO:tasks.rados.rados.0.smithi122.stdout:850: copy_from oid 25 from oid 41 current snap is 99 2024-09-15T03:49:27.326 INFO:tasks.rados.rados.0.smithi122.stdout:847: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:49:27.326 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 244 (ObjNum 283 snap 83 seq_num 283) dirty exists 2024-09-15T03:49:27.326 INFO:tasks.rados.rados.0.smithi122.stdout:849: finishing write tid 1 to smithi12223906-16 2024-09-15T03:49:27.327 INFO:tasks.rados.rados.0.smithi122.stdout:847: done (2 left) 2024-09-15T03:49:27.327 INFO:tasks.rados.rados.0.smithi122.stdout:851: rollback oid 36 current snap is 99 2024-09-15T03:49:27.327 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 36 to 94 2024-09-15T03:49:27.327 INFO:tasks.rados.rados.0.smithi122.stdout:852: delete oid 35 current snap is 99 2024-09-15T03:49:27.329 INFO:tasks.rados.rados.0.smithi122.stdout:849: finishing write tid 2 to smithi12223906-16 2024-09-15T03:49:27.333 INFO:tasks.rados.rados.0.smithi122.stdout:849: finishing write tid 3 to smithi12223906-16 2024-09-15T03:49:27.340 INFO:tasks.rados.rados.0.smithi122.stdout:852: done (3 left) 2024-09-15T03:49:27.340 INFO:tasks.rados.rados.0.smithi122.stdout:853: delete oid 17 current snap is 99 2024-09-15T03:49:27.342 INFO:tasks.rados.rados.0.smithi122.stdout:849: finishing write tid 4 to smithi12223906-16 2024-09-15T03:49:27.343 INFO:tasks.rados.rados.0.smithi122.stdout:850: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:49:27.343 INFO:tasks.rados.rados.0.smithi122.stdout:849: finishing write tid 5 to smithi12223906-16 2024-09-15T03:49:27.343 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 297 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:49:27.343 INFO:tasks.rados.rados.0.smithi122.stdout:849: left oid 16 (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:49:27.343 INFO:tasks.rados.rados.0.smithi122.stdout:851: finishing rollback tid 0 to smithi12223906-36 2024-09-15T03:49:27.343 INFO:tasks.rados.rados.0.smithi122.stdout:851: finishing rollback tid 1 to smithi12223906-36 2024-09-15T03:49:27.344 INFO:tasks.rados.rados.0.smithi122.stdout:850: finishing copy_from to smithi12223906-25 2024-09-15T03:49:27.344 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 110 (ObjNum 191 snap 53 seq_num 191) dirty exists 2024-09-15T03:49:27.344 INFO:tasks.rados.rados.0.smithi122.stdout:851: finishing rollback tid 2 to smithi12223906-36 2024-09-15T03:49:27.345 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 400 (ObjNum 281 snap 82 seq_num 281) dirty exists 2024-09-15T03:49:27.347 INFO:tasks.rados.rados.0.smithi122.stdout:849: done (3 left) 2024-09-15T03:49:27.348 INFO:tasks.rados.rados.0.smithi122.stdout:850: done (2 left) 2024-09-15T03:49:27.348 INFO:tasks.rados.rados.0.smithi122.stdout:851: done (1 left) 2024-09-15T03:49:27.348 INFO:tasks.rados.rados.0.smithi122.stdout:853: done (0 left) 2024-09-15T03:49:27.348 INFO:tasks.rados.rados.0.smithi122.stdout:854: snap_create 2024-09-15T03:49:28.295 INFO:tasks.rados.rados.0.smithi122.stdout:854: done (0 left) 2024-09-15T03:49:28.296 INFO:tasks.rados.rados.0.smithi122.stdout:855: read oid 10 snap -1 2024-09-15T03:49:28.296 INFO:tasks.rados.rados.0.smithi122.stdout:855: expect (ObjNum 316 snap 96 seq_num 316) 2024-09-15T03:49:28.296 INFO:tasks.rados.rados.0.smithi122.stdout:856: snap_remove snap 99 2024-09-15T03:49:29.308 INFO:tasks.rados.rados.0.smithi122.stdout:856: done (1 left) 2024-09-15T03:49:29.308 INFO:tasks.rados.rados.0.smithi122.stdout:857: copy_from oid 50 from oid 9 current snap is 100 2024-09-15T03:49:29.308 INFO:tasks.rados.rados.0.smithi122.stdout:855: expect (ObjNum 316 snap 96 seq_num 316) 2024-09-15T03:49:29.506 INFO:tasks.rados.rados.0.smithi122.stdout:857: finishing copy_from racing read to smithi12223906-50 2024-09-15T03:49:29.507 INFO:tasks.rados.rados.0.smithi122.stdout:857: finishing copy_from to smithi12223906-50 2024-09-15T03:49:29.507 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 277 (ObjNum 312 snap 96 seq_num 312) dirty exists 2024-09-15T03:49:29.507 INFO:tasks.rados.rados.0.smithi122.stdout:855: done (1 left) 2024-09-15T03:49:29.507 INFO:tasks.rados.rados.0.smithi122.stdout:857: done (0 left) 2024-09-15T03:49:29.507 INFO:tasks.rados.rados.0.smithi122.stdout:858: write oid 42 current snap is 100 2024-09-15T03:49:29.507 INFO:tasks.rados.rados.0.smithi122.stdout:858: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2024-09-15T03:49:29.519 INFO:tasks.rados.rados.0.smithi122.stdout:858: writing smithi12223906-42 from 413538 to 949883 tid 1 2024-09-15T03:49:29.525 INFO:tasks.rados.rados.0.smithi122.stdout:858: writing smithi12223906-42 from 1395221 to 1979002 tid 2 2024-09-15T03:49:29.532 INFO:tasks.rados.rados.0.smithi122.stdout:858: writing smithi12223906-42 from 2556103 to 3214720 tid 3 2024-09-15T03:49:29.532 INFO:tasks.rados.rados.0.smithi122.stdout:859: write (excl) oid 3 current snap is 100 2024-09-15T03:49:29.532 INFO:tasks.rados.rados.0.smithi122.stdout:859: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2024-09-15T03:49:29.539 INFO:tasks.rados.rados.0.smithi122.stdout:859: writing smithi12223906-3 from 430345 to 1158292 tid 1 2024-09-15T03:49:29.545 INFO:tasks.rados.rados.0.smithi122.stdout:859: writing smithi12223906-3 from 1570056 to 2297495 tid 2 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:859: writing smithi12223906-3 from 2699879 to 3169768 tid 3 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:860: rollback oid 30 current snap is 100 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 30 to 98 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: finishing write tid 1 to smithi12223906-42 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: finishing write tid 2 to smithi12223906-42 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: finishing write tid 3 to smithi12223906-42 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: finishing write tid 4 to smithi12223906-42 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: finishing write tid 5 to smithi12223906-42 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 404 (ObjNum 334 snap 100 seq_num 334) dirty exists 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: left oid 42 (ObjNum 334 snap 100 seq_num 334) 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:859: finishing write tid 1 to smithi12223906-3 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:859: finishing write tid 2 to smithi12223906-3 2024-09-15T03:49:29.551 INFO:tasks.rados.rados.0.smithi122.stdout:858: done (2 left) 2024-09-15T03:49:29.552 INFO:tasks.rados.rados.0.smithi122.stdout:861: delete oid 20 current snap is 100 2024-09-15T03:49:29.556 INFO:tasks.rados.rados.0.smithi122.stdout:861: done (2 left) 2024-09-15T03:49:29.556 INFO:tasks.rados.rados.0.smithi122.stdout:862: read oid 4 snap -1 2024-09-15T03:49:29.556 INFO:tasks.rados.rados.0.smithi122.stdout:862: expect (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:49:29.556 INFO:tasks.rados.rados.0.smithi122.stdout:863: snap_remove snap 94 2024-09-15T03:49:30.313 INFO:tasks.rados.rados.0.smithi122.stdout:863: done (3 left) 2024-09-15T03:49:30.313 INFO:tasks.rados.rados.0.smithi122.stdout:864: delete oid 35 current snap is 100 2024-09-15T03:49:30.315 INFO:tasks.rados.rados.0.smithi122.stdout:860: finishing rollback tid 0 to smithi12223906-30 2024-09-15T03:49:30.315 INFO:tasks.rados.rados.0.smithi122.stdout:860: finishing rollback tid 1 to smithi12223906-30 2024-09-15T03:49:30.315 INFO:tasks.rados.rados.0.smithi122.stdout:860: finishing rollback tid 2 to smithi12223906-30 2024-09-15T03:49:30.315 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 146 (ObjNum 279 snap 82 seq_num 279) dirty exists 2024-09-15T03:49:30.316 INFO:tasks.rados.rados.0.smithi122.stdout:859: finishing write tid 3 to smithi12223906-3 2024-09-15T03:49:30.316 INFO:tasks.rados.rados.0.smithi122.stdout:859: finishing write tid 4 to smithi12223906-3 2024-09-15T03:49:30.316 INFO:tasks.rados.rados.0.smithi122.stdout:859: finishing write tid 5 to smithi12223906-3 2024-09-15T03:49:30.316 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 408 (ObjNum 335 snap 100 seq_num 335) dirty exists 2024-09-15T03:49:30.316 INFO:tasks.rados.rados.0.smithi122.stdout:859: left oid 3 (ObjNum 335 snap 100 seq_num 335) 2024-09-15T03:49:30.316 INFO:tasks.rados.rados.0.smithi122.stdout:862: expect (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:49:30.622 INFO:tasks.rados.rados.0.smithi122.stdout:859: done (3 left) 2024-09-15T03:49:30.622 INFO:tasks.rados.rados.0.smithi122.stdout:860: done (2 left) 2024-09-15T03:49:30.623 INFO:tasks.rados.rados.0.smithi122.stdout:862: done (1 left) 2024-09-15T03:49:30.623 INFO:tasks.rados.rados.0.smithi122.stdout:864: done (0 left) 2024-09-15T03:49:30.623 INFO:tasks.rados.rados.0.smithi122.stdout:865: write oid 20 current snap is 100 2024-09-15T03:49:30.623 INFO:tasks.rados.rados.0.smithi122.stdout:865: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2024-09-15T03:49:30.628 INFO:tasks.rados.rados.0.smithi122.stdout:865: writing smithi12223906-20 from 480766 to 1034460 tid 1 2024-09-15T03:49:30.635 INFO:tasks.rados.rados.0.smithi122.stdout:865: writing smithi12223906-20 from 1629149 to 2220268 tid 2 2024-09-15T03:49:30.641 INFO:tasks.rados.rados.0.smithi122.stdout:865: writing smithi12223906-20 from 2782148 to 3202206 tid 3 2024-09-15T03:49:30.641 INFO:tasks.rados.rados.0.smithi122.stdout:866: rollback oid 4 current snap is 100 2024-09-15T03:49:30.641 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 98 2024-09-15T03:49:30.641 INFO:tasks.rados.rados.0.smithi122.stdout:865: finishing write tid 1 to smithi12223906-20 2024-09-15T03:49:30.641 INFO:tasks.rados.rados.0.smithi122.stdout:865: finishing write tid 2 to smithi12223906-20 2024-09-15T03:49:30.641 INFO:tasks.rados.rados.0.smithi122.stdout:867: snap_remove snap 85 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:867: done (2 left) 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:868: snap_create 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:865: finishing write tid 3 to smithi12223906-20 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:865: finishing write tid 4 to smithi12223906-20 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:865: finishing write tid 5 to smithi12223906-20 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 65 (ObjNum 338 snap 100 seq_num 338) dirty exists 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:865: left oid 20 (ObjNum 338 snap 100 seq_num 338) 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:866: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:49:31.338 INFO:tasks.rados.rados.0.smithi122.stdout:866: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:49:31.339 INFO:tasks.rados.rados.0.smithi122.stdout:866: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:49:31.339 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 184 (ObjNum 236 snap 65 seq_num 236) dirty exists 2024-09-15T03:49:31.767 INFO:tasks.rados.rados.0.smithi122.stdout:865: done (2 left) 2024-09-15T03:49:31.767 INFO:tasks.rados.rados.0.smithi122.stdout:866: done (1 left) 2024-09-15T03:49:31.767 INFO:tasks.rados.rados.0.smithi122.stdout:868: done (0 left) 2024-09-15T03:49:31.767 INFO:tasks.rados.rados.0.smithi122.stdout:869: read oid 15 snap -1 2024-09-15T03:49:31.767 INFO:tasks.rados.rados.0.smithi122.stdout:869: expect deleted 2024-09-15T03:49:31.767 INFO:tasks.rados.rados.0.smithi122.stdout:870: read oid 28 snap -1 2024-09-15T03:49:31.768 INFO:tasks.rados.rados.0.smithi122.stdout:870: expect (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:31.768 INFO:tasks.rados.rados.0.smithi122.stdout:871: delete oid 49 current snap is 101 2024-09-15T03:49:31.773 INFO:tasks.rados.rados.0.smithi122.stdout:870: expect (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:31.944 INFO:tasks.rados.rados.0.smithi122.stdout:869: done (2 left) 2024-09-15T03:49:31.945 INFO:tasks.rados.rados.0.smithi122.stdout:870: done (1 left) 2024-09-15T03:49:31.945 INFO:tasks.rados.rados.0.smithi122.stdout:871: done (0 left) 2024-09-15T03:49:31.945 INFO:tasks.rados.rados.0.smithi122.stdout:872: snap_remove snap 97 2024-09-15T03:49:32.780 INFO:tasks.rados.rados.0.smithi122.stdout:872: done (0 left) 2024-09-15T03:49:32.780 INFO:tasks.rados.rados.0.smithi122.stdout:873: read oid 42 snap -1 2024-09-15T03:49:32.780 INFO:tasks.rados.rados.0.smithi122.stdout:873: expect (ObjNum 334 snap 100 seq_num 334) 2024-09-15T03:49:32.780 INFO:tasks.rados.rados.0.smithi122.stdout:874: read oid 5 snap 98 2024-09-15T03:49:32.780 INFO:tasks.rados.rados.0.smithi122.stdout:874: expect (ObjNum 304 snap 96 seq_num 304) 2024-09-15T03:49:32.780 INFO:tasks.rados.rados.0.smithi122.stdout:875: write (excl) oid 22 current snap is 101 2024-09-15T03:49:32.781 INFO:tasks.rados.rados.0.smithi122.stdout:875: seq_num 340 ranges {514380=704192,1629760=478435,2604288=442602} 2024-09-15T03:49:32.791 INFO:tasks.rados.rados.0.smithi122.stdout:875: writing smithi12223906-22 from 514380 to 1218572 tid 1 2024-09-15T03:49:32.797 INFO:tasks.rados.rados.0.smithi122.stdout:875: writing smithi12223906-22 from 1629760 to 2108195 tid 2 2024-09-15T03:49:32.803 INFO:tasks.rados.rados.0.smithi122.stdout:875: writing smithi12223906-22 from 2604288 to 3046890 tid 3 2024-09-15T03:49:32.803 INFO:tasks.rados.rados.0.smithi122.stdout:876: read oid 24 snap -1 2024-09-15T03:49:32.803 INFO:tasks.rados.rados.0.smithi122.stdout:876: expect (ObjNum 303 snap 96 seq_num 303) 2024-09-15T03:49:32.803 INFO:tasks.rados.rados.0.smithi122.stdout:877: write oid 48 current snap is 101 2024-09-15T03:49:32.803 INFO:tasks.rados.rados.0.smithi122.stdout:877: seq_num 341 ranges {531187=779441,2088242=738446,3431711=299476} 2024-09-15T03:49:32.810 INFO:tasks.rados.rados.0.smithi122.stdout:877: writing smithi12223906-48 from 531187 to 1310628 tid 1 2024-09-15T03:49:32.818 INFO:tasks.rados.rados.0.smithi122.stdout:877: writing smithi12223906-48 from 2088242 to 2826688 tid 2 2024-09-15T03:49:32.823 INFO:tasks.rados.rados.0.smithi122.stdout:877: writing smithi12223906-48 from 3431711 to 3731187 tid 3 2024-09-15T03:49:32.824 INFO:tasks.rados.rados.0.smithi122.stdout:878: rollback oid 15 current snap is 101 2024-09-15T03:49:32.824 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 96 2024-09-15T03:49:32.824 INFO:tasks.rados.rados.0.smithi122.stdout:874: expect (ObjNum 304 snap 96 seq_num 304) 2024-09-15T03:49:32.967 INFO:tasks.rados.rados.0.smithi122.stdout:875: finishing write tid 1 to smithi12223906-22 2024-09-15T03:49:32.967 INFO:tasks.rados.rados.0.smithi122.stdout:875: finishing write tid 2 to smithi12223906-22 2024-09-15T03:49:32.967 INFO:tasks.rados.rados.0.smithi122.stdout:873: expect (ObjNum 334 snap 100 seq_num 334) 2024-09-15T03:49:33.082 INFO:tasks.rados.rados.0.smithi122.stdout:875: finishing write tid 3 to smithi12223906-22 2024-09-15T03:49:33.083 INFO:tasks.rados.rados.0.smithi122.stdout:875: finishing write tid 4 to smithi12223906-22 2024-09-15T03:49:33.083 INFO:tasks.rados.rados.0.smithi122.stdout:875: finishing write tid 5 to smithi12223906-22 2024-09-15T03:49:33.083 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 475 (ObjNum 340 snap 101 seq_num 340) dirty exists 2024-09-15T03:49:33.083 INFO:tasks.rados.rados.0.smithi122.stdout:875: left oid 22 (ObjNum 340 snap 101 seq_num 340) 2024-09-15T03:49:33.083 INFO:tasks.rados.rados.0.smithi122.stdout:877: finishing write tid 1 to smithi12223906-48 2024-09-15T03:49:33.083 INFO:tasks.rados.rados.0.smithi122.stdout:876: expect (ObjNum 303 snap 96 seq_num 303) 2024-09-15T03:49:33.264 INFO:tasks.rados.rados.0.smithi122.stdout:877: finishing write tid 2 to smithi12223906-48 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:878: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 0 (ObjNum 65 snap 0 seq_num 1953066355) dirty dne 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:877: finishing write tid 3 to smithi12223906-48 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:877: finishing write tid 4 to smithi12223906-48 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:877: finishing write tid 5 to smithi12223906-48 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 169 (ObjNum 341 snap 101 seq_num 341) dirty exists 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:877: left oid 48 (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:873: done (5 left) 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:874: done (4 left) 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:875: done (3 left) 2024-09-15T03:49:33.265 INFO:tasks.rados.rados.0.smithi122.stdout:876: done (2 left) 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:877: done (1 left) 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:878: done (0 left) 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:879: rollback oid 10 current snap is 101 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 96 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:880: read oid 45 snap -1 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:880: expect deleted 2024-09-15T03:49:33.266 INFO:tasks.rados.rados.0.smithi122.stdout:881: snap_create 2024-09-15T03:49:33.276 INFO:tasks.rados.rados.0.smithi122.stdout:879: finishing rollback tid 0 to smithi12223906-10 2024-09-15T03:49:33.277 INFO:tasks.rados.rados.0.smithi122.stdout:879: finishing rollback tid 1 to smithi12223906-10 2024-09-15T03:49:33.277 INFO:tasks.rados.rados.0.smithi122.stdout:879: finishing rollback tid 2 to smithi12223906-10 2024-09-15T03:49:33.277 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 173 (ObjNum 316 snap 96 seq_num 316) dirty exists 2024-09-15T03:49:33.787 INFO:tasks.rados.rados.0.smithi122.stdout:879: done (2 left) 2024-09-15T03:49:33.787 INFO:tasks.rados.rados.0.smithi122.stdout:880: done (1 left) 2024-09-15T03:49:33.787 INFO:tasks.rados.rados.0.smithi122.stdout:881: done (0 left) 2024-09-15T03:49:33.787 INFO:tasks.rados.rados.0.smithi122.stdout:882: write (excl) oid 23 current snap is 102 2024-09-15T03:49:33.787 INFO:tasks.rados.rados.0.smithi122.stdout:882: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2024-09-15T03:49:33.797 INFO:tasks.rados.rados.0.smithi122.stdout:882: writing smithi12223906-23 from 547994 to 1002684 tid 1 2024-09-15T03:49:33.804 INFO:tasks.rados.rados.0.smithi122.stdout:882: writing smithi12223906-23 from 1746724 to 2228828 tid 2 2024-09-15T03:49:33.809 INFO:tasks.rados.rados.0.smithi122.stdout:882: writing smithi12223906-23 from 2659134 to 3124280 tid 3 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:882: writing smithi12223906-23 from 3605976 to 3747994 tid 4 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:883: copy_from oid 12 from oid 32 current snap is 102 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:882: finishing write tid 1 to smithi12223906-23 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:882: finishing write tid 2 to smithi12223906-23 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:884: rollback oid 43 current snap is 102 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 98 2024-09-15T03:49:33.813 INFO:tasks.rados.rados.0.smithi122.stdout:885: write oid 31 current snap is 102 2024-09-15T03:49:33.814 INFO:tasks.rados.rados.0.smithi122.stdout:885: seq_num 343 ranges {564801=646292,1921559=742115,3202910=561891} 2024-09-15T03:49:33.819 INFO:tasks.rados.rados.0.smithi122.stdout:885: writing smithi12223906-31 from 564801 to 1211093 tid 1 2024-09-15T03:49:33.828 INFO:tasks.rados.rados.0.smithi122.stdout:885: writing smithi12223906-31 from 1921559 to 2663674 tid 2 2024-09-15T03:49:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:885: writing smithi12223906-31 from 3202910 to 3764801 tid 3 2024-09-15T03:49:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:886: read oid 46 snap -1 2024-09-15T03:49:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:886: expect (ObjNum 317 snap 96 seq_num 317) 2024-09-15T03:49:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:887: read oid 28 snap -1 2024-09-15T03:49:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:887: expect (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:888: snap_remove snap 96 2024-09-15T03:49:34.792 INFO:tasks.rados.rados.0.smithi122.stdout:888: done (6 left) 2024-09-15T03:49:34.792 INFO:tasks.rados.rados.0.smithi122.stdout:889: read oid 13 snap -1 2024-09-15T03:49:34.792 INFO:tasks.rados.rados.0.smithi122.stdout:889: expect (ObjNum 301 snap 95 seq_num 301) 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:890: delete oid 5 current snap is 102 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:882: finishing write tid 3 to smithi12223906-23 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:882: finishing write tid 4 to smithi12223906-23 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:884: finishing rollback tid 0 to smithi12223906-43 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:882: finishing write tid 5 to smithi12223906-23 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:882: finishing write tid 6 to smithi12223906-23 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 480 (ObjNum 342 snap 102 seq_num 342) dirty exists 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:882: left oid 23 (ObjNum 342 snap 102 seq_num 342) 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:884: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:884: finishing rollback tid 2 to smithi12223906-43 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 428 (ObjNum 285 snap 84 seq_num 285) dirty exists 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:885: finishing write tid 1 to smithi12223906-31 2024-09-15T03:49:34.793 INFO:tasks.rados.rados.0.smithi122.stdout:883: finishing copy_from racing read to smithi12223906-12 2024-09-15T03:49:34.794 INFO:tasks.rados.rados.0.smithi122.stdout:883: finishing copy_from to smithi12223906-12 2024-09-15T03:49:34.794 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 335 (ObjNum 277 snap 81 seq_num 277) dirty exists 2024-09-15T03:49:34.794 INFO:tasks.rados.rados.0.smithi122.stdout:885: finishing write tid 2 to smithi12223906-31 2024-09-15T03:49:34.794 INFO:tasks.rados.rados.0.smithi122.stdout:887: expect (ObjNum 308 snap 96 seq_num 308) 2024-09-15T03:49:34.985 INFO:tasks.rados.rados.0.smithi122.stdout:885: finishing write tid 3 to smithi12223906-31 2024-09-15T03:49:34.985 INFO:tasks.rados.rados.0.smithi122.stdout:885: finishing write tid 4 to smithi12223906-31 2024-09-15T03:49:34.985 INFO:tasks.rados.rados.0.smithi122.stdout:885: finishing write tid 5 to smithi12223906-31 2024-09-15T03:49:34.985 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 188 (ObjNum 343 snap 102 seq_num 343) dirty exists 2024-09-15T03:49:34.985 INFO:tasks.rados.rados.0.smithi122.stdout:885: left oid 31 (ObjNum 343 snap 102 seq_num 343) 2024-09-15T03:49:34.985 INFO:tasks.rados.rados.0.smithi122.stdout:882: done (7 left) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:883: done (6 left) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:884: done (5 left) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:885: done (4 left) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:887: done (3 left) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:890: done (2 left) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:891: read oid 22 snap -1 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:891: expect (ObjNum 340 snap 101 seq_num 340) 2024-09-15T03:49:34.986 INFO:tasks.rados.rados.0.smithi122.stdout:892: read oid 20 snap 100 2024-09-15T03:49:34.987 INFO:tasks.rados.rados.0.smithi122.stdout:892: expect (ObjNum 338 snap 100 seq_num 338) 2024-09-15T03:49:34.987 INFO:tasks.rados.rados.0.smithi122.stdout:893: delete oid 45 current snap is 102 2024-09-15T03:49:34.987 INFO:tasks.rados.rados.0.smithi122.stdout:886: expect (ObjNum 317 snap 96 seq_num 317) 2024-09-15T03:49:35.091 INFO:tasks.rados.rados.0.smithi122.stdout:889: expect (ObjNum 301 snap 95 seq_num 301) 2024-09-15T03:49:35.229 INFO:tasks.rados.rados.0.smithi122.stdout:892: expect (ObjNum 338 snap 100 seq_num 338) 2024-09-15T03:49:35.339 INFO:tasks.rados.rados.0.smithi122.stdout:891: expect (ObjNum 340 snap 101 seq_num 340) 2024-09-15T03:49:35.604 INFO:tasks.rados.rados.0.smithi122.stdout:886: done (4 left) 2024-09-15T03:49:35.604 INFO:tasks.rados.rados.0.smithi122.stdout:889: done (3 left) 2024-09-15T03:49:35.604 INFO:tasks.rados.rados.0.smithi122.stdout:891: done (2 left) 2024-09-15T03:49:35.604 INFO:tasks.rados.rados.0.smithi122.stdout:892: done (1 left) 2024-09-15T03:49:35.605 INFO:tasks.rados.rados.0.smithi122.stdout:893: done (0 left) 2024-09-15T03:49:35.605 INFO:tasks.rados.rados.0.smithi122.stdout:894: read oid 45 snap -1 2024-09-15T03:49:35.605 INFO:tasks.rados.rados.0.smithi122.stdout:894: expect deleted 2024-09-15T03:49:35.605 INFO:tasks.rados.rados.0.smithi122.stdout:895: delete oid 34 current snap is 102 2024-09-15T03:49:35.610 INFO:tasks.rados.rados.0.smithi122.stdout:894: done (1 left) 2024-09-15T03:49:35.611 INFO:tasks.rados.rados.0.smithi122.stdout:895: done (0 left) 2024-09-15T03:49:35.611 INFO:tasks.rados.rados.0.smithi122.stdout:896: snap_remove snap 100 2024-09-15T03:49:35.819 INFO:tasks.rados.rados.0.smithi122.stdout:896: done (0 left) 2024-09-15T03:49:35.819 INFO:tasks.rados.rados.0.smithi122.stdout:897: rollback oid 4 current snap is 102 2024-09-15T03:49:35.819 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 98 2024-09-15T03:49:35.819 INFO:tasks.rados.rados.0.smithi122.stdout:898: write (excl) oid 7 current snap is 102 2024-09-15T03:49:35.819 INFO:tasks.rados.rados.0.smithi122.stdout:898: seq_num 347 ranges {632029=547288,1639134=749453,2796249=437859,3653524=178505} 2024-09-15T03:49:35.830 INFO:tasks.rados.rados.0.smithi122.stdout:898: writing smithi12223906-7 from 632029 to 1179317 tid 1 2024-09-15T03:49:35.837 INFO:tasks.rados.rados.0.smithi122.stdout:898: writing smithi12223906-7 from 1639134 to 2388587 tid 2 2024-09-15T03:49:35.842 INFO:tasks.rados.rados.0.smithi122.stdout:898: writing smithi12223906-7 from 2796249 to 3234108 tid 3 2024-09-15T03:49:35.845 INFO:tasks.rados.rados.0.smithi122.stdout:898: writing smithi12223906-7 from 3653524 to 3832029 tid 4 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:897: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:899: read oid 16 snap -1 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:899: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:900: delete oid 25 current snap is 102 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:897: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:897: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 191 (ObjNum 236 snap 65 seq_num 236) dirty exists 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:898: finishing write tid 1 to smithi12223906-7 2024-09-15T03:49:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:898: finishing write tid 2 to smithi12223906-7 2024-09-15T03:49:35.848 INFO:tasks.rados.rados.0.smithi122.stdout:898: finishing write tid 3 to smithi12223906-7 2024-09-15T03:49:35.850 INFO:tasks.rados.rados.0.smithi122.stdout:898: finishing write tid 4 to smithi12223906-7 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:897: done (3 left) 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:900: done (2 left) 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:901: copy_from oid 29 from oid 48 current snap is 102 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:902: read oid 4 snap -1 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:902: expect (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:903: read oid 10 snap -1 2024-09-15T03:49:35.851 INFO:tasks.rados.rados.0.smithi122.stdout:903: expect (ObjNum 316 snap 96 seq_num 316) 2024-09-15T03:49:35.852 INFO:tasks.rados.rados.0.smithi122.stdout:904: write oid 14 current snap is 102 2024-09-15T03:49:35.852 INFO:tasks.rados.rados.0.smithi122.stdout:904: seq_num 349 ranges {665643=697786,2039745=753122,3534742=330901} 2024-09-15T03:49:35.859 INFO:tasks.rados.rados.0.smithi122.stdout:904: writing smithi12223906-14 from 665643 to 1363429 tid 1 2024-09-15T03:49:35.868 INFO:tasks.rados.rados.0.smithi122.stdout:904: writing smithi12223906-14 from 2039745 to 2792867 tid 2 2024-09-15T03:49:35.874 INFO:tasks.rados.rados.0.smithi122.stdout:904: writing smithi12223906-14 from 3534742 to 3865643 tid 3 2024-09-15T03:49:35.875 INFO:tasks.rados.rados.0.smithi122.stdout:905: delete oid 36 current snap is 102 2024-09-15T03:49:35.875 INFO:tasks.rados.rados.0.smithi122.stdout:899: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:49:36.053 INFO:tasks.rados.rados.0.smithi122.stdout:898: finishing write tid 5 to smithi12223906-7 2024-09-15T03:49:36.053 INFO:tasks.rados.rados.0.smithi122.stdout:898: finishing write tid 6 to smithi12223906-7 2024-09-15T03:49:36.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 197 (ObjNum 347 snap 102 seq_num 347) dirty exists 2024-09-15T03:49:36.053 INFO:tasks.rados.rados.0.smithi122.stdout:898: left oid 7 (ObjNum 347 snap 102 seq_num 347) 2024-09-15T03:49:36.053 INFO:tasks.rados.rados.0.smithi122.stdout:903: expect (ObjNum 316 snap 96 seq_num 316) 2024-09-15T03:49:36.245 INFO:tasks.rados.rados.0.smithi122.stdout:904: finishing write tid 1 to smithi12223906-14 2024-09-15T03:49:36.245 INFO:tasks.rados.rados.0.smithi122.stdout:902: expect (ObjNum 236 snap 65 seq_num 236) 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:901: finishing copy_from racing read to smithi12223906-29 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:904: finishing write tid 2 to smithi12223906-14 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:901: finishing copy_from to smithi12223906-29 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 169 (ObjNum 341 snap 101 seq_num 341) dirty exists 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:904: finishing write tid 3 to smithi12223906-14 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:904: finishing write tid 4 to smithi12223906-14 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:904: finishing write tid 5 to smithi12223906-14 2024-09-15T03:49:36.527 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 445 (ObjNum 349 snap 102 seq_num 349) dirty exists 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:904: left oid 14 (ObjNum 349 snap 102 seq_num 349) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:898: done (6 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:899: done (5 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:901: done (4 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:902: done (3 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:903: done (2 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:904: done (1 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:905: done (0 left) 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:906: write oid 24 current snap is 102 2024-09-15T03:49:36.528 INFO:tasks.rados.rados.0.smithi122.stdout:906: seq_num 351 ranges {699257=564637,1873062=640438,3305941=593316} 2024-09-15T03:49:36.544 INFO:tasks.rados.rados.0.smithi122.stdout:906: writing smithi12223906-24 from 699257 to 1263894 tid 1 2024-09-15T03:49:36.551 INFO:tasks.rados.rados.0.smithi122.stdout:906: writing smithi12223906-24 from 1873062 to 2513500 tid 2 2024-09-15T03:49:36.559 INFO:tasks.rados.rados.0.smithi122.stdout:906: writing smithi12223906-24 from 3305941 to 3899257 tid 3 2024-09-15T03:49:36.559 INFO:tasks.rados.rados.0.smithi122.stdout:907: read oid 22 snap -1 2024-09-15T03:49:36.559 INFO:tasks.rados.rados.0.smithi122.stdout:907: expect (ObjNum 340 snap 101 seq_num 340) 2024-09-15T03:49:36.559 INFO:tasks.rados.rados.0.smithi122.stdout:906: finishing write tid 1 to smithi12223906-24 2024-09-15T03:49:36.560 INFO:tasks.rados.rados.0.smithi122.stdout:908: copy_from oid 36 from oid 14 current snap is 102 2024-09-15T03:49:36.560 INFO:tasks.rados.rados.0.smithi122.stdout:906: finishing write tid 2 to smithi12223906-24 2024-09-15T03:49:36.560 INFO:tasks.rados.rados.0.smithi122.stdout:909: delete oid 8 current snap is 102 2024-09-15T03:49:36.563 INFO:tasks.rados.rados.0.smithi122.stdout:909: done (3 left) 2024-09-15T03:49:36.563 INFO:tasks.rados.rados.0.smithi122.stdout:910: copy_from oid 33 from oid 16 current snap is 102 2024-09-15T03:49:36.563 INFO:tasks.rados.rados.0.smithi122.stdout:911: read oid 19 snap -1 2024-09-15T03:49:36.564 INFO:tasks.rados.rados.0.smithi122.stdout:911: expect (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:49:36.564 INFO:tasks.rados.rados.0.smithi122.stdout:912: snap_remove snap 101 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:912: done (5 left) 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:913: copy_from oid 40 from oid 15 current snap is 102 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:906: finishing write tid 3 to smithi12223906-24 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:906: finishing write tid 4 to smithi12223906-24 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:906: finishing write tid 5 to smithi12223906-24 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 452 (ObjNum 351 snap 102 seq_num 351) dirty exists 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:906: left oid 24 (ObjNum 351 snap 102 seq_num 351) 2024-09-15T03:49:36.770 INFO:tasks.rados.rados.0.smithi122.stdout:911: expect (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:49:37.054 INFO:tasks.rados.rados.0.smithi122.stdout:908: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:49:37.054 INFO:tasks.rados.rados.0.smithi122.stdout:907: expect (ObjNum 340 snap 101 seq_num 340) 2024-09-15T03:49:37.067 INFO:tasks.mon_thrash:Sending CONT to mon f 2024-09-15T03:49:37.067 INFO:tasks.ceph.mon.f:Sent signal 18 2024-09-15T03:49:37.067 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:49:37.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:49:37.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: get_monmap_and_config 2024-09-15T03:49:37.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: monmap: 2024-09-15T03:49:37.181 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.181 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.181 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:37.180509+0000 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:37.180509+0000 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.182 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 1 Processor -- start 2024-09-15T03:49:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 1 -- start start 2024-09-15T03:49:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: init 2024-09-15T03:49:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: _add_conns ranks=[4,2,7,6,0,1,8,3,5] 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient(hunting): picked mon.noname-c con 0x7f32bc0a6a80 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient(hunting): picked mon.noname-b con 0x7f32bc09fb80 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient(hunting): picked mon.noname-i con 0x7f32bc138d30 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc04ca10 con 0x7f32bc09fb80 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc0a0f50 con 0x7f32bc0a6a80 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc133680 con 0x7f32bc138d30 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient(hunting): authenticate will time out at 1121.610380s 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32bb7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f32bc09fb80 0x7f32bc09ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:50212/0 (socket says 172.21.15.26:50212) 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32bb7fe700 1 -- 172.21.15.26:0/2985213039 learned_addr learned my addr 172.21.15.26:0/2985213039 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330558398 0 0) 0x7f32bc04ca10 con 0x7f32bc09fb80 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): my global_id is 134372 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4003570 con 0x7f32bc09fb80 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741679574 0 0) 0x7f32bc0a0f50 con 0x7f32bc0a6a80 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): my global_id is 174109 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc04ca10 con 0x7f32bc0a6a80 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735258454 0 0) 0x7f32bc133680 con 0x7f32bc138d30 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient(hunting): my global_id is 134314 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc0a0f50 con 0x7f32bc138d30 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1463709082 0 0) 0x7f32a4003570 con 0x7f32bc09fb80 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 >> v1:172.21.15.122:6792/0 conn(0x7f32bc138d30 legacy=0x7f32bc13b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 >> v1:172.21.15.26:6791/0 conn(0x7f32bc0a6a80 legacy=0x7f32bc132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 monclient: found mon.noname-b 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0a6e50 con 0x7f32bc09fb80 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32b00017b0 con 0x7f32bc09fb80 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: dump: 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.189 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.190 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 5 monclient: authenticate success, global_id 134372 2024-09-15T03:49:37.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32b0001a90 con 0x7f32bc09fb80 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 1 -- 172.21.15.26:0/2985213039 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32b0004c30 con 0x7f32bc09fb80 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32baffd700 10 monclient: dump: 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.192 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.193 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.179+0000 7f32c14bc700 4 set_mon_vals no callback set 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 10 monclient: shutdown 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 1 -- 172.21.15.26:0/2985213039 >> v1:172.21.15.26:6790/0 conn(0x7f32bc09fb80 legacy=0x7f32bc09ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 1 -- 172.21.15.26:0/2985213039 shutdown_connections 2024-09-15T03:49:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 1 -- 172.21.15.26:0/2985213039 >> 172.21.15.26:0/2985213039 conn(0x7f32bc06e380 msgr2=0x7f32bc0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 1 -- 172.21.15.26:0/2985213039 shutdown_connections 2024-09-15T03:49:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.183+0000 7f32c14bc700 1 -- 172.21.15.26:0/2985213039 wait complete. 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: monmap: 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 Processor -- start 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 -- start start 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: init 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: _add_conns ranks=[0,3,5,7,8,1,2,4,6] 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient(hunting): picked mon.noname-a con 0x7f32bc09fb80 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient(hunting): picked mon.noname-g con 0x7f32bc0a6a80 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient(hunting): picked mon.noname-h con 0x7f32bc138d30 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc09cfc0 con 0x7f32bc09fb80 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc11cba0 con 0x7f32bc0a6a80 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc11c8c0 con 0x7f32bc138d30 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32bbfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f32bc09fb80 0x7f32bc135b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47246/0 (socket says 172.21.15.26:47246) 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32bbfff700 1 -- 172.21.15.26:0/4005426448 learned_addr learned my addr 172.21.15.26:0/4005426448 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient(hunting): authenticate will time out at 1121.618481s 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395527883 0 0) 0x7f32bc09cfc0 con 0x7f32bc09fb80 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): my global_id is 4725 2024-09-15T03:49:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a8003570 con 0x7f32bc09fb80 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739856982 0 0) 0x7f32bc11cba0 con 0x7f32bc0a6a80 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): my global_id is 4782 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc09cfc0 con 0x7f32bc0a6a80 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829841888 0 0) 0x7f32bc11c8c0 con 0x7f32bc138d30 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): my global_id is 4901 2024-09-15T03:49:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc11cba0 con 0x7f32bc138d30 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2474007332 0 0) 0x7f32a8003570 con 0x7f32bc09fb80 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32bc11c8c0 con 0x7f32bc09fb80 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32ac002b30 con 0x7f32bc09fb80 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): dump: 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 263264466 0 0) 0x7f32bc09cfc0 con 0x7f32bc0a6a80 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32a8003570 con 0x7f32bc0a6a80 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32b0004b70 con 0x7f32bc0a6a80 2024-09-15T03:49:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): dump: 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2381191151 0 0) 0x7f32bc11cba0 con 0x7f32bc138d30 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32bc09cfc0 con 0x7f32bc138d30 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32b4001d40 con 0x7f32bc138d30 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient(hunting): dump: 2024-09-15T03:49:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1031388789 0 0) 0x7f32bc11c8c0 con 0x7f32bc09fb80 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 >> v1:172.21.15.122:6791/0 conn(0x7f32bc138d30 legacy=0x7f32bc14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 >> v1:172.21.15.122:6790/0 conn(0x7f32bc0a6a80 legacy=0x7f32bc136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 monclient: found mon.a 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc137360 con 0x7f32bc09fb80 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32a3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 5 monclient: authenticate success, global_id 4725 2024-09-15T03:49:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: _renew_subs 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc132920 con 0x7f32bc09fb80 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: _renew_subs 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.187+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc1380d0 con 0x7f32bc09fb80 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32ac0015e0 con 0x7f32bc09fb80 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32ac001b10 con 0x7f32bc09fb80 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 10 monclient: dump: 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.211 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.212 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32ba7fc700 4 set_mon_vals no callback set 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f32ac001cf0 con 0x7f32bc09fb80 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(225..225 src has 1..225) v4 ==== 4019+0+0 (unknown 3302849129 0 0) 0x7f32ac043f40 con 0x7f32bc09fb80 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32c14bc700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32c14bc700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32c14bc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c0050b0 con 0x7f32bc09fb80 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.191+0000 7f32bb7fe700 10 monclient: get_auth_request con 0x7f32a8037fb0 auth_method 0 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.195+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f32ac014dd0 con 0x7f32bc09fb80 2024-09-15T03:49:37.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.195+0000 7f32a3fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.195+0000 7f32a3fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:37.331 INFO:tasks.rados.rados.0.smithi122.stdout:908: finishing copy_from to smithi12223906-36 2024-09-15T03:49:37.331 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 445 (ObjNum 349 snap 102 seq_num 349) dirty exists 2024-09-15T03:49:37.331 INFO:tasks.rados.rados.0.smithi122.stdout:910: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:910: finishing copy_from to smithi12223906-33 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 297 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:913: finishing copy_from to smithi12223906-40 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:913: got expected ENOENT (src dne) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:913: finishing copy_from racing read to smithi12223906-40 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:906: done (5 left) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:907: done (4 left) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:908: done (3 left) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:910: done (2 left) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:911: done (1 left) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:913: done (0 left) 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:914: rollback oid 20 current snap is 102 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 98 2024-09-15T03:49:37.332 INFO:tasks.rados.rados.0.smithi122.stdout:915: read oid 17 snap -1 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:915: expect deleted 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:916: rollback oid 42 current snap is 102 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 98 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:917: rollback oid 44 current snap is 102 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 98 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:918: write oid 39 current snap is 102 2024-09-15T03:49:37.333 INFO:tasks.rados.rados.0.smithi122.stdout:918: seq_num 353 ranges {732871=715135,1873673=644107,3244434=688437} 2024-09-15T03:49:37.346 INFO:tasks.rados.rados.0.smithi122.stdout:918: writing smithi12223906-39 from 732871 to 1448006 tid 1 2024-09-15T03:49:37.353 INFO:tasks.rados.rados.0.smithi122.stdout:918: writing smithi12223906-39 from 1873673 to 2517780 tid 2 2024-09-15T03:49:37.361 INFO:tasks.rados.rados.0.smithi122.stdout:918: writing smithi12223906-39 from 3244434 to 3932871 tid 3 2024-09-15T03:49:37.361 INFO:tasks.rados.rados.0.smithi122.stdout:919: read oid 14 snap -1 2024-09-15T03:49:37.361 INFO:tasks.rados.rados.0.smithi122.stdout:919: expect (ObjNum 349 snap 102 seq_num 349) 2024-09-15T03:49:37.362 INFO:tasks.rados.rados.0.smithi122.stdout:920: write oid 13 current snap is 102 2024-09-15T03:49:37.362 INFO:tasks.rados.rados.0.smithi122.stdout:920: seq_num 354 ranges {749678=790384,2332155=504118,3271857=549469} 2024-09-15T03:49:37.370 INFO:tasks.rados.rados.0.smithi122.stdout:920: writing smithi12223906-13 from 749678 to 1540062 tid 1 2024-09-15T03:49:37.377 INFO:tasks.rados.rados.0.smithi122.stdout:920: writing smithi12223906-13 from 2332155 to 2836273 tid 2 2024-09-15T03:49:37.383 INFO:tasks.rados.rados.0.smithi122.stdout:920: writing smithi12223906-13 from 3271857 to 3821326 tid 3 2024-09-15T03:49:37.383 INFO:tasks.rados.rados.0.smithi122.stdout:921: rollback oid 30 current snap is 102 2024-09-15T03:49:37.383 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 30 to 98 2024-09-15T03:49:37.383 INFO:tasks.rados.rados.0.smithi122.stdout:917: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:49:37.383 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:49:37.383 INFO:tasks.rados.rados.0.smithi122.stdout:916: finishing rollback tid 0 to smithi12223906-42 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:916: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 465 (ObjNum 129 snap 0 seq_num 1842588656) dirty dne 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:914: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:914: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:914: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 74 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:918: finishing write tid 1 to smithi12223906-39 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:918: finishing write tid 2 to smithi12223906-39 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:918: finishing write tid 3 to smithi12223906-39 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:918: finishing write tid 4 to smithi12223906-39 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:918: finishing write tid 5 to smithi12223906-39 2024-09-15T03:49:37.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 470 (ObjNum 353 snap 102 seq_num 353) dirty exists 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:918: left oid 39 (ObjNum 353 snap 102 seq_num 353) 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:914: done (7 left) 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:915: done (6 left) 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:916: done (5 left) 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:917: done (4 left) 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:918: done (3 left) 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:922: write (excl) oid 1 current snap is 102 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:920: finishing write tid 1 to smithi12223906-13 2024-09-15T03:49:37.385 INFO:tasks.rados.rados.0.smithi122.stdout:922: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2024-09-15T03:49:37.390 INFO:tasks.rados.rados.0.smithi122.stdout:922: writing smithi12223906-1 from 766485 to 1232118 tid 1 2024-09-15T03:49:37.398 INFO:tasks.rados.rados.0.smithi122.stdout:922: writing smithi12223906-1 from 1990637 to 2638413 tid 2 2024-09-15T03:49:37.406 INFO:tasks.rados.rados.0.smithi122.stdout:922: writing smithi12223906-1 from 3299280 to 3966485 tid 3 2024-09-15T03:49:37.406 INFO:tasks.rados.rados.0.smithi122.stdout:923: write (excl) oid 49 current snap is 102 2024-09-15T03:49:37.406 INFO:tasks.rados.rados.0.smithi122.stdout:919: expect (ObjNum 349 snap 102 seq_num 349) 2024-09-15T03:49:37.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32c14bc700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:49:37.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32c14bc700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:49:37.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32c14bc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f329c0058e0 con 0x7f32bc09fb80 2024-09-15T03:49:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32a3fff700 1 -- 172.21.15.26:0/4005426448 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 1524661151) 0x7f32ac0149b0 con 0x7f32bc09fb80 2024-09-15T03:49:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32a3fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:49:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.431+0000 7f32a3fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:37.435 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":44,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:49:37.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 >> v1:172.21.15.122:6812/20895 conn(0x7f32a8037fb0 legacy=0x7f32a803a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 10 monclient: shutdown 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 >> v1:172.21.15.26:6789/0 conn(0x7f32bc09fb80 legacy=0x7f32bc135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 shutdown_connections 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 >> 172.21.15.26:0/4005426448 conn(0x7f32bc06e380 msgr2=0x7f32bc09de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 shutdown_connections 2024-09-15T03:49:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.435+0000 7f32c14bc700 1 -- 172.21.15.26:0/4005426448 wait complete. 2024-09-15T03:49:37.452 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:49:37.452 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:49:37.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.551+0000 7f34e6d28700 10 monclient: get_monmap_and_config 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.551+0000 7f34e6d28700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.551+0000 7f34e6d28700 10 monclient: monmap: 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:37.555369+0000 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:37.555369+0000 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.557 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 1 Processor -- start 2024-09-15T03:49:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 1 -- start start 2024-09-15T03:49:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient: init 2024-09-15T03:49:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient: _add_conns ranks=[8,6,7,1,3,5,0,4,2] 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient(hunting): picked mon.noname-e con 0x7f34e0064350 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient(hunting): picked mon.noname-d con 0x7f34e0132d20 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient(hunting): picked mon.noname-i con 0x7f34e01395b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e004ca10 con 0x7f34e0132d20 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e00a1e40 con 0x7f34e01395b0 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e0133150 con 0x7f34e0064350 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient(hunting): authenticate will time out at 1121.985437s 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e4ac4700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f34e0064350 0x7f34e006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56692/0 (socket says 172.21.15.26:56692) 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e4ac4700 1 -- 172.21.15.26:0/2212748323 learned_addr learned my addr 172.21.15.26:0/2212748323 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873730599 0 0) 0x7f34e0133150 con 0x7f34e0064350 2024-09-15T03:49:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): my global_id is 134351 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c8003570 con 0x7f34e0064350 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729197684 0 0) 0x7f34e004ca10 con 0x7f34e0132d20 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): my global_id is 4875 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e0133150 con 0x7f34e0132d20 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739069415 0 0) 0x7f34e00a1e40 con 0x7f34e01395b0 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient(hunting): my global_id is 134323 2024-09-15T03:49:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e004ca10 con 0x7f34e01395b0 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3444339615 0 0) 0x7f34c8003570 con 0x7f34e0064350 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 >> v1:172.21.15.122:6792/0 conn(0x7f34e01395b0 legacy=0x7f34e013b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 >> v1:172.21.15.26:6792/0 conn(0x7f34e0132d20 legacy=0x7f34e0137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 monclient: found mon.noname-e 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00a2120 con 0x7f34e0064350 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d0001b30 con 0x7f34e0064350 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: dump: 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 5 monclient: authenticate success, global_id 134351 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34d0001e10 con 0x7f34e0064350 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 1 -- 172.21.15.26:0/2212748323 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d0002000 con 0x7f34e0064350 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 4 set_mon_vals no callback set 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34df7fe700 10 monclient: dump: 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 10 monclient: shutdown 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.555+0000 7f34e6d28700 1 -- 172.21.15.26:0/2212748323 >> v1:172.21.15.26:6793/0 conn(0x7f34e0064350 legacy=0x7f34e006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- 172.21.15.26:0/2212748323 shutdown_connections 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- 172.21.15.26:0/2212748323 >> 172.21.15.26:0/2212748323 conn(0x7f34e009f3d0 msgr2=0x7f34e009f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- 172.21.15.26:0/2212748323 shutdown_connections 2024-09-15T03:49:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- 172.21.15.26:0/2212748323 wait complete. 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient: monmap: 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 Processor -- start 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- start start 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient: init 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient: _add_conns ranks=[6,7,8,2,1,5,4,0,3] 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient(hunting): picked mon.noname-d con 0x7f34e0064350 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient(hunting): picked mon.noname-i con 0x7f34e0132d20 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient(hunting): picked mon.noname-e con 0x7f34e01395b0 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e0136320 con 0x7f34e0064350 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e012d260 con 0x7f34e0132d20 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e4ac4700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f34e0064350 0x7f34e0133fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34540/0 (socket says 172.21.15.26:34540) 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e4ac4700 1 -- 172.21.15.26:0/2727558694 learned_addr learned my addr 172.21.15.26:0/2727558694 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e0136040 con 0x7f34e01395b0 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.559+0000 7f34e6d28700 10 monclient(hunting): authenticate will time out at 1121.991753s 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082031016 0 0) 0x7f34e0136320 con 0x7f34e0064350 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): my global_id is 4884 2024-09-15T03:49:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34cc0035a0 con 0x7f34e0064350 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39582630 0 0) 0x7f34e0136040 con 0x7f34e01395b0 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): my global_id is 134360 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e0136320 con 0x7f34e01395b0 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853125436 0 0) 0x7f34e012d260 con 0x7f34e0132d20 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): my global_id is 134332 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e0136040 con 0x7f34e0132d20 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3466970457 0 0) 0x7f34cc0035a0 con 0x7f34e0064350 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34e012d260 con 0x7f34e0064350 2024-09-15T03:49:37.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d0001af0 con 0x7f34e0064350 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): dump: 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2069527872 0 0) 0x7f34e0136320 con 0x7f34e01395b0 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34cc0035a0 con 0x7f34e01395b0 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d80019c0 con 0x7f34e01395b0 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): dump: 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.578 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.579 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3129802944 0 0) 0x7f34e012d260 con 0x7f34e0064350 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.26:6793/0 conn(0x7f34e01395b0 legacy=0x7f34e0149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.122:6792/0 conn(0x7f34e0132d20 legacy=0x7f34e01346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 monclient: found mon.d 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0136600 con 0x7f34e0064350 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 5 monclient: authenticate success, global_id 4884 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 10 monclient: _renew_subs 2024-09-15T03:49:37.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e01385a0 con 0x7f34e0064350 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 10 monclient: _renew_subs 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e0135280 con 0x7f34e0064350 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34d0001550 con 0x7f34e0064350 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d00020c0 con 0x7f34e0064350 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34deffd700 4 set_mon_vals no callback set 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: dump: 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.581 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.582 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f34c0006590 con 0x7f34c0002840 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f34d000ff20 con 0x7f34e0064350 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(225..225 src has 1..225) v4 ==== 4019+0+0 (unknown 3302849129 0 0) 0x7f34d0044410 con 0x7f34e0064350 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686656826 0 0) 0x7f34c0006590 con 0x7f34c0002840 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34e52c5700 10 monclient: get_auth_request con 0x7f34cc037c20 auth_method 0 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient(hunting): my global_id is 4734 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34e012d260 con 0x7f34c0002840 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3143999277 0 0) 0x7f34e012d260 con 0x7f34c0002840 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34c0006590 con 0x7f34c0002840 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d40021f0 con 0x7f34c0002840 2024-09-15T03:49:37.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.563+0000 7f34dcff9700 10 monclient: dump: 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.584 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1077510984 0 0) 0x7f34c0006590 con 0x7f34c0002840 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34c000a130 con 0x7f34c0002840 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f34d40045f0 con 0x7f34c0002840 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.26:6789/0 conn(0x7f34c0002840 legacy=0x7f34c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.567+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.26:6789/0 conn(0x7f34c0002840 legacy=0x7f34c0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34e6d28700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f34c0009640 con 0x7f34c00090b0 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93570903 0 0) 0x7f34c0009640 con 0x7f34c00090b0 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient(hunting): my global_id is 4743 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34c0006590 con 0x7f34c00090b0 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1826897781 0 0) 0x7f34c0006590 con 0x7f34c00090b0 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34c0009640 con 0x7f34c00090b0 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f34d4002d10 con 0x7f34c00090b0 2024-09-15T03:49:37.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient: dump: 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1222776440 0 0) 0x7f34c0009640 con 0x7f34c00090b0 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f34c00098f0 con 0x7f34c00090b0 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 3356457223) 0x7f34d40041d0 con 0x7f34c00090b0 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.26:6789/0 conn(0x7f34c00090b0 legacy=0x7f34c0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.579+0000 7f34dcff9700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.26:6789/0 conn(0x7f34c00090b0 legacy=0x7f34c0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:37.590 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.591 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:37.593 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.594 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:37.598 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:37.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:37.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.604 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:37.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:37.609 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:37.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.122:6812/20895 conn(0x7f34cc037c20 legacy=0x7f34cc03a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 10 monclient: shutdown 2024-09-15T03:49:37.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 >> v1:172.21.15.26:6792/0 conn(0x7f34e0064350 legacy=0x7f34e0133fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e52c5700 1 -- 172.21.15.26:0/2727558694 reap_dead start 2024-09-15T03:49:37.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 shutdown_connections 2024-09-15T03:49:37.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 >> 172.21.15.26:0/2727558694 conn(0x7f34e009f3d0 msgr2=0x7f34e009dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 shutdown_connections 2024-09-15T03:49:37.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.583+0000 7f34e6d28700 1 -- 172.21.15.26:0/2727558694 wait complete. 2024-09-15T03:49:37.611 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:37.611 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:49:37.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:49:37.674 INFO:tasks.rados.rados.0.smithi122.stdout:920: finishing write tid 2 to smithi12223906-13 2024-09-15T03:49:37.674 INFO:tasks.rados.rados.0.smithi122.stdout:923: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2024-09-15T03:49:37.681 INFO:tasks.rados.rados.0.smithi122.stdout:923: writing smithi12223906-49 from 783292 to 1324174 tid 1 2024-09-15T03:49:37.688 INFO:tasks.rados.rados.0.smithi122.stdout:923: writing smithi12223906-49 from 1932766 to 2440553 tid 2 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:923: writing smithi12223906-49 from 3210350 to 3782363 tid 3 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:919: done (4 left) 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:924: snap_create 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:921: finishing rollback tid 0 to smithi12223906-30 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:920: finishing write tid 3 to smithi12223906-13 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:920: finishing write tid 4 to smithi12223906-13 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:921: finishing rollback tid 1 to smithi12223906-30 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:920: finishing write tid 5 to smithi12223906-13 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 339 (ObjNum 354 snap 102 seq_num 354) dirty exists 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:920: left oid 13 (ObjNum 354 snap 102 seq_num 354) 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:921: finishing rollback tid 2 to smithi12223906-30 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 157 (ObjNum 279 snap 82 seq_num 279) dirty exists 2024-09-15T03:49:37.697 INFO:tasks.rados.rados.0.smithi122.stdout:922: finishing write tid 1 to smithi12223906-1 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:922: finishing write tid 2 to smithi12223906-1 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:922: finishing write tid 3 to smithi12223906-1 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:922: finishing write tid 4 to smithi12223906-1 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:922: finishing write tid 5 to smithi12223906-1 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 320 (ObjNum 355 snap 102 seq_num 355) dirty exists 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:922: left oid 1 (ObjNum 355 snap 102 seq_num 355) 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:923: finishing write tid 1 to smithi12223906-49 2024-09-15T03:49:37.698 INFO:tasks.rados.rados.0.smithi122.stdout:923: finishing write tid 2 to smithi12223906-49 2024-09-15T03:49:37.701 INFO:tasks.rados.rados.0.smithi122.stdout:923: finishing write tid 3 to smithi12223906-49 2024-09-15T03:49:37.701 INFO:tasks.rados.rados.0.smithi122.stdout:923: finishing write tid 4 to smithi12223906-49 2024-09-15T03:49:37.702 INFO:tasks.rados.rados.0.smithi122.stdout:923: finishing write tid 5 to smithi12223906-49 2024-09-15T03:49:37.702 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 295 (ObjNum 356 snap 102 seq_num 356) dirty exists 2024-09-15T03:49:37.702 INFO:tasks.rados.rados.0.smithi122.stdout:923: left oid 49 (ObjNum 356 snap 102 seq_num 356) 2024-09-15T03:49:37.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.723+0000 7f00ddba9700 10 monclient: get_monmap_and_config 2024-09-15T03:49:37.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.723+0000 7f00ddba9700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.723+0000 7f00ddba9700 10 monclient: monmap: 2024-09-15T03:49:37.727 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:37.726813+0000 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:37.726813+0000 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.728 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.729 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.729 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.723+0000 7f00ddba9700 1 Processor -- start 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.723+0000 7f00ddba9700 1 -- start start 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.723+0000 7f00ddba9700 10 monclient: init 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient: _add_conns ranks=[3,5,4,8,7,1,6,0,2] 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient(hunting): picked mon.noname-g con 0x7f00d80990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient(hunting): picked mon.noname-h con 0x7f00d80a1310 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient(hunting): picked mon.noname-c con 0x7f00d81335a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d804ca10 con 0x7f00d80990c0 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80a1b20 con 0x7f00d81335a0 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80a10b0 con 0x7f00d80a1310 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient(hunting): authenticate will time out at 1122.156472s 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d7fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f00d81335a0 0x7f00d813afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43058/0 (socket says 172.21.15.26:43058) 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d7fff700 1 -- 172.21.15.26:0/1582218089 learned_addr learned my addr 172.21.15.26:0/1582218089 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835992913 0 0) 0x7f00d80a1b20 con 0x7f00d81335a0 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): my global_id is 174118 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c4003570 con 0x7f00d81335a0 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1382735487 0 0) 0x7f00d804ca10 con 0x7f00d80990c0 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): my global_id is 4791 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d80a1b20 con 0x7f00d80990c0 2024-09-15T03:49:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114082314 0 0) 0x7f00d80a10b0 con 0x7f00d80a1310 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient(hunting): my global_id is 4910 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d804ca10 con 0x7f00d80a1310 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1399374955 0 0) 0x7f00d80a1b20 con 0x7f00d80990c0 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 >> v1:172.21.15.122:6791/0 conn(0x7f00d80a1310 legacy=0x7f00d8099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 >> v1:172.21.15.26:6791/0 conn(0x7f00d81335a0 legacy=0x7f00d813afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 monclient: found mon.noname-g 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8099ba0 con 0x7f00d80990c0 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c8001b10 con 0x7f00d80990c0 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 5 monclient: authenticate success, global_id 4791 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: dump: 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f00c8001e30 con 0x7f00d80990c0 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 4 set_mon_vals no callback set 2024-09-15T03:49:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 1 -- 172.21.15.26:0/1582218089 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c8002360 con 0x7f00d80990c0 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00d67fc700 10 monclient: dump: 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.738 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient: shutdown 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- 172.21.15.26:0/1582218089 >> v1:172.21.15.122:6790/0 conn(0x7f00d80990c0 legacy=0x7f00d8132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- 172.21.15.26:0/1582218089 shutdown_connections 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- 172.21.15.26:0/1582218089 >> 172.21.15.26:0/1582218089 conn(0x7f00d806e370 msgr2=0x7f00d80a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- 172.21.15.26:0/1582218089 shutdown_connections 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 1 -- 172.21.15.26:0/1582218089 wait complete. 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.727+0000 7f00ddba9700 10 monclient: monmap: 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.741 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 Processor -- start 2024-09-15T03:49:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 -- start start 2024-09-15T03:49:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: init 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: _add_conns ranks=[7,8,6,2,4,5,3,0,1] 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient(hunting): picked mon.noname-i con 0x7f00d80990c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient(hunting): picked mon.noname-e con 0x7f00d80a1310 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient(hunting): picked mon.noname-d con 0x7f00d81335a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80a1ce0 con 0x7f00d81335a0 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d809a010 con 0x7f00d80990c0 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d8121b50 con 0x7f00d80a1310 2024-09-15T03:49:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient(hunting): authenticate will time out at 1122.161207s 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00d7fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f00d81335a0 0x7f00d814a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34546/0 (socket says 172.21.15.26:34546) 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00d7fff700 1 -- 172.21.15.26:0/1720886820 learned_addr learned my addr 172.21.15.26:0/1720886820 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422097293 0 0) 0x7f00d80a1ce0 con 0x7f00d81335a0 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): my global_id is 4893 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b8003570 con 0x7f00d81335a0 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370555765 0 0) 0x7f00d809a010 con 0x7f00d80990c0 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): my global_id is 134341 2024-09-15T03:49:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d80a1ce0 con 0x7f00d80990c0 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451036918 0 0) 0x7f00d8121b50 con 0x7f00d80a1310 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): my global_id is 134369 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d809a010 con 0x7f00d80a1310 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2820789254 0 0) 0x7f00d809a010 con 0x7f00d80a1310 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00d8121b50 con 0x7f00d80a1310 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2210408445 0 0) 0x7f00d80a1ce0 con 0x7f00d80990c0 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00d809a010 con 0x7f00d80990c0 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c0001a60 con 0x7f00d80a1310 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): dump: 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3008172063 0 0) 0x7f00b8003570 con 0x7f00d81335a0 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00d80a1ce0 con 0x7f00d81335a0 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c8001ad0 con 0x7f00d80990c0 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): dump: 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00d0001960 con 0x7f00d81335a0 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient(hunting): dump: 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1116330992 0 0) 0x7f00d8121b50 con 0x7f00d80a1310 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.122:6792/0 conn(0x7f00d80990c0 legacy=0x7f00d8131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.26:6792/0 conn(0x7f00d81335a0 legacy=0x7f00d814a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 monclient: found mon.e 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8121e30 con 0x7f00d80a1310 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 5 monclient: authenticate success, global_id 134369 2024-09-15T03:49:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: _renew_subs 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d8120020 con 0x7f00d80a1310 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: _renew_subs 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d8120bf0 con 0x7f00d80a1310 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f00c0002cf0 con 0x7f00d80a1310 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c00022f0 con 0x7f00d80a1310 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00d5ffb700 4 set_mon_vals no callback set 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.731+0000 7f00cf7fe700 10 monclient: dump: 2024-09-15T03:49:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00ddba9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f00bc006590 con 0x7f00bc002840 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f00c00025a0 con 0x7f00d80a1310 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(225..225 src has 1..225) v4 ==== 4019+0+0 (unknown 3302849129 0 0) 0x7f00c00415a0 con 0x7f00d80a1310 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109349206 0 0) 0x7f00bc006590 con 0x7f00bc002840 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient(hunting): my global_id is 134381 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d8121b50 con 0x7f00bc002840 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00d7fff700 10 monclient: get_auth_request con 0x7f00b80380e0 auth_method 0 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 258865022 0 0) 0x7f00d8121b50 con 0x7f00bc002840 2024-09-15T03:49:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00bc006590 con 0x7f00bc002840 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c8001590 con 0x7f00bc002840 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient: dump: 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.757 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1004690510 0 0) 0x7f00bc006590 con 0x7f00bc002840 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00bc00a130 con 0x7f00bc002840 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f00c8003140 con 0x7f00bc002840 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.26:6790/0 conn(0x7f00bc002840 legacy=0x7f00bc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.735+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.26:6790/0 conn(0x7f00bc002840 legacy=0x7f00bc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f00bc007520 con 0x7f00bc0068d0 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528770009 0 0) 0x7f00bc007520 con 0x7f00bc0068d0 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient(hunting): my global_id is 134390 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc006590 con 0x7f00bc0068d0 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2805004513 0 0) 0x7f00bc006590 con 0x7f00bc0068d0 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00bc007520 con 0x7f00bc0068d0 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00c80038d0 con 0x7f00bc0068d0 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient: dump: 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2982724855 0 0) 0x7f00bc007520 con 0x7f00bc0068d0 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f00bc008760 con 0x7f00bc0068d0 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 290685682) 0x7f00c8004d30 con 0x7f00bc0068d0 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.26:6790/0 conn(0x7f00bc0068d0 legacy=0x7f00bc006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00cf7fe700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.26:6790/0 conn(0x7f00bc0068d0 legacy=0x7f00bc006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:37.762 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:37.763 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.764 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.765 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:37.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.767 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:37.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.770 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:37.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.774 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.775 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:37.778 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.780 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:37.781 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.122:6812/20895 conn(0x7f00b80380e0 legacy=0x7f00b803a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 10 monclient: shutdown 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 >> v1:172.21.15.26:6793/0 conn(0x7f00d80a1310 legacy=0x7f00d8147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00d7fff700 1 -- 172.21.15.26:0/1720886820 reap_dead start 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 shutdown_connections 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 >> 172.21.15.26:0/1720886820 conn(0x7f00d806e370 msgr2=0x7f00d80a4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 shutdown_connections 2024-09-15T03:49:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.747+0000 7f00ddba9700 1 -- 172.21.15.26:0/1720886820 wait complete. 2024-09-15T03:49:37.784 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:37.784 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:49:37.784 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:49:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient: get_monmap_and_config 2024-09-15T03:49:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient: monmap: 2024-09-15T03:49:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:37.896678+0000 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:37.896678+0000 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.899 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 1 Processor -- start 2024-09-15T03:49:37.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 1 -- start start 2024-09-15T03:49:37.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient: init 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient: _add_conns ranks=[3,6,2,0,7,5,4,1,8] 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient(hunting): picked mon.noname-g con 0x7f1af40a3860 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient(hunting): picked mon.noname-d con 0x7f1af409e0b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient(hunting): picked mon.noname-b con 0x7f1af409e480 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af404ca10 con 0x7f1af409e480 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af4098c50 con 0x7f1af40a3860 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af4098970 con 0x7f1af409e0b0 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af9014700 10 monclient(hunting): authenticate will time out at 1122.327681s 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1aebfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1af409e0b0 0x7f1af4134ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34556/0 (socket says 172.21.15.26:34556) 2024-09-15T03:49:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1af359e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f1af409e480 0x7f1af41382d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:50236/0 (socket says 172.21.15.26:50236) 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.895+0000 7f1aebfff700 1 -- 172.21.15.26:0/3312139755 learned_addr learned my addr 172.21.15.26:0/3312139755 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686800299 0 0) 0x7f1af404ca10 con 0x7f1af409e480 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): my global_id is 134399 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae0003570 con 0x7f1af409e480 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208330356 0 0) 0x7f1af4098970 con 0x7f1af409e0b0 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): my global_id is 4902 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af404ca10 con 0x7f1af409e0b0 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560681769 0 0) 0x7f1af4098c50 con 0x7f1af40a3860 2024-09-15T03:49:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient(hunting): my global_id is 4800 2024-09-15T03:49:37.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af4098970 con 0x7f1af40a3860 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651316459 0 0) 0x7f1ae0003570 con 0x7f1af409e480 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 >> v1:172.21.15.26:6792/0 conn(0x7f1af409e0b0 legacy=0x7f1af4134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 >> v1:172.21.15.122:6790/0 conn(0x7f1af40a3860 legacy=0x7f1af409d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 monclient: found mon.noname-b 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4098f30 con 0x7f1af409e480 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1aec001b50 con 0x7f1af409e480 2024-09-15T03:49:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 5 monclient: authenticate success, global_id 134399 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: dump: 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1aec001dc0 con 0x7f1af409e480 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 1 -- 172.21.15.26:0/3312139755 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1aec004ef0 con 0x7f1af409e480 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 4 set_mon_vals no callback set 2024-09-15T03:49:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af259c700 10 monclient: dump: 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 10 monclient: shutdown 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 1 -- 172.21.15.26:0/3312139755 >> v1:172.21.15.26:6790/0 conn(0x7f1af409e480 legacy=0x7f1af41382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 1 -- 172.21.15.26:0/3312139755 shutdown_connections 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 1 -- 172.21.15.26:0/3312139755 >> 172.21.15.26:0/3312139755 conn(0x7f1af4071350 msgr2=0x7f1af40a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 1 -- 172.21.15.26:0/3312139755 shutdown_connections 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.899+0000 7f1af9014700 1 -- 172.21.15.26:0/3312139755 wait complete. 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: build_initial_monmap 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: monmap: 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 Processor -- start 2024-09-15T03:49:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 -- start start 2024-09-15T03:49:37.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: init 2024-09-15T03:49:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: _add_conns ranks=[0,4,6,2,3,7,8,5,1] 2024-09-15T03:49:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient(hunting): picked mon.noname-a con 0x7f1af409e0b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient(hunting): picked mon.noname-c con 0x7f1af409e480 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient(hunting): picked mon.noname-d con 0x7f1af40a3860 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:37.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af406e1d0 con 0x7f1af409e0b0 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af41231b0 con 0x7f1af409e480 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af4122ed0 con 0x7f1af40a3860 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient(hunting): authenticate will time out at 1122.333436s 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af2d9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1af409e0b0 0x7f1af4141f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47280/0 (socket says 172.21.15.26:47280) 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af2d9d700 1 -- 172.21.15.26:0/1426980451 learned_addr learned my addr 172.21.15.26:0/1426980451 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041093105 0 0) 0x7f1af406e1d0 con 0x7f1af409e0b0 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): my global_id is 4752 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad4003570 con 0x7f1af409e0b0 2024-09-15T03:49:37.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053040847 0 0) 0x7f1af4122ed0 con 0x7f1af40a3860 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): my global_id is 4911 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af406e1d0 con 0x7f1af40a3860 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158987020 0 0) 0x7f1af41231b0 con 0x7f1af409e480 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): my global_id is 174127 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af4122ed0 con 0x7f1af409e480 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 438425783 0 0) 0x7f1ad4003570 con 0x7f1af409e0b0 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1af41231b0 con 0x7f1af409e0b0 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1ae4002ad0 con 0x7f1af409e0b0 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): dump: 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 904376399 0 0) 0x7f1af4122ed0 con 0x7f1af409e480 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ad4003570 con 0x7f1af409e480 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1adc001470 con 0x7f1af409e480 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient(hunting): dump: 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3436132889 0 0) 0x7f1af406e1d0 con 0x7f1af40a3860 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1af4122ed0 con 0x7f1af40a3860 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 125240290 0 0) 0x7f1af41231b0 con 0x7f1af409e0b0 2024-09-15T03:49:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6792/0 conn(0x7f1af40a3860 legacy=0x7f1af414a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6791/0 conn(0x7f1af409e480 legacy=0x7f1af4135db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 monclient: found mon.a 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4142660 con 0x7f1af409e0b0 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 5 monclient: authenticate success, global_id 4752 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: _renew_subs 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af41364c0 con 0x7f1af409e0b0 2024-09-15T03:49:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: _renew_subs 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af4122180 con 0x7f1af409e0b0 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1ae4001600 con 0x7f1af409e0b0 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1ae4001b30 con 0x7f1af409e0b0 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1af1d9b700 4 set_mon_vals no callback set 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.903+0000 7f1aeaffd700 10 monclient: dump: 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1af9014700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1ad8006590 con 0x7f1ad8002840 2024-09-15T03:49:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1ae4001d10 con 0x7f1af409e0b0 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(225..225 src has 1..225) v4 ==== 4019+0+0 (unknown 3302849129 0 0) 0x7f1ae4043fe0 con 0x7f1af409e0b0 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857920031 0 0) 0x7f1ad8006590 con 0x7f1ad8002840 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient(hunting): my global_id is 174136 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af41231b0 con 0x7f1ad8002840 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1af359e700 10 monclient: get_auth_request con 0x7f1ad4037f70 auth_method 0 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3967810944 0 0) 0x7f1af41231b0 con 0x7f1ad8002840 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ad8006590 con 0x7f1ad8002840 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1adc004b70 con 0x7f1ad8002840 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient: dump: 2024-09-15T03:49:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3269614679 0 0) 0x7f1ad8006590 con 0x7f1ad8002840 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ad800a130 con 0x7f1ad8002840 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1adc004fc0 con 0x7f1ad8002840 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6791/0 conn(0x7f1ad8002840 legacy=0x7f1ad8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.907+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6791/0 conn(0x7f1ad8002840 legacy=0x7f1ad8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1af9014700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1ad8009640 con 0x7f1ad80090b0 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123403822 0 0) 0x7f1ad8009640 con 0x7f1ad80090b0 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient(hunting): my global_id is 174145 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad8006590 con 0x7f1ad80090b0 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2180542731 0 0) 0x7f1ad8006590 con 0x7f1ad80090b0 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ad8009640 con 0x7f1ad80090b0 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1adc005800 con 0x7f1ad80090b0 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient: dump: 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2949669931 0 0) 0x7f1ad8009640 con 0x7f1ad80090b0 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1ad80098f0 con 0x7f1ad80090b0 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1870727145) 0x7f1adc005c20 con 0x7f1ad80090b0 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6791/0 conn(0x7f1ad80090b0 legacy=0x7f1ad8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.919+0000 7f1aeaffd700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6791/0 conn(0x7f1ad80090b0 legacy=0x7f1ad8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:37.931 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:37.932 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.934 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:37.935 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:37.936 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:37.937 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.938 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.939 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.940 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.941 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:37.942 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.943 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.944 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.946 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:37.948 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:37.949 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:37.950 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:37.951 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.122:6812/20895 conn(0x7f1ad4037f70 legacy=0x7f1ad403a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 10 monclient: shutdown 2024-09-15T03:49:37.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 >> v1:172.21.15.26:6789/0 conn(0x7f1af409e0b0 legacy=0x7f1af4141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af359e700 1 -- 172.21.15.26:0/1426980451 reap_dead start 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af359e700 1 -- 172.21.15.26:0/1426980451 reap_dead start 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 shutdown_connections 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 >> 172.21.15.26:0/1426980451 conn(0x7f1af4071350 msgr2=0x7f1af4063340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 shutdown_connections 2024-09-15T03:49:37.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:37.923+0000 7f1af9014700 1 -- 172.21.15.26:0/1426980451 wait complete. 2024-09-15T03:49:37.953 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:37.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:49:37.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:49:38.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.067+0000 7f5cb2b10700 10 monclient: get_monmap_and_config 2024-09-15T03:49:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.067+0000 7f5cb2b10700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.067+0000 7f5cb2b10700 10 monclient: monmap: 2024-09-15T03:49:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.152 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:38.070078+0000 2024-09-15T03:49:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:38.070078+0000 2024-09-15T03:49:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.153 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.154 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.067+0000 7f5cb2b10700 1 Processor -- start 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.067+0000 7f5cb2b10700 1 -- start start 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.067+0000 7f5cb2b10700 10 monclient: init 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient: _add_conns ranks=[5,6,4,8,7,2,1,3,0] 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient(hunting): picked mon.noname-h con 0x7f5cac12f270 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient(hunting): picked mon.noname-d con 0x7f5cac06d5b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient(hunting): picked mon.noname-c con 0x7f5cac1353b0 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:38.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac04ca10 con 0x7f5cac1353b0 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac06aab0 con 0x7f5cac12f270 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac09f050 con 0x7f5cac06d5b0 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient(hunting): authenticate will time out at 1122.500471s 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cabfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5cac06d5b0 0x7f5cac06d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34570/0 (socket says 172.21.15.26:34570) 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cabfff700 1 -- 172.21.15.26:0/1406045447 learned_addr learned my addr 172.21.15.26:0/1406045447 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711878663 0 0) 0x7f5cac09f050 con 0x7f5cac06d5b0 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): my global_id is 4920 2024-09-15T03:49:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c003570 con 0x7f5cac06d5b0 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264377564 0 0) 0x7f5cac04ca10 con 0x7f5cac1353b0 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): my global_id is 174154 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cac09f050 con 0x7f5cac1353b0 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112291397 0 0) 0x7f5cac06aab0 con 0x7f5cac12f270 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient(hunting): my global_id is 4919 2024-09-15T03:49:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cac04ca10 con 0x7f5cac12f270 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2834182567 0 0) 0x7f5cac09f050 con 0x7f5cac1353b0 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 >> v1:172.21.15.26:6792/0 conn(0x7f5cac06d5b0 legacy=0x7f5cac06d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 >> v1:172.21.15.122:6791/0 conn(0x7f5cac12f270 legacy=0x7f5cac12f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 monclient: found mon.noname-c 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac06a7d0 con 0x7f5cac1353b0 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca4001b10 con 0x7f5cac1353b0 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 5 monclient: authenticate success, global_id 174154 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: dump: 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ca4001db0 con 0x7f5cac1353b0 2024-09-15T03:49:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 1 -- 172.21.15.26:0/1406045447 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca40022e0 con 0x7f5cac1353b0 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 4 set_mon_vals no callback set 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cab7fe700 10 monclient: dump: 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient: shutdown 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- 172.21.15.26:0/1406045447 >> v1:172.21.15.26:6791/0 conn(0x7f5cac1353b0 legacy=0x7f5cac137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- 172.21.15.26:0/1406045447 shutdown_connections 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- 172.21.15.26:0/1406045447 >> 172.21.15.26:0/1406045447 conn(0x7f5cac063e10 msgr2=0x7f5cac0a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- 172.21.15.26:0/1406045447 shutdown_connections 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 -- 172.21.15.26:0/1406045447 wait complete. 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 10 monclient: monmap: 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:38.172 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.071+0000 7f5cb2b10700 1 Processor -- start 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- start start 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: init 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: _add_conns ranks=[2,7,8,5,4,3,1,0,6] 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient(hunting): picked mon.noname-b con 0x7f5cac06d5b0 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient(hunting): picked mon.noname-i con 0x7f5cac12f270 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient(hunting): picked mon.noname-e con 0x7f5cac1353b0 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac09cd70 con 0x7f5cac06d5b0 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac11b620 con 0x7f5cac12f270 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac134da0 con 0x7f5cac1353b0 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient(hunting): authenticate will time out at 1122.504575s 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb10ad700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5cac1353b0 0x7f5cac14a650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56710/0 (socket says 172.21.15.26:56710) 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb10ad700 1 -- 172.21.15.26:0/2319031294 learned_addr learned my addr 172.21.15.26:0/2319031294 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807875982 0 0) 0x7f5cac134da0 con 0x7f5cac1353b0 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): my global_id is 134378 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90003570 con 0x7f5cac1353b0 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055775137 0 0) 0x7f5cac09cd70 con 0x7f5cac06d5b0 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): my global_id is 134408 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cac134da0 con 0x7f5cac06d5b0 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 235826307 0 0) 0x7f5c90003570 con 0x7f5cac1353b0 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5cac09cd70 con 0x7f5cac1353b0 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca4002260 con 0x7f5cac1353b0 2024-09-15T03:49:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): dump: 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911741232 0 0) 0x7f5cac11b620 con 0x7f5cac12f270 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): my global_id is 134350 2024-09-15T03:49:38.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90003570 con 0x7f5cac12f270 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2956427545 0 0) 0x7f5cac09cd70 con 0x7f5cac1353b0 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.122:6792/0 conn(0x7f5cac12f270 legacy=0x7f5cac146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.26:6790/0 conn(0x7f5cac06d5b0 legacy=0x7f5cac12e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 monclient: found mon.e 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac11b340 con 0x7f5cac1353b0 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 5 monclient: authenticate success, global_id 134378 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: _renew_subs 2024-09-15T03:49:38.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac119f40 con 0x7f5cac1353b0 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: _renew_subs 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac1193b0 con 0x7f5cac1353b0 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ca40028c0 con 0x7f5cac1353b0 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca4002df0 con 0x7f5cac1353b0 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5caaffd700 4 set_mon_vals no callback set 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:38.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient: dump: 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c8c006590 con 0x7f5c8c002840 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5ca4002f50 con 0x7f5cac1353b0 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(225..225 src has 1..225) v4 ==== 4019+0+0 (unknown 3302849129 0 0) 0x7f5ca40443e0 con 0x7f5cac1353b0 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607429655 0 0) 0x7f5c8c006590 con 0x7f5c8c002840 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 10 monclient(hunting): my global_id is 4929 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cac09cd70 con 0x7f5c8c002840 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.075+0000 7f5cabfff700 10 monclient: get_auth_request con 0x7f5c900376e0 auth_method 0 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2205440175 0 0) 0x7f5cac09cd70 con 0x7f5c8c002840 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c8c006590 con 0x7f5c8c002840 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca0001970 con 0x7f5c8c002840 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 10 monclient: dump: 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3960514952 0 0) 0x7f5c8c006590 con 0x7f5c8c002840 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c8c00a130 con 0x7f5c8c002840 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5ca0001db0 con 0x7f5c8c002840 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.26:6792/0 conn(0x7f5c8c002840 legacy=0x7f5c8c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.079+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.26:6792/0 conn(0x7f5c8c002840 legacy=0x7f5c8c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5cb2b10700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c8c007520 con 0x7f5c8c0068d0 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815616965 0 0) 0x7f5c8c007520 con 0x7f5c8c0068d0 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 10 monclient(hunting): my global_id is 4938 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c8c006590 con 0x7f5c8c0068d0 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 260338797 0 0) 0x7f5c8c006590 con 0x7f5c8c0068d0 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c8c007520 con 0x7f5c8c0068d0 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca0002670 con 0x7f5c8c0068d0 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:38.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.087+0000 7f5ca8ff9700 10 monclient: dump: 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.188 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.190 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2534515109 0 0) 0x7f5c8c007520 con 0x7f5c8c0068d0 2024-09-15T03:49:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5c8c008760 con 0x7f5c8c0068d0 2024-09-15T03:49:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 2308662565) 0x7f5ca0002b20 con 0x7f5c8c0068d0 2024-09-15T03:49:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.26:6792/0 conn(0x7f5c8c0068d0 legacy=0x7f5c8c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5ca8ff9700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.26:6792/0 conn(0x7f5c8c0068d0 legacy=0x7f5c8c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:38.194 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:38.195 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.197 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:38.198 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.199 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:38.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.202 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:38.203 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.204 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.204 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.204 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.204 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.205 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.205 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.205 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.205 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:38.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:38.206 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.206 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:38.206 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.207 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.209 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:38.209 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.209 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.209 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.210 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.211 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:38.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:38.211 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.211 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.212 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:38.213 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:38.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.215 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:38.216 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.217 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.218 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:38.219 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.122:6812/20895 conn(0x7f5c900376e0 legacy=0x7f5c90039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 10 monclient: shutdown 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 >> v1:172.21.15.26:6793/0 conn(0x7f5cac1353b0 legacy=0x7f5cac14a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb10ad700 1 -- 172.21.15.26:0/2319031294 reap_dead start 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb10ad700 1 -- 172.21.15.26:0/2319031294 reap_dead start 2024-09-15T03:49:38.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 shutdown_connections 2024-09-15T03:49:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 >> 172.21.15.26:0/2319031294 conn(0x7f5cac063e10 msgr2=0x7f5cac09be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 shutdown_connections 2024-09-15T03:49:38.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.091+0000 7f5cb2b10700 1 -- 172.21.15.26:0/2319031294 wait complete. 2024-09-15T03:49:38.222 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:38.223 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:49:38.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:49:38.225 INFO:tasks.rados.rados.0.smithi122.stdout:920: done (4 left) 2024-09-15T03:49:38.225 INFO:tasks.rados.rados.0.smithi122.stdout:921: done (3 left) 2024-09-15T03:49:38.225 INFO:tasks.rados.rados.0.smithi122.stdout:922: done (2 left) 2024-09-15T03:49:38.225 INFO:tasks.rados.rados.0.smithi122.stdout:923: done (1 left) 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:924: done (0 left) 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:925: delete oid 35 current snap is 103 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:925: done (0 left) 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:926: read oid 9 snap -1 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:926: expect (ObjNum 312 snap 96 seq_num 312) 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:927: read oid 14 snap -1 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:927: expect (ObjNum 349 snap 102 seq_num 349) 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:928: write (excl) oid 32 current snap is 103 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:928: seq_num 358 ranges {416906=407733,1366083=511456} 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:928: writing smithi12223906-32 from 416906 to 824639 tid 1 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:928: writing smithi12223906-32 from 1366083 to 1877539 tid 2 2024-09-15T03:49:38.226 INFO:tasks.rados.rados.0.smithi122.stdout:929: rollback oid 17 current snap is 103 2024-09-15T03:49:38.227 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 102 2024-09-15T03:49:38.227 INFO:tasks.rados.rados.0.smithi122.stdout:926: expect (ObjNum 312 snap 96 seq_num 312) 2024-09-15T03:49:38.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient: get_monmap_and_config 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient: monmap: 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:38.340812+0000 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:38.340812+0000 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.345 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 1 Processor -- start 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 1 -- start start 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient: init 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient: _add_conns ranks=[6,0,3,4,5,7,8,1,2] 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient(hunting): picked mon.noname-d con 0x7f61b80a6880 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient(hunting): picked mon.noname-a con 0x7f61b809faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient(hunting): picked mon.noname-g con 0x7f61b8138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b804ca10 con 0x7f61b809faa0 2024-09-15T03:49:38.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b80a0580 con 0x7f61b8138d30 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b8133780 con 0x7f61b80a6880 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bf518700 10 monclient(hunting): authenticate will time out at 1122.770827s 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61be516700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f61b80a6880 0x7f61b8133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34588/0 (socket says 172.21.15.26:34588) 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bdd15700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61b809faa0 0x7f61b809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47290/0 (socket says 172.21.15.26:47290) 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61be516700 1 -- 172.21.15.26:0/3534579730 learned_addr learned my addr 172.21.15.26:0/3534579730 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437909302 0 0) 0x7f61b804ca10 con 0x7f61b809faa0 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): my global_id is 4761 2024-09-15T03:49:38.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a4003570 con 0x7f61b809faa0 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287557376 0 0) 0x7f61b8133780 con 0x7f61b80a6880 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): my global_id is 4947 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b804ca10 con 0x7f61b80a6880 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958909328 0 0) 0x7f61b0001490 con 0x7f61b8138d30 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 10 monclient(hunting): my global_id is 4809 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.339+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8133780 con 0x7f61b8138d30 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032813524 0 0) 0x7f61a4003570 con 0x7f61b809faa0 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 >> v1:172.21.15.26:6792/0 conn(0x7f61b80a6880 legacy=0x7f61b8133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 >> v1:172.21.15.122:6790/0 conn(0x7f61b8138d30 legacy=0x7f61b813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 monclient: found mon.noname-a 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80a6c50 con 0x7f61b809faa0 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 5 monclient: authenticate success, global_id 4761 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61b40017b0 con 0x7f61b809faa0 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: dump: 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.359 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f61b4001a90 con 0x7f61b809faa0 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:38.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 1 -- 172.21.15.26:0/3534579730 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61b4004b70 con 0x7f61b809faa0 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 4 set_mon_vals no callback set 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bd514700 10 monclient: dump: 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.361 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: shutdown 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- 172.21.15.26:0/3534579730 >> v1:172.21.15.26:6789/0 conn(0x7f61b809faa0 legacy=0x7f61b809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- 172.21.15.26:0/3534579730 shutdown_connections 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- 172.21.15.26:0/3534579730 >> 172.21.15.26:0/3534579730 conn(0x7f61b806e2a0 msgr2=0x7f61b80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- 172.21.15.26:0/3534579730 shutdown_connections 2024-09-15T03:49:38.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- 172.21.15.26:0/3534579730 wait complete. 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: monmap: 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.363 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 Processor -- start 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- start start 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: init 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient: _add_conns ranks=[2,3,0,6,7,4,1,5,8] 2024-09-15T03:49:38.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient(hunting): picked mon.noname-b con 0x7f61b809faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient(hunting): picked mon.noname-g con 0x7f61b80a6880 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient(hunting): picked mon.noname-a con 0x7f61b8138d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b80a18c0 con 0x7f61b8138d30 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b809cf10 con 0x7f61b809faa0 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b8137330 con 0x7f61b80a6880 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bf518700 10 monclient(hunting): authenticate will time out at 1122.774900s 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bed17700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61b8138d30 0x7f61b81366f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47298/0 (socket says 172.21.15.26:47298) 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61bed17700 1 -- 172.21.15.26:0/1381733254 learned_addr learned my addr 172.21.15.26:0/1381733254 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253207972 0 0) 0x7f61b80a18c0 con 0x7f61b8138d30 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): my global_id is 4770 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f619c003570 con 0x7f61b8138d30 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878410235 0 0) 0x7f61b809cf10 con 0x7f61b809faa0 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): my global_id is 134417 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b80a18c0 con 0x7f61b809faa0 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683222520 0 0) 0x7f61b8137330 con 0x7f61b80a6880 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 10 monclient(hunting): my global_id is 4818 2024-09-15T03:49:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b809cf10 con 0x7f61b80a6880 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.343+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3120602137 0 0) 0x7f61b80a18c0 con 0x7f61b809faa0 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61b8137330 con 0x7f61b809faa0 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61a8001be0 con 0x7f61b809faa0 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): dump: 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1148653039 0 0) 0x7f619c003570 con 0x7f61b8138d30 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61b80a18c0 con 0x7f61b8138d30 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61b0001e50 con 0x7f61b8138d30 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): dump: 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.369 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3251939276 0 0) 0x7f61b8137330 con 0x7f61b809faa0 2024-09-15T03:49:38.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.122:6790/0 conn(0x7f61b80a6880 legacy=0x7f61b8135fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.26:6789/0 conn(0x7f61b8138d30 legacy=0x7f61b81366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 monclient: found mon.b 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b811c850 con 0x7f61b809faa0 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 5 monclient: authenticate success, global_id 134417 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 10 monclient: _renew_subs 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b8137f00 con 0x7f61b809faa0 2024-09-15T03:49:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 10 monclient: _renew_subs 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8132a70 con 0x7f61b809faa0 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f61a8001210 con 0x7f61b809faa0 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61a8004ea0 con 0x7f61b809faa0 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bcd13700 4 set_mon_vals no callback set 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: dump: 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f61a800f8e0 con 0x7f61b809faa0 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f61a0006590 con 0x7f61a0002840 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 3991+0+0 (unknown 2894827671 0 0) 0x7f61a8043f30 con 0x7f61b809faa0 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336437446 0 0) 0x7f61a0006590 con 0x7f61a0002840 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient(hunting): my global_id is 134387 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8137330 con 0x7f61a0002840 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61bed17700 10 monclient: get_auth_request con 0x7f619c037dc0 auth_method 0 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1111309991 0 0) 0x7f61b8137330 con 0x7f61a0002840 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61a0006590 con 0x7f61a0002840 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61b4001210 con 0x7f61a0002840 2024-09-15T03:49:38.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 10 monclient: dump: 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.375 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3404780264 0 0) 0x7f61a0006590 con 0x7f61a0002840 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.347+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61a000a130 con 0x7f61a0002840 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.351+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f61b4001740 con 0x7f61a0002840 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.351+0000 7f61ae7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.351+0000 7f61ae7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.351+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.26:6793/0 conn(0x7f61a0002840 legacy=0x7f61a0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.351+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.26:6793/0 conn(0x7f61a0002840 legacy=0x7f61a0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.359+0000 7f61bf518700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.359+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f61a0009640 con 0x7f61a00090b0 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894775555 0 0) 0x7f61a0009640 con 0x7f61a00090b0 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient(hunting): my global_id is 134396 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a0006590 con 0x7f61a00090b0 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1232334270 0 0) 0x7f61a0006590 con 0x7f61a00090b0 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f61a0009640 con 0x7f61a00090b0 2024-09-15T03:49:38.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f61b4005530 con 0x7f61a00090b0 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient: dump: 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.380 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1780671982 0 0) 0x7f61a0009640 con 0x7f61a00090b0 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f61a00098f0 con 0x7f61a00090b0 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1560522777) 0x7f61b4007060 con 0x7f61a00090b0 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.26:6793/0 conn(0x7f61a00090b0 legacy=0x7f61a0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61ae7fc700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.26:6793/0 conn(0x7f61a00090b0 legacy=0x7f61a0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:38.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:38.383 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.384 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.387 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:38.388 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.389 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.390 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.391 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.395 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.396 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:38.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:38.398 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:38.399 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.399 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.399 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.399 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.399 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.400 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.400 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.400 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.401 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:38.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:38.401 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.402 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.402 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:38.402 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.403 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.403 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.403 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.403 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.404 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.404 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.404 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.404 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:38.405 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:38.405 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.405 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:38.406 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.406 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.406 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.407 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.407 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.407 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.407 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.407 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.408 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:38.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:38.408 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.408 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.409 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.409 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:38.409 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.409 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.410 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.410 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.410 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.414 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.414 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.415 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.415 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:38.415 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:38.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.416 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.416 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.416 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.417 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.417 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:38.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.417 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.417 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.418 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.418 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.419 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:38.419 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:38.423 INFO:tasks.rados.rados.0.smithi122.stdout:927: expect (ObjNum 349 snap 102 seq_num 349) 2024-09-15T03:49:38.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.122:6812/20895 conn(0x7f619c037dc0 legacy=0x7f619c03a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 10 monclient: shutdown 2024-09-15T03:49:38.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 >> v1:172.21.15.26:6790/0 conn(0x7f61b809faa0 legacy=0x7f61b81358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bed17700 1 -- 172.21.15.26:0/1381733254 reap_dead start 2024-09-15T03:49:38.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bed17700 1 -- 172.21.15.26:0/1381733254 reap_dead start 2024-09-15T03:49:38.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 shutdown_connections 2024-09-15T03:49:38.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 >> 172.21.15.26:0/1381733254 conn(0x7f61b806e2a0 msgr2=0x7f61b809d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 shutdown_connections 2024-09-15T03:49:38.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.363+0000 7f61bf518700 1 -- 172.21.15.26:0/1381733254 wait complete. 2024-09-15T03:49:38.426 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:38.426 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:49:38.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:49:38.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient: get_monmap_and_config 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient: monmap: 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:38.545182+0000 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:38.545182+0000 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.548 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 1 Processor -- start 2024-09-15T03:49:38.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 1 -- start start 2024-09-15T03:49:38.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient: init 2024-09-15T03:49:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient: _add_conns ranks=[7,6,8,4,2,3,1,5,0] 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient(hunting): picked mon.noname-i con 0x7fe2780a0090 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient(hunting): picked mon.noname-d con 0x7fe278099540 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient(hunting): picked mon.noname-e con 0x7fe278135c40 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27804ca10 con 0x7fe278099540 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27806e600 con 0x7fe2780a0090 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780a0460 con 0x7fe278135c40 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27d587700 10 monclient(hunting): authenticate will time out at 1122.975192s 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27cd86700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fe278135c40 0x7fe278138020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56738/0 (socket says 172.21.15.26:56738) 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe27cd86700 1 -- 172.21.15.26:0/880508045 learned_addr learned my addr 172.21.15.26:0/880508045 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe2777fe700 1 --1- 172.21.15.26:0/880508045 >> v1:172.21.15.26:6792/0 conn(0x7fe278099540 0x7fe27809dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34594/0 (socket says 172.21.15.26:34594) 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849372471 0 0) 0x7fe2780a0460 con 0x7fe278135c40 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 10 monclient(hunting): my global_id is 134405 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe260003570 con 0x7fe278135c40 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151890984 0 0) 0x7fe27804ca10 con 0x7fe278099540 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.543+0000 7fe276ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient(hunting): my global_id is 4956 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2780a0460 con 0x7fe278099540 2024-09-15T03:49:38.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728392337 0 0) 0x7fe27806e600 con 0x7fe2780a0090 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient(hunting): my global_id is 134359 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27804ca10 con 0x7fe2780a0090 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4015987046 0 0) 0x7fe260003570 con 0x7fe278135c40 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 >> v1:172.21.15.122:6792/0 conn(0x7fe2780a0090 legacy=0x7fe27809d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 >> v1:172.21.15.26:6792/0 conn(0x7fe278099540 legacy=0x7fe27809dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 monclient: found mon.noname-e 2024-09-15T03:49:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27809e6d0 con 0x7fe278135c40 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe2700017b0 con 0x7fe278135c40 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: dump: 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 5 monclient: authenticate success, global_id 134405 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe270001a90 con 0x7fe278135c40 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 4 set_mon_vals no callback set 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 1 -- 172.21.15.26:0/880508045 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe270004c30 con 0x7fe278135c40 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe276ffd700 10 monclient: dump: 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.564 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: shutdown 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- 172.21.15.26:0/880508045 >> v1:172.21.15.26:6793/0 conn(0x7fe278135c40 legacy=0x7fe278138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- 172.21.15.26:0/880508045 shutdown_connections 2024-09-15T03:49:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- 172.21.15.26:0/880508045 >> 172.21.15.26:0/880508045 conn(0x7fe2780a3430 msgr2=0x7fe2780a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- 172.21.15.26:0/880508045 shutdown_connections 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- 172.21.15.26:0/880508045 wait complete. 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: monmap: 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 Processor -- start 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- start start 2024-09-15T03:49:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: init 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient: _add_conns ranks=[6,4,2,1,8,7,5,0,3] 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient(hunting): picked mon.noname-d con 0x7fe278099540 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient(hunting): picked mon.noname-c con 0x7fe2780a0090 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient(hunting): picked mon.noname-b con 0x7fe278135c40 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe278130d70 con 0x7fe278135c40 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe278097bf0 con 0x7fe2780a0090 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe278097e20 con 0x7fe278099540 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27d587700 10 monclient(hunting): authenticate will time out at 1122.978866s 2024-09-15T03:49:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27cd86700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe278135c40 0x7fe278148100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:50258/0 (socket says 172.21.15.26:50258) 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe27cd86700 1 -- 172.21.15.26:0/557308963 learned_addr learned my addr 172.21.15.26:0/557308963 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe2777fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fe2780a0090 0x7fe27806ca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43102/0 (socket says 172.21.15.26:43102) 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970545432 0 0) 0x7fe278130d70 con 0x7fe278135c40 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): my global_id is 134426 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe264003570 con 0x7fe278135c40 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295288648 0 0) 0x7fe278097bf0 con 0x7fe2780a0090 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): my global_id is 174163 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe278130d70 con 0x7fe2780a0090 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968177697 0 0) 0x7fe278097e20 con 0x7fe278099540 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 10 monclient(hunting): my global_id is 4965 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe278097bf0 con 0x7fe278099540 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1138470749 0 0) 0x7fe264003570 con 0x7fe278135c40 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe278097e20 con 0x7fe278135c40 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.547+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2884800192 0 0) 0x7fe278097bf0 con 0x7fe278099540 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe264003570 con 0x7fe278099540 2024-09-15T03:49:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2698558897 0 0) 0x7fe278130d70 con 0x7fe2780a0090 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe278097bf0 con 0x7fe2780a0090 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe270004b70 con 0x7fe278135c40 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): dump: 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.572 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe26c0019e0 con 0x7fe278099540 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): dump: 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe268001890 con 0x7fe2780a0090 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): dump: 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3463385933 0 0) 0x7fe264003570 con 0x7fe278099540 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.26:6791/0 conn(0x7fe2780a0090 legacy=0x7fe27806ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.26:6790/0 conn(0x7fe278135c40 legacy=0x7fe278148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 monclient: found mon.d 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27806d150 con 0x7fe278099540 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 5 monclient: authenticate success, global_id 4965 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 10 monclient: _renew_subs 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278132320 con 0x7fe278099540 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 10 monclient: _renew_subs 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe278130770 con 0x7fe278099540 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe26c001b80 con 0x7fe278099540 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe26c002030 con 0x7fe278099540 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: dump: 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe2767fc700 4 set_mon_vals no callback set 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe258006590 con 0x7fe258002840 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe26c002210 con 0x7fe278099540 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 3991+0+0 (unknown 2894827671 0 0) 0x7fe26c0412c0 con 0x7fe278099540 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378915324 0 0) 0x7fe258006590 con 0x7fe258002840 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient(hunting): my global_id is 134263 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe264003570 con 0x7fe258002840 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe27cd86700 10 monclient: get_auth_request con 0x7fe2640382d0 auth_method 0 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1838583472 0 0) 0x7fe264003570 con 0x7fe258002840 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe258006590 con 0x7fe258002840 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe268004ef0 con 0x7fe258002840 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 10 monclient: dump: 2024-09-15T03:49:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4020214379 0 0) 0x7fe258006590 con 0x7fe258002840 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.551+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe25800a130 con 0x7fe258002840 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.555+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe268007200 con 0x7fe258002840 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.555+0000 7fe25ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.555+0000 7fe25ffff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.555+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.122:6789/0 conn(0x7fe258002840 legacy=0x7fe258004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.555+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.122:6789/0 conn(0x7fe258002840 legacy=0x7fe258004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:38.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe258007520 con 0x7fe2580068d0 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089801357 0 0) 0x7fe258007520 con 0x7fe2580068d0 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient(hunting): my global_id is 134272 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe258006590 con 0x7fe2580068d0 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3740524531 0 0) 0x7fe258006590 con 0x7fe2580068d0 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe258007520 con 0x7fe2580068d0 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe2680059d0 con 0x7fe2580068d0 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient: dump: 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.588 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1323935569 0 0) 0x7fe258007520 con 0x7fe2580068d0 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe258008760 con 0x7fe2580068d0 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1727409198) 0x7fe268006dd0 con 0x7fe2580068d0 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.122:6789/0 conn(0x7fe2580068d0 legacy=0x7fe258006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe25ffff700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.122:6789/0 conn(0x7fe2580068d0 legacy=0x7fe258006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.590 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:38.591 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:49:38.592 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.593 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:38.593 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:38.593 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.593 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.593 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.593 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.594 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.594 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.594 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.594 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.594 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.594 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.595 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:38.596 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:38.597 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.598 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:38.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.600 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.603 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.605 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.612 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.612 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.612 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.612 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:38.613 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:38.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.618 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.618 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:38.618 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.122:6812/20895 conn(0x7fe2640382d0 legacy=0x7fe26403a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 10 monclient: shutdown 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 >> v1:172.21.15.26:6792/0 conn(0x7fe278099540 legacy=0x7fe27806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27cd86700 1 -- 172.21.15.26:0/557308963 reap_dead start 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27cd86700 1 -- 172.21.15.26:0/557308963 reap_dead start 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 shutdown_connections 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 >> 172.21.15.26:0/557308963 conn(0x7fe2780a3430 msgr2=0x7fe27809cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 shutdown_connections 2024-09-15T03:49:38.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.563+0000 7fe27d587700 1 -- 172.21.15.26:0/557308963 wait complete. 2024-09-15T03:49:38.621 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:38.621 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:49:38.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:928: finishing write tid 1 to smithi12223906-32 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:928: finishing write tid 2 to smithi12223906-32 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:926: done (3 left) 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:927: done (2 left) 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:930: write oid 30 current snap is 103 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:929: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:49:38.651 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 282 (ObjNum 81 snap 0 seq_num 1) dirty dne 2024-09-15T03:49:38.652 INFO:tasks.rados.rados.0.smithi122.stdout:928: finishing write tid 3 to smithi12223906-32 2024-09-15T03:49:38.652 INFO:tasks.rados.rados.0.smithi122.stdout:930: seq_num 359 ranges {433713=482982,1424565=609148} 2024-09-15T03:49:38.655 INFO:tasks.rados.rados.0.smithi122.stdout:930: writing smithi12223906-30 from 433713 to 916695 tid 1 2024-09-15T03:49:38.662 INFO:tasks.rados.rados.0.smithi122.stdout:930: writing smithi12223906-30 from 1424565 to 2033713 tid 2 2024-09-15T03:49:38.662 INFO:tasks.rados.rados.0.smithi122.stdout:929: done (2 left) 2024-09-15T03:49:38.662 INFO:tasks.rados.rados.0.smithi122.stdout:931: delete oid 44 current snap is 103 2024-09-15T03:49:38.662 INFO:tasks.rados.rados.0.smithi122.stdout:928: finishing write tid 4 to smithi12223906-32 2024-09-15T03:49:38.662 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 499 (ObjNum 358 snap 103 seq_num 358) dirty exists 2024-09-15T03:49:38.662 INFO:tasks.rados.rados.0.smithi122.stdout:928: left oid 32 (ObjNum 358 snap 103 seq_num 358) 2024-09-15T03:49:38.665 INFO:tasks.rados.rados.0.smithi122.stdout:928: done (2 left) 2024-09-15T03:49:38.665 INFO:tasks.rados.rados.0.smithi122.stdout:931: done (1 left) 2024-09-15T03:49:38.665 INFO:tasks.rados.rados.0.smithi122.stdout:932: snap_remove snap 98 2024-09-15T03:49:38.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.739+0000 7f37e6a85700 10 monclient: get_monmap_and_config 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.739+0000 7f37e6a85700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.739+0000 7f37e6a85700 10 monclient: monmap: 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:38.743126+0000 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:38.743126+0000 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.744 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.745 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 1 Processor -- start 2024-09-15T03:49:38.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 1 -- start start 2024-09-15T03:49:38.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient: init 2024-09-15T03:49:38.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient: _add_conns ranks=[5,3,7,8,4,1,6,0,2] 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient(hunting): picked mon.noname-h con 0x7f37e012f270 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient(hunting): picked mon.noname-g con 0x7f37e006d5b0 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient(hunting): picked mon.noname-i con 0x7f37e01353b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e004ca10 con 0x7f37e006d5b0 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e006aab0 con 0x7f37e012f270 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e009f050 con 0x7f37e01353b0 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e6a85700 10 monclient(hunting): authenticate will time out at 1123.174422s 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e4821700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f37e012f270 0x7f37e012f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:33242/0 (socket says 172.21.15.26:33242) 2024-09-15T03:49:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e5022700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f37e01353b0 0x7f37e0137790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:59604/0 (socket says 172.21.15.26:59604) 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37e4821700 1 -- 172.21.15.26:0/2117457207 learned_addr learned my addr 172.21.15.26:0/2117457207 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37dffff700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f37e006d5b0 0x7f37e006d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:52472/0 (socket says 172.21.15.26:52472) 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2442631956 0 0) 0x7f37e004ca10 con 0x7f37e006d5b0 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): my global_id is 4827 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c8003570 con 0x7f37e006d5b0 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430324540 0 0) 0x7f37e009f050 con 0x7f37e01353b0 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): my global_id is 134368 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e004ca10 con 0x7f37e01353b0 2024-09-15T03:49:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951816396 0 0) 0x7f37e006aab0 con 0x7f37e012f270 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient(hunting): my global_id is 4928 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e009f050 con 0x7f37e012f270 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 255362808 0 0) 0x7f37c8003570 con 0x7f37e006d5b0 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 >> v1:172.21.15.122:6792/0 conn(0x7f37e01353b0 legacy=0x7f37e0137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 >> v1:172.21.15.122:6791/0 conn(0x7f37e012f270 legacy=0x7f37e012f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 monclient: found mon.noname-g 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e006a7d0 con 0x7f37e006d5b0 2024-09-15T03:49:38.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.743+0000 7f37df7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 5 monclient: authenticate success, global_id 4827 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d40016e0 con 0x7f37e006d5b0 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: dump: 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.755 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.756 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37d40019c0 con 0x7f37e006d5b0 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 4 set_mon_vals no callback set 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 1 -- 172.21.15.26:0/2117457207 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d4004b70 con 0x7f37e006d5b0 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37df7fe700 10 monclient: dump: 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 10 monclient: shutdown 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 -- 172.21.15.26:0/2117457207 >> v1:172.21.15.122:6790/0 conn(0x7f37e006d5b0 legacy=0x7f37e006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 -- 172.21.15.26:0/2117457207 shutdown_connections 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 -- 172.21.15.26:0/2117457207 >> 172.21.15.26:0/2117457207 conn(0x7f37e0063e10 msgr2=0x7f37e00a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 -- 172.21.15.26:0/2117457207 shutdown_connections 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 -- 172.21.15.26:0/2117457207 wait complete. 2024-09-15T03:49:38.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 10 monclient: monmap: 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.760 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.761 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.761 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.761 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.761 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.761 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.761 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 Processor -- start 2024-09-15T03:49:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 1 -- start start 2024-09-15T03:49:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.747+0000 7f37e6a85700 10 monclient: init 2024-09-15T03:49:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient: _add_conns ranks=[8,1,7,4,3,5,2,0,6] 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient(hunting): picked mon.noname-e con 0x7f37e006d5b0 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient(hunting): picked mon.noname-f con 0x7f37e012f270 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient(hunting): picked mon.noname-i con 0x7f37e01353b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e009aa40 con 0x7f37e012f270 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e0069d50 con 0x7f37e01353b0 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e009cd70 con 0x7f37e006d5b0 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient(hunting): authenticate will time out at 1123.180875s 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e4821700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f37e006d5b0 0x7f37e013ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56752/0 (socket says 172.21.15.26:56752) 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e4821700 1 -- 172.21.15.26:0/3546436398 learned_addr learned my addr 172.21.15.26:0/3546436398 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705682839 0 0) 0x7f37e009cd70 con 0x7f37e006d5b0 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): my global_id is 134414 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37cc003570 con 0x7f37e006d5b0 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238494638 0 0) 0x7f37e0069d50 con 0x7f37e01353b0 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): my global_id is 134377 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e009cd70 con 0x7f37e01353b0 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759317719 0 0) 0x7f37e009aa40 con 0x7f37e012f270 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): my global_id is 134281 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e0069d50 con 0x7f37e012f270 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3672926385 0 0) 0x7f37cc003570 con 0x7f37e006d5b0 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37e009aa40 con 0x7f37e006d5b0 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d00019e0 con 0x7f37e006d5b0 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): dump: 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.770 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3590150438 0 0) 0x7f37e009cd70 con 0x7f37e01353b0 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37cc003570 con 0x7f37e01353b0 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d8001d00 con 0x7f37e01353b0 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient(hunting): dump: 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2979918194 0 0) 0x7f37e009aa40 con 0x7f37e006d5b0 2024-09-15T03:49:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6792/0 conn(0x7f37e01353b0 legacy=0x7f37e0146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6789/0 conn(0x7f37e012f270 legacy=0x7f37e0142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 monclient: found mon.e 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e009bff0 con 0x7f37e006d5b0 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 5 monclient: authenticate success, global_id 134414 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient: _renew_subs 2024-09-15T03:49:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e012e6e0 con 0x7f37e006d5b0 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient: _renew_subs 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e013e500 con 0x7f37e006d5b0 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37d0001d90 con 0x7f37e006d5b0 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d0004ee0 con 0x7f37e006d5b0 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37deffd700 4 set_mon_vals no callback set 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.751+0000 7f37dcff9700 10 monclient: dump: 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.776 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37e6a85700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f37d00050c0 con 0x7f37e006d5b0 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f37c0006590 con 0x7f37c0002840 2024-09-15T03:49:38.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 3991+0+0 (unknown 2894827671 0 0) 0x7f37d0043ff0 con 0x7f37e006d5b0 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37e5022700 10 monclient: get_auth_request con 0x7f37cc037bd0 auth_method 0 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16124595 0 0) 0x7f37c0006590 con 0x7f37c0002840 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient(hunting): my global_id is 4836 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e009aa40 con 0x7f37c0002840 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 656766943 0 0) 0x7f37e009aa40 con 0x7f37c0002840 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37c0006590 con 0x7f37c0002840 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d4001660 con 0x7f37c0002840 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:38.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient: dump: 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.779 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3113680793 0 0) 0x7f37c0006590 con 0x7f37c0002840 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37c000a130 con 0x7f37c0002840 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f37d40053a0 con 0x7f37c0002840 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6790/0 conn(0x7f37c0002840 legacy=0x7f37c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.755+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6790/0 conn(0x7f37c0002840 legacy=0x7f37c0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37e6a85700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:38.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f37c0007520 con 0x7f37c00068d0 2024-09-15T03:49:38.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954723736 0 0) 0x7f37c0007520 con 0x7f37c00068d0 2024-09-15T03:49:38.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient(hunting): my global_id is 4845 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c0006590 con 0x7f37c00068d0 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 965114035 0 0) 0x7f37c0006590 con 0x7f37c00068d0 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37c0007520 con 0x7f37c00068d0 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37d4005c90 con 0x7f37c00068d0 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient: dump: 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.783 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.784 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3174517568 0 0) 0x7f37c0007520 con 0x7f37c00068d0 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f37c0008760 con 0x7f37c00068d0 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1515836672) 0x7f37d4008390 con 0x7f37c00068d0 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:38.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6790/0 conn(0x7f37c00068d0 legacy=0x7f37c0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.767+0000 7f37dcff9700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6790/0 conn(0x7f37c00068d0 legacy=0x7f37c0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:38.786 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:38.787 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.788 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.789 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:38.790 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.791 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.792 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.794 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.795 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.796 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.798 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.799 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.800 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.802 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.804 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:38.805 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.808 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.809 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.810 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 6 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:38.811 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:38.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.122:6812/20895 conn(0x7f37cc037bd0 legacy=0x7f37cc03a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 10 monclient: shutdown 2024-09-15T03:49:38.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 >> v1:172.21.15.26:6793/0 conn(0x7f37e006d5b0 legacy=0x7f37e013ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e5022700 1 -- 172.21.15.26:0/3546436398 reap_dead start 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e5022700 1 -- 172.21.15.26:0/3546436398 reap_dead start 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 shutdown_connections 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 >> 172.21.15.26:0/3546436398 conn(0x7f37e0063e10 msgr2=0x7f37e006c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 shutdown_connections 2024-09-15T03:49:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.771+0000 7f37e6a85700 1 -- 172.21.15.26:0/3546436398 wait complete. 2024-09-15T03:49:38.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:38.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:49:38.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:49:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient: get_monmap_and_config 2024-09-15T03:49:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient: monmap: 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:38.929116+0000 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:38.929116+0000 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.931 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 1 Processor -- start 2024-09-15T03:49:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 1 -- start start 2024-09-15T03:49:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient: init 2024-09-15T03:49:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient: _add_conns ranks=[6,7,1,4,0,2,5,8,3] 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient(hunting): picked mon.noname-d con 0x7f29c81335e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient(hunting): picked mon.noname-i con 0x7f29c809fc80 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient(hunting): picked mon.noname-f con 0x7f29c80a0050 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c804ca10 con 0x7f29c80a0050 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c806eab0 con 0x7f29c81335e0 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80a1330 con 0x7f29c809fc80 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c7fff700 10 monclient(hunting): authenticate will time out at 1123.358819s 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c6ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f29c81335e0 0x7f29c8132da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34612/0 (socket says 172.21.15.26:34612) 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c6ffd700 1 -- 172.21.15.26:0/74615267 learned_addr learned my addr 172.21.15.26:0/74615267 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367711025 0 0) 0x7f29c806eab0 con 0x7f29c81335e0 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): my global_id is 4974 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b0003570 con 0x7f29c81335e0 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141186656 0 0) 0x7f29c80a1330 con 0x7f29c809fc80 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): my global_id is 134386 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c806eab0 con 0x7f29c809fc80 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268692021 0 0) 0x7f29c804ca10 con 0x7f29c80a0050 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 10 monclient(hunting): my global_id is 134290 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c80a1330 con 0x7f29c80a0050 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.927+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2618799967 0 0) 0x7f29b0003570 con 0x7f29c81335e0 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 >> v1:172.21.15.122:6792/0 conn(0x7f29c809fc80 legacy=0x7f29c8137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 >> v1:172.21.15.122:6789/0 conn(0x7f29c80a0050 legacy=0x7f29c813b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 monclient: found mon.noname-d 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80a10d0 con 0x7f29c81335e0 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 5 monclient: authenticate success, global_id 4974 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29c0001b30 con 0x7f29c81335e0 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: dump: 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.939 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.940 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f29c0001da0 con 0x7f29c81335e0 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 4 set_mon_vals no callback set 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 1 -- 172.21.15.26:0/74615267 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29c0004fb0 con 0x7f29c81335e0 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c5ffb700 10 monclient: dump: 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: shutdown 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- 172.21.15.26:0/74615267 >> v1:172.21.15.26:6792/0 conn(0x7f29c81335e0 legacy=0x7f29c8132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- 172.21.15.26:0/74615267 shutdown_connections 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- 172.21.15.26:0/74615267 >> 172.21.15.26:0/74615267 conn(0x7f29c806e6e0 msgr2=0x7f29c80a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- 172.21.15.26:0/74615267 shutdown_connections 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- 172.21.15.26:0/74615267 wait complete. 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: build_initial_monmap 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: monmap: 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:38.943 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 Processor -- start 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- start start 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: init 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient: _add_conns ranks=[0,7,2,8,1,6,5,3,4] 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient(hunting): picked mon.noname-a con 0x7f29c809fc80 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient(hunting): picked mon.noname-i con 0x7f29c80a0050 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient(hunting): picked mon.noname-b con 0x7f29c81335e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c80a1b70 con 0x7f29c809fc80 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c8120030 con 0x7f29c81335e0 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c809cf10 con 0x7f29c80a0050 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c7fff700 10 monclient(hunting): authenticate will time out at 1123.363931s 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c6ffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29c809fc80 0x7f29c812fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47306/0 (socket says 172.21.15.26:47306) 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c6ffd700 1 -- 172.21.15.26:0/2231953291 learned_addr learned my addr 172.21.15.26:0/2231953291 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.931+0000 7f29c77fe700 1 --1- 172.21.15.26:0/2231953291 >> v1:172.21.15.26:6790/0 conn(0x7f29c81335e0 0x7f29c814a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:50262/0 (socket says 172.21.15.26:50262) 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386574214 0 0) 0x7f29c80a1b70 con 0x7f29c809fc80 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): my global_id is 4779 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b4003570 con 0x7f29c809fc80 2024-09-15T03:49:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480130707 0 0) 0x7f29c809cf10 con 0x7f29c80a0050 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): my global_id is 134395 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c80a1b70 con 0x7f29c80a0050 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257484367 0 0) 0x7f29c8120030 con 0x7f29c81335e0 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): my global_id is 134435 2024-09-15T03:49:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c809cf10 con 0x7f29c81335e0 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3323125587 0 0) 0x7f29b4003570 con 0x7f29c809fc80 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29c8120030 con 0x7f29c809fc80 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29c0001bb0 con 0x7f29c809fc80 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): dump: 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1321220959 0 0) 0x7f29c809cf10 con 0x7f29c81335e0 2024-09-15T03:49:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29b4003570 con 0x7f29c81335e0 2024-09-15T03:49:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29b8001a20 con 0x7f29c81335e0 2024-09-15T03:49:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): dump: 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3599330171 0 0) 0x7f29c80a1b70 con 0x7f29c80a0050 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29c809cf10 con 0x7f29c80a0050 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29bc0019e0 con 0x7f29c80a0050 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): dump: 2024-09-15T03:49:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2066096113 0 0) 0x7f29c8120030 con 0x7f29c809fc80 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.122:6792/0 conn(0x7f29c80a0050 legacy=0x7f29c8146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.26:6790/0 conn(0x7f29c81335e0 legacy=0x7f29c814a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 monclient: found mon.a 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c811fd50 con 0x7f29c809fc80 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 5 monclient: authenticate success, global_id 4779 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 10 monclient: _renew_subs 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c811ee60 con 0x7f29c809fc80 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 10 monclient: _renew_subs 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c8133ec0 con 0x7f29c809fc80 2024-09-15T03:49:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f29c00051e0 con 0x7f29c809fc80 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29c0005690 con 0x7f29c809fc80 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c57fa700 4 set_mon_vals no callback set 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient: dump: 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f29a8006590 con 0x7f29a8002840 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f29c0010050 con 0x7f29c809fc80 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 3991+0+0 (unknown 2894827671 0 0) 0x7f29c0044390 con 0x7f29c809fc80 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955671594 0 0) 0x7f29a8006590 con 0x7f29a8002840 2024-09-15T03:49:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 10 monclient(hunting): my global_id is 4937 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c8120030 con 0x7f29a8002840 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.935+0000 7f29c77fe700 10 monclient: get_auth_request con 0x7f29b4038060 auth_method 0 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 214410592 0 0) 0x7f29c8120030 con 0x7f29a8002840 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29a8006590 con 0x7f29a8002840 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29bc0021b0 con 0x7f29a8002840 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 10 monclient: dump: 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2919127216 0 0) 0x7f29a8006590 con 0x7f29a8002840 2024-09-15T03:49:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29a800a130 con 0x7f29a8002840 2024-09-15T03:49:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f29bc0044b0 con 0x7f29a8002840 2024-09-15T03:49:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.122:6791/0 conn(0x7f29a8002840 legacy=0x7f29a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.939+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.122:6791/0 conn(0x7f29a8002840 legacy=0x7f29a8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.947+0000 7f29c7fff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.947+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f29a8007520 con 0x7f29a80068d0 2024-09-15T03:49:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516076758 0 0) 0x7f29a8007520 con 0x7f29a80068d0 2024-09-15T03:49:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient(hunting): my global_id is 4946 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a8006590 con 0x7f29a80068d0 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2405278963 0 0) 0x7f29a8006590 con 0x7f29a80068d0 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29a8007520 con 0x7f29a80068d0 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f29bc002c00 con 0x7f29a80068d0 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient: dump: 2024-09-15T03:49:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 686710044 0 0) 0x7f29a8007520 con 0x7f29a80068d0 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f29a8008760 con 0x7f29a80068d0 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1393434268) 0x7f29bc004210 con 0x7f29a80068d0 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.122:6791/0 conn(0x7f29a80068d0 legacy=0x7f29a8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29aeffd700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.122:6791/0 conn(0x7f29a80068d0 legacy=0x7f29a8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:38.970 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:38.971 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.972 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.973 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:38.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:38.975 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:38.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.977 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:38.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.981 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:38.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.985 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:38.987 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.988 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:38.989 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:38.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:38.990 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:38.990 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:38.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.122:6812/20895 conn(0x7f29b4038060 legacy=0x7f29b403a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 10 monclient: shutdown 2024-09-15T03:49:38.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:38.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 >> v1:172.21.15.26:6789/0 conn(0x7f29c809fc80 legacy=0x7f29c812fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:38.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c77fe700 1 -- 172.21.15.26:0/2231953291 reap_dead start 2024-09-15T03:49:38.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c77fe700 1 -- 172.21.15.26:0/2231953291 reap_dead start 2024-09-15T03:49:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 shutdown_connections 2024-09-15T03:49:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 >> 172.21.15.26:0/2231953291 conn(0x7f29c806e6e0 msgr2=0x7f29c8130fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 shutdown_connections 2024-09-15T03:49:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:38.951+0000 7f29c7fff700 1 -- 172.21.15.26:0/2231953291 wait complete. 2024-09-15T03:49:38.992 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:38.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:49:38.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:49:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.107+0000 7f55817f7700 10 monclient: get_monmap_and_config 2024-09-15T03:49:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.107+0000 7f55817f7700 10 monclient: build_initial_monmap 2024-09-15T03:49:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.107+0000 7f55817f7700 10 monclient: monmap: 2024-09-15T03:49:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:39.110922+0000 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:39.110922+0000 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:39.113 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 1 Processor -- start 2024-09-15T03:49:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 1 -- start start 2024-09-15T03:49:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient: init 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient: _add_conns ranks=[6,8,3,1,2,7,4,5,0] 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient(hunting): picked mon.noname-d con 0x7f557c070de0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient(hunting): picked mon.noname-e con 0x7f557c06cdc0 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient(hunting): picked mon.noname-g con 0x7f557c137060 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c04ca10 con 0x7f557c137060 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c06baa0 con 0x7f557c070de0 2024-09-15T03:49:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c06c3e0 con 0x7f557c06cdc0 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f557affd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f557c070de0 0x7f557c09d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34614/0 (socket says 172.21.15.26:34614) 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f557affd700 1 -- 172.21.15.26:0/4257578593 learned_addr learned my addr 172.21.15.26:0/4257578593 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient(hunting): authenticate will time out at 1123.542207s 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325528957 0 0) 0x7f557c06baa0 con 0x7f557c070de0 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): my global_id is 4983 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5564003570 con 0x7f557c070de0 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11449212 0 0) 0x7f557c06c3e0 con 0x7f557c06cdc0 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): my global_id is 134423 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c06baa0 con 0x7f557c06cdc0 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300128046 0 0) 0x7f557c04ca10 con 0x7f557c137060 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient(hunting): my global_id is 4854 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c06c3e0 con 0x7f557c137060 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703272113 0 0) 0x7f5564003570 con 0x7f557c070de0 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 >> v1:172.21.15.26:6793/0 conn(0x7f557c06cdc0 legacy=0x7f557c09d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 >> v1:172.21.15.122:6790/0 conn(0x7f557c137060 legacy=0x7f557c139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 monclient: found mon.noname-d 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:49:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0711b0 con 0x7f557c070de0 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f556c0017b0 con 0x7f557c070de0 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: dump: 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 5 monclient: authenticate success, global_id 4983 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:39.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f556c001a90 con 0x7f557c070de0 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 1 -- 172.21.15.26:0/4257578593 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f556c004c30 con 0x7f557c070de0 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f55817f7700 4 set_mon_vals no callback set 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.111+0000 7f5579ffb700 10 monclient: dump: 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.122 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient: shutdown 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- 172.21.15.26:0/4257578593 >> v1:172.21.15.26:6792/0 conn(0x7f557c070de0 legacy=0x7f557c09d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- 172.21.15.26:0/4257578593 shutdown_connections 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- 172.21.15.26:0/4257578593 >> 172.21.15.26:0/4257578593 conn(0x7f557c063e10 msgr2=0x7f557c0a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- 172.21.15.26:0/4257578593 shutdown_connections 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- 172.21.15.26:0/4257578593 wait complete. 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient: build_initial_monmap 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient: monmap: 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 Processor -- start 2024-09-15T03:49:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- start start 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient: init 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient: _add_conns ranks=[3,1,7,0,6,4,8,5,2] 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient(hunting): picked mon.noname-g con 0x7f557c06cdc0 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient(hunting): picked mon.noname-f con 0x7f557c070de0 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 10 monclient(hunting): picked mon.noname-i con 0x7f557c137060 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.115+0000 7f55817f7700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c11eff0 con 0x7f557c070de0 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c11ed10 con 0x7f557c06cdc0 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c11f6f0 con 0x7f557c137060 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 monclient(hunting): authenticate will time out at 1123.548069s 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f557affd700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f557c06cdc0 0x7f557c11b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:52498/0 (socket says 172.21.15.26:52498) 2024-09-15T03:49:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f557affd700 1 -- 172.21.15.26:0/2722204557 learned_addr learned my addr 172.21.15.26:0/2722204557 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f557a7fc700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f557c070de0 0x7f557c11bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:55384/0 (socket says 172.21.15.26:55384) 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756060033 0 0) 0x7f557c11f6f0 con 0x7f557c137060 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): my global_id is 134404 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5568003570 con 0x7f557c137060 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984173994 0 0) 0x7f557c11ed10 con 0x7f557c06cdc0 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): my global_id is 4863 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c11f6f0 con 0x7f557c06cdc0 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891879155 0 0) 0x7f557c11eff0 con 0x7f557c070de0 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): my global_id is 134299 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c11ed10 con 0x7f557c070de0 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2750937494 0 0) 0x7f5568003570 con 0x7f557c137060 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f557c11eff0 con 0x7f557c137060 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5574001d40 con 0x7f557c137060 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.134 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 924587959 0 0) 0x7f557c11f6f0 con 0x7f557c06cdc0 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5568003570 con 0x7f557c06cdc0 2024-09-15T03:49:39.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f556c004b70 con 0x7f557c06cdc0 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2268185631 0 0) 0x7f557c11ed10 con 0x7f557c070de0 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f557c11f6f0 con 0x7f557c070de0 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f55700019e0 con 0x7f557c070de0 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.137 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.138 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2976952197 0 0) 0x7f557c11eff0 con 0x7f557c137060 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6790/0 conn(0x7f557c06cdc0 legacy=0x7f557c11b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6789/0 conn(0x7f557c070de0 legacy=0x7f557c11bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 monclient: found mon.i 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c11ea30 con 0x7f557c137060 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 5 monclient: authenticate success, global_id 134404 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 monclient: _renew_subs 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:49:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c131f90 con 0x7f557c137060 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 monclient: _renew_subs 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c11db40 con 0x7f557c137060 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5574001fe0 con 0x7f557c137060 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5574002510 con 0x7f557c137060 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55797fa700 4 set_mon_vals no callback set 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f5562ffd700 10 monclient: dump: 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.119+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f555c006590 con 0x7f555c002840 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f55740026f0 con 0x7f557c137060 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(226..226 src has 1..226) v4 ==== 3991+0+0 (unknown 2894827671 0 0) 0x7f55740418f0 con 0x7f557c137060 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233917202 0 0) 0x7f555c006590 con 0x7f555c002840 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient(hunting): my global_id is 134413 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c11eff0 con 0x7f555c002840 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f557a7fc700 10 monclient: get_auth_request con 0x7f5568038010 auth_method 0 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 720339652 0 0) 0x7f557c11eff0 con 0x7f555c002840 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f555c006590 con 0x7f555c002840 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f556c001860 con 0x7f555c002840 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient: dump: 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.143 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2419732822 0 0) 0x7f555c006590 con 0x7f555c002840 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f555c00a130 con 0x7f555c002840 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f556c0052b0 con 0x7f555c002840 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6792/0 conn(0x7f555c002840 legacy=0x7f555c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.123+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6792/0 conn(0x7f555c002840 legacy=0x7f555c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f555c009640 con 0x7f555c0090b0 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536867661 0 0) 0x7f555c009640 con 0x7f555c0090b0 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient(hunting): my global_id is 134422 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c006590 con 0x7f555c0090b0 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2254434107 0 0) 0x7f555c006590 con 0x7f555c0090b0 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f555c009640 con 0x7f555c0090b0 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f556c005bc0 con 0x7f555c0090b0 2024-09-15T03:49:39.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient: dump: 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.147 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1496897617 0 0) 0x7f555c009640 con 0x7f555c0090b0 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f555c0098f0 con 0x7f555c0090b0 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 72587799) 0x7f556c0082a0 con 0x7f555c0090b0 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6792/0 conn(0x7f555c0090b0 legacy=0x7f555c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f5562ffd700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6792/0 conn(0x7f555c0090b0 legacy=0x7f555c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:39.149 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 2, 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:39.150 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:39.151 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:39.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:39.153 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.155 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.156 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:39.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.160 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.162 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:39.163 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.164 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.165 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.166 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:39.167 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6812/20895 conn(0x7f5568038010 legacy=0x7f556803a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 10 monclient: shutdown 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 >> v1:172.21.15.122:6792/0 conn(0x7f557c137060 legacy=0x7f557c11c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f557b7fe700 1 -- 172.21.15.26:0/2722204557 reap_dead start 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f557b7fe700 1 -- 172.21.15.26:0/2722204557 reap_dead start 2024-09-15T03:49:39.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 shutdown_connections 2024-09-15T03:49:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 >> 172.21.15.26:0/2722204557 conn(0x7f557c063e10 msgr2=0x7f557c1367e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 shutdown_connections 2024-09-15T03:49:39.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.135+0000 7f55817f7700 1 -- 172.21.15.26:0/2722204557 wait complete. 2024-09-15T03:49:39.170 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:39.170 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T03:49:39.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T03:49:39.176 INFO:tasks.rados.rados.0.smithi122.stdout:932: done (1 left) 2024-09-15T03:49:39.176 INFO:tasks.rados.rados.0.smithi122.stdout:933: write oid 7 current snap is 103 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:930: finishing write tid 1 to smithi12223906-30 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:930: finishing write tid 2 to smithi12223906-30 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:930: finishing write tid 3 to smithi12223906-30 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:930: finishing write tid 4 to smithi12223906-30 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 161 (ObjNum 359 snap 103 seq_num 359) dirty exists 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:930: left oid 30 (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:49:39.177 INFO:tasks.rados.rados.0.smithi122.stdout:933: seq_num 361 ranges {467327=633480,1825176=242151} 2024-09-15T03:49:39.183 INFO:tasks.rados.rados.0.smithi122.stdout:933: writing smithi12223906-7 from 467327 to 1100807 tid 1 2024-09-15T03:49:39.189 INFO:tasks.rados.rados.0.smithi122.stdout:933: writing smithi12223906-7 from 1825176 to 2067327 tid 2 2024-09-15T03:49:39.190 INFO:tasks.rados.rados.0.smithi122.stdout:930: done (1 left) 2024-09-15T03:49:39.190 INFO:tasks.rados.rados.0.smithi122.stdout:934: copy_from oid 6 from oid 9 current snap is 103 2024-09-15T03:49:39.190 INFO:tasks.rados.rados.0.smithi122.stdout:935: delete oid 5 current snap is 103 2024-09-15T03:49:39.193 INFO:tasks.rados.rados.0.smithi122.stdout:935: done (2 left) 2024-09-15T03:49:39.193 INFO:tasks.rados.rados.0.smithi122.stdout:936: delete oid 39 current snap is 103 2024-09-15T03:49:39.194 INFO:tasks.rados.rados.0.smithi122.stdout:933: finishing write tid 1 to smithi12223906-7 2024-09-15T03:49:39.196 INFO:tasks.rados.rados.0.smithi122.stdout:933: finishing write tid 2 to smithi12223906-7 2024-09-15T03:49:39.196 INFO:tasks.rados.rados.0.smithi122.stdout:933: finishing write tid 3 to smithi12223906-7 2024-09-15T03:49:39.197 INFO:tasks.rados.rados.0.smithi122.stdout:933: finishing write tid 4 to smithi12223906-7 2024-09-15T03:49:39.198 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 204 (ObjNum 361 snap 103 seq_num 361) dirty exists 2024-09-15T03:49:39.198 INFO:tasks.rados.rados.0.smithi122.stdout:933: left oid 7 (ObjNum 361 snap 103 seq_num 361) 2024-09-15T03:49:39.198 INFO:tasks.rados.rados.0.smithi122.stdout:933: done (2 left) 2024-09-15T03:49:39.199 INFO:tasks.rados.rados.0.smithi122.stdout:936: done (1 left) 2024-09-15T03:49:39.199 INFO:tasks.rados.rados.0.smithi122.stdout:937: read oid 30 snap -1 2024-09-15T03:49:39.199 INFO:tasks.rados.rados.0.smithi122.stdout:937: expect (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:49:39.199 INFO:tasks.rados.rados.0.smithi122.stdout:938: write (excl) oid 13 current snap is 103 2024-09-15T03:49:39.199 INFO:tasks.rados.rados.0.smithi122.stdout:938: seq_num 364 ranges {517748=459227,1600622=406110} 2024-09-15T03:49:39.205 INFO:tasks.rados.rados.0.smithi122.stdout:938: writing smithi12223906-13 from 517748 to 976975 tid 1 2024-09-15T03:49:39.213 INFO:tasks.rados.rados.0.smithi122.stdout:938: writing smithi12223906-13 from 1600622 to 2006732 tid 2 2024-09-15T03:49:39.213 INFO:tasks.rados.rados.0.smithi122.stdout:939: read oid 47 snap -1 2024-09-15T03:49:39.213 INFO:tasks.rados.rados.0.smithi122.stdout:939: expect (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:49:39.213 INFO:tasks.rados.rados.0.smithi122.stdout:940: delete oid 15 current snap is 103 2024-09-15T03:49:39.213 INFO:tasks.rados.rados.0.smithi122.stdout:934: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:49:39.213 INFO:tasks.rados.rados.0.smithi122.stdout:937: expect (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:49:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.275+0000 7f32e7fff700 10 monclient: get_monmap_and_config 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.275+0000 7f32e7fff700 10 monclient: build_initial_monmap 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.275+0000 7f32e7fff700 10 monclient: monmap: 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:39.278033+0000 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:39.278033+0000 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:39.280 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.275+0000 7f32e7fff700 1 Processor -- start 2024-09-15T03:49:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.275+0000 7f32e7fff700 1 -- start start 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.275+0000 7f32e7fff700 10 monclient: init 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient: _add_conns ranks=[8,0,3,7,5,1,6,2,4] 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient(hunting): picked mon.noname-e con 0x7f32e809e070 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient(hunting): picked mon.noname-a con 0x7f32e80987c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient(hunting): picked mon.noname-g con 0x7f32e8136e80 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e804ca10 con 0x7f32e80987c0 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e80994e0 con 0x7f32e8136e80 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e809e9b0 con 0x7f32e809e070 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient(hunting): authenticate will time out at 1123.708659s 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e67fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f32e80987c0 0x7f32e8098b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47316/0 (socket says 172.21.15.26:47316) 2024-09-15T03:49:39.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e67fc700 1 -- 172.21.15.26:0/2579922716 learned_addr learned my addr 172.21.15.26:0/2579922716 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703720148 0 0) 0x7f32e804ca10 con 0x7f32e80987c0 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): my global_id is 4788 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d0003570 con 0x7f32e80987c0 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958796058 0 0) 0x7f32e809e9b0 con 0x7f32e809e070 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): my global_id is 134432 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e804ca10 con 0x7f32e809e070 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177305678 0 0) 0x7f32e80994e0 con 0x7f32e8136e80 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient(hunting): my global_id is 4872 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e809e9b0 con 0x7f32e8136e80 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1501153808 0 0) 0x7f32d0003570 con 0x7f32e80987c0 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 >> v1:172.21.15.26:6793/0 conn(0x7f32e809e070 legacy=0x7f32e809d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 >> v1:172.21.15.122:6790/0 conn(0x7f32e8136e80 legacy=0x7f32e8139260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 monclient: found mon.noname-a 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e809dbd0 con 0x7f32e80987c0 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:39.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 5 monclient: authenticate success, global_id 4788 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32dc0017b0 con 0x7f32e80987c0 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: dump: 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.293 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32dc001a90 con 0x7f32e80987c0 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:39.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 4 set_mon_vals no callback set 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 1 -- 172.21.15.26:0/2579922716 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32dc004c30 con 0x7f32e80987c0 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e5ffb700 10 monclient: dump: 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 10 monclient: shutdown 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- 172.21.15.26:0/2579922716 >> v1:172.21.15.26:6789/0 conn(0x7f32e80987c0 legacy=0x7f32e8098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- 172.21.15.26:0/2579922716 shutdown_connections 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- 172.21.15.26:0/2579922716 >> 172.21.15.26:0/2579922716 conn(0x7f32e80a24e0 msgr2=0x7f32e80a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- 172.21.15.26:0/2579922716 shutdown_connections 2024-09-15T03:49:39.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.279+0000 7f32e7fff700 1 -- 172.21.15.26:0/2579922716 wait complete. 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: build_initial_monmap 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: monmap: 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:39.297 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 Processor -- start 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 -- start start 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: init 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: _add_conns ranks=[4,8,2,1,6,3,0,5,7] 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient(hunting): picked mon.noname-c con 0x7f32e80987c0 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient(hunting): picked mon.noname-e con 0x7f32e809e070 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient(hunting): picked mon.noname-b con 0x7f32e8136e80 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e8130bd0 con 0x7f32e8136e80 2024-09-15T03:49:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e806dfe0 con 0x7f32e80987c0 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e806c920 con 0x7f32e809e070 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient(hunting): authenticate will time out at 1123.713189s 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e6ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f32e80987c0 0x7f32e806ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43110/0 (socket says 172.21.15.26:43110) 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e6ffd700 1 -- 172.21.15.26:0/35696830 learned_addr learned my addr 172.21.15.26:0/35696830 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160527011 0 0) 0x7f32e806dfe0 con 0x7f32e80987c0 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): my global_id is 174172 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d4003570 con 0x7f32e80987c0 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425574333 0 0) 0x7f32e806c920 con 0x7f32e809e070 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): my global_id is 134441 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e806dfe0 con 0x7f32e809e070 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889761189 0 0) 0x7f32e8130bd0 con 0x7f32e8136e80 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): my global_id is 134444 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e806c920 con 0x7f32e8136e80 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2878944856 0 0) 0x7f32d4003570 con 0x7f32e80987c0 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32e8130bd0 con 0x7f32e80987c0 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32d8001d10 con 0x7f32e80987c0 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): dump: 2024-09-15T03:49:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3935432531 0 0) 0x7f32e806c920 con 0x7f32e8136e80 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32d4003570 con 0x7f32e8136e80 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32e0001cc0 con 0x7f32e8136e80 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient(hunting): dump: 2024-09-15T03:49:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4155775488 0 0) 0x7f32e8130bd0 con 0x7f32e80987c0 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 >> v1:172.21.15.26:6793/0 conn(0x7f32e809e070 legacy=0x7f32e806b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 >> v1:172.21.15.26:6790/0 conn(0x7f32e8136e80 legacy=0x7f32e806bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 monclient: found mon.c 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e806dd00 con 0x7f32e80987c0 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 5 monclient: authenticate success, global_id 174172 2024-09-15T03:49:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: _renew_subs 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e806cda0 con 0x7f32e80987c0 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: _renew_subs 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e806d050 con 0x7f32e80987c0 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f32d8001780 con 0x7f32e80987c0 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f32d80025f0 con 0x7f32e80987c0 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32e57fa700 4 set_mon_vals no callback set 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.283+0000 7f32ceffd700 10 monclient: dump: 2024-09-15T03:49:39.305 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f32d800fb20 con 0x7f32e80987c0 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32e7fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32e7fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32e7fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c80050b0 con 0x7f32e80987c0 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 4019+0+0 (unknown 3202300147 0 0) 0x7f32d8043fc0 con 0x7f32e80987c0 2024-09-15T03:49:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.287+0000 7f32e67fc700 10 monclient: get_auth_request con 0x7f32d4037d50 auth_method 0 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:934: finishing copy_from to smithi12223906-6 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 277 (ObjNum 312 snap 96 seq_num 312) dirty exists 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:938: finishing write tid 1 to smithi12223906-13 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:938: finishing write tid 2 to smithi12223906-13 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:938: finishing write tid 3 to smithi12223906-13 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:938: finishing write tid 4 to smithi12223906-13 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 342 (ObjNum 364 snap 103 seq_num 364) dirty exists 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:938: left oid 13 (ObjNum 364 snap 103 seq_num 364) 2024-09-15T03:49:39.308 INFO:tasks.rados.rados.0.smithi122.stdout:939: expect (ObjNum 227 snap 62 seq_num 227) 2024-09-15T03:49:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.295+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f32d8014960 con 0x7f32e80987c0 2024-09-15T03:49:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.295+0000 7f32ceffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.295+0000 7f32ceffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:39.478 INFO:tasks.rados.rados.0.smithi122.stdout:934: done (4 left) 2024-09-15T03:49:39.478 INFO:tasks.rados.rados.0.smithi122.stdout:937: done (3 left) 2024-09-15T03:49:39.478 INFO:tasks.rados.rados.0.smithi122.stdout:938: done (2 left) 2024-09-15T03:49:39.478 INFO:tasks.rados.rados.0.smithi122.stdout:939: done (1 left) 2024-09-15T03:49:39.479 INFO:tasks.rados.rados.0.smithi122.stdout:940: done (0 left) 2024-09-15T03:49:39.479 INFO:tasks.rados.rados.0.smithi122.stdout:941: read oid 18 snap 102 2024-09-15T03:49:39.479 INFO:tasks.rados.rados.0.smithi122.stdout:941: expect (ObjNum 302 snap 96 seq_num 302) 2024-09-15T03:49:39.479 INFO:tasks.rados.rados.0.smithi122.stdout:942: write (excl) oid 45 current snap is 103 2024-09-15T03:49:39.479 INFO:tasks.rados.rados.0.smithi122.stdout:942: seq_num 366 ranges {551362=726078,1717586=409779} 2024-09-15T03:49:39.495 INFO:tasks.rados.rados.0.smithi122.stdout:942: writing smithi12223906-45 from 551362 to 1277440 tid 1 2024-09-15T03:49:39.499 INFO:tasks.rados.rados.0.smithi122.stdout:942: writing smithi12223906-45 from 1717586 to 2127365 tid 2 2024-09-15T03:49:39.499 INFO:tasks.rados.rados.0.smithi122.stdout:943: read oid 27 snap -1 2024-09-15T03:49:39.499 INFO:tasks.rados.rados.0.smithi122.stdout:943: expect (ObjNum 309 snap 96 seq_num 309) 2024-09-15T03:49:39.500 INFO:tasks.rados.rados.0.smithi122.stdout:944: write oid 16 current snap is 103 2024-09-15T03:49:39.500 INFO:tasks.rados.rados.0.smithi122.stdout:944: seq_num 367 ranges {568169=401327,1376068=669790} 2024-09-15T03:49:39.505 INFO:tasks.rados.rados.0.smithi122.stdout:944: writing smithi12223906-16 from 568169 to 969496 tid 1 2024-09-15T03:49:39.511 INFO:tasks.rados.rados.0.smithi122.stdout:944: writing smithi12223906-16 from 1376068 to 2045858 tid 2 2024-09-15T03:49:39.512 INFO:tasks.rados.rados.0.smithi122.stdout:945: write oid 14 current snap is 103 2024-09-15T03:49:39.512 INFO:tasks.rados.rados.0.smithi122.stdout:945: seq_num 368 ranges {584976=476576,1834550=350426} 2024-09-15T03:49:39.517 INFO:tasks.rados.rados.0.smithi122.stdout:945: writing smithi12223906-14 from 584976 to 1061552 tid 1 2024-09-15T03:49:39.524 INFO:tasks.rados.rados.0.smithi122.stdout:945: writing smithi12223906-14 from 1834550 to 2184976 tid 2 2024-09-15T03:49:39.524 INFO:tasks.rados.rados.0.smithi122.stdout:946: write (excl) oid 21 current snap is 103 2024-09-15T03:49:39.524 INFO:tasks.rados.rados.0.smithi122.stdout:946: seq_num 369 ranges {601783=551825,1776679=425104} 2024-09-15T03:49:39.530 INFO:tasks.rados.rados.0.smithi122.stdout:946: writing smithi12223906-21 from 601783 to 1153608 tid 1 2024-09-15T03:49:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.527+0000 7f32e7fff700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T03:49:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.527+0000 7f32e7fff700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T03:49:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.527+0000 7f32e7fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:39.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.527+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f32c8005b70 con 0x7f32e80987c0 2024-09-15T03:49:39.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f32ceffd700 1 -- 172.21.15.26:0/35696830 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f32d8014780 con 0x7f32e80987c0 2024-09-15T03:49:39.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f32ceffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T03:49:39.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f32ceffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:39.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=12,kv=8,logm=54} crc {auth=371558463,config=3140514009,health=1017346500,kv=2519098299,logm=2506376475}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=24,config=1,health=11,kv=7,logm=57} crc {auth=160341891,config=1228700967,health=3172947059,kv=1358559433,logm=1325440517}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=11,kv=8,logm=56} crc {auth=50401405,config=3140514009,health=1288994568,kv=2519098299,logm=3765989608}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=22,config=2,health=12,kv=5,logm=59} crc {auth=3246815654,config=3140514009,health=1017346500,kv=1124123498,logm=1828522571}) 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=23,config=2,health=12,kv=6,logm=57} crc {auth=4236518273,config=3140514009,health=1017346500,kv=777372614,logm=56353739}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=7,logm=57} crc {auth=1435913397,config=3140514009,health=1017346500,kv=2519098299,logm=257160248}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2203604951,config=3140514009,health=1017346500,kv=2519098299,logm=162944103}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=371558463,config=3140514009,health=1189301077,kv=2519098299,logm=2671708011}) 2024-09-15T03:49:39.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.531+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=2342806867,config=3140514009,health=3188742911,kv=2519098299,logm=2617166909}) 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 >> v1:172.21.15.122:6812/20895 conn(0x7f32d4037d50 legacy=0x7f32d403a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 10 monclient: shutdown 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 >> v1:172.21.15.26:6791/0 conn(0x7f32e80987c0 legacy=0x7f32e806ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 shutdown_connections 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 >> 172.21.15.26:0/35696830 conn(0x7f32e80a24e0 msgr2=0x7f32e8097ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 shutdown_connections 2024-09-15T03:49:39.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:39.535+0000 7f32e7fff700 1 -- 172.21.15.26:0/35696830 wait complete. 2024-09-15T03:49:39.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3805883984}) 2024-09-15T03:49:39.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1116758417}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1682340261}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1551407070}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4081366611}) 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3413588883}) 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1108706848}) 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3113335273}) 2024-09-15T03:49:39.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.539+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3233389063}) 2024-09-15T03:49:39.546 INFO:tasks.rados.rados.0.smithi122.stdout:946: writing smithi12223906-21 from 1776679 to 2201783 tid 2 2024-09-15T03:49:39.546 INFO:tasks.rados.rados.0.smithi122.stdout:947: delete oid 26 current snap is 103 2024-09-15T03:49:39.546 INFO:tasks.rados.rados.0.smithi122.stdout:941: expect (ObjNum 302 snap 96 seq_num 302) 2024-09-15T03:49:39.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3231535600}) 2024-09-15T03:49:39.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2666439335}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4112710236}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=737013762}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=509969639}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1060277156}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4174291706}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3102674816}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3031180681}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=73720902}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=209161920}) 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1205500763}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1427289378}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4226615964}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2403472513}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2473395733}) 2024-09-15T03:49:39.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057384012}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.543+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1373066539}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2343317686}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3399870670}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1380024632}) 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1775778927}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1828593964}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1482828153}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3059949806}) 2024-09-15T03:49:39.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2056060444}) 2024-09-15T03:49:39.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=721577063}) 2024-09-15T03:49:39.559 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T03:49:39.583 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.583 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.583 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4289855054}) 2024-09-15T03:49:39.583 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.583 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4247318141}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=542620684}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.547+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1248779033}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1450907268}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.584 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2953154575}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2511835218}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2890538683}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2670314902}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1726514100}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=878006889}) 2024-09-15T03:49:39.585 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2588343902}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3199294840}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=582997851}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4183805385}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.586 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1350805978}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2229727033}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.551+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3088238664}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4026449638}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=818261600}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2575665519}) 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.587 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=885918493}) 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1111156146}) 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1676246314}) 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.588 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4132999668}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1990189503}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3602191690}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=901564938}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=306306743}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2081620641}) 2024-09-15T03:49:39.589 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2917318749}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3300736727}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4077456669}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1631621981}) 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.590 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1091228218}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.555+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3194751127}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4211715715}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3664948668}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4184157960}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3040528839}) 2024-09-15T03:49:39.591 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2399359956}) 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=49693620}) 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2234113822}) 2024-09-15T03:49:39.592 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=358415672}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1890069025}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=857903695}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2557674681}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=452000633}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.593 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2726481781}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2041137238}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=160889610}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3113905645}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3293572192}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.559+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1697718374}) 2024-09-15T03:49:39.594 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2335540341}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=201631714}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1981199208}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1815109110}) 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.595 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1049818467}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1311433540}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3299902660}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4176586222}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1339832984}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=67,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3} crc {logm=3542373456,mdsmap=1531151339,mgr=465081112,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1225837693,mon_config_key=733955277,monmap=4273209975}) 2024-09-15T03:49:39.596 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=62,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=5} crc {logm=213208544,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1225837693,mon_config_key=733955277,monmap=4273209975,osd_metadata=231491462}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=2} crc {logm=3923773625,mdsmap=1531151339,mgr=2380166094,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1225837693,mon_config_key=733955277,monmap=2173287178}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=63,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=4,monmap=3,osd_metadata=6} crc {logm=545552070,mdsmap=3585761405,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1225837693,mon_config_key=1595646671,monmap=4273209975,osd_metadata=2302341569}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.597 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=4,monmap=2,osd_metadata=5} crc {logm=2593714519,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1225837693,mon_config_key=3784792071,monmap=3140514009,osd_metadata=231491462}) 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=68,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=10,mon_config_key=4,monmap=2,osd_metadata=4} crc {logm=4146140782,mdsmap=1531151339,mgr=1925365903,mgr_command_descs=47663135,mgrstat=870525124,mon_config_key=1595646671,monmap=3140514009,osd_metadata=2435132412}) 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=68,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=1} crc {logm=2867325196,mdsmap=1531151339,mgr=991478979,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=3223037967,mon_config_key=733955277,monmap=4273209975,osd_metadata=2273602603}) 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.598 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=66,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=3} crc {logm=3347497600,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1972861301,mon_config_key=733955277,monmap=4273209975,osd_metadata=39359367}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.563+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=64,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=4} crc {logm=2484930013,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=883607424,mon_config_key=733955277,monmap=4273209975,osd_metadata=2435132412}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=39,osdmap=57} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=1977555109,osdmap=3034669418}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=37,osdmap=59} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3261164656,osdmap=2957276987}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=39,osdmap=57} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3427551407,osdmap=951129349}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.599 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=39,osdmap=57} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=487393378,osdmap=929030498}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=38,osdmap=58} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=530084066,osdmap=2080180490}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=39,osdmap=57} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3712494190,osdmap=3763043067}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3680114364,osdmap=3132918567}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3103153968,osdmap=332151000}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.567+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_metadata=3,osd_pg_creating=1,osd_snap=41,osdmap=55} crc {osd_metadata=1680283130,osd_pg_creating=2856894291,osd_snap=3680114364,osdmap=3598162738}) 2024-09-15T03:49:39.600 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2994288987}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=28071874}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=416189987}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3341714395}) 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.601 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1965343077}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3547876133}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=189810256}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1194896717}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1962202701}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.602 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3831882494}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=104430172}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3177621820}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.571+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3263527944}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1918348050}) 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.603 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1912638385}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1068316906}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1535092799}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2104914271}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=85} crc {osdmap=4070148660}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=89} crc {osdmap=562152147}) 2024-09-15T03:49:39.604 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=86} crc {osdmap=3427894129}) 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=86} crc {osdmap=3902954481}) 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=90} crc {osdmap=3203304728}) 2024-09-15T03:49:39.605 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=88} crc {osdmap=2691299612}) 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=91} crc {osdmap=1779092272}) 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=87} crc {osdmap=1913844984}) 2024-09-15T03:49:39.606 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:49:39.575+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=90} crc {osdmap=2137747236}) 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:942: finishing write tid 1 to smithi12223906-45 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:942: finishing write tid 2 to smithi12223906-45 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:942: finishing write tid 3 to smithi12223906-45 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:942: finishing write tid 4 to smithi12223906-45 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 345 (ObjNum 366 snap 103 seq_num 366) dirty exists 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:942: left oid 45 (ObjNum 366 snap 103 seq_num 366) 2024-09-15T03:49:39.653 INFO:tasks.rados.rados.0.smithi122.stdout:943: expect (ObjNum 309 snap 96 seq_num 309) 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:944: finishing write tid 1 to smithi12223906-16 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:944: finishing write tid 2 to smithi12223906-16 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:944: finishing write tid 3 to smithi12223906-16 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:941: done (6 left) 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:942: done (5 left) 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:943: done (4 left) 2024-09-15T03:49:39.859 INFO:tasks.rados.rados.0.smithi122.stdout:947: done (3 left) 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:948: read oid 48 snap 102 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:948: expect (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:949: snap_create 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:944: finishing write tid 4 to smithi12223906-16 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 329 (ObjNum 367 snap 103 seq_num 367) dirty exists 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:944: left oid 16 (ObjNum 367 snap 103 seq_num 367) 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:945: finishing write tid 1 to smithi12223906-14 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:945: finishing write tid 2 to smithi12223906-14 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:945: finishing write tid 3 to smithi12223906-14 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:945: finishing write tid 4 to smithi12223906-14 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 486 (ObjNum 368 snap 103 seq_num 368) dirty exists 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:945: left oid 14 (ObjNum 368 snap 103 seq_num 368) 2024-09-15T03:49:39.860 INFO:tasks.rados.rados.0.smithi122.stdout:946: finishing write tid 1 to smithi12223906-21 2024-09-15T03:49:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:946: finishing write tid 2 to smithi12223906-21 2024-09-15T03:49:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:946: finishing write tid 3 to smithi12223906-21 2024-09-15T03:49:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:946: finishing write tid 4 to smithi12223906-21 2024-09-15T03:49:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 188 (ObjNum 369 snap 103 seq_num 369) dirty exists 2024-09-15T03:49:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:946: left oid 21 (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:49:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:948: expect (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:49:40.560 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:49:40.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:49:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient: get_monmap_and_config 2024-09-15T03:49:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient: build_initial_monmap 2024-09-15T03:49:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient: monmap: 2024-09-15T03:49:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:40.672643+0000 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:40.672643+0000 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:40.675 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 1 Processor -- start 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 1 -- start start 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient: init 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient: _add_conns ranks=[4,2,7,5,8,3,6,0,1] 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient(hunting): picked mon.noname-c con 0x7f88a4099da0 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient(hunting): picked mon.noname-b con 0x7f88a409b330 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient(hunting): picked mon.noname-i con 0x7f88a4135670 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a404ca10 con 0x7f88a409b330 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a409b700 con 0x7f88a4099da0 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a409dcf0 con 0x7f88a4135670 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a3fff700 10 monclient(hunting): authenticate will time out at 1125.103594s 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a2ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f88a4099da0 0x7f88a409ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43114/0 (socket says 172.21.15.26:43114) 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.671+0000 7f88a2ffd700 1 -- 172.21.15.26:0/2821076249 learned_addr learned my addr 172.21.15.26:0/2821076249 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886163123 0 0) 0x7f88a409b700 con 0x7f88a4099da0 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient(hunting): my global_id is 174181 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f888c004680 con 0x7f88a4099da0 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007920927 0 0) 0x7f88a404ca10 con 0x7f88a409b330 2024-09-15T03:49:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient(hunting): my global_id is 134453 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a409b700 con 0x7f88a409b330 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766172392 0 0) 0x7f888c004680 con 0x7f88a4099da0 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 >> v1:172.21.15.122:6792/0 conn(0x7f88a4135670 legacy=0x7f88a4137a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 >> v1:172.21.15.26:6790/0 conn(0x7f88a409b330 legacy=0x7f88a409d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 monclient: found mon.noname-c 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a409ea70 con 0x7f88a4099da0 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 5 monclient: authenticate success, global_id 174181 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f88940017b0 con 0x7f88a4099da0 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: dump: 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8894001a90 con 0x7f88a4099da0 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 1 -- 172.21.15.26:0/2821076249 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8894001c80 con 0x7f88a4099da0 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 4 set_mon_vals no callback set 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a1ffb700 10 monclient: dump: 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: shutdown 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- 172.21.15.26:0/2821076249 >> v1:172.21.15.26:6791/0 conn(0x7f88a4099da0 legacy=0x7f88a409ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- 172.21.15.26:0/2821076249 shutdown_connections 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- 172.21.15.26:0/2821076249 >> 172.21.15.26:0/2821076249 conn(0x7f88a40a24e0 msgr2=0x7f88a40a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- 172.21.15.26:0/2821076249 shutdown_connections 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- 172.21.15.26:0/2821076249 wait complete. 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: build_initial_monmap 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: monmap: 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:40.690 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 Processor -- start 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- start start 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: init 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient: _add_conns ranks=[3,5,6,0,2,4,7,8,1] 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient(hunting): picked mon.noname-g con 0x7f88a409b330 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient(hunting): picked mon.noname-h con 0x7f88a4135670 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient(hunting): picked mon.noname-d con 0x7f88a412d890 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a412eee0 con 0x7f88a409b330 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a412e0f0 con 0x7f88a4135670 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a412f1c0 con 0x7f88a412d890 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a3fff700 10 monclient(hunting): authenticate will time out at 1125.107599s 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a37fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f88a412d890 0x7f88a414a2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34622/0 (socket says 172.21.15.26:34622) 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.675+0000 7f88a37fe700 1 -- 172.21.15.26:0/3283006113 learned_addr learned my addr 172.21.15.26:0/3283006113 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860618315 0 0) 0x7f88a412f1c0 con 0x7f88a412d890 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): my global_id is 4992 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8890003570 con 0x7f88a412d890 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181308240 0 0) 0x7f88a412e0f0 con 0x7f88a4135670 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): my global_id is 4955 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a412f1c0 con 0x7f88a4135670 2024-09-15T03:49:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030548959 0 0) 0x7f88a412eee0 con 0x7f88a409b330 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): my global_id is 4881 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a412e0f0 con 0x7f88a409b330 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4263463489 0 0) 0x7f8890003570 con 0x7f88a412d890 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88a412eee0 con 0x7f88a412d890 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 662033507 0 0) 0x7f88a412f1c0 con 0x7f88a4135670 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8890003570 con 0x7f88a4135670 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f889c001d80 con 0x7f88a412d890 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): dump: 2024-09-15T03:49:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3565203546 0 0) 0x7f88a412e0f0 con 0x7f88a409b330 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88a412f1c0 con 0x7f88a409b330 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8898001b30 con 0x7f88a4135670 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): dump: 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8894001770 con 0x7f88a409b330 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient(hunting): dump: 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3040954523 0 0) 0x7f88a412eee0 con 0x7f88a412d890 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 >> v1:172.21.15.122:6791/0 conn(0x7f88a4135670 legacy=0x7f88a4133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 >> v1:172.21.15.122:6790/0 conn(0x7f88a409b330 legacy=0x7f88a4141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 monclient: found mon.d 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a412f4a0 con 0x7f88a412d890 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: _finish_auth 0 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 5 monclient: authenticate success, global_id 4992 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 monclient: _renew_subs 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a41343f0 con 0x7f88a412d890 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 monclient: _renew_subs 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a4142e10 con 0x7f88a412d890 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f889c001f70 con 0x7f88a412d890 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a412d500 con 0x7f88a412d890 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f889c0028f0 con 0x7f88a412d890 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f888affd700 10 monclient: dump: 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.679+0000 7f88a17fa700 4 set_mon_vals no callback set 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.683+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f889c002a50 con 0x7f88a412d890 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.683+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(227..227 src has 1..227) v4 ==== 4019+0+0 (unknown 3202300147 0 0) 0x7f889c044190 con 0x7f88a412d890 2024-09-15T03:49:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.683+0000 7f88a2ffd700 10 monclient: get_auth_request con 0x7f8890037fd0 auth_method 0 2024-09-15T03:49:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.687+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f889c014fb0 con 0x7f88a412d890 2024-09-15T03:49:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.687+0000 7f888affd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.687+0000 7f888affd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:40.846 INFO:tasks.rados.rados.0.smithi122.stdout:944: done (4 left) 2024-09-15T03:49:40.846 INFO:tasks.rados.rados.0.smithi122.stdout:945: done (3 left) 2024-09-15T03:49:40.846 INFO:tasks.rados.rados.0.smithi122.stdout:946: done (2 left) 2024-09-15T03:49:40.846 INFO:tasks.rados.rados.0.smithi122.stdout:948: done (1 left) 2024-09-15T03:49:40.847 INFO:tasks.rados.rados.0.smithi122.stdout:949: done (0 left) 2024-09-15T03:49:40.847 INFO:tasks.rados.rados.0.smithi122.stdout:950: rollback oid 6 current snap is 104 2024-09-15T03:49:40.847 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 6 to 102 2024-09-15T03:49:40.847 INFO:tasks.rados.rados.0.smithi122.stdout:951: write (excl) oid 17 current snap is 104 2024-09-15T03:49:40.847 INFO:tasks.rados.rados.0.smithi122.stdout:951: seq_num 371 ranges {635397=702323,1893643=341754} 2024-09-15T03:49:40.856 INFO:tasks.rados.rados.0.smithi122.stdout:951: writing smithi12223906-17 from 635397 to 1337720 tid 1 2024-09-15T03:49:40.861 INFO:tasks.rados.rados.0.smithi122.stdout:951: writing smithi12223906-17 from 1893643 to 2235397 tid 2 2024-09-15T03:49:40.862 INFO:tasks.rados.rados.0.smithi122.stdout:952: write (excl) oid 12 current snap is 104 2024-09-15T03:49:40.862 INFO:tasks.rados.rados.0.smithi122.stdout:950: finishing rollback tid 0 to smithi12223906-6 2024-09-15T03:49:40.862 INFO:tasks.rados.rados.0.smithi122.stdout:952: seq_num 372 ranges {652204=777572,1952125=300079} 2024-09-15T03:49:40.869 INFO:tasks.rados.rados.0.smithi122.stdout:952: writing smithi12223906-12 from 652204 to 1429776 tid 1 2024-09-15T03:49:40.875 INFO:tasks.rados.rados.0.smithi122.stdout:952: writing smithi12223906-12 from 1952125 to 2252204 tid 2 2024-09-15T03:49:40.875 INFO:tasks.rados.rados.0.smithi122.stdout:953: copy_from oid 2 from oid 34 current snap is 104 2024-09-15T03:49:40.875 INFO:tasks.rados.rados.0.smithi122.stdout:950: finishing rollback tid 1 to smithi12223906-6 2024-09-15T03:49:40.876 INFO:tasks.rados.rados.0.smithi122.stdout:950: finishing rollback tid 2 to smithi12223906-6 2024-09-15T03:49:40.876 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 280 (ObjNum 322 snap 97 seq_num 322) dirty exists 2024-09-15T03:49:40.876 INFO:tasks.rados.rados.0.smithi122.stdout:950: done (3 left) 2024-09-15T03:49:40.876 INFO:tasks.rados.rados.0.smithi122.stdout:954: read oid 40 snap -1 2024-09-15T03:49:40.876 INFO:tasks.rados.rados.0.smithi122.stdout:954: expect (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:49:40.876 INFO:tasks.rados.rados.0.smithi122.stdout:955: snap_remove snap 102 2024-09-15T03:49:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f88a3fff700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:49:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f88a3fff700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:49:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f88a3fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f88a404bbf0 con 0x7f88a412d890 2024-09-15T03:49:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f888affd700 1 -- 172.21.15.26:0/3283006113 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 3620904192) 0x7f889c0149b0 con 0x7f88a412d890 2024-09-15T03:49:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f888affd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:49:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.923+0000 7f888affd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:40.927 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":44,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:49:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 >> v1:172.21.15.122:6812/20895 conn(0x7f8890037fd0 legacy=0x7f889003a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 10 monclient: shutdown 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 >> v1:172.21.15.26:6792/0 conn(0x7f88a412d890 legacy=0x7f88a414a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 shutdown_connections 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 >> 172.21.15.26:0/3283006113 conn(0x7f88a40a24e0 msgr2=0x7f88a4134890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 shutdown_connections 2024-09-15T03:49:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:40.927+0000 7f88a3fff700 1 -- 172.21.15.26:0/3283006113 wait complete. 2024-09-15T03:49:40.944 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:49:40.944 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:49:40.944 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:49:40.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:49:41.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.047+0000 7f917bfff700 10 monclient: get_monmap_and_config 2024-09-15T03:49:41.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.047+0000 7f917bfff700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.047+0000 7f917bfff700 10 monclient: monmap: 2024-09-15T03:49:41.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:41.050288+0000 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:41.050288+0000 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.052 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.053 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.053 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.053 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.053 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.053 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.047+0000 7f917bfff700 1 Processor -- start 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.047+0000 7f917bfff700 1 -- start start 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.047+0000 7f917bfff700 10 monclient: init 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient: _add_conns ranks=[6,3,5,4,1,0,8,7,2] 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient(hunting): picked mon.noname-d con 0x7f917c132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient(hunting): picked mon.noname-g con 0x7f917c09faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient(hunting): picked mon.noname-h con 0x7f917c138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c04ca10 con 0x7f917c09faa0 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c09d5d0 con 0x7f917c138d40 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c133620 con 0x7f917c132b40 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient(hunting): authenticate will time out at 1125.480347s 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917affd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f917c132b40 0x7f917c132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34626/0 (socket says 172.21.15.26:34626) 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917affd700 1 -- 172.21.15.26:0/1104545687 learned_addr learned my addr 172.21.15.26:0/1104545687 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737451017 0 0) 0x7f917c133620 con 0x7f917c132b40 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): my global_id is 5001 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9164003570 con 0x7f917c132b40 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936176830 0 0) 0x7f917c09d5d0 con 0x7f917c138d40 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): my global_id is 4964 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c133620 con 0x7f917c138d40 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362623475 0 0) 0x7f917c04ca10 con 0x7f917c09faa0 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient(hunting): my global_id is 4890 2024-09-15T03:49:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c09d5d0 con 0x7f917c09faa0 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997839382 0 0) 0x7f9164003570 con 0x7f917c132b40 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 >> v1:172.21.15.122:6791/0 conn(0x7f917c138d40 legacy=0x7f917c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 >> v1:172.21.15.122:6790/0 conn(0x7f917c09faa0 legacy=0x7f917c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 monclient: found mon.noname-d 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0a0cc0 con 0x7f917c132b40 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 5 monclient: authenticate success, global_id 5001 2024-09-15T03:49:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f916c0017b0 con 0x7f917c132b40 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: dump: 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f916c001a20 con 0x7f917c132b40 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 1 -- 172.21.15.26:0/1104545687 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f916c004c30 con 0x7f917c132b40 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 4 set_mon_vals no callback set 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f9179ffb700 10 monclient: dump: 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient: shutdown 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- 172.21.15.26:0/1104545687 >> v1:172.21.15.26:6792/0 conn(0x7f917c132b40 legacy=0x7f917c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- 172.21.15.26:0/1104545687 shutdown_connections 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- 172.21.15.26:0/1104545687 >> 172.21.15.26:0/1104545687 conn(0x7f917c06e370 msgr2=0x7f917c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- 172.21.15.26:0/1104545687 shutdown_connections 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 1 -- 172.21.15.26:0/1104545687 wait complete. 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.051+0000 7f917bfff700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: monmap: 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.065 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 Processor -- start 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 -- start start 2024-09-15T03:49:41.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: init 2024-09-15T03:49:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: _add_conns ranks=[1,4,3,2,5,7,0,6,8] 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient(hunting): picked mon.noname-f con 0x7f917c09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f917c132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient(hunting): picked mon.noname-g con 0x7f917c138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c09cfa0 con 0x7f917c09faa0 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c09ccc0 con 0x7f917c138d40 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c11cec0 con 0x7f917c132b40 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient(hunting): authenticate will time out at 1125.486440s 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917a7fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f917c132b40 0x7f917c1315f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43120/0 (socket says 172.21.15.26:43120) 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917a7fc700 1 -- 172.21.15.26:0/3087981335 learned_addr learned my addr 172.21.15.26:0/3087981335 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131371960 0 0) 0x7f917c11cec0 con 0x7f917c132b40 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): my global_id is 174190 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91680035a0 con 0x7f917c132b40 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605255109 0 0) 0x7f917c09ccc0 con 0x7f917c138d40 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): my global_id is 4899 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c11cec0 con 0x7f917c138d40 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166743094 0 0) 0x7f917c09cfa0 con 0x7f917c09faa0 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): my global_id is 134308 2024-09-15T03:49:41.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c09ccc0 con 0x7f917c09faa0 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1170523707 0 0) 0x7f91680035a0 con 0x7f917c132b40 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f917c09cfa0 con 0x7f917c132b40 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9170002b70 con 0x7f917c132b40 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3298367019 0 0) 0x7f917c09cfa0 con 0x7f917c132b40 2024-09-15T03:49:41.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.122:6790/0 conn(0x7f917c138d40 legacy=0x7f917c14a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.122:6789/0 conn(0x7f917c09faa0 legacy=0x7f917c130ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 monclient: found mon.c 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c131d00 con 0x7f917c132b40 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 5 monclient: authenticate success, global_id 174190 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: _renew_subs 2024-09-15T03:49:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c1378a0 con 0x7f917c132b40 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9170001620 con 0x7f917c132b40 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: _renew_subs 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f91797fa700 4 set_mon_vals no callback set 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9170001b50 con 0x7f917c132b40 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c137ce0 con 0x7f917c132b40 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.055+0000 7f9162ffd700 10 monclient: dump: 2024-09-15T03:49:41.076 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.077 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f917bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f917c04bbf0 con 0x7f917c136880 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29576209 0 0) 0x7f917c04bbf0 con 0x7f917c136880 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient(hunting): my global_id is 4797 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c09cfa0 con 0x7f917c136880 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9170001d30 con 0x7f917c132b40 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3991+0+0 (unknown 3854576550 0 0) 0x7f9170043ff0 con 0x7f917c132b40 2024-09-15T03:49:41.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1476499913 0 0) 0x7f917c09cfa0 con 0x7f917c136880 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f917c04bbf0 con 0x7f917c136880 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f916c001770 con 0x7f917c136880 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient: dump: 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.079 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f917b7fe700 10 monclient: get_auth_request con 0x7f91680377b0 auth_method 0 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4114288361 0 0) 0x7f917c04bbf0 con 0x7f917c136880 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f917c06fac0 con 0x7f917c136880 2024-09-15T03:49:41.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f916c004830 con 0x7f917c136880 2024-09-15T03:49:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.26:6789/0 conn(0x7f917c136880 legacy=0x7f917c09aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.059+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.26:6789/0 conn(0x7f917c136880 legacy=0x7f917c09aa80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f917c136c50 con 0x7f917c02ad10 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964404499 0 0) 0x7f917c136c50 con 0x7f917c02ad10 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient(hunting): my global_id is 4806 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f917c04bbf0 con 0x7f917c02ad10 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3168505169 0 0) 0x7f917c04bbf0 con 0x7f917c02ad10 2024-09-15T03:49:41.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f917c136c50 con 0x7f917c02ad10 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f916c005340 con 0x7f917c02ad10 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient: dump: 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2543314567 0 0) 0x7f917c136c50 con 0x7f917c02ad10 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f917c0a1410 con 0x7f917c02ad10 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 3262230861) 0x7f916c007580 con 0x7f917c02ad10 2024-09-15T03:49:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.26:6789/0 conn(0x7f917c02ad10 legacy=0x7f917c09b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f9162ffd700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.26:6789/0 conn(0x7f917c02ad10 legacy=0x7f917c09b190 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:41.086 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.088 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.089 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.090 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.096 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:41.097 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.098 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.099 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.100 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.101 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.103 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.104 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.105 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.106 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.107 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.108 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:41.108 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.122:6812/20895 conn(0x7f91680377b0 legacy=0x7f9168039c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 10 monclient: shutdown 2024-09-15T03:49:41.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 >> v1:172.21.15.26:6791/0 conn(0x7f917c132b40 legacy=0x7f917c1315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917b7fe700 1 -- 172.21.15.26:0/3087981335 reap_dead start 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917b7fe700 1 -- 172.21.15.26:0/3087981335 reap_dead start 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 shutdown_connections 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 >> 172.21.15.26:0/3087981335 conn(0x7f917c06e370 msgr2=0x7f917c09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 shutdown_connections 2024-09-15T03:49:41.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.075+0000 7f917bfff700 1 -- 172.21.15.26:0/3087981335 wait complete. 2024-09-15T03:49:41.110 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:41.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:49:41.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:49:41.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.219+0000 7faa0972f700 10 monclient: get_monmap_and_config 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.219+0000 7faa0972f700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.219+0000 7faa0972f700 10 monclient: monmap: 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:41.222901+0000 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:41.222901+0000 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.225 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.219+0000 7faa0972f700 1 Processor -- start 2024-09-15T03:49:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.219+0000 7faa0972f700 1 -- start start 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.219+0000 7faa0972f700 10 monclient: init 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient: _add_conns ranks=[6,4,2,1,3,0,8,7,5] 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient(hunting): picked mon.noname-d con 0x7faa04132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient(hunting): picked mon.noname-c con 0x7faa0409faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient(hunting): picked mon.noname-b con 0x7faa04138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0404ca10 con 0x7faa04138d40 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0409d5d0 con 0x7faa0409faa0 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa04133620 con 0x7faa04132b40 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient(hunting): authenticate will time out at 1125.652590s 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa08f2e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7faa04138d40 0x7faa0413b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:50276/0 (socket says 172.21.15.26:50276) 2024-09-15T03:49:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa08f2e700 1 -- 172.21.15.26:0/1670130970 learned_addr learned my addr 172.21.15.26:0/1670130970 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311624552 0 0) 0x7faa0409d5d0 con 0x7faa0409faa0 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): my global_id is 174199 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f0003570 con 0x7faa0409faa0 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461047122 0 0) 0x7faa04133620 con 0x7faa04132b40 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): my global_id is 5010 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0409d5d0 con 0x7faa04132b40 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121940918 0 0) 0x7faa0404ca10 con 0x7faa04138d40 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient(hunting): my global_id is 134462 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa04133620 con 0x7faa04138d40 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190041216 0 0) 0x7fa9f0003570 con 0x7faa0409faa0 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 >> v1:172.21.15.26:6792/0 conn(0x7faa04132b40 legacy=0x7faa04132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 >> v1:172.21.15.26:6790/0 conn(0x7faa04138d40 legacy=0x7faa0413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 monclient: found mon.noname-c 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040a0cc0 con 0x7faa0409faa0 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 5 monclient: authenticate success, global_id 174199 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9ec0017b0 con 0x7faa0409faa0 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: dump: 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa9ec001a20 con 0x7faa0409faa0 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 1 -- 172.21.15.26:0/1670130970 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9ec002240 con 0x7faa0409faa0 2024-09-15T03:49:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 4 set_mon_vals no callback set 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa02ffd700 10 monclient: dump: 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient: shutdown 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- 172.21.15.26:0/1670130970 >> v1:172.21.15.26:6791/0 conn(0x7faa0409faa0 legacy=0x7faa0409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- 172.21.15.26:0/1670130970 shutdown_connections 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- 172.21.15.26:0/1670130970 >> 172.21.15.26:0/1670130970 conn(0x7faa0406e370 msgr2=0x7faa040a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- 172.21.15.26:0/1670130970 shutdown_connections 2024-09-15T03:49:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 1 -- 172.21.15.26:0/1670130970 wait complete. 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.223+0000 7faa0972f700 10 monclient: monmap: 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 Processor -- start 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 -- start start 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: init 2024-09-15T03:49:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: _add_conns ranks=[0,6,8,3,1,2,7,5,4] 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient(hunting): picked mon.noname-a con 0x7faa0409faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient(hunting): picked mon.noname-d con 0x7faa04132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient(hunting): picked mon.noname-e con 0x7faa04138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa04138aa0 con 0x7faa0409faa0 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0409ccc0 con 0x7faa04132b40 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0409cfa0 con 0x7faa04138d40 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient(hunting): authenticate will time out at 1125.657093s 2024-09-15T03:49:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa03fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa0409faa0 0x7faa04142790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47338/0 (socket says 172.21.15.26:47338) 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa03fff700 1 -- 172.21.15.26:0/918988597 learned_addr learned my addr 172.21.15.26:0/918988597 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286833318 0 0) 0x7faa0409ccc0 con 0x7faa04132b40 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): my global_id is 5019 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e4003570 con 0x7faa04132b40 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297507958 0 0) 0x7faa04138aa0 con 0x7faa0409faa0 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): my global_id is 4815 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0409ccc0 con 0x7faa0409faa0 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831425831 0 0) 0x7faa0409cfa0 con 0x7faa04138d40 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): my global_id is 134450 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa04138aa0 con 0x7faa04138d40 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1517225123 0 0) 0x7faa0409ccc0 con 0x7faa0409faa0 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa0409cfa0 con 0x7faa0409faa0 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9f4001b20 con 0x7faa0409faa0 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): dump: 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2797856096 0 0) 0x7fa9e4003570 con 0x7faa04132b40 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faa0409ccc0 con 0x7faa04132b40 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9ec001ed0 con 0x7faa04132b40 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient(hunting): dump: 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 748427892 0 0) 0x7faa04138aa0 con 0x7faa04138d40 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9e4003570 con 0x7faa04138d40 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3661488214 0 0) 0x7faa0409ccc0 con 0x7faa04132b40 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6793/0 conn(0x7faa04138d40 legacy=0x7faa0414ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6789/0 conn(0x7faa0409faa0 legacy=0x7faa04142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 monclient: found mon.d 2024-09-15T03:49:41.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa041378a0 con 0x7faa04132b40 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 5 monclient: authenticate success, global_id 5019 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: _renew_subs 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0411ce10 con 0x7faa04132b40 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: _renew_subs 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faa04137390 con 0x7faa04132b40 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa9ec001860 con 0x7faa04132b40 2024-09-15T03:49:41.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9ec001200 con 0x7faa04132b40 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa027fc700 4 set_mon_vals no callback set 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7fa9fbfff700 10 monclient: dump: 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.252 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.227+0000 7faa0972f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa9e8006590 con 0x7fa9e8002840 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa9ec00d0b0 con 0x7faa04132b40 2024-09-15T03:49:41.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3991+0+0 (unknown 3854576550 0 0) 0x7fa9ec0416a0 con 0x7faa04132b40 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926853081 0 0) 0x7fa9e8006590 con 0x7fa9e8002840 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient(hunting): my global_id is 134471 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa0409ccc0 con 0x7fa9e8002840 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7faa08f2e700 10 monclient: get_auth_request con 0x7fa9e4037ea0 auth_method 0 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1866353298 0 0) 0x7faa0409ccc0 con 0x7fa9e8002840 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9e8006590 con 0x7fa9e8002840 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9f40025f0 con 0x7fa9e8002840 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient: dump: 2024-09-15T03:49:41.254 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.255 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3716359860 0 0) 0x7fa9e8006590 con 0x7fa9e8002840 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9e800a130 con 0x7fa9e8002840 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa9f4002be0 con 0x7fa9e8002840 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6790/0 conn(0x7fa9e8002840 legacy=0x7fa9e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.231+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6790/0 conn(0x7fa9e8002840 legacy=0x7fa9e8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.239+0000 7faa0972f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.239+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa9e8009640 con 0x7fa9e80090b0 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523199991 0 0) 0x7fa9e8009640 con 0x7fa9e80090b0 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient(hunting): my global_id is 134480 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e8006590 con 0x7fa9e80090b0 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2626632655 0 0) 0x7fa9e8006590 con 0x7fa9e80090b0 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9e8009640 con 0x7fa9e80090b0 2024-09-15T03:49:41.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9f4005d40 con 0x7fa9e80090b0 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient: dump: 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2184125010 0 0) 0x7fa9e8009640 con 0x7fa9e80090b0 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa9e80098f0 con 0x7fa9e80090b0 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 2153034521) 0x7fa9f40060a0 con 0x7fa9e80090b0 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6790/0 conn(0x7fa9e80090b0 legacy=0x7fa9e8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7fa9fbfff700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6790/0 conn(0x7fa9e80090b0 legacy=0x7fa9e8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:41.264 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:41.265 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.266 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.267 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.268 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.269 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.270 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.273 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:41.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.275 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.277 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.278 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:41.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.282 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:41.282 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.122:6812/20895 conn(0x7fa9e4037ea0 legacy=0x7fa9e403a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 10 monclient: shutdown 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 >> v1:172.21.15.26:6792/0 conn(0x7faa04132b40 legacy=0x7faa041473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa08f2e700 1 -- 172.21.15.26:0/918988597 reap_dead start 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 shutdown_connections 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 >> 172.21.15.26:0/918988597 conn(0x7faa0406e370 msgr2=0x7faa0409da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 shutdown_connections 2024-09-15T03:49:41.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.243+0000 7faa0972f700 1 -- 172.21.15.26:0/918988597 wait complete. 2024-09-15T03:49:41.284 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:41.284 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:49:41.284 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:49:41.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient: get_monmap_and_config 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient: monmap: 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:41.397051+0000 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:41.397051+0000 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.399 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 1 Processor -- start 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 1 -- start start 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient: init 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient: _add_conns ranks=[1,7,5,4,3,6,0,2,8] 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient(hunting): picked mon.noname-f con 0x7f3dfc132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient(hunting): picked mon.noname-i con 0x7f3dfc09faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient(hunting): picked mon.noname-h con 0x7f3dfc138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc04ca10 con 0x7f3dfc132b40 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc09d5d0 con 0x7f3dfc138d40 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc133620 con 0x7f3dfc09faa0 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfbfff700 10 monclient(hunting): authenticate will time out at 1125.827267s 2024-09-15T03:49:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfb7fe700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f3dfc138d40 0x7f3dfc13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:33280/0 (socket says 172.21.15.26:33280) 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfb7fe700 1 -- 172.21.15.26:0/1706664720 learned_addr learned my addr 172.21.15.26:0/1706664720 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.395+0000 7f3dfaffd700 1 --1- 172.21.15.26:0/1706664720 >> v1:172.21.15.122:6789/0 conn(0x7f3dfc132b40 0x7f3dfc132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:55404/0 (socket says 172.21.15.26:55404) 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237493256 0 0) 0x7f3dfc04ca10 con 0x7f3dfc132b40 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): my global_id is 134317 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de4003570 con 0x7f3dfc132b40 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225950629 0 0) 0x7f3dfc09d5d0 con 0x7f3dfc138d40 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): my global_id is 4973 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc04ca10 con 0x7f3dfc138d40 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223973797 0 0) 0x7f3dfc133620 con 0x7f3dfc09faa0 2024-09-15T03:49:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient(hunting): my global_id is 134440 2024-09-15T03:49:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc09d5d0 con 0x7f3dfc09faa0 2024-09-15T03:49:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2524522187 0 0) 0x7f3de4003570 con 0x7f3dfc132b40 2024-09-15T03:49:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 >> v1:172.21.15.122:6792/0 conn(0x7f3dfc09faa0 legacy=0x7f3dfc09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 >> v1:172.21.15.122:6791/0 conn(0x7f3dfc138d40 legacy=0x7f3dfc13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 monclient: found mon.noname-f 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc0a0cc0 con 0x7f3dfc132b40 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 5 monclient: authenticate success, global_id 134317 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3df00017b0 con 0x7f3dfc132b40 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: dump: 2024-09-15T03:49:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3df00019a0 con 0x7f3dfc132b40 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 1 -- 172.21.15.26:0/1706664720 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3df0004830 con 0x7f3dfc132b40 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 4 set_mon_vals no callback set 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3df9ffb700 10 monclient: dump: 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 10 monclient: shutdown 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1706664720 >> v1:172.21.15.122:6789/0 conn(0x7f3dfc132b40 legacy=0x7f3dfc132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1706664720 shutdown_connections 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1706664720 >> 172.21.15.26:0/1706664720 conn(0x7f3dfc06e370 msgr2=0x7f3dfc0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1706664720 shutdown_connections 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1706664720 wait complete. 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 10 monclient: monmap: 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 Processor -- start 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 1 -- start start 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.399+0000 7f3dfbfff700 10 monclient: init 2024-09-15T03:49:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient: _add_conns ranks=[6,0,3,5,4,2,1,8,7] 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient(hunting): picked mon.noname-d con 0x7f3dfc09faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient(hunting): picked mon.noname-a con 0x7f3dfc132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient(hunting): picked mon.noname-g con 0x7f3dfc138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc09cfa0 con 0x7f3dfc132b40 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc09ccc0 con 0x7f3dfc138d40 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dfc137c50 con 0x7f3dfc09faa0 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient(hunting): authenticate will time out at 1125.832448s 2024-09-15T03:49:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfaffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f3dfc09faa0 0x7f3dfc142370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34656/0 (socket says 172.21.15.26:34656) 2024-09-15T03:49:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfa7fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3dfc132b40 0x7f3dfc142a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47342/0 (socket says 172.21.15.26:47342) 2024-09-15T03:49:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfaffd700 1 -- 172.21.15.26:0/1579562128 learned_addr learned my addr 172.21.15.26:0/1579562128 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197964475 0 0) 0x7f3dfc137c50 con 0x7f3dfc09faa0 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): my global_id is 5028 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3de8003570 con 0x7f3dfc09faa0 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2670098512 0 0) 0x7f3dfc09cfa0 con 0x7f3dfc132b40 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): my global_id is 4824 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc137c50 con 0x7f3dfc132b40 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009632005 0 0) 0x7f3dfc09ccc0 con 0x7f3dfc138d40 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): my global_id is 4908 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc09cfa0 con 0x7f3dfc138d40 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1074495717 0 0) 0x7f3de8003570 con 0x7f3dfc09faa0 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3dfc09ccc0 con 0x7f3dfc09faa0 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3df00016b0 con 0x7f3dfc09faa0 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3900757395 0 0) 0x7f3dfc09cfa0 con 0x7f3dfc138d40 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3de8003570 con 0x7f3dfc138d40 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3df4001c80 con 0x7f3dfc138d40 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.422 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3106278987 0 0) 0x7f3dfc137c50 con 0x7f3dfc132b40 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3dfc09cfa0 con 0x7f3dfc132b40 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3dec0019e0 con 0x7f3dfc132b40 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.424 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 551736495 0 0) 0x7f3dfc09ccc0 con 0x7f3dfc09faa0 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.122:6790/0 conn(0x7f3dfc138d40 legacy=0x7f3dfc14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.26:6789/0 conn(0x7f3dfc132b40 legacy=0x7f3dfc142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 monclient: found mon.d 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dfc099c00 con 0x7f3dfc09faa0 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 5 monclient: authenticate success, global_id 5028 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient: _renew_subs 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dfc098970 con 0x7f3dfc09faa0 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient: _renew_subs 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dfc097c40 con 0x7f3dfc09faa0 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3df0004e70 con 0x7f3dfc09faa0 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3df00053a0 con 0x7f3dfc09faa0 2024-09-15T03:49:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3df97fa700 4 set_mon_vals no callback set 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 10 monclient: dump: 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.427 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.403+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3df00055c0 con 0x7f3dfc09faa0 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3dfbfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3ddc006590 con 0x7f3ddc002840 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3991+0+0 (unknown 3854576550 0 0) 0x7f3df0044230 con 0x7f3dfc09faa0 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3dfb7fe700 10 monclient: get_auth_request con 0x7f3de8038140 auth_method 0 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171704069 0 0) 0x7f3ddc006590 con 0x7f3ddc002840 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient(hunting): my global_id is 174208 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc09ccc0 con 0x7f3ddc002840 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1631933045 0 0) 0x7f3dfc09ccc0 con 0x7f3ddc002840 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3ddc006590 con 0x7f3ddc002840 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3dec001f40 con 0x7f3ddc002840 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient: dump: 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.429 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3739532376 0 0) 0x7f3ddc006590 con 0x7f3ddc002840 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ddc00a130 con 0x7f3ddc002840 2024-09-15T03:49:41.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3dec005810 con 0x7f3ddc002840 2024-09-15T03:49:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.26:6791/0 conn(0x7f3ddc002840 legacy=0x7f3ddc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.407+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.26:6791/0 conn(0x7f3ddc002840 legacy=0x7f3ddc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3ddc009640 con 0x7f3ddc0090b0 2024-09-15T03:49:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525894990 0 0) 0x7f3ddc009640 con 0x7f3ddc0090b0 2024-09-15T03:49:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient(hunting): my global_id is 174217 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ddc006590 con 0x7f3ddc0090b0 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1668905673 0 0) 0x7f3ddc006590 con 0x7f3ddc0090b0 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3ddc009640 con 0x7f3ddc0090b0 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3dec002960 con 0x7f3ddc0090b0 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient: dump: 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.433 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.434 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1641024113 0 0) 0x7f3ddc009640 con 0x7f3ddc0090b0 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3ddc0098f0 con 0x7f3ddc0090b0 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 4270208514) 0x7f3dec007cf0 con 0x7f3ddc0090b0 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.26:6791/0 conn(0x7f3ddc0090b0 legacy=0x7f3ddc008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3de2ffd700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.26:6791/0 conn(0x7f3ddc0090b0 legacy=0x7f3ddc008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:41.436 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.437 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.438 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:41.439 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.440 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:41.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.444 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.445 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:41.448 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.449 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.454 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:41.455 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:41.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.122:6812/20895 conn(0x7f3de8038140 legacy=0x7f3de803a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 10 monclient: shutdown 2024-09-15T03:49:41.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 >> v1:172.21.15.26:6792/0 conn(0x7f3dfc09faa0 legacy=0x7f3dfc142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfb7fe700 1 -- 172.21.15.26:0/1579562128 reap_dead start 2024-09-15T03:49:41.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 shutdown_connections 2024-09-15T03:49:41.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 >> 172.21.15.26:0/1579562128 conn(0x7f3dfc06e370 msgr2=0x7f3dfc09e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 shutdown_connections 2024-09-15T03:49:41.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.419+0000 7f3dfbfff700 1 -- 172.21.15.26:0/1579562128 wait complete. 2024-09-15T03:49:41.457 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:41.457 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:49:41.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:49:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.575+0000 7f5aca19d700 10 monclient: get_monmap_and_config 2024-09-15T03:49:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.575+0000 7f5aca19d700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.575+0000 7f5aca19d700 10 monclient: monmap: 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:41.578874+0000 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:41.578874+0000 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.581 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.575+0000 7f5aca19d700 1 Processor -- start 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.575+0000 7f5aca19d700 1 -- start start 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.575+0000 7f5aca19d700 10 monclient: init 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient: _add_conns ranks=[5,8,4,0,6,7,2,3,1] 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient(hunting): picked mon.noname-h con 0x7f5ac40a2a70 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient(hunting): picked mon.noname-e con 0x7f5ac409e380 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient(hunting): picked mon.noname-c con 0x7f5ac409bfc0 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac404ca10 con 0x7f5ac409bfc0 2024-09-15T03:49:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac409d3e0 con 0x7f5ac40a2a70 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac4098880 con 0x7f5ac409e380 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient(hunting): authenticate will time out at 1126.008280s 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5ac999c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f5ac409bfc0 0x7f5ac4138d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43154/0 (socket says 172.21.15.26:43154) 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5ac899a700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5ac409e380 0x7f5ac4098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56796/0 (socket says 172.21.15.26:56796) 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5ac999c700 1 -- 172.21.15.26:0/1899623020 learned_addr learned my addr 172.21.15.26:0/1899623020 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351395056 0 0) 0x7f5ac4098880 con 0x7f5ac409e380 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): my global_id is 134459 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ab0003570 con 0x7f5ac409e380 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575733336 0 0) 0x7f5ac409d3e0 con 0x7f5ac40a2a70 2024-09-15T03:49:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): my global_id is 4982 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac4098880 con 0x7f5ac40a2a70 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271132742 0 0) 0x7f5ac404ca10 con 0x7f5ac409bfc0 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient(hunting): my global_id is 174226 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac409d3e0 con 0x7f5ac409bfc0 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4051354321 0 0) 0x7f5ab0003570 con 0x7f5ac409e380 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 >> v1:172.21.15.122:6791/0 conn(0x7f5ac40a2a70 legacy=0x7f5ac4097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 >> v1:172.21.15.26:6791/0 conn(0x7f5ac409bfc0 legacy=0x7f5ac4138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 monclient: found mon.noname-e 2024-09-15T03:49:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac409d100 con 0x7f5ac409e380 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 5 monclient: authenticate success, global_id 134459 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ab4001730 con 0x7f5ac409e380 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: dump: 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.589 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ab4001950 con 0x7f5ac409e380 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 1 -- 172.21.15.26:0/1899623020 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ab4004830 con 0x7f5ac409e380 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 4 set_mon_vals no callback set 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5abbfff700 10 monclient: dump: 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient: shutdown 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- 172.21.15.26:0/1899623020 >> v1:172.21.15.26:6793/0 conn(0x7f5ac409e380 legacy=0x7f5ac4098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- 172.21.15.26:0/1899623020 shutdown_connections 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- 172.21.15.26:0/1899623020 >> 172.21.15.26:0/1899623020 conn(0x7f5ac4063cd0 msgr2=0x7f5ac40a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- 172.21.15.26:0/1899623020 shutdown_connections 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 1 -- 172.21.15.26:0/1899623020 wait complete. 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.579+0000 7f5aca19d700 10 monclient: monmap: 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 Processor -- start 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 -- start start 2024-09-15T03:49:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: init 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: _add_conns ranks=[8,7,2,1,4,6,0,3,5] 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient(hunting): picked mon.noname-e con 0x7f5ac409bfc0 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient(hunting): picked mon.noname-i con 0x7f5ac409e380 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient(hunting): picked mon.noname-b con 0x7f5ac40a2a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac406df30 con 0x7f5ac40a2a70 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac409cad0 con 0x7f5ac409e380 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac409ad80 con 0x7f5ac409bfc0 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient(hunting): authenticate will time out at 1126.013492s 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ac919b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5ac409bfc0 0x7f5ac4141db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56800/0 (socket says 172.21.15.26:56800) 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ac919b700 1 -- 172.21.15.26:0/3837477677 learned_addr learned my addr 172.21.15.26:0/3837477677 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515005939 0 0) 0x7f5ac409ad80 con 0x7f5ac409bfc0 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): my global_id is 134468 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aa8003570 con 0x7f5ac409bfc0 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711800477 0 0) 0x7f5ac406df30 con 0x7f5ac40a2a70 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): my global_id is 134489 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac409ad80 con 0x7f5ac40a2a70 2024-09-15T03:49:41.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236044760 0 0) 0x7f5ac409cad0 con 0x7f5ac409e380 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): my global_id is 134449 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac406df30 con 0x7f5ac409e380 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 20619669 0 0) 0x7f5aa8003570 con 0x7f5ac409bfc0 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ac409cad0 con 0x7f5ac409bfc0 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ac0002aa0 con 0x7f5ac409bfc0 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): dump: 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.601 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.602 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1728740804 0 0) 0x7f5ac406df30 con 0x7f5ac409e380 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5aa8003570 con 0x7f5ac409e380 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ab40016b0 con 0x7f5ac409e380 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): dump: 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2307560907 0 0) 0x7f5ac409ad80 con 0x7f5ac40a2a70 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ac406df30 con 0x7f5ac40a2a70 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5abc001d60 con 0x7f5ac40a2a70 2024-09-15T03:49:41.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient(hunting): dump: 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1616449854 0 0) 0x7f5ac409cad0 con 0x7f5ac409bfc0 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.122:6792/0 conn(0x7f5ac409e380 legacy=0x7f5ac4146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.26:6790/0 conn(0x7f5ac40a2a70 legacy=0x7f5ac414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 monclient: found mon.e 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac4122840 con 0x7f5ac409bfc0 2024-09-15T03:49:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 5 monclient: authenticate success, global_id 134468 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: _renew_subs 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac41218e0 con 0x7f5ac409bfc0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: _renew_subs 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac4123c70 con 0x7f5ac409bfc0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ac0001610 con 0x7f5ac409bfc0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ac0001b40 con 0x7f5ac409bfc0 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5abb7fe700 4 set_mon_vals no callback set 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.583+0000 7f5ab97fa700 10 monclient: dump: 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.608 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5ac0001d20 con 0x7f5ac409bfc0 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(228..228 src has 1..228) v4 ==== 3991+0+0 (unknown 3854576550 0 0) 0x7f5ac0043f90 con 0x7f5ac409bfc0 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5aca19d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5ac4122d60 con 0x7f5ac409a310 2024-09-15T03:49:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ac899a700 10 monclient: get_auth_request con 0x7f5aa8038060 auth_method 0 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731292037 0 0) 0x7f5ac4122d60 con 0x7f5ac409a310 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient(hunting): my global_id is 5037 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac409cad0 con 0x7f5ac409a310 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3173083891 0 0) 0x7f5ac409cad0 con 0x7f5ac409a310 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ac4122d60 con 0x7f5ac409a310 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5abc004ef0 con 0x7f5ac409a310 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:41.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient: dump: 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2733505466 0 0) 0x7f5ac4122d60 con 0x7f5ac409a310 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ac409a6e0 con 0x7f5ac409a310 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5abc005230 con 0x7f5ac409a310 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.26:6792/0 conn(0x7f5ac409a310 legacy=0x7f5ac405e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.587+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.26:6792/0 conn(0x7f5ac409a310 legacy=0x7f5ac405e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5ac404bbf0 con 0x7f5ac402ae80 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550523572 0 0) 0x7f5ac404bbf0 con 0x7f5ac402ae80 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient(hunting): my global_id is 5046 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ac4122d60 con 0x7f5ac402ae80 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1684462653 0 0) 0x7f5ac4122d60 con 0x7f5ac402ae80 2024-09-15T03:49:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ac404bbf0 con 0x7f5ac402ae80 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5abc005a70 con 0x7f5ac402ae80 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient: dump: 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 288706093 0 0) 0x7f5ac404bbf0 con 0x7f5ac402ae80 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5ac4066de0 con 0x7f5ac402ae80 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 2436004413) 0x7f5abc00ada0 con 0x7f5ac402ae80 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.26:6792/0 conn(0x7f5ac402ae80 legacy=0x7f5ac4119490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ab97fa700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.26:6792/0 conn(0x7f5ac402ae80 legacy=0x7f5ac4119490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:41.619 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.620 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.621 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:41.622 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.623 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.624 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.625 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.626 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.629 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.630 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.632 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.633 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.635 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.636 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:49:41.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.638 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:41.639 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.122:6812/20895 conn(0x7f5aa8038060 legacy=0x7f5aa803a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 10 monclient: shutdown 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 >> v1:172.21.15.26:6793/0 conn(0x7f5ac409bfc0 legacy=0x7f5ac4141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5ac999c700 1 -- 172.21.15.26:0/3837477677 reap_dead start 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 shutdown_connections 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 >> 172.21.15.26:0/3837477677 conn(0x7f5ac4063cd0 msgr2=0x7f5ac406c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 shutdown_connections 2024-09-15T03:49:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.599+0000 7f5aca19d700 1 -- 172.21.15.26:0/3837477677 wait complete. 2024-09-15T03:49:41.641 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:41.641 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:49:41.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:49:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.755+0000 7f45b286e700 10 monclient: get_monmap_and_config 2024-09-15T03:49:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.755+0000 7f45b286e700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.755+0000 7f45b286e700 10 monclient: monmap: 2024-09-15T03:49:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:41.759712+0000 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:41.759712+0000 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.761 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 1 Processor -- start 2024-09-15T03:49:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 1 -- start start 2024-09-15T03:49:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient: init 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient: _add_conns ranks=[2,6,1,7,4,8,0,5,3] 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient(hunting): picked mon.noname-b con 0x7f45ac097a60 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient(hunting): picked mon.noname-d con 0x7f45ac132610 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient(hunting): picked mon.noname-f con 0x7f45ac136110 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac04ca10 con 0x7f45ac136110 2024-09-15T03:49:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac098b90 con 0x7f45ac097a60 2024-09-15T03:49:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0993a0 con 0x7f45ac132610 2024-09-15T03:49:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b286e700 10 monclient(hunting): authenticate will time out at 1126.190989s 2024-09-15T03:49:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b206d700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f45ac136110 0x7f45ac1384f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:55408/0 (socket says 172.21.15.26:55408) 2024-09-15T03:49:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b106b700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f45ac132610 0x7f45ac1349f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34682/0 (socket says 172.21.15.26:34682) 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b206d700 1 -- 172.21.15.26:0/1768279722 learned_addr learned my addr 172.21.15.26:0/1768279722 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b186c700 1 --1- 172.21.15.26:0/1768279722 >> v1:172.21.15.26:6790/0 conn(0x7f45ac097a60 0x7f45ac097e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:50314/0 (socket says 172.21.15.26:50314) 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757416163 0 0) 0x7f45ac0993a0 con 0x7f45ac132610 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b086a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b086a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b086a700 10 monclient(hunting): my global_id is 5055 2024-09-15T03:49:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.759+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4594003570 con 0x7f45ac132610 2024-09-15T03:49:41.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362191639 0 0) 0x7f45ac098b90 con 0x7f45ac097a60 2024-09-15T03:49:41.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient(hunting): my global_id is 134498 2024-09-15T03:49:41.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac0993a0 con 0x7f45ac097a60 2024-09-15T03:49:41.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2057756436 0 0) 0x7f45ac0993a0 con 0x7f45ac097a60 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 >> v1:172.21.15.26:6792/0 conn(0x7f45ac132610 legacy=0x7f45ac1349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 >> v1:172.21.15.122:6789/0 conn(0x7f45ac136110 legacy=0x7f45ac1384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 monclient: found mon.noname-b 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac099680 con 0x7f45ac097a60 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45a4001790 con 0x7f45ac097a60 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:49:41.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: dump: 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.773 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 5 monclient: authenticate success, global_id 134498 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f45a4001ab0 con 0x7f45ac097a60 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 4 set_mon_vals no callback set 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 1 -- 172.21.15.26:0/1768279722 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45a4001ce0 con 0x7f45ac097a60 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b086a700 10 monclient: dump: 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.774 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.775 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:955: done (4 left) 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:956: read oid 20 snap -1 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:956: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:957: snap_create 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:951: finishing write tid 1 to smithi12223906-17 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:951: finishing write tid 2 to smithi12223906-17 2024-09-15T03:49:41.776 INFO:tasks.rados.rados.0.smithi122.stdout:951: finishing write tid 3 to smithi12223906-17 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:951: finishing write tid 4 to smithi12223906-17 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 303 (ObjNum 371 snap 104 seq_num 371) dirty exists 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:951: left oid 17 (ObjNum 371 snap 104 seq_num 371) 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:952: finishing write tid 1 to smithi12223906-12 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:952: finishing write tid 2 to smithi12223906-12 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:952: finishing write tid 3 to smithi12223906-12 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:953: finishing copy_from to smithi12223906-2 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:953: got expected ENOENT (src dne) 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:952: finishing write tid 4 to smithi12223906-12 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 339 (ObjNum 372 snap 104 seq_num 372) dirty exists 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:952: left oid 12 (ObjNum 372 snap 104 seq_num 372) 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:953: finishing copy_from racing read to smithi12223906-2 2024-09-15T03:49:41.777 INFO:tasks.rados.rados.0.smithi122.stdout:954: expect (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:49:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 10 monclient: shutdown 2024-09-15T03:49:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 1 -- 172.21.15.26:0/1768279722 >> v1:172.21.15.26:6790/0 conn(0x7f45ac097a60 legacy=0x7f45ac097e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 1 -- 172.21.15.26:0/1768279722 shutdown_connections 2024-09-15T03:49:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.767+0000 7f45b286e700 1 -- 172.21.15.26:0/1768279722 >> 172.21.15.26:0/1768279722 conn(0x7f45ac063cd0 msgr2=0x7f45ac0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.771+0000 7f45b286e700 1 -- 172.21.15.26:0/1768279722 shutdown_connections 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.771+0000 7f45b286e700 1 -- 172.21.15.26:0/1768279722 wait complete. 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.771+0000 7f45b286e700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.771+0000 7f45b286e700 10 monclient: monmap: 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.771+0000 7f45b286e700 1 Processor -- start 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 1 -- start start 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient: init 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient: _add_conns ranks=[1,8,4,5,0,2,7,6,3] 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient(hunting): picked mon.noname-f con 0x7f45ac097a60 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient(hunting): picked mon.noname-e con 0x7f45ac132610 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient(hunting): picked mon.noname-c con 0x7f45ac136110 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac06d8c0 con 0x7f45ac097a60 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac138a50 con 0x7f45ac136110 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac09bea0 con 0x7f45ac132610 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b286e700 10 monclient(hunting): authenticate will time out at 1126.205138s 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b106b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f45ac132610 0x7f45ac144e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56814/0 (socket says 172.21.15.26:56814) 2024-09-15T03:49:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.775+0000 7f45b106b700 1 -- 172.21.15.26:0/2780405435 learned_addr learned my addr 172.21.15.26:0/2780405435 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740865811 0 0) 0x7f45ac138a50 con 0x7f45ac136110 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): my global_id is 174235 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4598003570 con 0x7f45ac136110 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320795581 0 0) 0x7f45ac09bea0 con 0x7f45ac132610 2024-09-15T03:49:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): my global_id is 134477 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac138a50 con 0x7f45ac132610 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3733075178 0 0) 0x7f45ac138a50 con 0x7f45ac132610 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45ac09bea0 con 0x7f45ac132610 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45a0001a00 con 0x7f45ac132610 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): dump: 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.786 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3511775522 0 0) 0x7f45ac09bea0 con 0x7f45ac132610 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.26:6791/0 conn(0x7f45ac136110 legacy=0x7f45ac1485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.122:6789/0 conn(0x7f45ac097a60 legacy=0x7f45ac131840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 monclient: found mon.e 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac09ba50 con 0x7f45ac132610 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 5 monclient: authenticate success, global_id 134477 2024-09-15T03:49:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 10 monclient: _renew_subs 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac06d340 con 0x7f45ac132610 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 10 monclient: _renew_subs 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45b286e700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac06c8f0 con 0x7f45ac132610 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f45a0001cb0 con 0x7f45ac132610 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f45a0004ea0 con 0x7f45ac132610 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45a9ffb700 10 monclient: dump: 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.789 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.791 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.779+0000 7f45ab7fe700 4 set_mon_vals no callback set 2024-09-15T03:49:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f45a000f860 con 0x7f45ac132610 2024-09-15T03:49:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 4063+0+0 (unknown 3248377167 0 0) 0x7f45a0043f40 con 0x7f45ac132610 2024-09-15T03:49:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45b186c700 10 monclient: get_auth_request con 0x7f45980377f0 auth_method 0 2024-09-15T03:49:41.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45937fe700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f458c006590 con 0x7f458c002840 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048286557 0 0) 0x7f458c006590 con 0x7f458c002840 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 10 monclient(hunting): my global_id is 134486 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac09bea0 con 0x7f458c002840 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2800404529 0 0) 0x7f45ac09bea0 con 0x7f458c002840 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f458c006590 con 0x7f458c002840 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f459c001a60 con 0x7f458c002840 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.783+0000 7f45a9ffb700 10 monclient: dump: 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.793 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.794 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2601130518 0 0) 0x7f458c006590 con 0x7f458c002840 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f458c00a130 con 0x7f458c002840 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f459c002020 con 0x7f458c002840 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.26:6793/0 conn(0x7f458c002840 legacy=0x7f458c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.787+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.26:6793/0 conn(0x7f458c002840 legacy=0x7f458c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45937fe700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f458c007520 con 0x7f458c0068d0 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288168897 0 0) 0x7f458c007520 con 0x7f458c0068d0 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient(hunting): my global_id is 134495 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f458c006590 con 0x7f458c0068d0 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3547221871 0 0) 0x7f458c006590 con 0x7f458c0068d0 2024-09-15T03:49:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f458c007520 con 0x7f458c0068d0 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f459c005160 con 0x7f458c0068d0 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient: dump: 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2425084279 0 0) 0x7f458c007520 con 0x7f458c0068d0 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f458c008760 con 0x7f458c0068d0 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 2001617769) 0x7f459c007360 con 0x7f458c0068d0 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.26:6793/0 conn(0x7f458c0068d0 legacy=0x7f458c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.799+0000 7f45a9ffb700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.26:6793/0 conn(0x7f458c0068d0 legacy=0x7f458c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:41.807 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.808 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.809 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.811 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.812 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.813 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.814 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:41.815 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.816 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.818 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.819 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.822 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.823 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:49:41.826 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.827 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.827 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:41.827 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.122:6812/20895 conn(0x7f45980377f0 legacy=0x7f4598039ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 10 monclient: shutdown 2024-09-15T03:49:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 >> v1:172.21.15.26:6793/0 conn(0x7f45ac132610 legacy=0x7f45ac144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45b206d700 1 -- 172.21.15.26:0/2780405435 reap_dead start 2024-09-15T03:49:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45b206d700 1 -- 172.21.15.26:0/2780405435 reap_dead start 2024-09-15T03:49:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 shutdown_connections 2024-09-15T03:49:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 >> 172.21.15.26:0/2780405435 conn(0x7f45ac063cd0 msgr2=0x7f45ac06f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 shutdown_connections 2024-09-15T03:49:41.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.803+0000 7f45937fe700 1 -- 172.21.15.26:0/2780405435 wait complete. 2024-09-15T03:49:41.829 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:41.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:49:41.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:49:41.872 INFO:tasks.rados.rados.0.smithi122.stdout:956: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:49:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.935+0000 7fa5d0b97700 10 monclient: get_monmap_and_config 2024-09-15T03:49:41.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.935+0000 7fa5d0b97700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.935+0000 7fa5d0b97700 10 monclient: monmap: 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:41.937915+0000 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:41.937915+0000 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.939 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.935+0000 7fa5d0b97700 1 Processor -- start 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.935+0000 7fa5d0b97700 1 -- start start 2024-09-15T03:49:41.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.935+0000 7fa5d0b97700 10 monclient: init 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient: _add_conns ranks=[0,1,6,2,8,7,4,5,3] 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient(hunting): picked mon.noname-a con 0x7fa5cc070fb0 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient(hunting): picked mon.noname-f con 0x7fa5cc09de00 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient(hunting): picked mon.noname-d con 0x7fa5cc09e8e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc04ca10 con 0x7fa5cc070fb0 2024-09-15T03:49:41.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc09b120 con 0x7fa5cc09de00 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5cbfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fa5cc09e8e0 0x7fa5cc137d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34698/0 (socket says 172.21.15.26:34698) 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5cbfff700 1 -- 172.21.15.26:0/4260577472 learned_addr learned my addr 172.21.15.26:0/4260577472 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5cb7fe700 1 --1- 172.21.15.26:0/4260577472 >> v1:172.21.15.26:6789/0 conn(0x7fa5cc070fb0 0x7fa5cc09c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47348/0 (socket says 172.21.15.26:47348) 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- 172.21.15.26:0/4260577472 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc09b400 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient(hunting): authenticate will time out at 1126.369439s 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500450226 0 0) 0x7fa5cc04ca10 con 0x7fa5cc070fb0 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): my global_id is 4833 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b8003570 con 0x7fa5cc070fb0 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409886420 0 0) 0x7fa5cc09b400 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): my global_id is 5064 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5cc04ca10 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488497540 0 0) 0x7fa5cc09b120 con 0x7fa5cc09de00 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient(hunting): my global_id is 134335 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5cc09b400 con 0x7fa5cc09de00 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397732574 0 0) 0x7fa5b8003570 con 0x7fa5cc070fb0 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 >> v1:172.21.15.26:6792/0 conn(0x7fa5cc09e8e0 legacy=0x7fa5cc137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 >> v1:172.21.15.122:6789/0 conn(0x7fa5cc09de00 legacy=0x7fa5cc09e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 monclient: found mon.noname-a 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc13c490 con 0x7fa5cc070fb0 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5bc0016e0 con 0x7fa5cc070fb0 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: dump: 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.947 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 5 monclient: authenticate success, global_id 4833 2024-09-15T03:49:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5bc0018d0 con 0x7fa5cc070fb0 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 1 -- 172.21.15.26:0/4260577472 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5bc004830 con 0x7fa5cc070fb0 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 4 set_mon_vals no callback set 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5caffd700 10 monclient: dump: 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient: shutdown 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- 172.21.15.26:0/4260577472 >> v1:172.21.15.26:6789/0 conn(0x7fa5cc070fb0 legacy=0x7fa5cc09c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- 172.21.15.26:0/4260577472 shutdown_connections 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- 172.21.15.26:0/4260577472 >> 172.21.15.26:0/4260577472 conn(0x7fa5cc0a2ea0 msgr2=0x7fa5cc09eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- 172.21.15.26:0/4260577472 shutdown_connections 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 1 -- 172.21.15.26:0/4260577472 wait complete. 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.939+0000 7fa5d0b97700 10 monclient: build_initial_monmap 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: monmap: 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:41.952 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 Processor -- start 2024-09-15T03:49:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 -- start start 2024-09-15T03:49:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: init 2024-09-15T03:49:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: _add_conns ranks=[7,0,2,8,1,5,6,3,4] 2024-09-15T03:49:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient(hunting): picked mon.noname-i con 0x7fa5cc09de00 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient(hunting): picked mon.noname-a con 0x7fa5cc09e8e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient(hunting): picked mon.noname-b con 0x7fa5cc06e3c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc13c620 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc06b910 con 0x7fa5cc06e3c0 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc09f910 con 0x7fa5cc09de00 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient(hunting): authenticate will time out at 1126.373223s 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c3fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa5cc09e8e0 0x7fa5cc13fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47350/0 (socket says 172.21.15.26:47350) 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c3fff700 1 -- 172.21.15.26:0/2958749302 learned_addr learned my addr 172.21.15.26:0/2958749302 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5cb7fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6792/0 conn(0x7fa5cc09de00 legacy=0x7fa5cc13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5cb7fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6792/0 conn(0x7fa5cc09de00 legacy=0x7fa5cc13f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:49:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5cb7fe700 1 --1- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6792/0 conn(0x7fa5cc09de00 0x7fa5cc13f7e0 :-1 s=OPENED pgs=109 cs=1 l=1).handle_message read tag failed 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5cb7fe700 1 --1- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6792/0 conn(0x7fa5cc09de00 0x7fa5cc13f7e0 :-1 s=OPENED pgs=109 cs=1 l=1).fault on lossy channel, failing 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590737536 0 0) 0x7fa5cc13c620 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): my global_id is 4842 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac003570 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.122:6792/0 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052137232 0 0) 0x7fa5c4001550 con 0x7fa5cc06e3c0 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): my global_id is 134507 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5cc13c620 con 0x7fa5cc06e3c0 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4215614455 0 0) 0x7fa5ac003570 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5c4001550 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5b4002b70 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): dump: 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3577559243 0 0) 0x7fa5cc13c620 con 0x7fa5cc06e3c0 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5ac003570 con 0x7fa5cc06e3c0 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5c4001e90 con 0x7fa5cc06e3c0 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient(hunting): dump: 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3358952456 0 0) 0x7fa5c4001550 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6792/0 conn(0x7fa5cc09de00 legacy=0x7fa5cc13f7e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.26:6790/0 conn(0x7fa5cc06e3c0 legacy=0x7fa5cc1478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 monclient: found mon.a 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc06d620 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: _finish_auth 0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 5 monclient: authenticate success, global_id 4842 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: _renew_subs 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc06c040 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: _renew_subs 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:49:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc06d340 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5b40016e0 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5b4001b90 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5ca7fc700 4 set_mon_vals no callback set 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.943+0000 7fa5c37fe700 10 monclient: dump: 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.965 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5d0b97700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa5b0006590 con 0x7fa5b0002840 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa5b4001d70 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 4063+0+0 (unknown 3248377167 0 0) 0x7fa5b4044090 con 0x7fa5cc09e8e0 2024-09-15T03:49:41.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644028054 0 0) 0x7fa5b0006590 con 0x7fa5b0002840 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient(hunting): my global_id is 134344 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c4001550 con 0x7fa5b0002840 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5cbfff700 10 monclient: get_auth_request con 0x7fa5ac037b80 auth_method 0 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 599802365 0 0) 0x7fa5c4001550 con 0x7fa5b0002840 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5b0006590 con 0x7fa5b0002840 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5bc0014a0 con 0x7fa5b0002840 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient: dump: 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 423246719 0 0) 0x7fa5b0006590 con 0x7fa5b0002840 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5b000a130 con 0x7fa5b0002840 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa5bc004b00 con 0x7fa5b0002840 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6789/0 conn(0x7fa5b0002840 legacy=0x7fa5b0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.947+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6789/0 conn(0x7fa5b0002840 legacy=0x7fa5b0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5d0b97700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa5b0009640 con 0x7fa5b00090b0 2024-09-15T03:49:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154666200 0 0) 0x7fa5b0009640 con 0x7fa5b00090b0 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 10 monclient(hunting): my global_id is 134353 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b0006590 con 0x7fa5b00090b0 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 654070322 0 0) 0x7fa5b0006590 con 0x7fa5b00090b0 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5b0009640 con 0x7fa5b00090b0 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5bc0052c0 con 0x7fa5b00090b0 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 10 monclient: dump: 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1699165700 0 0) 0x7fa5b0009640 con 0x7fa5b00090b0 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.959+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa5b00098f0 con 0x7fa5b00090b0 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 272496507) 0x7fa5bc007480 con 0x7fa5b00090b0 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5c37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5c37fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6789/0 conn(0x7fa5b00090b0 legacy=0x7fa5b0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5c37fe700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6789/0 conn(0x7fa5b00090b0 legacy=0x7fa5b0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.973 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.974 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.975 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.976 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:41.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:41.978 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.980 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.982 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:41.986 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.987 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.989 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:41.990 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:41.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:41.992 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:41.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.122:6812/20895 conn(0x7fa5ac037b80 legacy=0x7fa5ac03a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 10 monclient: shutdown 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 >> v1:172.21.15.26:6789/0 conn(0x7fa5cc09e8e0 legacy=0x7fa5cc13fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5cbfff700 1 -- 172.21.15.26:0/2958749302 reap_dead start 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5cbfff700 1 -- 172.21.15.26:0/2958749302 reap_dead start 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 shutdown_connections 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 >> 172.21.15.26:0/2958749302 conn(0x7fa5cc0a2ea0 msgr2=0x7fa5cc0a1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 shutdown_connections 2024-09-15T03:49:41.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:41.963+0000 7fa5d0b97700 1 -- 172.21.15.26:0/2958749302 wait complete. 2024-09-15T03:49:41.994 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:41.995 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:49:41.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.103+0000 7faa9ffff700 10 monclient: get_monmap_and_config 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.103+0000 7faa9ffff700 10 monclient: build_initial_monmap 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.103+0000 7faa9ffff700 10 monclient: monmap: 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:42.107521+0000 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:42.107521+0000 2024-09-15T03:49:42.108 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:42.109 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 Processor -- start 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- start start 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient: init 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient: _add_conns ranks=[0,4,7,6,5,8,3,2,1] 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient(hunting): picked mon.noname-a con 0x7faaa00a2a70 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient(hunting): picked mon.noname-c con 0x7faaa009e380 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient(hunting): picked mon.noname-i con 0x7faaa009bfc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:42.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa004ca10 con 0x7faaa00a2a70 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa009d3e0 con 0x7faaa009e380 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa0098880 con 0x7faaa009bfc0 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient(hunting): authenticate will time out at 1126.536995s 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa97fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7faaa009e380 0x7faaa0098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43170/0 (socket says 172.21.15.26:43170) 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa97fff700 1 -- 172.21.15.26:0/533834940 learned_addr learned my addr 172.21.15.26:0/533834940 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871598674 0 0) 0x7faaa004ca10 con 0x7faaa00a2a70 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient(hunting): my global_id is 4851 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa8c003570 con 0x7faaa00a2a70 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737035521 0 0) 0x7faaa009d3e0 con 0x7faaa009e380 2024-09-15T03:49:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient(hunting): my global_id is 174244 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa004ca10 con 0x7faaa009e380 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728762354 0 0) 0x7faa8c003570 con 0x7faaa00a2a70 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 >> v1:172.21.15.122:6792/0 conn(0x7faaa009bfc0 legacy=0x7faaa0138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 >> v1:172.21.15.26:6791/0 conn(0x7faaa009e380 legacy=0x7faaa0098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 monclient: found mon.noname-a 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa009d100 con 0x7faaa00a2a70 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa900017b0 con 0x7faaa00a2a70 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: dump: 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 5 monclient: authenticate success, global_id 4851 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa90001a50 con 0x7faaa00a2a70 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 1 -- 172.21.15.26:0/533834940 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa90001c40 con 0x7faaa00a2a70 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 4 set_mon_vals no callback set 2024-09-15T03:49:42.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9e7fc700 10 monclient: dump: 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.118 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 10 monclient: shutdown 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- 172.21.15.26:0/533834940 >> v1:172.21.15.26:6789/0 conn(0x7faaa00a2a70 legacy=0x7faaa0097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- 172.21.15.26:0/533834940 shutdown_connections 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- 172.21.15.26:0/533834940 >> 172.21.15.26:0/533834940 conn(0x7faaa0063cd0 msgr2=0x7faaa00a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- 172.21.15.26:0/533834940 shutdown_connections 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.107+0000 7faa9ffff700 1 -- 172.21.15.26:0/533834940 wait complete. 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: build_initial_monmap 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: monmap: 2024-09-15T03:49:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:42.121 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 Processor -- start 2024-09-15T03:49:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 -- start start 2024-09-15T03:49:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: init 2024-09-15T03:49:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: _add_conns ranks=[7,1,0,3,5,2,4,8,6] 2024-09-15T03:49:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient(hunting): picked mon.noname-i con 0x7faaa009bfc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient(hunting): picked mon.noname-f con 0x7faaa009e380 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient(hunting): picked mon.noname-a con 0x7faaa00a2a70 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa011b4b0 con 0x7faaa00a2a70 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa006df30 con 0x7faaa009e380 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa011c4a0 con 0x7faaa009bfc0 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient(hunting): authenticate will time out at 1126.541896s 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9f7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faaa00a2a70 0x7faaa0147dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47376/0 (socket says 172.21.15.26:47376) 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9f7fe700 1 -- 172.21.15.26:0/626887050 learned_addr learned my addr 172.21.15.26:0/626887050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa97fff700 1 --1- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6789/0 conn(0x7faaa009e380 0x7faaa0140270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:55442/0 (socket says 172.21.15.26:55442) 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711509535 0 0) 0x7faaa011b4b0 con 0x7faaa00a2a70 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): my global_id is 4860 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa80003570 con 0x7faaa00a2a70 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428975467 0 0) 0x7faaa011c4a0 con 0x7faaa009bfc0 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): my global_id is 134467 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa011b4b0 con 0x7faaa009bfc0 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570435975 0 0) 0x7faaa006df30 con 0x7faaa009e380 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): my global_id is 134362 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa011c4a0 con 0x7faaa009e380 2024-09-15T03:49:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1098047362 0 0) 0x7faaa011b4b0 con 0x7faaa009bfc0 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaa006df30 con 0x7faaa009bfc0 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa90001bc0 con 0x7faaa009bfc0 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3778183656 0 0) 0x7faa80003570 con 0x7faaa00a2a70 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaa011b4b0 con 0x7faaa00a2a70 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa98001e20 con 0x7faaa00a2a70 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:49:42.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient(hunting): dump: 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 604576930 0 0) 0x7faaa006df30 con 0x7faaa009bfc0 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6789/0 conn(0x7faaa009e380 legacy=0x7faaa0140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.26:6789/0 conn(0x7faaa00a2a70 legacy=0x7faaa0147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 monclient: found mon.i 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa011d6a0 con 0x7faaa009bfc0 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: _finish_auth 0 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 5 monclient: authenticate success, global_id 134467 2024-09-15T03:49:42.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: _renew_subs 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa011ca30 con 0x7faaa009bfc0 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faa900024a0 con 0x7faaa009bfc0 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa90002950 con 0x7faaa009bfc0 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9dffb700 4 set_mon_vals no callback set 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa96ffd700 10 monclient: dump: 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.133 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: _renew_subs 2024-09-15T03:49:42.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.111+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa011c170 con 0x7faaa009bfc0 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa9ffff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7faaa004bbf0 con 0x7faaa0119e50 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7faa9000d3e0 con 0x7faaa009bfc0 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 4063+0+0 (unknown 3248377167 0 0) 0x7faa90041720 con 0x7faaa009bfc0 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517654040 0 0) 0x7faaa004bbf0 con 0x7faaa0119e50 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient(hunting): my global_id is 4917 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa006df30 con 0x7faaa0119e50 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa9f7fe700 10 monclient: get_auth_request con 0x7faa80037d30 auth_method 0 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2931817082 0 0) 0x7faaa006df30 con 0x7faaa0119e50 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaa004bbf0 con 0x7faaa0119e50 2024-09-15T03:49:42.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa88004b70 con 0x7faaa0119e50 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient: dump: 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.136 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2405376127 0 0) 0x7faaa004bbf0 con 0x7faaa0119e50 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faaa009cad0 con 0x7faaa0119e50 2024-09-15T03:49:42.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7faa88005110 con 0x7faaa0119e50 2024-09-15T03:49:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6790/0 conn(0x7faaa0119e50 legacy=0x7faaa00696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.115+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6790/0 conn(0x7faaa0119e50 legacy=0x7faaa00696d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.123+0000 7faa9ffff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.123+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7faaa0066de0 con 0x7faaa002ad10 2024-09-15T03:49:42.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108912296 0 0) 0x7faaa0066de0 con 0x7faaa002ad10 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient(hunting): my global_id is 4926 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa004bbf0 con 0x7faaa002ad10 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1884554777 0 0) 0x7faaa004bbf0 con 0x7faaa002ad10 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faaa0066de0 con 0x7faaa002ad10 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faa88005940 con 0x7faaa002ad10 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient: dump: 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.140 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 414416143 0 0) 0x7faaa0066de0 con 0x7faaa002ad10 2024-09-15T03:49:42.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faaa011a8e0 con 0x7faaa002ad10 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1155173854) 0x7faa88005d30 con 0x7faaa002ad10 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6790/0 conn(0x7faaa002ad10 legacy=0x7faaa0069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa96ffd700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6790/0 conn(0x7faaa002ad10 legacy=0x7faaa0069de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:42.142 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:49:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.144 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:42.145 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:42.146 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.147 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:42.148 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.149 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.150 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.151 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:42.152 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.155 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.156 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:42.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.160 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 6 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:42.161 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:42.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6812/20895 conn(0x7faa80037d30 legacy=0x7faa8003a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 10 monclient: shutdown 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 >> v1:172.21.15.122:6792/0 conn(0x7faaa009bfc0 legacy=0x7faaa013fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9f7fe700 1 -- 172.21.15.26:0/626887050 reap_dead start 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9f7fe700 1 -- 172.21.15.26:0/626887050 reap_dead start 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 shutdown_connections 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 >> 172.21.15.26:0/626887050 conn(0x7faaa0063cd0 msgr2=0x7faaa0097b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 shutdown_connections 2024-09-15T03:49:42.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.127+0000 7faa9ffff700 1 -- 172.21.15.26:0/626887050 wait complete. 2024-09-15T03:49:42.164 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:42.164 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:49:42.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:49:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient: get_monmap_and_config 2024-09-15T03:49:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient: build_initial_monmap 2024-09-15T03:49:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient: monmap: 2024-09-15T03:49:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:42.273596+0000 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:42.273596+0000 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:42.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 1 Processor -- start 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 1 -- start start 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient: init 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient: _add_conns ranks=[4,6,3,1,8,5,0,7,2] 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient(hunting): picked mon.noname-c con 0x7f7be40a6880 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient(hunting): picked mon.noname-d con 0x7f7be409faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:42.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient(hunting): picked mon.noname-g con 0x7f7be4138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be404ca10 con 0x7f7be4138d30 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40a0580 con 0x7f7be40a6880 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be4133780 con 0x7f7be409faa0 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7be90d6700 10 monclient(hunting): authenticate will time out at 1126.703513s 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7bdbfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f7be409faa0 0x7f7be409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34714/0 (socket says 172.21.15.26:34714) 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.271+0000 7f7bdbfff700 1 -- 172.21.15.26:0/4249551486 learned_addr learned my addr 172.21.15.26:0/4249551486 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707461588 0 0) 0x7f7be4133780 con 0x7f7be409faa0 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): my global_id is 5073 2024-09-15T03:49:42.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd0003570 con 0x7f7be409faa0 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928127237 0 0) 0x7f7bd4001110 con 0x7f7be40a6880 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): my global_id is 174253 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be4133780 con 0x7f7be40a6880 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726543429 0 0) 0x7f7be404ca10 con 0x7f7be4138d30 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient(hunting): my global_id is 4935 2024-09-15T03:49:42.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd4001110 con 0x7f7be4138d30 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967337234 0 0) 0x7f7bd0003570 con 0x7f7be409faa0 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 >> v1:172.21.15.26:6791/0 conn(0x7f7be40a6880 legacy=0x7f7be4133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 >> v1:172.21.15.122:6790/0 conn(0x7f7be4138d30 legacy=0x7f7be413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 monclient: found mon.noname-d 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40a6c50 con 0x7f7be409faa0 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: _finish_auth 0 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 5 monclient: authenticate success, global_id 5073 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7bcc0017b0 con 0x7f7be409faa0 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: dump: 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.283 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:42.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7bcc0019d0 con 0x7f7be409faa0 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 4 set_mon_vals no callback set 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 1 -- 172.21.15.26:0/4249551486 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7bcc0048f0 con 0x7f7be409faa0 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be37fe700 10 monclient: dump: 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: shutdown 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- 172.21.15.26:0/4249551486 >> v1:172.21.15.26:6792/0 conn(0x7f7be409faa0 legacy=0x7f7be409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- 172.21.15.26:0/4249551486 shutdown_connections 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- 172.21.15.26:0/4249551486 >> 172.21.15.26:0/4249551486 conn(0x7f7be406e2a0 msgr2=0x7f7be40a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- 172.21.15.26:0/4249551486 shutdown_connections 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- 172.21.15.26:0/4249551486 wait complete. 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: build_initial_monmap 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: monmap: 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:42.289 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 Processor -- start 2024-09-15T03:49:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- start start 2024-09-15T03:49:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: init 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient: _add_conns ranks=[5,4,1,3,2,0,7,8,6] 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient(hunting): picked mon.noname-h con 0x7f7be409faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient(hunting): picked mon.noname-c con 0x7f7be40a6880 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient(hunting): picked mon.noname-f con 0x7f7be4138d30 addr v1:172.21.15.122:6789/0 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be409a9f0 con 0x7f7be4138d30 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be4098880 con 0x7f7be40a6880 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be4098be0 con 0x7f7be409faa0 2024-09-15T03:49:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be90d6700 10 monclient(hunting): authenticate will time out at 1126.707367s 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7bdbfff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f7be40a6880 0x7f7be4098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43180/0 (socket says 172.21.15.26:43180) 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7bdbfff700 1 -- 172.21.15.26:0/1066432373 learned_addr learned my addr 172.21.15.26:0/1066432373 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084671301 0 0) 0x7f7be4098880 con 0x7f7be40a6880 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be0ff9700 10 monclient(hunting): my global_id is 174262 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.275+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc4003570 con 0x7f7be40a6880 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502388631 0 0) 0x7f7be4098be0 con 0x7f7be409faa0 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): my global_id is 4991 2024-09-15T03:49:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be4098880 con 0x7f7be409faa0 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3948654450 0 0) 0x7f7bc4003570 con 0x7f7be40a6880 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7be4098be0 con 0x7f7be40a6880 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7bcc0016b0 con 0x7f7be40a6880 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): dump: 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473262343 0 0) 0x7f7be409a9f0 con 0x7f7be4138d30 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): my global_id is 134371 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc4003570 con 0x7f7be4138d30 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1981026038 0 0) 0x7f7be4098be0 con 0x7f7be40a6880 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6791/0 conn(0x7f7be409faa0 legacy=0x7f7be4097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6789/0 conn(0x7f7be4138d30 legacy=0x7f7be414a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 monclient: found mon.c 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be409a710 con 0x7f7be40a6880 2024-09-15T03:49:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: _finish_auth 0 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 5 monclient: authenticate success, global_id 174262 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 10 monclient: _renew_subs 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be409c920 con 0x7f7be40a6880 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 10 monclient: _renew_subs 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be409b3d0 con 0x7f7be40a6880 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7bcc004910 con 0x7f7be40a6880 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be27fc700 4 set_mon_vals no callback set 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7bcc0051b0 con 0x7f7be40a6880 2024-09-15T03:49:42.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient: dump: 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f7bcc00fb70 con 0x7f7be40a6880 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7bc8006590 con 0x7f7bc8002840 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 4063+0+0 (unknown 3248377167 0 0) 0x7f7bcc044190 con 0x7f7be40a6880 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be88d5700 10 monclient: get_auth_request con 0x7f7bc40375f0 auth_method 0 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861777999 0 0) 0x7f7bc8006590 con 0x7f7bc8002840 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 10 monclient(hunting): my global_id is 5000 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.279+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be4098be0 con 0x7f7bc8002840 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 172218011 0 0) 0x7f7be4098be0 con 0x7f7bc8002840 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bc8006590 con 0x7f7bc8002840 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7bd40023d0 con 0x7f7bc8002840 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 10 monclient: dump: 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3350088688 0 0) 0x7f7bc8006590 con 0x7f7bc8002840 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bc800a130 con 0x7f7bc8002840 2024-09-15T03:49:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7bd4004760 con 0x7f7bc8002840 2024-09-15T03:49:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6791/0 conn(0x7f7bc8002840 legacy=0x7f7bc8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.283+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6791/0 conn(0x7f7bc8002840 legacy=0x7f7bc8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:42.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.291+0000 7f7be90d6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:42.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.291+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7bc8009640 con 0x7f7bc80090b0 2024-09-15T03:49:42.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252256533 0 0) 0x7f7bc8009640 con 0x7f7bc80090b0 2024-09-15T03:49:42.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient(hunting): my global_id is 5009 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc8006590 con 0x7f7bc80090b0 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 704652692 0 0) 0x7f7bc8006590 con 0x7f7bc80090b0 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bc8009640 con 0x7f7bc80090b0 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7bd4002f30 con 0x7f7bc80090b0 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient: dump: 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.307 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3729758983 0 0) 0x7f7bc8009640 con 0x7f7bc80090b0 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7bc80098f0 con 0x7f7bc80090b0 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3255381879) 0x7f7bd4004a10 con 0x7f7bc80090b0 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6791/0 conn(0x7f7bc80090b0 legacy=0x7f7bc8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be0ff9700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6791/0 conn(0x7f7bc80090b0 legacy=0x7f7bc8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:42.309 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.311 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:42.312 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:42.314 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:42.317 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:42.318 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.319 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:42.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.323 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:42.326 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.327 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:42.328 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.122:6812/20895 conn(0x7f7bc40375f0 legacy=0x7f7bc4039aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 10 monclient: shutdown 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 >> v1:172.21.15.26:6791/0 conn(0x7f7be40a6880 legacy=0x7f7be4098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be88d5700 1 -- 172.21.15.26:0/1066432373 reap_dead start 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be88d5700 1 -- 172.21.15.26:0/1066432373 reap_dead start 2024-09-15T03:49:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 shutdown_connections 2024-09-15T03:49:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 >> 172.21.15.26:0/1066432373 conn(0x7f7be406e2a0 msgr2=0x7f7be4067880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 shutdown_connections 2024-09-15T03:49:42.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.295+0000 7f7be90d6700 1 -- 172.21.15.26:0/1066432373 wait complete. 2024-09-15T03:49:42.331 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:42.331 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:49:42.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient: get_monmap_and_config 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient: build_initial_monmap 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient: monmap: 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:42.445396+0000 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:42.445396+0000 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:42.447 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 1 Processor -- start 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 1 -- start start 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient: init 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient: _add_conns ranks=[7,4,0,3,6,2,8,5,1] 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient(hunting): picked mon.noname-i con 0x7fb2a01335e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient(hunting): picked mon.noname-c con 0x7fb2a009fc80 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:42.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient(hunting): picked mon.noname-a con 0x7fb2a00a0050 addr v1:172.21.15.26:6789/0 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a004ca10 con 0x7fb2a00a0050 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a006eab0 con 0x7fb2a009fc80 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00a1330 con 0x7fb2a01335e0 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29f59e700 10 monclient(hunting): authenticate will time out at 1126.875115s 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29dd9b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fb2a009fc80 0x7fb2a0137ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43182/0 (socket says 172.21.15.26:43182) 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29dd9b700 1 -- 172.21.15.26:0/2721884426 learned_addr learned my addr 172.21.15.26:0/2721884426 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960850961 0 0) 0x7fb2a006eab0 con 0x7fb2a009fc80 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 10 monclient(hunting): my global_id is 174271 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb280003570 con 0x7fb2a009fc80 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763152275 0 0) 0x7fb2a004ca10 con 0x7fb2a00a0050 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 10 monclient(hunting): my global_id is 4869 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.443+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a006eab0 con 0x7fb2a00a0050 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171512192 0 0) 0x7fb2a00a1330 con 0x7fb2a01335e0 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient(hunting): my global_id is 134476 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a004ca10 con 0x7fb2a01335e0 2024-09-15T03:49:42.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843208935 0 0) 0x7fb280003570 con 0x7fb2a009fc80 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 >> v1:172.21.15.122:6792/0 conn(0x7fb2a01335e0 legacy=0x7fb2a0132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 >> v1:172.21.15.26:6789/0 conn(0x7fb2a00a0050 legacy=0x7fb2a013b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: _finish_hunting 0 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 monclient: found mon.noname-c 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00a10d0 con 0x7fb2a009fc80 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: _finish_auth 0 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:42.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 5 monclient: authenticate success, global_id 174271 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2880017b0 con 0x7fb2a009fc80 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: dump: 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.454 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb288001a20 con 0x7fb2a009fc80 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:42.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 1 -- 172.21.15.26:0/2721884426 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb288004c30 con 0x7fb2a009fc80 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 4 set_mon_vals no callback set 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29d59a700 10 monclient: dump: 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.456 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: shutdown 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- 172.21.15.26:0/2721884426 >> v1:172.21.15.26:6791/0 conn(0x7fb2a009fc80 legacy=0x7fb2a0137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- 172.21.15.26:0/2721884426 shutdown_connections 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- 172.21.15.26:0/2721884426 >> 172.21.15.26:0/2721884426 conn(0x7fb2a006e6e0 msgr2=0x7fb2a00a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:42.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- 172.21.15.26:0/2721884426 shutdown_connections 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- 172.21.15.26:0/2721884426 wait complete. 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: build_initial_monmap 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: monmap: 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:42.458 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:42.459 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:42.459 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:42.459 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:42.459 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:42.459 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 Processor -- start 2024-09-15T03:49:42.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- start start 2024-09-15T03:49:42.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: init 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient: _add_conns ranks=[8,2,5,4,7,1,0,6,3] 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient(hunting): picked mon.noname-e con 0x7fb2a009fc80 addr v1:172.21.15.26:6793/0 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient(hunting): picked mon.noname-b con 0x7fb2a00a0050 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient(hunting): picked mon.noname-h con 0x7fb2a01335e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00a1b70 con 0x7fb2a00a0050 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a009cf10 con 0x7fb2a01335e0 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a011e1d0 con 0x7fb2a009fc80 2024-09-15T03:49:42.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29f59e700 10 monclient(hunting): authenticate will time out at 1126.879328s 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29e59c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fb2a009fc80 0x7fb2a012fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:56840/0 (socket says 172.21.15.26:56840) 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb29e59c700 1 -- 172.21.15.26:0/2377046574 learned_addr learned my addr 172.21.15.26:0/2377046574 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455682114 0 0) 0x7fb2a011e1d0 con 0x7fb2a009fc80 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb2967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb2967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb2967fc700 10 monclient(hunting): my global_id is 134504 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.447+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb284003570 con 0x7fb2a009fc80 2024-09-15T03:49:42.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027040713 0 0) 0x7fb2a00a1b70 con 0x7fb2a00a0050 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): my global_id is 134516 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a011e1d0 con 0x7fb2a00a0050 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539057856 0 0) 0x7fb2a009cf10 con 0x7fb2a01335e0 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): my global_id is 5018 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a00a1b70 con 0x7fb2a01335e0 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3981662729 0 0) 0x7fb284003570 con 0x7fb2a009fc80 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2a009cf10 con 0x7fb2a009fc80 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb290001e00 con 0x7fb2a009fc80 2024-09-15T03:49:42.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): dump: 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.468 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 512598215 0 0) 0x7fb2a011e1d0 con 0x7fb2a00a0050 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb284003570 con 0x7fb2a00a0050 2024-09-15T03:49:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb288001770 con 0x7fb2a00a0050 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): dump: 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3692955653 0 0) 0x7fb2a009cf10 con 0x7fb2a009fc80 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.122:6791/0 conn(0x7fb2a01335e0 legacy=0x7fb2a014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.26:6790/0 conn(0x7fb2a00a0050 legacy=0x7fb2a0146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: _finish_hunting 0 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 monclient: found mon.e 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a011fd00 con 0x7fb2a009fc80 2024-09-15T03:49:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: _finish_auth 0 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 5 monclient: authenticate success, global_id 134504 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 10 monclient: _renew_subs 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0142820 con 0x7fb2a009fc80 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 10 monclient: _renew_subs 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a011eda0 con 0x7fb2a009fc80 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb290002320 con 0x7fb2a009fc80 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:49:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb290002850 con 0x7fb2a009fc80 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: dump: 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29cd99700 4 set_mon_vals no callback set 2024-09-15T03:49:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb2900029b0 con 0x7fb2a009fc80 2024-09-15T03:49:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(229..229 src has 1..229) v4 ==== 4063+0+0 (unknown 3248377167 0 0) 0x7fb290044200 con 0x7fb2a009fc80 2024-09-15T03:49:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb278006590 con 0x7fb278002840 2024-09-15T03:49:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb29dd9b700 10 monclient: get_auth_request con 0x7fb284037cb0 auth_method 0 2024-09-15T03:49:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705693710 0 0) 0x7fb278006590 con 0x7fb278002840 2024-09-15T03:49:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient(hunting): my global_id is 134485 2024-09-15T03:49:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a009cf10 con 0x7fb278002840 2024-09-15T03:49:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2209893485 0 0) 0x7fb2a009cf10 con 0x7fb278002840 2024-09-15T03:49:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb278006590 con 0x7fb278002840 2024-09-15T03:49:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2980019a0 con 0x7fb278002840 2024-09-15T03:49:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.451+0000 7fb2967fc700 10 monclient: dump: 2024-09-15T03:49:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1065269779 0 0) 0x7fb278006590 con 0x7fb278002840 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb27800a130 con 0x7fb278002840 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb298001e80 con 0x7fb278002840 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:49:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.122:6792/0 conn(0x7fb278002840 legacy=0x7fb278004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.455+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.122:6792/0 conn(0x7fb278002840 legacy=0x7fb278004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb29f59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:49:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb278009620 con 0x7fb278009090 2024-09-15T03:49:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262350417 0 0) 0x7fb278009620 con 0x7fb278009090 2024-09-15T03:49:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb2967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb2967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb2967fc700 10 monclient(hunting): my global_id is 134494 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.463+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb278006590 con 0x7fb278009090 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1942184993 0 0) 0x7fb278006590 con 0x7fb278009090 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb278009620 con 0x7fb278009090 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb298002630 con 0x7fb278009090 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 10 monclient: dump: 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:49:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:49:42.483 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 128948967 0 0) 0x7fb278009620 con 0x7fb278009090 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb2780098d0 con 0x7fb278009090 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3832269546) 0x7fb298002820 con 0x7fb278009090 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.122:6792/0 conn(0x7fb278009090 legacy=0x7fb278008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb2967fc700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.122:6792/0 conn(0x7fb278009090 legacy=0x7fb278008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:49:42.484 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 44, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:49:42.485 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.486 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.487 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.487 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.487 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.487 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.487 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.487 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:49:42.488 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:49:42.488 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.488 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.488 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:42.490 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:49:42.491 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.492 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:49:42.493 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:49:42.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.497 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:49:42.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.501 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:49:42.504 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:49:42.505 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.506 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:49:42.507 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:49:42.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.122:6812/20895 conn(0x7fb284037cb0 legacy=0x7fb28403a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 10 monclient: shutdown 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29ed9d700 1 -- 172.21.15.26:0/2377046574 reap_dead start 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 >> v1:172.21.15.26:6793/0 conn(0x7fb2a009fc80 legacy=0x7fb2a012fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 shutdown_connections 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 >> 172.21.15.26:0/2377046574 conn(0x7fb2a006e6e0 msgr2=0x7fb2a006f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 shutdown_connections 2024-09-15T03:49:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:42.467+0000 7fb29f59e700 1 -- 172.21.15.26:0/2377046574 wait complete. 2024-09-15T03:49:42.509 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:49:42.510 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'f', 'g'] 2024-09-15T03:49:42.510 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b', 'e'] 2024-09-15T03:49:42.510 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T03:49:42.510 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T03:49:42.510 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T03:49:42.510 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:49:42.526 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T03:49:42.526 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T03:49:42.526 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T03:49:42.527 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T03:49:42.527 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:49:42.543 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T03:49:42.543 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T03:49:42.544 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:49:42.544 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T03:49:42.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:49:42.564 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T03:49:42.564 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T03:49:42.565 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T03:49:42.565 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-09-15T03:49:42.565 INFO:tasks.ceph.mon.e:Sent signal 19 2024-09-15T03:49:42.565 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:49:57.568 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T03:49:57.569 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T03:49:57.570 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-09-15T03:49:57.570 INFO:tasks.ceph.mon.e:Sent signal 18 2024-09-15T03:49:57.571 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-15T03:49:57.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:49:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient: get_monmap_and_config 2024-09-15T03:49:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient: build_initial_monmap 2024-09-15T03:49:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient: monmap: 2024-09-15T03:49:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:49:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:49:57.672609+0000 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:49:57.672609+0000 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:49:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:49:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:49:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:49:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:49:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:49:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:49:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:49:57.677 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:49:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 1 Processor -- start 2024-09-15T03:49:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 1 -- start start 2024-09-15T03:49:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient: init 2024-09-15T03:49:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient: _reopen_session rank -1 2024-09-15T03:49:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient: _add_conns ranks=[6,4,2,0,7,8,5,3,1] 2024-09-15T03:49:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient(hunting): picked mon.noname-d con 0x7f2f480715a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:49:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient(hunting): picked mon.noname-c con 0x7f2f4809f640 addr v1:172.21.15.26:6791/0 2024-09-15T03:49:57.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient(hunting): picked mon.noname-b con 0x7f2f48099c90 addr v1:172.21.15.26:6790/0 2024-09-15T03:49:57.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4804ca10 con 0x7f2f48099c90 2024-09-15T03:49:57.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f48098ca0 con 0x7f2f4809f640 2024-09-15T03:49:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f48071970 con 0x7f2f480715a0 2024-09-15T03:49:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient(hunting): _renew_subs 2024-09-15T03:49:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4cdc2700 10 monclient(hunting): authenticate will time out at 1142.103064s 2024-09-15T03:49:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f45d9b700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f2f4809f640 legacy=0x7f2f4809e190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:49:57.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f46d9d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f2f48099c90 0x7f2f48137d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:45114/0 (socket says 172.21.15.26:45114) 2024-09-15T03:49:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f4659c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f2f480715a0 0x7f2f4809da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43320/0 (socket says 172.21.15.26:43320) 2024-09-15T03:49:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.671+0000 7f2f46d9d700 1 -- 172.21.15.26:0/1919191256 learned_addr learned my addr 172.21.15.26:0/1919191256 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:49:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:57.875+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f4809f640 legacy=0x7f2f4809e190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:49:58.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:58.279+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f4809f640 legacy=0x7f2f4809e190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:49:59.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:49:59.083+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f4809f640 legacy=0x7f2f4809e190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:00.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient(hunting): tick 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 monclient(hunting): continuing hunt 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6792/0 conn(0x7f2f480715a0 legacy=0x7f2f4809da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f4809f640 legacy=0x7f2f4809e190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6790/0 conn(0x7f2f48099c90 legacy=0x7f2f48137d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient: _add_conns ranks=[1,3,7,0,5,8] 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-f con 0x7f2f28003800 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-g con 0x7f2f28007480 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-i con 0x7f2f2800b200 addr v1:172.21.15.122:6792/0 2024-09-15T03:50:00.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4809e8a0 con 0x7f2f28003800 2024-09-15T03:50:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f48098ca0 con 0x7f2f28007480 2024-09-15T03:50:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6790/0 conn(0x7f2f28007480 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f4659c700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6789/0 conn(0x7f2f28003800 legacy=0x7f2f28005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f28011010 con 0x7f2f2800b200 2024-09-15T03:50:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.671+0000 7f2f37fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.875+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6790/0 conn(0x7f2f28007480 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:00.875+0000 7f2f4659c700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6789/0 conn(0x7f2f28003800 legacy=0x7f2f28005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:01.279+0000 7f2f4659c700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6789/0 conn(0x7f2f28003800 legacy=0x7f2f28005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:01.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:01.279+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6790/0 conn(0x7f2f28007480 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:02.083+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6790/0 conn(0x7f2f28007480 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:02.083+0000 7f2f4659c700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6789/0 conn(0x7f2f28003800 legacy=0x7f2f28005c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient(hunting): tick 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 monclient(hunting): continuing hunt 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6792/0 conn(0x7f2f2800b200 legacy=0x7f2f2800d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6790/0 conn(0x7f2f28007480 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6789/0 conn(0x7f2f28003800 legacy=0x7f2f28005c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:03.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient: _add_conns ranks=[0,8,5] 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-a con 0x7f2f28010090 addr v1:172.21.15.26:6789/0 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-e con 0x7f2f2800eee0 addr v1:172.21.15.26:6793/0 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f46d9d700 1 -- 172.21.15.26:0/1919191256 reap_dead start 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-h con 0x7f2f2800fa50 addr v1:172.21.15.122:6791/0 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f28012260 con 0x7f2f28010090 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4809e8a0 con 0x7f2f2800fa50 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f48098ca0 con 0x7f2f2800eee0 2024-09-15T03:50:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:03.675+0000 7f2f37fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:04.887+0000 7f2f4659c700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6789/0 conn(0x7f2f28010090 legacy=0x7f2f280104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T03:50:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:04.887+0000 7f2f4659c700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6789/0 conn(0x7f2f28010090 legacy=0x7f2f280104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:50:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:04.887+0000 7f2f4659c700 1 --1- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6789/0 conn(0x7f2f28010090 0x7f2f280104c0 :-1 s=OPENED pgs=258 cs=1 l=1).handle_message read tag failed 2024-09-15T03:50:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:04.887+0000 7f2f4659c700 1 --1- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6789/0 conn(0x7f2f28010090 0x7f2f280104c0 :-1 s=OPENED pgs=258 cs=1 l=1).fault on lossy channel, failing 2024-09-15T03:50:04.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:04.887+0000 7f2f4559a700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.26:6789/0 2024-09-15T03:50:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient(hunting): tick 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 monclient(hunting): continuing hunt 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6793/0 conn(0x7f2f2800eee0 legacy=0x7f2f2800f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6791/0 conn(0x7f2f2800fa50 legacy=0x7f2f28019590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6789/0 conn(0x7f2f28010090 legacy=0x7f2f280104c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient: _add_conns ranks=[0,4,7,5,6,1,2,8,3] 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-a con 0x7f2f28010090 addr v1:172.21.15.26:6789/0 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-c con 0x7f2f2800a350 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient(hunting): picked mon.noname-i con 0x7f2f2800a7c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2800a100 con 0x7f2f28010090 2024-09-15T03:50:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2800ad70 con 0x7f2f2800a350 2024-09-15T03:50:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2800af70 con 0x7f2f2800a7c0 2024-09-15T03:50:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f37fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.675+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f2800a350 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:06.879+0000 7f2f45d9b700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f2800a350 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:07.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20945407 0 0) 0x7f2f2800a100 con 0x7f2f28010090 2024-09-15T03:50:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient(hunting): my global_id is 4878 2024-09-15T03:50:07.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f30003570 con 0x7f2f28010090 2024-09-15T03:50:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896695664 0 0) 0x7f2f2800af70 con 0x7f2f2800a7c0 2024-09-15T03:50:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient(hunting): my global_id is 134503 2024-09-15T03:50:07.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2800a100 con 0x7f2f2800a7c0 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3227793954 0 0) 0x7f2f30003570 con 0x7f2f28010090 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.122:6792/0 conn(0x7f2f2800a7c0 legacy=0x7f2f2800d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6791/0 conn(0x7f2f2800a350 legacy=0x7f2f28009930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient: _finish_hunting 0 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 monclient: found mon.noname-a 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:07.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f28005c30 con 0x7f2f28010090 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient: _finish_auth 0 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f380017a0 con 0x7f2f28010090 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4cdc2700 5 monclient: authenticate success, global_id 4878 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4cdc2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4559a700 10 monclient: dump: 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.257 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.251+0000 7f2f4cdc2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f380019c0 con 0x7f2f28010090 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4559a700 1 -- 172.21.15.26:0/1919191256 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f38004b70 con 0x7f2f28010090 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4559a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4559a700 10 monclient: dump: 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.265 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:07.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 4 set_mon_vals no callback set 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 10 monclient: shutdown 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/1919191256 >> v1:172.21.15.26:6789/0 conn(0x7f2f28010090 legacy=0x7f2f280104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f46d9d700 1 -- 172.21.15.26:0/1919191256 reap_dead start 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/1919191256 shutdown_connections 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/1919191256 >> 172.21.15.26:0/1919191256 conn(0x7f2f480a3740 msgr2=0x7f2f48070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/1919191256 shutdown_connections 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/1919191256 wait complete. 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 10 monclient: build_initial_monmap 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 10 monclient: monmap: 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:07.267 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 Processor -- start 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 1 -- start start 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.255+0000 7f2f4cdc2700 10 monclient: init 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _add_conns ranks=[8,1,3,6,4,5,7,2,0] 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient(hunting): picked mon.noname-e con 0x7f2f48131940 addr v1:172.21.15.26:6793/0 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient(hunting): picked mon.noname-f con 0x7f2f48131d10 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient(hunting): picked mon.noname-g con 0x7f2f4811c270 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4811d520 con 0x7f2f48131d10 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4806e730 con 0x7f2f4811c270 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f48097fd0 con 0x7f2f48131940 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient(hunting): authenticate will time out at 1151.688509s 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f45d9b700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f2f48131d10 legacy=0x7f2f4811bb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:07.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f46d9d700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f2f4811c270 legacy=0x7f2f48146e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4659c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f2f48131940 0x7f2f4811b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38184/0 (socket says 172.21.15.26:38184) 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4659c700 1 -- 172.21.15.26:0/3887242975 learned_addr learned my addr 172.21.15.26:0/3887242975 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124332555 0 0) 0x7f2f48097fd0 con 0x7f2f48131940 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): my global_id is 134513 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f28003590 con 0x7f2f48131940 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1533822422 0 0) 0x7f2f28003590 con 0x7f2f48131940 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f48097fd0 con 0x7f2f48131940 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f38001760 con 0x7f2f48131940 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): dump: 2024-09-15T03:50:07.270 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.271 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3724453776 0 0) 0x7f2f48097fd0 con 0x7f2f48131940 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 >> v1:172.21.15.122:6790/0 conn(0x7f2f4811c270 legacy=0x7f2f48146e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 >> v1:172.21.15.122:6789/0 conn(0x7f2f48131d10 legacy=0x7f2f4811bb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: _finish_hunting 0 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 monclient: found mon.e 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4811db70 con 0x7f2f48131940 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: _finish_auth 0 2024-09-15T03:50:07.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 5 monclient: authenticate success, global_id 134513 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _renew_subs 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4811ea60 con 0x7f2f48131940 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _renew_subs 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4813e8b0 con 0x7f2f48131940 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f38004c70 con 0x7f2f48131940 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f380051a0 con 0x7f2f48131940 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f44d99700 4 set_mon_vals no callback set 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:50:07.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 10 monclient: dump: 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.274 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2f38005300 con 0x7f2f48131940 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0050b0 con 0x7f2f48131940 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3927+0+0 (unknown 1270649229 0 0) 0x7f2f38043fe0 con 0x7f2f48131940 2024-09-15T03:50:07.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.259+0000 7f2f45d9b700 10 monclient: get_auth_request con 0x7f2f28038f80 auth_method 0 2024-09-15T03:50:07.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.263+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f2f38014ee0 con 0x7f2f48131940 2024-09-15T03:50:07.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.263+0000 7f2f367fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:07.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.263+0000 7f2f367fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f4cdc2700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:50:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f4cdc2700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:50:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f4cdc2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2f2c0058e0 con 0x7f2f48131940 2024-09-15T03:50:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f367fc700 1 -- 172.21.15.26:0/3887242975 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2682 (unknown 254746657 0 2301355371) 0x7f2f380148e0 con 0x7f2f48131940 2024-09-15T03:50:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f367fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:50:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.491+0000 7f2f367fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:07.494 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":52,"quorum":[0,2,5,6,7,8],"quorum_names":["a","b","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 >> v1:172.21.15.122:6812/20895 conn(0x7f2f28038f80 legacy=0x7f2f2803b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 10 monclient: shutdown 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 >> v1:172.21.15.26:6793/0 conn(0x7f2f48131940 legacy=0x7f2f4811b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 shutdown_connections 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 >> 172.21.15.26:0/3887242975 conn(0x7f2f480a3740 msgr2=0x7f2f4809fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 shutdown_connections 2024-09-15T03:50:07.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.495+0000 7f2f4cdc2700 1 -- 172.21.15.26:0/3887242975 wait complete. 2024-09-15T03:50:07.513 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-15T03:50:07.513 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-09-15T03:50:07.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.615+0000 7f2c6af3b700 10 monclient: get_monmap_and_config 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.615+0000 7f2c6af3b700 10 monclient: build_initial_monmap 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.615+0000 7f2c6af3b700 10 monclient: monmap: 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:50:07.619241+0000 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:50:07.619241+0000 2024-09-15T03:50:07.620 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:07.621 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.615+0000 7f2c6af3b700 1 Processor -- start 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- start start 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient: init 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient: _add_conns ranks=[3,0,4,2,6,8,1,7,5] 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient(hunting): picked mon.noname-g con 0x7f2c640990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient(hunting): picked mon.noname-a con 0x7f2c640a1310 addr v1:172.21.15.26:6789/0 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient(hunting): picked mon.noname-c con 0x7f2c641335a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6404ca10 con 0x7f2c640a1310 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640a1b20 con 0x7f2c640990c0 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640a10b0 con 0x7f2c641335a0 2024-09-15T03:50:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient(hunting): authenticate will time out at 1152.048856s 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c694d8700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f2c641335a0 legacy=0x7f2c6413afe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c68cd7700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f2c640990c0 legacy=0x7f2c64132e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c63fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2c640a1310 0x7f2c64099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60170/0 (socket says 172.21.15.26:60170) 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c63fff700 1 -- 172.21.15.26:0/2335364573 learned_addr learned my addr 172.21.15.26:0/2335364573 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852293571 0 0) 0x7f2c6404ca10 con 0x7f2c640a1310 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient(hunting): my global_id is 4887 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50003570 con 0x7f2c640a1310 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055671131 0 0) 0x7f2c50003570 con 0x7f2c640a1310 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 >> v1:172.21.15.26:6791/0 conn(0x7f2c641335a0 legacy=0x7f2c6413afe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 >> v1:172.21.15.122:6790/0 conn(0x7f2c640990c0 legacy=0x7f2c64132e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: _finish_hunting 0 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 monclient: found mon.noname-a 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64099ba0 con 0x7f2c640a1310 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: _finish_auth 0 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c4c0017b0 con 0x7f2c640a1310 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 5 monclient: authenticate success, global_id 4887 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: dump: 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.629 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c4c001ad0 con 0x7f2c640a1310 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 4 set_mon_vals no callback set 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 1 -- 172.21.15.26:0/2335364573 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c4c004c30 con 0x7f2c640a1310 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c637fe700 10 monclient: dump: 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 10 monclient: shutdown 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2335364573 >> v1:172.21.15.26:6789/0 conn(0x7f2c640a1310 legacy=0x7f2c64099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2335364573 shutdown_connections 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2335364573 >> 172.21.15.26:0/2335364573 conn(0x7f2c6406e370 msgr2=0x7f2c640a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2335364573 shutdown_connections 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.619+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2335364573 wait complete. 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: build_initial_monmap 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: monmap: 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:07.632 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:07.634 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 Processor -- start 2024-09-15T03:50:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 -- start start 2024-09-15T03:50:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: init 2024-09-15T03:50:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: _add_conns ranks=[8,6,2,3,5,7,1,0,4] 2024-09-15T03:50:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient(hunting): picked mon.noname-e con 0x7f2c640990c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient(hunting): picked mon.noname-d con 0x7f2c640a1310 addr v1:172.21.15.26:6792/0 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient(hunting): picked mon.noname-b con 0x7f2c641335a0 addr v1:172.21.15.26:6790/0 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c64130690 con 0x7f2c641335a0 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6412f450 con 0x7f2c640a1310 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c641303b0 con 0x7f2c640990c0 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient(hunting): authenticate will time out at 1152.053727s 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c68cd7700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f2c640990c0 0x7f2c64141a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38200/0 (socket says 172.21.15.26:38200) 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c68cd7700 1 -- 172.21.15.26:0/2867521903 learned_addr learned my addr 172.21.15.26:0/2867521903 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129260863 0 0) 0x7f2c6412f450 con 0x7f2c640a1310 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): my global_id is 5082 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c44003570 con 0x7f2c640a1310 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774437878 0 0) 0x7f2c641303b0 con 0x7f2c640990c0 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): my global_id is 134522 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c6412f450 con 0x7f2c640990c0 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370893558 0 0) 0x7f2c64130690 con 0x7f2c641335a0 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): my global_id is 134525 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c641303b0 con 0x7f2c641335a0 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1781374576 0 0) 0x7f2c44003570 con 0x7f2c640a1310 2024-09-15T03:50:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c64130690 con 0x7f2c640a1310 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1410845209 0 0) 0x7f2c6412f450 con 0x7f2c640990c0 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c44003570 con 0x7f2c640990c0 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c4c001770 con 0x7f2c640a1310 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): dump: 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c54004300 con 0x7f2c640990c0 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:50:07.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient(hunting): dump: 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3290478745 0 0) 0x7f2c44003570 con 0x7f2c640990c0 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6792/0 conn(0x7f2c640a1310 legacy=0x7f2c64132180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6790/0 conn(0x7f2c641335a0 legacy=0x7f2c64149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 monclient: found mon.e 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64130970 con 0x7f2c640990c0 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: _finish_auth 0 2024-09-15T03:50:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 5 monclient: authenticate success, global_id 134522 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: _renew_subs 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64142160 con 0x7f2c640990c0 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: _renew_subs 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c64132a30 con 0x7f2c640990c0 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2c54004460 con 0x7f2c640990c0 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c54004990 con 0x7f2c640990c0 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c62ffd700 4 set_mon_vals no callback set 2024-09-15T03:50:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.623+0000 7f2c60ff9700 10 monclient: dump: 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c6af3b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2c48006590 con 0x7f2c48002840 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679945558 0 0) 0x7f2c48006590 con 0x7f2c48002840 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient(hunting): my global_id is 4896 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c44003570 con 0x7f2c48002840 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2367355796 0 0) 0x7f2c44003570 con 0x7f2c48002840 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c48006590 con 0x7f2c48002840 2024-09-15T03:50:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c4c001230 con 0x7f2c48002840 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient: dump: 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2c54004b70 con 0x7f2c640990c0 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(230..230 src has 1..230) v4 ==== 3927+0+0 (unknown 1270649229 0 0) 0x7f2c54043da0 con 0x7f2c640990c0 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1028016189 0 0) 0x7f2c48006590 con 0x7f2c48002840 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c4800a130 con 0x7f2c48002840 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c694d8700 10 monclient: get_auth_request con 0x7f2c44037ed0 auth_method 0 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2c4c007610 con 0x7f2c48002840 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6789/0 conn(0x7f2c48002840 legacy=0x7f2c48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.627+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6789/0 conn(0x7f2c48002840 legacy=0x7f2c48004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:07.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c6af3b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:50:07.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2c48007550 con 0x7f2c48006900 2024-09-15T03:50:07.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816929598 0 0) 0x7f2c48007550 con 0x7f2c48006900 2024-09-15T03:50:07.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c60ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c60ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c60ff9700 10 monclient(hunting): my global_id is 4905 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.639+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c48006590 con 0x7f2c48006900 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1716699602 0 0) 0x7f2c48006590 con 0x7f2c48006900 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c48007550 con 0x7f2c48006900 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2c4c005d60 con 0x7f2c48006900 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 10 monclient: dump: 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3245544344 0 0) 0x7f2c48007550 con 0x7f2c48006900 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2c480087a0 con 0x7f2c48006900 2024-09-15T03:50:07.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 1010615323) 0x7f2c4c007290 con 0x7f2c48006900 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6789/0 conn(0x7f2c48006900 legacy=0x7f2c48006d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c60ff9700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6789/0 conn(0x7f2c48006900 legacy=0x7f2c48006d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 52, 2024-09-15T03:50:07.655 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:50:07.656 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:07.657 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.658 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:50:07.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:07.660 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.661 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:50:07.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:50:07.665 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:50:07.666 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.669 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:07.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.671 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.672 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:07.673 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:07.674 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:50:07.675 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:50:07.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.122:6812/20895 conn(0x7f2c44037ed0 legacy=0x7f2c4403a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 10 monclient: shutdown 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 >> v1:172.21.15.26:6793/0 conn(0x7f2c640990c0 legacy=0x7f2c64141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c694d8700 1 -- 172.21.15.26:0/2867521903 reap_dead start 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 shutdown_connections 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 >> 172.21.15.26:0/2867521903 conn(0x7f2c6406e370 msgr2=0x7f2c64067b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 shutdown_connections 2024-09-15T03:50:07.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.643+0000 7f2c6af3b700 1 -- 172.21.15.26:0/2867521903 wait complete. 2024-09-15T03:50:07.677 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:50:07.677 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-09-15T03:50:07.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:50:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient: get_monmap_and_config 2024-09-15T03:50:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient: build_initial_monmap 2024-09-15T03:50:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient: monmap: 2024-09-15T03:50:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.794 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:50:07.793269+0000 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:50:07.793269+0000 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:07.795 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:07.796 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:07.796 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 1 Processor -- start 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 1 -- start start 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient: init 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient: _add_conns ranks=[4,0,8,5,2,7,6,1,3] 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient(hunting): picked mon.noname-c con 0x7fdb5c0990c0 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient(hunting): picked mon.noname-a con 0x7fdb5c0a1310 addr v1:172.21.15.26:6789/0 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient(hunting): picked mon.noname-e con 0x7fdb5c1335a0 addr v1:172.21.15.26:6793/0 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c04ca10 con 0x7fdb5c0a1310 2024-09-15T03:50:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c0a1b20 con 0x7fdb5c0990c0 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c0a10b0 con 0x7fdb5c1335a0 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb6473f700 10 monclient(hunting): authenticate will time out at 1152.222612s 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb624db700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c132e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb61cda700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdb5c0a1310 0x7fdb5c099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60202/0 (socket says 172.21.15.26:60202) 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb61cda700 1 -- 172.21.15.26:0/4109021025 learned_addr learned my addr 172.21.15.26:0/4109021025 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961005801 0 0) 0x7fdb5c04ca10 con 0x7fdb5c0a1310 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 10 monclient(hunting): my global_id is 4914 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb48003570 con 0x7fdb5c0a1310 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611312427 0 0) 0x7fdb5c0a10b0 con 0x7fdb5c1335a0 2024-09-15T03:50:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 10 monclient(hunting): my global_id is 134531 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c04ca10 con 0x7fdb5c1335a0 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.791+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3225431798 0 0) 0x7fdb48003570 con 0x7fdb5c0a1310 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 >> v1:172.21.15.26:6793/0 conn(0x7fdb5c1335a0 legacy=0x7fdb5c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c132e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: _finish_hunting 0 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 monclient: found mon.noname-a 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c099ba0 con 0x7fdb5c0a1310 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: _finish_auth 0 2024-09-15T03:50:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 5 monclient: authenticate success, global_id 4914 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb580017b0 con 0x7fdb5c0a1310 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: dump: 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb58001a10 con 0x7fdb5c0a1310 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 4 set_mon_vals no callback set 2024-09-15T03:50:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 1 -- 172.21.15.26:0/4109021025 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb58004c30 con 0x7fdb5c0a1310 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb614d9700 10 monclient: dump: 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 10 monclient: shutdown 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 -- 172.21.15.26:0/4109021025 >> v1:172.21.15.26:6789/0 conn(0x7fdb5c0a1310 legacy=0x7fdb5c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 -- 172.21.15.26:0/4109021025 shutdown_connections 2024-09-15T03:50:07.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 -- 172.21.15.26:0/4109021025 >> 172.21.15.26:0/4109021025 conn(0x7fdb5c06e370 msgr2=0x7fdb5c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 -- 172.21.15.26:0/4109021025 shutdown_connections 2024-09-15T03:50:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 -- 172.21.15.26:0/4109021025 wait complete. 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 10 monclient: build_initial_monmap 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 10 monclient: monmap: 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 Processor -- start 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 1 -- start start 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.795+0000 7fdb6473f700 10 monclient: init 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient: _add_conns ranks=[4,1,3,6,0,8,5,7,2] 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient(hunting): picked mon.noname-c con 0x7fdb5c0990c0 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient(hunting): picked mon.noname-f con 0x7fdb5c0a1310 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb624db700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c1321c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient(hunting): picked mon.noname-g con 0x7fdb5c1335a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c0a1ce0 con 0x7fdb5c0a1310 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c12f720 con 0x7fdb5c1335a0 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c09a010 con 0x7fdb5c0990c0 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb6473f700 10 monclient(hunting): authenticate will time out at 1152.228512s 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb61cda700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fdb5c0a1310 legacy=0x7fdb5c12bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:07.799+0000 7fdb62cdc700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fdb5c1335a0 legacy=0x7fdb5c12c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:08.003+0000 7fdb624db700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c1321c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:08.003+0000 7fdb62cdc700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fdb5c1335a0 legacy=0x7fdb5c12c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:08.003+0000 7fdb61cda700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fdb5c0a1310 legacy=0x7fdb5c12bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:951: done (5 left) 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:952: done (4 left) 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:953: done (3 left) 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:954: done (2 left) 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:956: done (1 left) 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:957: done (0 left) 2024-09-15T03:50:08.263 INFO:tasks.rados.rados.0.smithi122.stdout:958: delete oid 49 current snap is 105 2024-09-15T03:50:08.269 INFO:tasks.rados.rados.0.smithi122.stdout:958: done (0 left) 2024-09-15T03:50:08.270 INFO:tasks.rados.rados.0.smithi122.stdout:959: read oid 25 snap -1 2024-09-15T03:50:08.270 INFO:tasks.rados.rados.0.smithi122.stdout:959: expect deleted 2024-09-15T03:50:08.270 INFO:tasks.rados.rados.0.smithi122.stdout:960: snap_remove snap 104 2024-09-15T03:50:08.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:08.407+0000 7fdb624db700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c1321c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:08.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:08.407+0000 7fdb62cdc700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fdb5c1335a0 legacy=0x7fdb5c12c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:08.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:08.407+0000 7fdb61cda700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fdb5c0a1310 legacy=0x7fdb5c12bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:09.211+0000 7fdb624db700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c1321c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:09.211+0000 7fdb62cdc700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fdb5c1335a0 legacy=0x7fdb5c12c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:09.211+0000 7fdb61cda700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fdb5c0a1310 legacy=0x7fdb5c12bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:09.273 INFO:tasks.rados.rados.0.smithi122.stdout:960: done (1 left) 2024-09-15T03:50:09.273 INFO:tasks.rados.rados.0.smithi122.stdout:961: snap_remove snap 103 2024-09-15T03:50:10.284 INFO:tasks.rados.rados.0.smithi122.stdout:961: done (1 left) 2024-09-15T03:50:10.285 INFO:tasks.rados.rados.0.smithi122.stdout:962: rollback oid 22 current snap is 105 2024-09-15T03:50:10.285 INFO:tasks.rados.rados.0.smithi122.stdout:962: done (1 left) 2024-09-15T03:50:10.285 INFO:tasks.rados.rados.0.smithi122.stdout:963: delete oid 36 current snap is 105 2024-09-15T03:50:10.297 INFO:tasks.rados.rados.0.smithi122.stdout:959: done (1 left) 2024-09-15T03:50:10.297 INFO:tasks.rados.rados.0.smithi122.stdout:963: done (0 left) 2024-09-15T03:50:10.297 INFO:tasks.rados.rados.0.smithi122.stdout:964: copy_from oid 24 from oid 33 current snap is 105 2024-09-15T03:50:10.298 INFO:tasks.rados.rados.0.smithi122.stdout:965: read oid 34 snap -1 2024-09-15T03:50:10.298 INFO:tasks.rados.rados.0.smithi122.stdout:965: expect deleted 2024-09-15T03:50:10.298 INFO:tasks.rados.rados.0.smithi122.stdout:966: rollback oid 8 current snap is 105 2024-09-15T03:50:10.298 INFO:tasks.rados.rados.0.smithi122.stdout:966: done (2 left) 2024-09-15T03:50:10.298 INFO:tasks.rados.rados.0.smithi122.stdout:967: write (excl) oid 13 current snap is 105 2024-09-15T03:50:10.298 INFO:tasks.rados.rados.0.smithi122.stdout:967: seq_num 375 ranges {702625=719672,2127571=175054} 2024-09-15T03:50:10.305 INFO:tasks.rados.rados.0.smithi122.stdout:967: writing smithi12223906-13 from 702625 to 1422297 tid 1 2024-09-15T03:50:10.311 INFO:tasks.rados.rados.0.smithi122.stdout:967: writing smithi12223906-13 from 2127571 to 2302625 tid 2 2024-09-15T03:50:10.311 INFO:tasks.rados.rados.0.smithi122.stdout:968: delete oid 25 current snap is 105 2024-09-15T03:50:10.311 INFO:tasks.rados.rados.0.smithi122.stdout:964: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:50:10.313 INFO:tasks.rados.rados.0.smithi122.stdout:967: finishing write tid 1 to smithi12223906-13 2024-09-15T03:50:10.314 INFO:tasks.rados.rados.0.smithi122.stdout:964: finishing copy_from to smithi12223906-24 2024-09-15T03:50:10.314 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 297 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:964: done (3 left) 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:965: done (2 left) 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:968: done (1 left) 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:969: snap_create 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:967: finishing write tid 2 to smithi12223906-13 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:967: finishing write tid 3 to smithi12223906-13 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:967: finishing write tid 4 to smithi12223906-13 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 348 (ObjNum 375 snap 105 seq_num 375) dirty exists 2024-09-15T03:50:10.315 INFO:tasks.rados.rados.0.smithi122.stdout:967: left oid 13 (ObjNum 375 snap 105 seq_num 375) 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient(hunting): tick 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 monclient(hunting): continuing hunt 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fdb5c0990c0 legacy=0x7fdb5c1321c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fdb5c1335a0 legacy=0x7fdb5c12c1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fdb5c0a1310 legacy=0x7fdb5c12bab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient: _add_conns ranks=[7,6,2,0,5,8] 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient(hunting): picked mon.noname-i con 0x7fdb440037c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient(hunting): picked mon.noname-d con 0x7fdb44007420 addr v1:172.21.15.26:6792/0 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient(hunting): picked mon.noname-b con 0x7fdb4400b1a0 addr v1:172.21.15.26:6790/0 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c12c8d0 con 0x7fdb4400b1a0 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c0a1ce0 con 0x7fdb44007420 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c09a010 con 0x7fdb440037c0 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb517fa700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:10.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb61cda700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fdb44007420 0x7fdb440098d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58882/0 (socket says 172.21.15.26:58882) 2024-09-15T03:50:10.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb61cda700 1 -- 172.21.15.26:0/2554715566 learned_addr learned my addr 172.21.15.26:0/2554715566 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:10.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616107096 0 0) 0x7fdb5c12c8d0 con 0x7fdb4400b1a0 2024-09-15T03:50:10.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:10.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:10.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): my global_id is 134534 2024-09-15T03:50:10.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb40003570 con 0x7fdb4400b1a0 2024-09-15T03:50:10.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429758335 0 0) 0x7fdb5c0a1ce0 con 0x7fdb44007420 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): my global_id is 5091 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c12c8d0 con 0x7fdb44007420 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946039430 0 0) 0x7fdb5c09a010 con 0x7fdb440037c0 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:10.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): my global_id is 134512 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c0a1ce0 con 0x7fdb440037c0 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1257049466 0 0) 0x7fdb40003570 con 0x7fdb4400b1a0 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb5c09a010 con 0x7fdb4400b1a0 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb540045c0 con 0x7fdb4400b1a0 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): dump: 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.812 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2026243129 0 0) 0x7fdb5c0a1ce0 con 0x7fdb440037c0 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb40003570 con 0x7fdb440037c0 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3601966195 0 0) 0x7fdb5c12c8d0 con 0x7fdb44007420 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb5c0a1ce0 con 0x7fdb44007420 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb4c0043a0 con 0x7fdb440037c0 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:50:10.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): dump: 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.815 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb58004c30 con 0x7fdb44007420 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient(hunting): dump: 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.816 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2803916055 0 0) 0x7fdb5c09a010 con 0x7fdb4400b1a0 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.122:6792/0 conn(0x7fdb440037c0 legacy=0x7fdb44005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.26:6792/0 conn(0x7fdb44007420 legacy=0x7fdb440098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient: _finish_hunting 0 2024-09-15T03:50:10.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 monclient: found mon.b 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb62cdc700 1 -- 172.21.15.26:0/2554715566 reap_dead start 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44011f90 con 0x7fdb4400b1a0 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient: _finish_auth 0 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:10.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 5 monclient: authenticate success, global_id 134534 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 10 monclient: _renew_subs 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c131170 con 0x7fdb4400b1a0 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 10 monclient: _renew_subs 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c130280 con 0x7fdb4400b1a0 2024-09-15T03:50:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdb54004bd0 con 0x7fdb4400b1a0 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb54005100 con 0x7fdb4400b1a0 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.799+0000 7fdb60cd8700 4 set_mon_vals no callback set 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient: dump: 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.822 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.823 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb6473f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdb38006590 con 0x7fdb38002840 2024-09-15T03:50:10.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdb540052e0 con 0x7fdb4400b1a0 2024-09-15T03:50:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(233..233 src has 1..233) v4 ==== 3987+0+0 (unknown 1681408700 0 0) 0x7fdb54043f70 con 0x7fdb4400b1a0 2024-09-15T03:50:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757548104 0 0) 0x7fdb38006590 con 0x7fdb38002840 2024-09-15T03:50:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:10.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient(hunting): my global_id is 134543 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c09a010 con 0x7fdb38002840 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb61cda700 10 monclient: get_auth_request con 0x7fdb40037f60 auth_method 0 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 692350542 0 0) 0x7fdb5c09a010 con 0x7fdb38002840 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb38006590 con 0x7fdb38002840 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb4c004a70 con 0x7fdb38002840 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 10 monclient: dump: 2024-09-15T03:50:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.827 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1508205111 0 0) 0x7fdb38006590 con 0x7fdb38002840 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.803+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb3800a130 con 0x7fdb38002840 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.807+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdb4c0050b0 con 0x7fdb38002840 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.807+0000 7fdb527fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.807+0000 7fdb527fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.807+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.26:6790/0 conn(0x7fdb38002840 legacy=0x7fdb38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:10.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.807+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.26:6790/0 conn(0x7fdb38002840 legacy=0x7fdb38004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb6473f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:50:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdb38007520 con 0x7fdb380068d0 2024-09-15T03:50:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81532748 0 0) 0x7fdb38007520 con 0x7fdb380068d0 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 10 monclient(hunting): my global_id is 134552 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb38006590 con 0x7fdb380068d0 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1316472827 0 0) 0x7fdb38006590 con 0x7fdb380068d0 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb38007520 con 0x7fdb380068d0 2024-09-15T03:50:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdb4c0058c0 con 0x7fdb380068d0 2024-09-15T03:50:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.815+0000 7fdb527fc700 10 monclient: dump: 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2956105809 0 0) 0x7fdb38007520 con 0x7fdb380068d0 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdb38008760 con 0x7fdb380068d0 2024-09-15T03:50:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 3791143783) 0x7fdb4c005ca0 con 0x7fdb380068d0 2024-09-15T03:50:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:50:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.26:6790/0 conn(0x7fdb380068d0 legacy=0x7fdb38006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb527fc700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.26:6790/0 conn(0x7fdb380068d0 legacy=0x7fdb38006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:10.839 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:50:10.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 52, 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:50:10.840 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:10.841 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:10.842 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:10.843 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:10.844 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:10.844 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:10.844 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:10.844 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:10.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:50:10.845 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:10.846 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:10.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:50:10.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:50:10.846 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:50:10.846 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:50:10.846 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:50:10.847 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:50:10.847 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:50:10.847 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:10.847 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:50:10.847 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:10.847 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:50:10.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:50:10.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:50:10.852 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.852 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.852 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.852 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:50:10.854 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.855 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.856 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:50:10.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:50:10.864 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.866 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:10.867 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:10.868 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 5 2024-09-15T03:50:10.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:10.868 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:10.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:10.868 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:50:10.868 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:50:10.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.122:6812/20895 conn(0x7fdb40037f60 legacy=0x7fdb4003a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 10 monclient: shutdown 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 >> v1:172.21.15.26:6790/0 conn(0x7fdb4400b1a0 legacy=0x7fdb4400d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 shutdown_connections 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 >> 172.21.15.26:0/2554715566 conn(0x7fdb5c06e370 msgr2=0x7fdb5c0a5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 shutdown_connections 2024-09-15T03:50:10.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.819+0000 7fdb6473f700 1 -- 172.21.15.26:0/2554715566 wait complete. 2024-09-15T03:50:10.870 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:50:10.871 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-15T03:50:10.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:50:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.987+0000 7f8b2ec33700 10 monclient: get_monmap_and_config 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.987+0000 7f8b2ec33700 10 monclient: build_initial_monmap 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.987+0000 7f8b2ec33700 10 monclient: monmap: 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:50:10.990893+0000 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:50:10.990893+0000 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:10.993 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 Processor -- start 2024-09-15T03:50:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- start start 2024-09-15T03:50:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient: init 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient: _add_conns ranks=[1,6,3,2,7,5,4,0,8] 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient(hunting): picked mon.noname-f con 0x7f8b2809e440 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient(hunting): picked mon.noname-d con 0x7f8b2812ffc0 addr v1:172.21.15.26:6792/0 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient(hunting): picked mon.noname-g con 0x7f8b28130390 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2804ca10 con 0x7f8b2809e440 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2809a0f0 con 0x7f8b28130390 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2809e810 con 0x7f8b2812ffc0 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient(hunting): authenticate will time out at 1155.421208s 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b27fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8b2812ffc0 0x7f8b28134be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58892/0 (socket says 172.21.15.26:58892) 2024-09-15T03:50:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2d1d0700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f8b28130390 legacy=0x7f8b28138310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2c9cf700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f8b2809e440 legacy=0x7f8b2812f8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b27fff700 1 -- 172.21.15.26:0/867669492 learned_addr learned my addr 172.21.15.26:0/867669492 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091563376 0 0) 0x7f8b2809e810 con 0x7f8b2812ffc0 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient(hunting): my global_id is 5100 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b10003570 con 0x7f8b2812ffc0 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3249195260 0 0) 0x7f8b10003570 con 0x7f8b2812ffc0 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 >> v1:172.21.15.122:6790/0 conn(0x7f8b28130390 legacy=0x7f8b28138310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 >> v1:172.21.15.122:6789/0 conn(0x7f8b2809e440 legacy=0x7f8b2812f8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: _finish_hunting 0 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 monclient: found mon.noname-d 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2809c2a0 con 0x7f8b2812ffc0 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: _finish_auth 0 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b1c0017d0 con 0x7f8b2812ffc0 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: dump: 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 5 monclient: authenticate success, global_id 5100 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b1c001af0 con 0x7f8b2812ffc0 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 1 -- 172.21.15.26:0/867669492 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b1c004c30 con 0x7f8b2812ffc0 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 4 set_mon_vals no callback set 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b277fe700 10 monclient: dump: 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 10 monclient: shutdown 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- 172.21.15.26:0/867669492 >> v1:172.21.15.26:6792/0 conn(0x7f8b2812ffc0 legacy=0x7f8b28134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- 172.21.15.26:0/867669492 shutdown_connections 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- 172.21.15.26:0/867669492 >> 172.21.15.26:0/867669492 conn(0x7f8b28063f30 msgr2=0x7f8b280a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- 172.21.15.26:0/867669492 shutdown_connections 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.991+0000 7f8b2ec33700 1 -- 172.21.15.26:0/867669492 wait complete. 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient: monmap: 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 1 Processor -- start 2024-09-15T03:50:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 1 -- start start 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient: init 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient: _add_conns ranks=[5,1,7,2,4,8,3,6,0] 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient(hunting): picked mon.noname-h con 0x7f8b2809e440 addr v1:172.21.15.122:6791/0 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient(hunting): picked mon.noname-f con 0x7f8b2812ffc0 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient(hunting): picked mon.noname-i con 0x7f8b28130390 addr v1:172.21.15.122:6792/0 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b28099720 con 0x7f8b2812ffc0 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b280994f0 con 0x7f8b2809e440 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b28120110 con 0x7f8b28130390 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2ec33700 10 monclient(hunting): authenticate will time out at 1155.426416s 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b27fff700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f8b2812ffc0 legacy=0x7f8b2812ea50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2c9cf700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f8b2809e440 0x7f8b28141a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:48480/0 (socket says 172.21.15.26:48480) 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b2c9cf700 1 -- 172.21.15.26:0/36367763 learned_addr learned my addr 172.21.15.26:0/36367763 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916443573 0 0) 0x7f8b280994f0 con 0x7f8b2809e440 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): my global_id is 5027 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b14003570 con 0x7f8b2809e440 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721191183 0 0) 0x7f8b28120110 con 0x7f8b28130390 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): my global_id is 134521 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b280994f0 con 0x7f8b28130390 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3360793293 0 0) 0x7f8b280994f0 con 0x7f8b28130390 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b28120110 con 0x7f8b28130390 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 281802375 0 0) 0x7f8b14003570 con 0x7f8b2809e440 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b280994f0 con 0x7f8b2809e440 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b200045b0 con 0x7f8b28130390 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.995+0000 7f8b24ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): dump: 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.011 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b18004380 con 0x7f8b2809e440 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): dump: 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3768872944 0 0) 0x7f8b28120110 con 0x7f8b28130390 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.122:6791/0 conn(0x7f8b2809e440 legacy=0x7f8b28141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.122:6789/0 conn(0x7f8b2812ffc0 legacy=0x7f8b2812ea50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 monclient: found mon.i 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28121bd0 con 0x7f8b28130390 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 5 monclient: authenticate success, global_id 134521 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 10 monclient: _renew_subs 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b28142870 con 0x7f8b28130390 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 10 monclient: _renew_subs 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2812f300 con 0x7f8b28130390 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8b20004810 con 0x7f8b28130390 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b20004d40 con 0x7f8b28130390 2024-09-15T03:50:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b26ffd700 4 set_mon_vals no callback set 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: dump: 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8b20004ea0 con 0x7f8b28130390 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b08006590 con 0x7f8b08002840 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(233..233 src has 1..233) v4 ==== 3987+0+0 (unknown 1681408700 0 0) 0x7f8b20044080 con 0x7f8b28130390 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566423207 0 0) 0x7f8b08006590 con 0x7f8b08002840 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient(hunting): my global_id is 5109 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b28120110 con 0x7f8b08002840 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b27fff700 10 monclient: get_auth_request con 0x7f8b14037c20 auth_method 0 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2471020060 0 0) 0x7f8b28120110 con 0x7f8b08002840 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b08006590 con 0x7f8b08002840 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b18004b10 con 0x7f8b08002840 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 10 monclient: dump: 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.018 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3289949673 0 0) 0x7f8b08006590 con 0x7f8b08002840 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:10.999+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b0800a130 con 0x7f8b08002840 2024-09-15T03:50:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.003+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8b18005150 con 0x7f8b08002840 2024-09-15T03:50:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.003+0000 7f8b24ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.003+0000 7f8b24ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.003+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.26:6792/0 conn(0x7f8b08002840 legacy=0x7f8b08004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.003+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.26:6792/0 conn(0x7f8b08002840 legacy=0x7f8b08004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:50:11.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8b08007520 con 0x7f8b080068d0 2024-09-15T03:50:11.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362738208 0 0) 0x7f8b08007520 con 0x7f8b080068d0 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient(hunting): my global_id is 5118 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b08006590 con 0x7f8b080068d0 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3162425458 0 0) 0x7f8b08006590 con 0x7f8b080068d0 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8b08007520 con 0x7f8b080068d0 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8b18005980 con 0x7f8b080068d0 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient: dump: 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.022 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 515590641 0 0) 0x7f8b08007520 con 0x7f8b080068d0 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8b08008760 con 0x7f8b080068d0 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7367 (unknown 0 0 2098065541) 0x7f8b18005ae0 con 0x7f8b080068d0 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.26:6792/0 conn(0x7f8b080068d0 legacy=0x7f8b08006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b24ff9700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.26:6792/0 conn(0x7f8b080068d0 legacy=0x7f8b08006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:11.024 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 52, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:50:11.025 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.027 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.028 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:50:11.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.030 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.032 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.033 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.034 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.037 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.044 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.045 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.045 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.045 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:50:11.045 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:50:11.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.122:6812/20895 conn(0x7f8b14037c20 legacy=0x7f8b1403a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 10 monclient: shutdown 2024-09-15T03:50:11.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 >> v1:172.21.15.122:6792/0 conn(0x7f8b28130390 legacy=0x7f8b28149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2d1d0700 1 -- 172.21.15.26:0/36367763 reap_dead start 2024-09-15T03:50:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2d1d0700 1 -- 172.21.15.26:0/36367763 reap_dead start 2024-09-15T03:50:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 shutdown_connections 2024-09-15T03:50:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 >> 172.21.15.26:0/36367763 conn(0x7f8b28063f30 msgr2=0x7f8b2806f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 shutdown_connections 2024-09-15T03:50:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.011+0000 7f8b2ec33700 1 -- 172.21.15.26:0/36367763 wait complete. 2024-09-15T03:50:11.047 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:50:11.047 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-15T03:50:11.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient: get_monmap_and_config 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient: monmap: 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:50:11.166286+0000 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:50:11.166286+0000 2024-09-15T03:50:11.167 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.168 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 1 Processor -- start 2024-09-15T03:50:11.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 1 -- start start 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient: init 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient: _add_conns ranks=[5,1,4,2,7,0,6,3,8] 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient(hunting): picked mon.noname-h con 0x7fb14c1335e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient(hunting): picked mon.noname-f con 0x7fb14c09fc80 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient(hunting): picked mon.noname-c con 0x7fb14c0a0050 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c04ca10 con 0x7fb14c09fc80 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c06eab0 con 0x7fb14c0a0050 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0a1330 con 0x7fb14c1335e0 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151c13700 10 monclient(hunting): authenticate will time out at 1155.595833s 2024-09-15T03:50:11.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb151412700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7fb14c0a0050 legacy=0x7fb14c13b5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.163+0000 7fb14bfff700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fb14c09fc80 legacy=0x7fb14c137ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb150c11700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7fb14c1335e0 0x7fb14c132da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:48486/0 (socket says 172.21.15.26:48486) 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb150c11700 1 -- 172.21.15.26:0/2139849424 learned_addr learned my addr 172.21.15.26:0/2139849424 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207294824 0 0) 0x7fb14c0a1330 con 0x7fb14c1335e0 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient(hunting): my global_id is 5036 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb134003570 con 0x7fb14c1335e0 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3005415107 0 0) 0x7fb134003570 con 0x7fb14c1335e0 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 >> v1:172.21.15.26:6791/0 conn(0x7fb14c0a0050 legacy=0x7fb14c13b5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 >> v1:172.21.15.122:6789/0 conn(0x7fb14c09fc80 legacy=0x7fb14c137ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 monclient: found mon.noname-h 2024-09-15T03:50:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c0a10d0 con 0x7fb14c1335e0 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 5 monclient: authenticate success, global_id 5036 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb13c001b30 con 0x7fb14c1335e0 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: dump: 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.174 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb13c001de0 con 0x7fb14c1335e0 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 4 set_mon_vals no callback set 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 1 -- 172.21.15.26:0/2139849424 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb13c004fb0 con 0x7fb14c1335e0 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb14b7fe700 10 monclient: dump: 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 10 monclient: shutdown 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 -- 172.21.15.26:0/2139849424 >> v1:172.21.15.122:6791/0 conn(0x7fb14c1335e0 legacy=0x7fb14c132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 -- 172.21.15.26:0/2139849424 shutdown_connections 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 -- 172.21.15.26:0/2139849424 >> 172.21.15.26:0/2139849424 conn(0x7fb14c06e6e0 msgr2=0x7fb14c0a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 -- 172.21.15.26:0/2139849424 shutdown_connections 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 -- 172.21.15.26:0/2139849424 wait complete. 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 10 monclient: monmap: 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.179 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 Processor -- start 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 1 -- start start 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.167+0000 7fb151c13700 10 monclient: init 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient: _add_conns ranks=[6,3,5,8,7,4,2,0,1] 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient(hunting): picked mon.noname-d con 0x7fb14c09fc80 addr v1:172.21.15.26:6792/0 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient(hunting): picked mon.noname-g con 0x7fb14c0a0050 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient(hunting): picked mon.noname-h con 0x7fb14c1335e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c11e7e0 con 0x7fb14c0a0050 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c09cf10 con 0x7fb14c1335e0 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0a1b70 con 0x7fb14c09fc80 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient(hunting): authenticate will time out at 1155.600628s 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb14bfff700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fb14c0a0050 legacy=0x7fb14c146cd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb150c11700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fb14c09fc80 0x7fb14c12fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58924/0 (socket says 172.21.15.26:58924) 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb150c11700 1 -- 172.21.15.26:0/1963493801 learned_addr learned my addr 172.21.15.26:0/1963493801 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190649133 0 0) 0x7fb14c0a1b70 con 0x7fb14c09fc80 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): my global_id is 5127 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb138003570 con 0x7fb14c09fc80 2024-09-15T03:50:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586477432 0 0) 0x7fb14c09cf10 con 0x7fb14c1335e0 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): my global_id is 5045 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb14c0a1b70 con 0x7fb14c1335e0 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 951101796 0 0) 0x7fb138003570 con 0x7fb14c09fc80 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb14c09cf10 con 0x7fb14c09fc80 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb13c001af0 con 0x7fb14c09fc80 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): dump: 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.187 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3768676850 0 0) 0x7fb14c09cf10 con 0x7fb14c09fc80 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.122:6791/0 conn(0x7fb14c1335e0 legacy=0x7fb14c14a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.122:6790/0 conn(0x7fb14c0a0050 legacy=0x7fb14c146cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 monclient: found mon.d 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c1203f0 con 0x7fb14c09fc80 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 5 monclient: authenticate success, global_id 5127 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient: _renew_subs 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:50:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c11e020 con 0x7fb14c09fc80 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient: _renew_subs 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c130110 con 0x7fb14c09fc80 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb13c002010 con 0x7fb14c09fc80 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb13c0057d0 con 0x7fb14c09fc80 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb14affd700 4 set_mon_vals no callback set 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 10 monclient: dump: 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.191 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb13c010190 con 0x7fb14c09fc80 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.171+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb12c006590 con 0x7fb12c002840 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(233..233 src has 1..233) v4 ==== 3987+0+0 (unknown 1681408700 0 0) 0x7fb13c044640 con 0x7fb14c09fc80 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140484861 0 0) 0x7fb12c006590 con 0x7fb12c002840 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient(hunting): my global_id is 134540 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb14c09cf10 con 0x7fb12c002840 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb151412700 10 monclient: get_auth_request con 0x7fb138037560 auth_method 0 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 495560338 0 0) 0x7fb14c09cf10 con 0x7fb12c002840 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb12c006590 con 0x7fb12c002840 2024-09-15T03:50:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb140004150 con 0x7fb12c002840 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient: dump: 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2452065631 0 0) 0x7fb12c006590 con 0x7fb12c002840 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb12c00a130 con 0x7fb12c002840 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb140006540 con 0x7fb12c002840 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.26:6793/0 conn(0x7fb12c002840 legacy=0x7fb12c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.175+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.26:6793/0 conn(0x7fb12c002840 legacy=0x7fb12c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb151c13700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb12c009640 con 0x7fb12c0090b0 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491964791 0 0) 0x7fb12c009640 con 0x7fb12c0090b0 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient(hunting): my global_id is 134549 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb12c006590 con 0x7fb12c0090b0 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1068068009 0 0) 0x7fb12c006590 con 0x7fb12c0090b0 2024-09-15T03:50:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb12c009640 con 0x7fb12c0090b0 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb140004cb0 con 0x7fb12c0090b0 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient: dump: 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 535191115 0 0) 0x7fb12c009640 con 0x7fb12c0090b0 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb12c0098f0 con 0x7fb12c0090b0 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 2758382202) 0x7fb1400061b0 con 0x7fb12c0090b0 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.26:6793/0 conn(0x7fb12c0090b0 legacy=0x7fb12c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.187+0000 7fb148ff9700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.26:6793/0 conn(0x7fb12c0090b0 legacy=0x7fb12c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 52, 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:50:11.199 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.200 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.201 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:50:11.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.203 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:50:11.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.206 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.208 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:50:11.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.210 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.213 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.214 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.216 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.217 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:50:11.218 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.218 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.218 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:50:11.218 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:50:11.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.122:6812/20895 conn(0x7fb138037560 legacy=0x7fb138039a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 10 monclient: shutdown 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 >> v1:172.21.15.26:6792/0 conn(0x7fb14c09fc80 legacy=0x7fb14c12fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151412700 1 -- 172.21.15.26:0/1963493801 reap_dead start 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151412700 1 -- 172.21.15.26:0/1963493801 reap_dead start 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 shutdown_connections 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 >> 172.21.15.26:0/1963493801 conn(0x7fb14c06e6e0 msgr2=0x7fb14c130c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 shutdown_connections 2024-09-15T03:50:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.191+0000 7fb151c13700 1 -- 172.21.15.26:0/1963493801 wait complete. 2024-09-15T03:50:11.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:50:11.221 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-15T03:50:11.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:50:11.294 INFO:tasks.rados.rados.0.smithi122.stdout:967: done (1 left) 2024-09-15T03:50:11.294 INFO:tasks.rados.rados.0.smithi122.stdout:969: done (0 left) 2024-09-15T03:50:11.294 INFO:tasks.rados.rados.0.smithi122.stdout:970: copy_from oid 24 from oid 26 current snap is 106 2024-09-15T03:50:11.295 INFO:tasks.rados.rados.0.smithi122.stdout:971: snap_create 2024-09-15T03:50:11.297 INFO:tasks.rados.rados.0.smithi122.stdout:970: finishing copy_from to smithi12223906-24 2024-09-15T03:50:11.297 INFO:tasks.rados.rados.0.smithi122.stdout:970: got expected ENOENT (src dne) 2024-09-15T03:50:11.298 INFO:tasks.rados.rados.0.smithi122.stdout:970: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient: get_monmap_and_config 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient: monmap: 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:50:11.341976+0000 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:50:11.341976+0000 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.343 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.344 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 1 Processor -- start 2024-09-15T03:50:11.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 1 -- start start 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient: init 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient: _add_conns ranks=[5,2,8,7,1,4,3,0,6] 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient(hunting): picked mon.noname-h con 0x7f81b80a6a70 addr v1:172.21.15.122:6791/0 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient(hunting): picked mon.noname-b con 0x7f81b8099020 addr v1:172.21.15.26:6790/0 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient(hunting): picked mon.noname-e con 0x7f81b8138cc0 addr v1:172.21.15.26:6793/0 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b804ca10 con 0x7f81b8099020 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b8098c20 con 0x7f81b80a6a70 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80a2280 con 0x7f81b8138cc0 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81be5fd700 10 monclient(hunting): authenticate will time out at 1155.771325s 2024-09-15T03:50:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81bcdfa700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f81b8099020 0x7f81b80993f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:53018/0 (socket says 172.21.15.26:53018) 2024-09-15T03:50:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81bcdfa700 1 -- 172.21.15.26:0/1911693550 learned_addr learned my addr 172.21.15.26:0/1911693550 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81bddfc700 1 --1- 172.21.15.26:0/1911693550 >> v1:172.21.15.26:6793/0 conn(0x7f81b8138cc0 0x7f81b813b0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38232/0 (socket says 172.21.15.26:38232) 2024-09-15T03:50:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081800788 0 0) 0x7f81b804ca10 con 0x7f81b8099020 2024-09-15T03:50:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81b7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81b7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.339+0000 7f81b7fff700 10 monclient(hunting): my global_id is 134561 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a0003570 con 0x7f81b8099020 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476818773 0 0) 0x7f81b80a2280 con 0x7f81b8138cc0 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient(hunting): my global_id is 134558 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b804ca10 con 0x7f81b8138cc0 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774275281 0 0) 0x7f81b8098c20 con 0x7f81b80a6a70 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient(hunting): my global_id is 5054 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b80a2280 con 0x7f81b80a6a70 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814619239 0 0) 0x7f81a0003570 con 0x7f81b8099020 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 >> v1:172.21.15.26:6793/0 conn(0x7f81b8138cc0 legacy=0x7f81b813b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 >> v1:172.21.15.122:6791/0 conn(0x7f81b80a6a70 legacy=0x7f81b8098510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 monclient: found mon.noname-b 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80a2840 con 0x7f81b8099020 2024-09-15T03:50:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 5 monclient: authenticate success, global_id 134561 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81ac001b30 con 0x7f81b8099020 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: dump: 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.362 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81ac001e10 con 0x7f81b8099020 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 1 -- 172.21.15.26:0/1911693550 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81ac004fb0 con 0x7f81b8099020 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 4 set_mon_vals no callback set 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81b7fff700 10 monclient: dump: 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.363 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.364 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 10 monclient: shutdown 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 -- 172.21.15.26:0/1911693550 >> v1:172.21.15.26:6790/0 conn(0x7f81b8099020 legacy=0x7f81b80993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 -- 172.21.15.26:0/1911693550 shutdown_connections 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 -- 172.21.15.26:0/1911693550 >> 172.21.15.26:0/1911693550 conn(0x7f81b806e1d0 msgr2=0x7f81b80a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 -- 172.21.15.26:0/1911693550 shutdown_connections 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 -- 172.21.15.26:0/1911693550 wait complete. 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 10 monclient: monmap: 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 Processor -- start 2024-09-15T03:50:11.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 1 -- start start 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.343+0000 7f81be5fd700 10 monclient: init 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient: _add_conns ranks=[0,5,1,7,6,8,3,2,4] 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient(hunting): picked mon.noname-a con 0x7f81b8099020 addr v1:172.21.15.26:6789/0 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient(hunting): picked mon.noname-h con 0x7f81b80a6a70 addr v1:172.21.15.122:6791/0 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient(hunting): picked mon.noname-f con 0x7f81b8138cc0 addr v1:172.21.15.122:6789/0 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b8099d60 con 0x7f81b8099020 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80a0460 con 0x7f81b8138cc0 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80a1770 con 0x7f81b80a6a70 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient(hunting): authenticate will time out at 1155.776340s 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81bddfc700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f81b8138cc0 legacy=0x7f81b814a170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81bd5fb700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f81b8099020 0x7f81b809d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60208/0 (socket says 172.21.15.26:60208) 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81bd5fb700 1 -- 172.21.15.26:0/2592962712 learned_addr learned my addr 172.21.15.26:0/2592962712 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734158625 0 0) 0x7f81b8099d60 con 0x7f81b8099020 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): my global_id is 4923 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a4003570 con 0x7f81b8099020 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139090751 0 0) 0x7f81b80a1770 con 0x7f81b80a6a70 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): my global_id is 5063 2024-09-15T03:50:11.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b8099d60 con 0x7f81b80a6a70 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3002090901 0 0) 0x7f81a4003570 con 0x7f81b8099020 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81b80a1770 con 0x7f81b8099020 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81a8001b20 con 0x7f81b8099020 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): dump: 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 749947573 0 0) 0x7f81b8099d60 con 0x7f81b80a6a70 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81a4003570 con 0x7f81b80a6a70 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81ac004ef0 con 0x7f81b80a6a70 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient(hunting): dump: 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.371 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3341880899 0 0) 0x7f81b80a1770 con 0x7f81b8099020 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6791/0 conn(0x7f81b80a6a70 legacy=0x7f81b809dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6789/0 conn(0x7f81b8138cc0 legacy=0x7f81b814a170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 monclient: found mon.a 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b809e3c0 con 0x7f81b8099020 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 5 monclient: authenticate success, global_id 4923 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient: _renew_subs 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:50:11.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b809e9e0 con 0x7f81b8099020 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient: _renew_subs 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b809f570 con 0x7f81b8099020 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81a8002040 con 0x7f81b8099020 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81a8002570 con 0x7f81b8099020 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b6ffd700 4 set_mon_vals no callback set 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 10 monclient: dump: 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.374 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.375 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f81a8002750 con 0x7f81b8099020 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(234..234 src has 1..234) v4 ==== 3959+0+0 (unknown 1275905413 0 0) 0x7f81a8043fc0 con 0x7f81b8099020 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8198006590 con 0x7f8198002840 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.347+0000 7f81bcdfa700 10 monclient: get_auth_request con 0x7f81a4037bf0 auth_method 0 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75285035 0 0) 0x7f8198006590 con 0x7f8198002840 2024-09-15T03:50:11.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient(hunting): my global_id is 5072 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b80a1770 con 0x7f8198002840 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2004731545 0 0) 0x7f81b80a1770 con 0x7f8198002840 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8198006590 con 0x7f8198002840 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81b0001b60 con 0x7f8198002840 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient: dump: 2024-09-15T03:50:11.377 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.378 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2703646075 0 0) 0x7f8198006590 con 0x7f8198002840 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f819800a130 con 0x7f8198002840 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f81b0004830 con 0x7f8198002840 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6791/0 conn(0x7f8198002840 legacy=0x7f8198004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.351+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6791/0 conn(0x7f8198002840 legacy=0x7f8198004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.359+0000 7f81be5fd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.359+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8198007520 con 0x7f81980068d0 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985648368 0 0) 0x7f8198007520 con 0x7f81980068d0 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient(hunting): my global_id is 5081 2024-09-15T03:50:11.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8198006590 con 0x7f81980068d0 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1811029081 0 0) 0x7f8198006590 con 0x7f81980068d0 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8198007520 con 0x7f81980068d0 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81b0005010 con 0x7f81980068d0 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient: dump: 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.384 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4029940346 0 0) 0x7f8198007520 con 0x7f81980068d0 2024-09-15T03:50:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8198008760 con 0x7f81980068d0 2024-09-15T03:50:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 4033577017) 0x7f81b0007280 con 0x7f81980068d0 2024-09-15T03:50:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:50:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6791/0 conn(0x7f81980068d0 legacy=0x7f8198006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81b57fa700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6791/0 conn(0x7f81980068d0 legacy=0x7f8198006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.388 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:50:11.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:11.388 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:11.388 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:50:11.388 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 52, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:50:11.389 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.390 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:50:11.392 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.393 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.396 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:50:11.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.400 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.401 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.403 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.404 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.405 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.405 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.405 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.405 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.405 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.406 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.407 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.408 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.409 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.409 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.409 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.409 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:50:11.409 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:50:11.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.122:6812/20895 conn(0x7f81a4037bf0 legacy=0x7f81a403a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 10 monclient: shutdown 2024-09-15T03:50:11.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 >> v1:172.21.15.26:6789/0 conn(0x7f81b8099020 legacy=0x7f81b809d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81bddfc700 1 -- 172.21.15.26:0/2592962712 reap_dead start 2024-09-15T03:50:11.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 shutdown_connections 2024-09-15T03:50:11.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 >> 172.21.15.26:0/2592962712 conn(0x7f81b806e1d0 msgr2=0x7f81b80a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 shutdown_connections 2024-09-15T03:50:11.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.363+0000 7f81be5fd700 1 -- 172.21.15.26:0/2592962712 wait complete. 2024-09-15T03:50:11.412 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:50:11.412 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-09-15T03:50:11.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: get_monmap_and_config 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: monmap: 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:50:11.532049+0000 2024-09-15T03:50:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:50:11.532049+0000 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.536 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 1 Processor -- start 2024-09-15T03:50:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 1 -- start start 2024-09-15T03:50:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: init 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: _add_conns ranks=[3,4,2,8,0,1,5,7,6] 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient(hunting): picked mon.noname-g con 0x7f44100990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient(hunting): picked mon.noname-c con 0x7f44100a1310 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient(hunting): picked mon.noname-b con 0x7f44101335a0 addr v1:172.21.15.26:6790/0 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441004ca10 con 0x7f44101335a0 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100a1b20 con 0x7f44100990c0 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100a10b0 con 0x7f44100a1310 2024-09-15T03:50:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient(hunting): authenticate will time out at 1155.962001s 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415847700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f44100a1310 legacy=0x7f4410099490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4416048700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f44100990c0 legacy=0x7f4410132e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4416849700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f44101335a0 0x7f441013afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:53020/0 (socket says 172.21.15.26:53020) 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4416849700 1 -- 172.21.15.26:0/1497802821 learned_addr learned my addr 172.21.15.26:0/1497802821 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073379700 0 0) 0x7f441004ca10 con 0x7f44101335a0 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient(hunting): my global_id is 134570 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43fc003570 con 0x7f44101335a0 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3443759585 0 0) 0x7f43fc003570 con 0x7f44101335a0 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 >> v1:172.21.15.26:6791/0 conn(0x7f44100a1310 legacy=0x7f4410099490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 >> v1:172.21.15.122:6790/0 conn(0x7f44100990c0 legacy=0x7f4410132e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 monclient: found mon.noname-b 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4410099ba0 con 0x7f44101335a0 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f44080017b0 con 0x7f44101335a0 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 5 monclient: authenticate success, global_id 134570 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: dump: 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:50:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4408001ad0 con 0x7f44101335a0 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 10 monclient: get_monmap_and_config success 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f44182ac700 4 set_mon_vals no callback set 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 1 -- 172.21.15.26:0/1497802821 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4408004c30 con 0x7f44101335a0 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.531+0000 7f4415046700 10 monclient: dump: 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient: shutdown 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- 172.21.15.26:0/1497802821 >> v1:172.21.15.26:6790/0 conn(0x7f44101335a0 legacy=0x7f441013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- 172.21.15.26:0/1497802821 shutdown_connections 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- 172.21.15.26:0/1497802821 >> 172.21.15.26:0/1497802821 conn(0x7f441006e370 msgr2=0x7f44100a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- 172.21.15.26:0/1497802821 shutdown_connections 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- 172.21.15.26:0/1497802821 wait complete. 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient: build_initial_monmap 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient: monmap: 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:50:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:50:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 Processor -- start 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- start start 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient: init 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient: _reopen_session rank -1 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient: _add_conns ranks=[8,2,4,7,6,1,0,3,5] 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient(hunting): picked mon.noname-e con 0x7f44100990c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:50:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient(hunting): picked mon.noname-b con 0x7f44100a1310 addr v1:172.21.15.26:6790/0 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient(hunting): picked mon.noname-c con 0x7f44101335a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100a1ce0 con 0x7f44100a1310 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f4416849700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f44101335a0 legacy=0x7f441012c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441009a010 con 0x7f44101335a0 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f4415847700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f44100a1310 0x7f441012bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:53034/0 (socket says 172.21.15.26:53034) 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f4415847700 1 -- 172.21.15.26:0/1840602717 learned_addr learned my addr 172.21.15.26:0/1840602717 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:50:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f4416048700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f44100990c0 0x7f4410132160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:38238/0 (socket says 172.21.15.26:38238) 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441012c8d0 con 0x7f44100990c0 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient(hunting): _renew_subs 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44182ac700 10 monclient(hunting): authenticate will time out at 1155.967653s 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214544414 0 0) 0x7f44100a1ce0 con 0x7f44100a1310 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.535+0000 7f44067fc700 10 monclient(hunting): my global_id is 134579 2024-09-15T03:50:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f4003570 con 0x7f44100a1310 2024-09-15T03:50:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631801685 0 0) 0x7f441012c8d0 con 0x7f44100990c0 2024-09-15T03:50:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): my global_id is 134567 2024-09-15T03:50:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44100a1ce0 con 0x7f44100990c0 2024-09-15T03:50:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3711181466 0 0) 0x7f43f4003570 con 0x7f44100a1310 2024-09-15T03:50:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f441012c8d0 con 0x7f44100a1310 2024-09-15T03:50:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4400004350 con 0x7f44100a1310 2024-09-15T03:50:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:50:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): dump: 2024-09-15T03:50:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.568 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:50:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3190912418 0 0) 0x7f44100a1ce0 con 0x7f44100990c0 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43f4003570 con 0x7f44100990c0 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f440c004670 con 0x7f44100990c0 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): dump: 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3034626876 0 0) 0x7f441012c8d0 con 0x7f44100a1310 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.26:6793/0 conn(0x7f44100990c0 legacy=0x7f4410132160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.26:6791/0 conn(0x7f44101335a0 legacy=0x7f441012c1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: _finish_hunting 0 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 monclient: found mon.b 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44101311d0 con 0x7f44100a1310 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: _finish_auth 0 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 5 monclient: authenticate success, global_id 134579 2024-09-15T03:50:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 10 monclient: _renew_subs 2024-09-15T03:50:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:50:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44101314b0 con 0x7f44100a1310 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 10 monclient: _renew_subs 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44101302e0 con 0x7f44100a1310 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4400004630 con 0x7f44100a1310 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4400004b60 con 0x7f44100a1310 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:50:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f4414845700 4 set_mon_vals no callback set 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient: dump: 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4400004d40 con 0x7f44100a1310 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f43f8006590 con 0x7f43f8002840 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(234..234 src has 1..234) v4 ==== 3959+0+0 (unknown 1275905413 0 0) 0x7f4400043f30 con 0x7f44100a1310 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f4416849700 10 monclient: get_auth_request con 0x7f43f4037b80 auth_method 0 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784332750 0 0) 0x7f43f8006590 con 0x7f43f8002840 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 10 monclient(hunting): my global_id is 134530 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.539+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f441012c8d0 con 0x7f43f8002840 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 6532953 0 0) 0x7f441012c8d0 con 0x7f43f8002840 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43f8006590 con 0x7f43f8002840 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f440c004f10 con 0x7f43f8002840 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 10 monclient: dump: 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2187605135 0 0) 0x7f43f8006590 con 0x7f43f8002840 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43f800a130 con 0x7f43f8002840 2024-09-15T03:50:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f440c007210 con 0x7f43f8002840 2024-09-15T03:50:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:50:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:50:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.122:6792/0 conn(0x7f43f8002840 legacy=0x7f43f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.543+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.122:6792/0 conn(0x7f43f8002840 legacy=0x7f43f8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44182ac700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:50:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f43f8007520 con 0x7f43f80068d0 2024-09-15T03:50:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773361043 0 0) 0x7f43f8007520 con 0x7f43f80068d0 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 10 monclient(hunting): my global_id is 134539 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f8006590 con 0x7f43f80068d0 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2714275229 0 0) 0x7f43f8006590 con 0x7f43f80068d0 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43f8007520 con 0x7f43f80068d0 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f440c005a60 con 0x7f43f80068d0 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 10 monclient: dump: 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:50:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:50:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3406052091 0 0) 0x7f43f8007520 con 0x7f43f80068d0 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.551+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43f8008760 con 0x7f43f80068d0 2024-09-15T03:50:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7365 (unknown 0 0 2355829486) 0x7f440c00a200 con 0x7f43f80068d0 2024-09-15T03:50:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44067fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:50:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44067fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:50:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.122:6792/0 conn(0x7f43f80068d0 legacy=0x7f43f8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44067fc700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.122:6792/0 conn(0x7f43f80068d0 legacy=0x7f43f8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:50:11.585 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 52, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:50:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:50:11.587 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.587 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:50:11.587 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.588 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.589 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:50:11.590 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:50:11.591 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.592 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.593 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:50:11.594 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.595 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:50:11.597 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:50:11.602 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:50:11.604 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.604 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:50:11.605 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.605 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.606 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.606 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:50:11.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:50:11.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.608 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:50:11.608 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.609 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.609 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.609 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:50:11.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:50:11.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.611 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.611 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:50:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:50:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:50:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:50:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.612 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:50:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:50:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.613 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:50:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:50:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:50:11.614 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:50:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:50:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:50:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:50:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:50:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:50:11.615 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:50:11.616 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:50:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.122:6812/20895 conn(0x7f43f4037b80 legacy=0x7f43f403a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 10 monclient: shutdown 2024-09-15T03:50:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:50:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 >> v1:172.21.15.26:6790/0 conn(0x7f44100a1310 legacy=0x7f441012bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:50:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f4416849700 1 -- 172.21.15.26:0/1840602717 reap_dead start 2024-09-15T03:50:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f4416849700 1 -- 172.21.15.26:0/1840602717 reap_dead start 2024-09-15T03:50:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 shutdown_connections 2024-09-15T03:50:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 >> 172.21.15.26:0/1840602717 conn(0x7f441006e370 msgr2=0x7f44100a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:50:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 shutdown_connections 2024-09-15T03:50:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:50:11.555+0000 7f44182ac700 1 -- 172.21.15.26:0/1840602717 wait complete. 2024-09-15T03:50:11.622 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:50:11.622 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:50:11.775 INFO:tasks.rados.rados.0.smithi122.stdout:970: done (1 left) 2024-09-15T03:50:11.775 INFO:tasks.rados.rados.0.smithi122.stdout:971: done (0 left) 2024-09-15T03:50:11.775 INFO:tasks.rados.rados.0.smithi122.stdout:972: write (excl) oid 28 current snap is 107 2024-09-15T03:50:11.775 INFO:tasks.rados.rados.0.smithi122.stdout:972: seq_num 377 ranges {736239=470170,1728182=571782} 2024-09-15T03:50:11.785 INFO:tasks.rados.rados.0.smithi122.stdout:972: writing smithi12223906-28 from 736239 to 1206409 tid 1 2024-09-15T03:50:11.793 INFO:tasks.rados.rados.0.smithi122.stdout:972: writing smithi12223906-28 from 1728182 to 2299964 tid 2 2024-09-15T03:50:11.793 INFO:tasks.rados.rados.0.smithi122.stdout:973: snap_remove snap 106 2024-09-15T03:50:12.781 INFO:tasks.rados.rados.0.smithi122.stdout:973: done (1 left) 2024-09-15T03:50:12.781 INFO:tasks.rados.rados.0.smithi122.stdout:974: read oid 40 snap -1 2024-09-15T03:50:12.781 INFO:tasks.rados.rados.0.smithi122.stdout:974: expect (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:50:12.782 INFO:tasks.rados.rados.0.smithi122.stdout:975: rollback oid 11 current snap is 107 2024-09-15T03:50:12.782 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 11 to 105 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:972: finishing write tid 1 to smithi12223906-28 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:972: finishing write tid 2 to smithi12223906-28 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:972: finishing write tid 3 to smithi12223906-28 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:972: finishing write tid 4 to smithi12223906-28 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 210 (ObjNum 377 snap 107 seq_num 377) dirty exists 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:972: left oid 28 (ObjNum 377 snap 107 seq_num 377) 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:972: done (2 left) 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:976: rollback oid 4 current snap is 107 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 105 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:977: read oid 24 snap -1 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:977: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:12.783 INFO:tasks.rados.rados.0.smithi122.stdout:978: snap_create 2024-09-15T03:50:12.790 INFO:tasks.rados.rados.0.smithi122.stdout:974: expect (ObjNum 295 snap 89 seq_num 295) 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:975: finishing rollback tid 0 to smithi12223906-11 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:975: finishing rollback tid 1 to smithi12223906-11 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:975: finishing rollback tid 2 to smithi12223906-11 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 335 (ObjNum 296 snap 90 seq_num 296) dirty exists 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:976: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:976: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:976: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 194 (ObjNum 236 snap 65 seq_num 236) dirty exists 2024-09-15T03:50:12.883 INFO:tasks.rados.rados.0.smithi122.stdout:977: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:13.781 INFO:tasks.rados.rados.0.smithi122.stdout:974: done (4 left) 2024-09-15T03:50:13.781 INFO:tasks.rados.rados.0.smithi122.stdout:975: done (3 left) 2024-09-15T03:50:13.781 INFO:tasks.rados.rados.0.smithi122.stdout:976: done (2 left) 2024-09-15T03:50:13.781 INFO:tasks.rados.rados.0.smithi122.stdout:977: done (1 left) 2024-09-15T03:50:13.781 INFO:tasks.rados.rados.0.smithi122.stdout:978: done (0 left) 2024-09-15T03:50:13.782 INFO:tasks.rados.rados.0.smithi122.stdout:979: read oid 35 snap -1 2024-09-15T03:50:13.782 INFO:tasks.rados.rados.0.smithi122.stdout:979: expect deleted 2024-09-15T03:50:13.782 INFO:tasks.rados.rados.0.smithi122.stdout:980: read oid 2 snap -1 2024-09-15T03:50:13.782 INFO:tasks.rados.rados.0.smithi122.stdout:980: expect deleted 2024-09-15T03:50:13.782 INFO:tasks.rados.rados.0.smithi122.stdout:981: write oid 23 current snap is 108 2024-09-15T03:50:13.782 INFO:tasks.rados.rados.0.smithi122.stdout:981: seq_num 378 ranges {753046=545419,1786664=431793} 2024-09-15T03:50:13.792 INFO:tasks.rados.rados.0.smithi122.stdout:981: writing smithi12223906-23 from 753046 to 1298465 tid 1 2024-09-15T03:50:13.798 INFO:tasks.rados.rados.0.smithi122.stdout:981: writing smithi12223906-23 from 1786664 to 2218457 tid 2 2024-09-15T03:50:13.798 INFO:tasks.rados.rados.0.smithi122.stdout:982: snap_create 2024-09-15T03:50:13.798 INFO:tasks.rados.rados.0.smithi122.stdout:981: finishing write tid 1 to smithi12223906-23 2024-09-15T03:50:13.803 INFO:tasks.rados.rados.0.smithi122.stdout:981: finishing write tid 2 to smithi12223906-23 2024-09-15T03:50:13.803 INFO:tasks.rados.rados.0.smithi122.stdout:981: finishing write tid 3 to smithi12223906-23 2024-09-15T03:50:13.804 INFO:tasks.rados.rados.0.smithi122.stdout:981: finishing write tid 4 to smithi12223906-23 2024-09-15T03:50:13.804 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 512 (ObjNum 378 snap 108 seq_num 378) dirty exists 2024-09-15T03:50:13.804 INFO:tasks.rados.rados.0.smithi122.stdout:981: left oid 23 (ObjNum 378 snap 108 seq_num 378) 2024-09-15T03:50:14.793 INFO:tasks.rados.rados.0.smithi122.stdout:979: done (3 left) 2024-09-15T03:50:14.793 INFO:tasks.rados.rados.0.smithi122.stdout:980: done (2 left) 2024-09-15T03:50:14.793 INFO:tasks.rados.rados.0.smithi122.stdout:981: done (1 left) 2024-09-15T03:50:14.793 INFO:tasks.rados.rados.0.smithi122.stdout:982: done (0 left) 2024-09-15T03:50:14.793 INFO:tasks.rados.rados.0.smithi122.stdout:983: snap_create 2024-09-15T03:50:15.803 INFO:tasks.rados.rados.0.smithi122.stdout:983: done (0 left) 2024-09-15T03:50:15.803 INFO:tasks.rados.rados.0.smithi122.stdout:984: snap_create 2024-09-15T03:50:16.811 INFO:tasks.rados.rados.0.smithi122.stdout:984: done (0 left) 2024-09-15T03:50:16.812 INFO:tasks.rados.rados.0.smithi122.stdout:985: write oid 3 current snap is 111 2024-09-15T03:50:16.812 INFO:tasks.rados.rados.0.smithi122.stdout:985: seq_num 379 ranges {769853=620668,1845146=524707} 2024-09-15T03:50:16.821 INFO:tasks.rados.rados.0.smithi122.stdout:985: writing smithi12223906-3 from 769853 to 1390521 tid 1 2024-09-15T03:50:16.828 INFO:tasks.rados.rados.0.smithi122.stdout:985: writing smithi12223906-3 from 1845146 to 2369853 tid 2 2024-09-15T03:50:16.828 INFO:tasks.rados.rados.0.smithi122.stdout:986: rollback oid 34 current snap is 111 2024-09-15T03:50:16.828 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 108 2024-09-15T03:50:16.828 INFO:tasks.rados.rados.0.smithi122.stdout:987: snap_remove snap 109 2024-09-15T03:50:17.827 INFO:tasks.rados.rados.0.smithi122.stdout:985: finishing write tid 1 to smithi12223906-3 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:986: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 0 (ObjNum 657 snap 0 seq_num 1457507936) dirty dne 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:985: finishing write tid 2 to smithi12223906-3 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:985: finishing write tid 3 to smithi12223906-3 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:985: finishing write tid 4 to smithi12223906-3 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 497 (ObjNum 379 snap 111 seq_num 379) dirty exists 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:985: left oid 3 (ObjNum 379 snap 111 seq_num 379) 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:985: done (2 left) 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:986: done (1 left) 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:987: done (0 left) 2024-09-15T03:50:17.828 INFO:tasks.rados.rados.0.smithi122.stdout:988: rollback oid 29 current snap is 111 2024-09-15T03:50:17.829 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 29 to 108 2024-09-15T03:50:17.829 INFO:tasks.rados.rados.0.smithi122.stdout:989: snap_create 2024-09-15T03:50:17.839 INFO:tasks.rados.rados.0.smithi122.stdout:988: finishing rollback tid 0 to smithi12223906-29 2024-09-15T03:50:17.841 INFO:tasks.rados.rados.0.smithi122.stdout:988: finishing rollback tid 1 to smithi12223906-29 2024-09-15T03:50:17.842 INFO:tasks.rados.rados.0.smithi122.stdout:988: finishing rollback tid 2 to smithi12223906-29 2024-09-15T03:50:17.842 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 216 (ObjNum 341 snap 101 seq_num 341) dirty exists 2024-09-15T03:50:18.839 INFO:tasks.rados.rados.0.smithi122.stdout:988: done (1 left) 2024-09-15T03:50:18.839 INFO:tasks.rados.rados.0.smithi122.stdout:989: done (0 left) 2024-09-15T03:50:18.839 INFO:tasks.rados.rados.0.smithi122.stdout:990: read oid 24 snap -1 2024-09-15T03:50:18.839 INFO:tasks.rados.rados.0.smithi122.stdout:990: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:18.839 INFO:tasks.rados.rados.0.smithi122.stdout:991: snap_remove snap 110 2024-09-15T03:50:19.853 INFO:tasks.rados.rados.0.smithi122.stdout:991: done (1 left) 2024-09-15T03:50:19.858 INFO:tasks.rados.rados.0.smithi122.stdout:992: rollback oid 46 current snap is 112 2024-09-15T03:50:19.858 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 46 to 105 2024-09-15T03:50:19.858 INFO:tasks.rados.rados.0.smithi122.stdout:993: write oid 28 current snap is 112 2024-09-15T03:50:19.858 INFO:tasks.rados.rados.0.smithi122.stdout:993: seq_num 380 ranges {786660=695917,1903628=435462} 2024-09-15T03:50:19.863 INFO:tasks.rados.rados.0.smithi122.stdout:993: writing smithi12223906-28 from 786660 to 1482577 tid 1 2024-09-15T03:50:19.868 INFO:tasks.rados.rados.0.smithi122.stdout:993: writing smithi12223906-28 from 1903628 to 2339090 tid 2 2024-09-15T03:50:19.868 INFO:tasks.rados.rados.0.smithi122.stdout:994: rollback oid 7 current snap is 112 2024-09-15T03:50:19.868 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 111 2024-09-15T03:50:19.868 INFO:tasks.rados.rados.0.smithi122.stdout:990: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:992: finishing rollback tid 0 to smithi12223906-46 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:992: finishing rollback tid 1 to smithi12223906-46 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:992: finishing rollback tid 2 to smithi12223906-46 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 351 (ObjNum 317 snap 96 seq_num 317) dirty exists 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:993: finishing write tid 1 to smithi12223906-28 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:993: finishing write tid 2 to smithi12223906-28 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:993: finishing write tid 3 to smithi12223906-28 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:994: finishing rollback tid 0 to smithi12223906-7 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:993: finishing write tid 4 to smithi12223906-28 2024-09-15T03:50:20.055 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 222 (ObjNum 380 snap 112 seq_num 380) dirty exists 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:993: left oid 28 (ObjNum 380 snap 112 seq_num 380) 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:990: done (3 left) 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:992: done (2 left) 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:993: done (1 left) 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:995: write (excl) oid 13 current snap is 112 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:994: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:994: finishing rollback tid 2 to smithi12223906-7 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 226 (ObjNum 361 snap 103 seq_num 361) dirty exists 2024-09-15T03:50:20.056 INFO:tasks.rados.rados.0.smithi122.stdout:995: seq_num 381 ranges {403467=487519,1562110=579120} 2024-09-15T03:50:20.066 INFO:tasks.rados.rados.0.smithi122.stdout:995: writing smithi12223906-13 from 403467 to 890986 tid 1 2024-09-15T03:50:20.076 INFO:tasks.rados.rados.0.smithi122.stdout:995: writing smithi12223906-13 from 1562110 to 2141230 tid 2 2024-09-15T03:50:20.076 INFO:tasks.rados.rados.0.smithi122.stdout:994: done (1 left) 2024-09-15T03:50:20.076 INFO:tasks.rados.rados.0.smithi122.stdout:996: delete oid 18 current snap is 112 2024-09-15T03:50:20.076 INFO:tasks.rados.rados.0.smithi122.stdout:995: finishing write tid 1 to smithi12223906-13 2024-09-15T03:50:20.082 INFO:tasks.rados.rados.0.smithi122.stdout:995: finishing write tid 2 to smithi12223906-13 2024-09-15T03:50:20.082 INFO:tasks.rados.rados.0.smithi122.stdout:995: finishing write tid 3 to smithi12223906-13 2024-09-15T03:50:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:995: finishing write tid 4 to smithi12223906-13 2024-09-15T03:50:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 354 (ObjNum 381 snap 112 seq_num 381) dirty exists 2024-09-15T03:50:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:995: left oid 13 (ObjNum 381 snap 112 seq_num 381) 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:995: done (1 left) 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:996: done (0 left) 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:997: read oid 20 snap 111 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:997: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:998: read oid 31 snap 105 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:998: expect (ObjNum 343 snap 102 seq_num 343) 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:999: read oid 1 snap -1 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:999: expect (ObjNum 355 snap 102 seq_num 355) 2024-09-15T03:50:20.084 INFO:tasks.rados.rados.0.smithi122.stdout:1000: read oid 39 snap -1 2024-09-15T03:50:20.085 INFO:tasks.rados.rados.0.smithi122.stdout:1000: expect deleted 2024-09-15T03:50:20.085 INFO:tasks.rados.rados.0.smithi122.stdout:1001: write (excl) oid 16 current snap is 112 2024-09-15T03:50:20.085 INFO:tasks.rados.rados.0.smithi122.stdout:1001: seq_num 383 ranges {437081=638017,1679074=582789} 2024-09-15T03:50:20.091 INFO:tasks.rados.rados.0.smithi122.stdout:1001: writing smithi12223906-16 from 437081 to 1075098 tid 1 2024-09-15T03:50:20.100 INFO:tasks.rados.rados.0.smithi122.stdout:1001: writing smithi12223906-16 from 1679074 to 2261863 tid 2 2024-09-15T03:50:20.100 INFO:tasks.rados.rados.0.smithi122.stdout:1002: read oid 13 snap -1 2024-09-15T03:50:20.100 INFO:tasks.rados.rados.0.smithi122.stdout:1002: expect (ObjNum 381 snap 112 seq_num 381) 2024-09-15T03:50:20.100 INFO:tasks.rados.rados.0.smithi122.stdout:1003: delete oid 5 current snap is 112 2024-09-15T03:50:20.100 INFO:tasks.rados.rados.0.smithi122.stdout:997: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:20.349 INFO:tasks.rados.rados.0.smithi122.stdout:997: done (6 left) 2024-09-15T03:50:20.350 INFO:tasks.rados.rados.0.smithi122.stdout:1000: done (5 left) 2024-09-15T03:50:20.350 INFO:tasks.rados.rados.0.smithi122.stdout:1003: done (4 left) 2024-09-15T03:50:20.350 INFO:tasks.rados.rados.0.smithi122.stdout:1004: snap_remove snap 107 2024-09-15T03:50:20.869 INFO:tasks.rados.rados.0.smithi122.stdout:1004: done (4 left) 2024-09-15T03:50:20.869 INFO:tasks.rados.rados.0.smithi122.stdout:1005: read oid 27 snap -1 2024-09-15T03:50:20.870 INFO:tasks.rados.rados.0.smithi122.stdout:1005: expect (ObjNum 309 snap 96 seq_num 309) 2024-09-15T03:50:20.870 INFO:tasks.rados.rados.0.smithi122.stdout:1006: snap_create 2024-09-15T03:50:20.870 INFO:tasks.rados.rados.0.smithi122.stdout:999: expect (ObjNum 355 snap 102 seq_num 355) 2024-09-15T03:50:21.049 INFO:tasks.rados.rados.0.smithi122.stdout:998: expect (ObjNum 343 snap 102 seq_num 343) 2024-09-15T03:50:21.165 INFO:tasks.rados.rados.0.smithi122.stdout:1001: finishing write tid 1 to smithi12223906-16 2024-09-15T03:50:21.165 INFO:tasks.rados.rados.0.smithi122.stdout:1001: finishing write tid 2 to smithi12223906-16 2024-09-15T03:50:21.165 INFO:tasks.rados.rados.0.smithi122.stdout:1001: finishing write tid 3 to smithi12223906-16 2024-09-15T03:50:21.166 INFO:tasks.rados.rados.0.smithi122.stdout:1002: expect (ObjNum 381 snap 112 seq_num 381) 2024-09-15T03:50:21.289 INFO:tasks.rados.rados.0.smithi122.stdout:1001: finishing write tid 4 to smithi12223906-16 2024-09-15T03:50:21.289 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 339 (ObjNum 383 snap 112 seq_num 383) dirty exists 2024-09-15T03:50:21.289 INFO:tasks.rados.rados.0.smithi122.stdout:1001: left oid 16 (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:50:21.289 INFO:tasks.rados.rados.0.smithi122.stdout:1005: expect (ObjNum 309 snap 96 seq_num 309) 2024-09-15T03:50:21.787 INFO:tasks.rados.rados.0.smithi122.stdout:998: done (5 left) 2024-09-15T03:50:21.787 INFO:tasks.rados.rados.0.smithi122.stdout:999: done (4 left) 2024-09-15T03:50:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:1001: done (3 left) 2024-09-15T03:50:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:1002: done (2 left) 2024-09-15T03:50:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:1005: done (1 left) 2024-09-15T03:50:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:1006: done (0 left) 2024-09-15T03:50:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:1007: write (excl) oid 34 current snap is 113 2024-09-15T03:50:21.788 INFO:tasks.rados.rados.0.smithi122.stdout:1007: seq_num 385 ranges {470695=788515,1679685=586458} 2024-09-15T03:50:21.797 INFO:tasks.rados.rados.0.smithi122.stdout:1007: writing smithi12223906-34 from 470695 to 1259210 tid 1 2024-09-15T03:50:21.802 INFO:tasks.rados.rados.0.smithi122.stdout:1007: writing smithi12223906-34 from 1679685 to 2266143 tid 2 2024-09-15T03:50:21.803 INFO:tasks.rados.rados.0.smithi122.stdout:1008: snap_remove snap 105 2024-09-15T03:50:22.791 INFO:tasks.rados.rados.0.smithi122.stdout:1007: finishing write tid 1 to smithi12223906-34 2024-09-15T03:50:22.792 INFO:tasks.rados.rados.0.smithi122.stdout:1007: finishing write tid 2 to smithi12223906-34 2024-09-15T03:50:22.792 INFO:tasks.rados.rados.0.smithi122.stdout:1007: finishing write tid 3 to smithi12223906-34 2024-09-15T03:50:22.792 INFO:tasks.rados.rados.0.smithi122.stdout:1007: finishing write tid 4 to smithi12223906-34 2024-09-15T03:50:22.792 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 342 (ObjNum 385 snap 113 seq_num 385) dirty exists 2024-09-15T03:50:22.792 INFO:tasks.rados.rados.0.smithi122.stdout:1007: left oid 34 (ObjNum 385 snap 113 seq_num 385) 2024-09-15T03:50:22.793 INFO:tasks.rados.rados.0.smithi122.stdout:1007: done (1 left) 2024-09-15T03:50:22.793 INFO:tasks.rados.rados.0.smithi122.stdout:1008: done (0 left) 2024-09-15T03:50:22.793 INFO:tasks.rados.rados.0.smithi122.stdout:1009: read oid 12 snap 112 2024-09-15T03:50:22.793 INFO:tasks.rados.rados.0.smithi122.stdout:1009: expect (ObjNum 372 snap 104 seq_num 372) 2024-09-15T03:50:22.793 INFO:tasks.rados.rados.0.smithi122.stdout:1010: read oid 14 snap -1 2024-09-15T03:50:22.794 INFO:tasks.rados.rados.0.smithi122.stdout:1010: expect (ObjNum 368 snap 103 seq_num 368) 2024-09-15T03:50:22.794 INFO:tasks.rados.rados.0.smithi122.stdout:1011: copy_from oid 50 from oid 20 current snap is 113 2024-09-15T03:50:22.794 INFO:tasks.rados.rados.0.smithi122.stdout:1012: snap_remove snap 111 2024-09-15T03:50:23.793 INFO:tasks.rados.rados.0.smithi122.stdout:1012: done (3 left) 2024-09-15T03:50:23.794 INFO:tasks.rados.rados.0.smithi122.stdout:1013: copy_from oid 41 from oid 26 current snap is 113 2024-09-15T03:50:23.794 INFO:tasks.rados.rados.0.smithi122.stdout:1009: expect (ObjNum 372 snap 104 seq_num 372) 2024-09-15T03:50:23.936 INFO:tasks.rados.rados.0.smithi122.stdout:1010: expect (ObjNum 368 snap 103 seq_num 368) 2024-09-15T03:50:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:1011: finishing copy_from racing read to smithi12223906-50 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1011: finishing copy_from to smithi12223906-50 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 74 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1013: finishing copy_from to smithi12223906-41 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1013: got expected ENOENT (src dne) 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1013: finishing copy_from racing read to smithi12223906-41 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1009: done (3 left) 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1010: done (2 left) 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1011: done (1 left) 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1013: done (0 left) 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1014: read oid 42 snap 108 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1014: expect deleted 2024-09-15T03:50:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:1015: snap_remove snap 112 2024-09-15T03:50:24.884 INFO:tasks.rados.rados.0.smithi122.stdout:1015: done (1 left) 2024-09-15T03:50:24.884 INFO:tasks.rados.rados.0.smithi122.stdout:1016: read oid 38 snap -1 2024-09-15T03:50:24.884 INFO:tasks.rados.rados.0.smithi122.stdout:1016: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:24.884 INFO:tasks.rados.rados.0.smithi122.stdout:1017: read oid 13 snap 108 2024-09-15T03:50:24.885 INFO:tasks.rados.rados.0.smithi122.stdout:1017: expect (ObjNum 375 snap 105 seq_num 375) 2024-09-15T03:50:24.885 INFO:tasks.rados.rados.0.smithi122.stdout:1018: write (excl) oid 12 current snap is 113 2024-09-15T03:50:24.885 INFO:tasks.rados.rados.0.smithi122.stdout:1018: seq_num 386 ranges {487502=463764,1738167=446469} 2024-09-15T03:50:24.894 INFO:tasks.rados.rados.0.smithi122.stdout:1018: writing smithi12223906-12 from 487502 to 951266 tid 1 2024-09-15T03:50:24.902 INFO:tasks.rados.rados.0.smithi122.stdout:1018: writing smithi12223906-12 from 1738167 to 2184636 tid 2 2024-09-15T03:50:24.902 INFO:tasks.rados.rados.0.smithi122.stdout:1019: delete oid 2 current snap is 113 2024-09-15T03:50:24.903 INFO:tasks.rados.rados.0.smithi122.stdout:1017: expect (ObjNum 375 snap 105 seq_num 375) 2024-09-15T03:50:25.067 INFO:tasks.rados.rados.0.smithi122.stdout:1016: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:25.255 INFO:tasks.rados.rados.0.smithi122.stdout:1018: finishing write tid 1 to smithi12223906-12 2024-09-15T03:50:25.255 INFO:tasks.rados.rados.0.smithi122.stdout:1018: finishing write tid 2 to smithi12223906-12 2024-09-15T03:50:25.255 INFO:tasks.rados.rados.0.smithi122.stdout:1018: finishing write tid 3 to smithi12223906-12 2024-09-15T03:50:25.255 INFO:tasks.rados.rados.0.smithi122.stdout:1018: finishing write tid 4 to smithi12223906-12 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 345 (ObjNum 386 snap 113 seq_num 386) dirty exists 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1018: left oid 12 (ObjNum 386 snap 113 seq_num 386) 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1014: done (4 left) 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1016: done (3 left) 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1017: done (2 left) 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1018: done (1 left) 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1019: done (0 left) 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1020: copy_from oid 3 from oid 47 current snap is 113 2024-09-15T03:50:25.256 INFO:tasks.rados.rados.0.smithi122.stdout:1021: write (excl) oid 49 current snap is 113 2024-09-15T03:50:25.257 INFO:tasks.rados.rados.0.smithi122.stdout:1021: seq_num 388 ranges {521116=730615,1971484=549632} 2024-09-15T03:50:25.272 INFO:tasks.rados.rados.0.smithi122.stdout:1021: writing smithi12223906-49 from 521116 to 1251731 tid 1 2024-09-15T03:50:25.280 INFO:tasks.rados.rados.0.smithi122.stdout:1021: writing smithi12223906-49 from 1971484 to 2521116 tid 2 2024-09-15T03:50:25.281 INFO:tasks.rados.rados.0.smithi122.stdout:1022: read oid 44 snap -1 2024-09-15T03:50:25.281 INFO:tasks.rados.rados.0.smithi122.stdout:1022: expect deleted 2024-09-15T03:50:25.281 INFO:tasks.rados.rados.0.smithi122.stdout:1023: rollback oid 35 current snap is 113 2024-09-15T03:50:25.281 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 108 2024-09-15T03:50:25.281 INFO:tasks.rados.rados.0.smithi122.stdout:1020: finishing copy_from racing read to smithi12223906-3 2024-09-15T03:50:25.282 INFO:tasks.rados.rados.0.smithi122.stdout:1020: finishing copy_from to smithi12223906-3 2024-09-15T03:50:25.282 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 174 (ObjNum 227 snap 62 seq_num 227) dirty exists 2024-09-15T03:50:25.282 INFO:tasks.rados.rados.0.smithi122.stdout:1021: finishing write tid 1 to smithi12223906-49 2024-09-15T03:50:25.282 INFO:tasks.rados.rados.0.smithi122.stdout:1020: done (3 left) 2024-09-15T03:50:25.282 INFO:tasks.rados.rados.0.smithi122.stdout:1024: write (excl) oid 3 current snap is 113 2024-09-15T03:50:25.283 INFO:tasks.rados.rados.0.smithi122.stdout:1024: seq_num 389 ranges {537923=405864,1513613=593796} 2024-09-15T03:50:25.285 INFO:tasks.rados.rados.0.smithi122.stdout:1024: writing smithi12223906-3 from 537923 to 943787 tid 1 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1024: writing smithi12223906-3 from 1513613 to 2107409 tid 2 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1025: read oid 2 snap 108 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1025: expect deleted 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1026: read oid 1 snap 108 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1026: expect (ObjNum 355 snap 102 seq_num 355) 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1027: read oid 30 snap -1 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1027: expect (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1028: write oid 22 current snap is 113 2024-09-15T03:50:25.292 INFO:tasks.rados.rados.0.smithi122.stdout:1028: seq_num 390 ranges {554730=481113,1572095=737454} 2024-09-15T03:50:25.298 INFO:tasks.rados.rados.0.smithi122.stdout:1028: writing smithi12223906-22 from 554730 to 1035843 tid 1 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1028: writing smithi12223906-22 from 1572095 to 2309549 tid 2 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1029: delete oid 10 current snap is 113 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1023: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 0 (ObjNum 961 snap 0 seq_num 872751936) dirty dne 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1021: finishing write tid 2 to smithi12223906-49 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1021: finishing write tid 3 to smithi12223906-49 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1021: finishing write tid 4 to smithi12223906-49 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 313 (ObjNum 388 snap 113 seq_num 388) dirty exists 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1021: left oid 49 (ObjNum 388 snap 113 seq_num 388) 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1024: finishing write tid 1 to smithi12223906-3 2024-09-15T03:50:25.306 INFO:tasks.rados.rados.0.smithi122.stdout:1027: expect (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:50:25.389 INFO:tasks.rados.rados.0.smithi122.stdout:1024: finishing write tid 2 to smithi12223906-3 2024-09-15T03:50:25.389 INFO:tasks.rados.rados.0.smithi122.stdout:1024: finishing write tid 3 to smithi12223906-3 2024-09-15T03:50:25.390 INFO:tasks.rados.rados.0.smithi122.stdout:1024: finishing write tid 4 to smithi12223906-3 2024-09-15T03:50:25.390 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 509 (ObjNum 389 snap 113 seq_num 389) dirty exists 2024-09-15T03:50:25.390 INFO:tasks.rados.rados.0.smithi122.stdout:1024: left oid 3 (ObjNum 389 snap 113 seq_num 389) 2024-09-15T03:50:25.390 INFO:tasks.rados.rados.0.smithi122.stdout:1026: expect (ObjNum 355 snap 102 seq_num 355) 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1028: finishing write tid 1 to smithi12223906-22 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1028: finishing write tid 2 to smithi12223906-22 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1028: finishing write tid 3 to smithi12223906-22 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1028: finishing write tid 4 to smithi12223906-22 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 534 (ObjNum 390 snap 113 seq_num 390) dirty exists 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1028: left oid 22 (ObjNum 390 snap 113 seq_num 390) 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1021: done (8 left) 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1022: done (7 left) 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1023: done (6 left) 2024-09-15T03:50:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:1024: done (5 left) 2024-09-15T03:50:25.552 INFO:tasks.rados.rados.0.smithi122.stdout:1025: done (4 left) 2024-09-15T03:50:25.552 INFO:tasks.rados.rados.0.smithi122.stdout:1026: done (3 left) 2024-09-15T03:50:25.552 INFO:tasks.rados.rados.0.smithi122.stdout:1027: done (2 left) 2024-09-15T03:50:25.552 INFO:tasks.rados.rados.0.smithi122.stdout:1028: done (1 left) 2024-09-15T03:50:25.552 INFO:tasks.rados.rados.0.smithi122.stdout:1029: done (0 left) 2024-09-15T03:50:25.552 INFO:tasks.rados.rados.0.smithi122.stdout:1030: snap_remove snap 108 2024-09-15T03:50:25.890 INFO:tasks.rados.rados.0.smithi122.stdout:1030: done (0 left) 2024-09-15T03:50:25.890 INFO:tasks.rados.rados.0.smithi122.stdout:1031: read oid 23 snap -1 2024-09-15T03:50:25.891 INFO:tasks.rados.rados.0.smithi122.stdout:1031: expect (ObjNum 378 snap 108 seq_num 378) 2024-09-15T03:50:25.894 INFO:tasks.rados.rados.0.smithi122.stdout:1032: snap_create 2024-09-15T03:50:25.907 INFO:tasks.rados.rados.0.smithi122.stdout:1031: expect (ObjNum 378 snap 108 seq_num 378) 2024-09-15T03:50:26.899 INFO:tasks.rados.rados.0.smithi122.stdout:1031: done (1 left) 2024-09-15T03:50:26.899 INFO:tasks.rados.rados.0.smithi122.stdout:1032: done (0 left) 2024-09-15T03:50:26.900 INFO:tasks.rados.rados.0.smithi122.stdout:1033: rollback oid 19 current snap is 114 2024-09-15T03:50:26.900 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 113 2024-09-15T03:50:26.900 INFO:tasks.rados.rados.0.smithi122.stdout:1034: copy_from oid 43 from oid 13 current snap is 114 2024-09-15T03:50:26.900 INFO:tasks.rados.rados.0.smithi122.stdout:1035: snap_create 2024-09-15T03:50:26.910 INFO:tasks.rados.rados.0.smithi122.stdout:1033: finishing rollback tid 0 to smithi12223906-19 2024-09-15T03:50:26.913 INFO:tasks.rados.rados.0.smithi122.stdout:1033: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:50:26.913 INFO:tasks.rados.rados.0.smithi122.stdout:1033: finishing rollback tid 2 to smithi12223906-19 2024-09-15T03:50:26.913 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 357 (ObjNum 232 snap 65 seq_num 232) dirty exists 2024-09-15T03:50:26.914 INFO:tasks.rados.rados.0.smithi122.stdout:1034: finishing copy_from racing read to smithi12223906-43 2024-09-15T03:50:26.915 INFO:tasks.rados.rados.0.smithi122.stdout:1034: finishing copy_from to smithi12223906-43 2024-09-15T03:50:26.916 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 354 (ObjNum 381 snap 112 seq_num 381) dirty exists 2024-09-15T03:50:27.905 INFO:tasks.rados.rados.0.smithi122.stdout:1033: done (2 left) 2024-09-15T03:50:27.905 INFO:tasks.rados.rados.0.smithi122.stdout:1034: done (1 left) 2024-09-15T03:50:27.905 INFO:tasks.rados.rados.0.smithi122.stdout:1035: done (0 left) 2024-09-15T03:50:27.905 INFO:tasks.rados.rados.0.smithi122.stdout:1036: write (excl) oid 40 current snap is 115 2024-09-15T03:50:27.905 INFO:tasks.rados.rados.0.smithi122.stdout:1036: seq_num 392 ranges {588344=631611,1689059=741123} 2024-09-15T03:50:27.913 INFO:tasks.rados.rados.0.smithi122.stdout:1036: writing smithi12223906-40 from 588344 to 1219955 tid 1 2024-09-15T03:50:27.921 INFO:tasks.rados.rados.0.smithi122.stdout:1036: writing smithi12223906-40 from 1689059 to 2430182 tid 2 2024-09-15T03:50:27.921 INFO:tasks.rados.rados.0.smithi122.stdout:1037: write (excl) oid 47 current snap is 115 2024-09-15T03:50:27.921 INFO:tasks.rados.rados.0.smithi122.stdout:1036: finishing write tid 1 to smithi12223906-40 2024-09-15T03:50:27.921 INFO:tasks.rados.rados.0.smithi122.stdout:1037: seq_num 393 ranges {605151=706860,2031188=573963} 2024-09-15T03:50:27.928 INFO:tasks.rados.rados.0.smithi122.stdout:1037: writing smithi12223906-47 from 605151 to 1312011 tid 1 2024-09-15T03:50:27.935 INFO:tasks.rados.rados.0.smithi122.stdout:1037: writing smithi12223906-47 from 2031188 to 2605151 tid 2 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:1038: snap_create 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:1036: finishing write tid 2 to smithi12223906-40 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:1036: finishing write tid 3 to smithi12223906-40 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:1036: finishing write tid 4 to smithi12223906-40 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 169 (ObjNum 392 snap 115 seq_num 392) dirty exists 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:1036: left oid 40 (ObjNum 392 snap 115 seq_num 392) 2024-09-15T03:50:27.936 INFO:tasks.rados.rados.0.smithi122.stdout:1037: finishing write tid 1 to smithi12223906-47 2024-09-15T03:50:27.940 INFO:tasks.rados.rados.0.smithi122.stdout:1037: finishing write tid 2 to smithi12223906-47 2024-09-15T03:50:27.940 INFO:tasks.rados.rados.0.smithi122.stdout:1037: finishing write tid 3 to smithi12223906-47 2024-09-15T03:50:27.940 INFO:tasks.rados.rados.0.smithi122.stdout:1037: finishing write tid 4 to smithi12223906-47 2024-09-15T03:50:27.941 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 184 (ObjNum 393 snap 115 seq_num 393) dirty exists 2024-09-15T03:50:27.941 INFO:tasks.rados.rados.0.smithi122.stdout:1037: left oid 47 (ObjNum 393 snap 115 seq_num 393) 2024-09-15T03:50:28.926 INFO:tasks.rados.rados.0.smithi122.stdout:1036: done (2 left) 2024-09-15T03:50:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1037: done (1 left) 2024-09-15T03:50:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1038: done (0 left) 2024-09-15T03:50:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1039: read oid 42 snap -1 2024-09-15T03:50:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1039: expect deleted 2024-09-15T03:50:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1040: write (excl) oid 45 current snap is 116 2024-09-15T03:50:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1040: seq_num 394 ranges {621958=782109,2089670=532288} 2024-09-15T03:50:28.939 INFO:tasks.rados.rados.0.smithi122.stdout:1040: writing smithi12223906-45 from 621958 to 1404067 tid 1 2024-09-15T03:50:28.946 INFO:tasks.rados.rados.0.smithi122.stdout:1040: writing smithi12223906-45 from 2089670 to 2621958 tid 2 2024-09-15T03:50:28.947 INFO:tasks.rados.rados.0.smithi122.stdout:1041: read oid 50 snap -1 2024-09-15T03:50:28.947 INFO:tasks.rados.rados.0.smithi122.stdout:1041: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:28.947 INFO:tasks.rados.rados.0.smithi122.stdout:1042: snap_remove snap 115 2024-09-15T03:50:29.940 INFO:tasks.rados.rados.0.smithi122.stdout:1042: done (3 left) 2024-09-15T03:50:29.940 INFO:tasks.rados.rados.0.smithi122.stdout:1043: snap_remove snap 113 2024-09-15T03:50:30.951 INFO:tasks.rados.rados.0.smithi122.stdout:1043: done (3 left) 2024-09-15T03:50:30.951 INFO:tasks.rados.rados.0.smithi122.stdout:1044: read oid 19 snap -1 2024-09-15T03:50:30.951 INFO:tasks.rados.rados.0.smithi122.stdout:1044: expect (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:50:30.952 INFO:tasks.rados.rados.0.smithi122.stdout:1045: snap_remove snap 114 2024-09-15T03:50:31.793 INFO:tasks.rados.rados.0.smithi122.stdout:1045: done (4 left) 2024-09-15T03:50:31.793 INFO:tasks.rados.rados.0.smithi122.stdout:1046: read oid 9 snap -1 2024-09-15T03:50:31.794 INFO:tasks.rados.rados.0.smithi122.stdout:1046: expect (ObjNum 312 snap 96 seq_num 312) 2024-09-15T03:50:31.794 INFO:tasks.rados.rados.0.smithi122.stdout:1047: copy_from oid 37 from oid 21 current snap is 116 2024-09-15T03:50:31.794 INFO:tasks.rados.rados.0.smithi122.stdout:1048: read oid 3 snap -1 2024-09-15T03:50:31.794 INFO:tasks.rados.rados.0.smithi122.stdout:1048: expect (ObjNum 389 snap 113 seq_num 389) 2024-09-15T03:50:31.794 INFO:tasks.rados.rados.0.smithi122.stdout:1049: write oid 12 current snap is 116 2024-09-15T03:50:31.794 INFO:tasks.rados.rados.0.smithi122.stdout:1049: seq_num 395 ranges {638765=457358,1748152=604803} 2024-09-15T03:50:31.803 INFO:tasks.rados.rados.0.smithi122.stdout:1049: writing smithi12223906-12 from 638765 to 1096123 tid 1 2024-09-15T03:50:31.811 INFO:tasks.rados.rados.0.smithi122.stdout:1049: writing smithi12223906-12 from 1748152 to 2352955 tid 2 2024-09-15T03:50:31.811 INFO:tasks.rados.rados.0.smithi122.stdout:1050: read oid 30 snap -1 2024-09-15T03:50:31.811 INFO:tasks.rados.rados.0.smithi122.stdout:1050: expect (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:50:31.811 INFO:tasks.rados.rados.0.smithi122.stdout:1051: write (excl) oid 5 current snap is 116 2024-09-15T03:50:31.811 INFO:tasks.rados.rados.0.smithi122.stdout:1051: seq_num 396 ranges {655572=648960,1922987=732585} 2024-09-15T03:50:31.819 INFO:tasks.rados.rados.0.smithi122.stdout:1051: writing smithi12223906-5 from 655572 to 1304532 tid 1 2024-09-15T03:50:31.827 INFO:tasks.rados.rados.0.smithi122.stdout:1051: writing smithi12223906-5 from 1922987 to 2655572 tid 2 2024-09-15T03:50:31.827 INFO:tasks.rados.rados.0.smithi122.stdout:1052: copy_from oid 6 from oid 33 current snap is 116 2024-09-15T03:50:31.827 INFO:tasks.rados.rados.0.smithi122.stdout:1040: finishing write tid 1 to smithi12223906-45 2024-09-15T03:50:31.827 INFO:tasks.rados.rados.0.smithi122.stdout:1040: finishing write tid 2 to smithi12223906-45 2024-09-15T03:50:31.827 INFO:tasks.rados.rados.0.smithi122.stdout:1040: finishing write tid 3 to smithi12223906-45 2024-09-15T03:50:31.827 INFO:tasks.rados.rados.0.smithi122.stdout:1040: finishing write tid 4 to smithi12223906-45 2024-09-15T03:50:31.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 360 (ObjNum 394 snap 116 seq_num 394) dirty exists 2024-09-15T03:50:31.828 INFO:tasks.rados.rados.0.smithi122.stdout:1040: left oid 45 (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:50:31.828 INFO:tasks.rados.rados.0.smithi122.stdout:1039: done (10 left) 2024-09-15T03:50:31.828 INFO:tasks.rados.rados.0.smithi122.stdout:1040: done (9 left) 2024-09-15T03:50:31.828 INFO:tasks.rados.rados.0.smithi122.stdout:1053: snap_create 2024-09-15T03:50:31.828 INFO:tasks.rados.rados.0.smithi122.stdout:1041: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:32.078 INFO:tasks.rados.rados.0.smithi122.stdout:1044: expect (ObjNum 232 snap 65 seq_num 232) 2024-09-15T03:50:32.362 INFO:tasks.rados.rados.0.smithi122.stdout:1046: expect (ObjNum 312 snap 96 seq_num 312) 2024-09-15T03:50:32.553 INFO:tasks.rados.rados.0.smithi122.stdout:1048: expect (ObjNum 389 snap 113 seq_num 389) 2024-09-15T03:50:32.683 INFO:tasks.rados.rados.0.smithi122.stdout:1049: finishing write tid 1 to smithi12223906-12 2024-09-15T03:50:32.683 INFO:tasks.rados.rados.0.smithi122.stdout:1047: finishing copy_from racing read to smithi12223906-37 2024-09-15T03:50:32.683 INFO:tasks.rados.rados.0.smithi122.stdout:1050: expect (ObjNum 359 snap 103 seq_num 359) 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:1049: finishing write tid 2 to smithi12223906-12 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:1049: finishing write tid 3 to smithi12223906-12 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:1049: finishing write tid 4 to smithi12223906-12 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 348 (ObjNum 395 snap 116 seq_num 395) dirty exists 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:1049: left oid 12 (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:1047: finishing copy_from to smithi12223906-37 2024-09-15T03:50:32.759 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 188 (ObjNum 369 snap 103 seq_num 369) dirty exists 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1051: finishing write tid 1 to smithi12223906-5 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1051: finishing write tid 2 to smithi12223906-5 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1051: finishing write tid 3 to smithi12223906-5 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1051: finishing write tid 4 to smithi12223906-5 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 187 (ObjNum 396 snap 116 seq_num 396) dirty exists 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1051: left oid 5 (ObjNum 396 snap 116 seq_num 396) 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1052: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:1052: finishing copy_from to smithi12223906-6 2024-09-15T03:50:32.760 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 297 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:50:32.793 INFO:tasks.rados.rados.0.smithi122.stdout:1041: done (9 left) 2024-09-15T03:50:32.793 INFO:tasks.rados.rados.0.smithi122.stdout:1044: done (8 left) 2024-09-15T03:50:32.794 INFO:tasks.rados.rados.0.smithi122.stdout:1046: done (7 left) 2024-09-15T03:50:32.794 INFO:tasks.rados.rados.0.smithi122.stdout:1047: done (6 left) 2024-09-15T03:50:32.794 INFO:tasks.rados.rados.0.smithi122.stdout:1048: done (5 left) 2024-09-15T03:50:32.794 INFO:tasks.rados.rados.0.smithi122.stdout:1049: done (4 left) 2024-09-15T03:50:32.795 INFO:tasks.rados.rados.0.smithi122.stdout:1050: done (3 left) 2024-09-15T03:50:32.795 INFO:tasks.rados.rados.0.smithi122.stdout:1051: done (2 left) 2024-09-15T03:50:32.795 INFO:tasks.rados.rados.0.smithi122.stdout:1052: done (1 left) 2024-09-15T03:50:32.795 INFO:tasks.rados.rados.0.smithi122.stdout:1053: done (0 left) 2024-09-15T03:50:32.795 INFO:tasks.rados.rados.0.smithi122.stdout:1054: write oid 36 current snap is 117 2024-09-15T03:50:32.796 INFO:tasks.rados.rados.0.smithi122.stdout:1054: seq_num 397 ranges {672379=724209,1865116=608472} 2024-09-15T03:50:32.805 INFO:tasks.rados.rados.0.smithi122.stdout:1054: writing smithi12223906-36 from 672379 to 1396588 tid 1 2024-09-15T03:50:32.811 INFO:tasks.rados.rados.0.smithi122.stdout:1054: writing smithi12223906-36 from 1865116 to 2473588 tid 2 2024-09-15T03:50:32.811 INFO:tasks.rados.rados.0.smithi122.stdout:1055: snap_create 2024-09-15T03:50:32.811 INFO:tasks.rados.rados.0.smithi122.stdout:1054: finishing write tid 1 to smithi12223906-36 2024-09-15T03:50:32.814 INFO:tasks.rados.rados.0.smithi122.stdout:1054: finishing write tid 2 to smithi12223906-36 2024-09-15T03:50:32.814 INFO:tasks.rados.rados.0.smithi122.stdout:1054: finishing write tid 3 to smithi12223906-36 2024-09-15T03:50:32.814 INFO:tasks.rados.rados.0.smithi122.stdout:1054: finishing write tid 4 to smithi12223906-36 2024-09-15T03:50:32.815 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 518 (ObjNum 397 snap 117 seq_num 397) dirty exists 2024-09-15T03:50:32.815 INFO:tasks.rados.rados.0.smithi122.stdout:1054: left oid 36 (ObjNum 397 snap 117 seq_num 397) 2024-09-15T03:50:33.806 INFO:tasks.rados.rados.0.smithi122.stdout:1054: done (1 left) 2024-09-15T03:50:33.807 INFO:tasks.rados.rados.0.smithi122.stdout:1055: done (0 left) 2024-09-15T03:50:33.807 INFO:tasks.rados.rados.0.smithi122.stdout:1056: write (excl) oid 5 current snap is 118 2024-09-15T03:50:33.807 INFO:tasks.rados.rados.0.smithi122.stdout:1056: seq_num 398 ranges {689186=799458,1923598=752130} 2024-09-15T03:50:33.821 INFO:tasks.rados.rados.0.smithi122.stdout:1056: writing smithi12223906-5 from 689186 to 1488644 tid 1 2024-09-15T03:50:33.828 INFO:tasks.rados.rados.0.smithi122.stdout:1056: writing smithi12223906-5 from 1923598 to 2675728 tid 2 2024-09-15T03:50:33.828 INFO:tasks.rados.rados.0.smithi122.stdout:1057: delete oid 11 current snap is 118 2024-09-15T03:50:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:1057: done (1 left) 2024-09-15T03:50:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:1058: delete oid 9 current snap is 118 2024-09-15T03:50:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:1056: finishing write tid 1 to smithi12223906-5 2024-09-15T03:50:33.835 INFO:tasks.rados.rados.0.smithi122.stdout:1056: finishing write tid 2 to smithi12223906-5 2024-09-15T03:50:33.836 INFO:tasks.rados.rados.0.smithi122.stdout:1056: finishing write tid 3 to smithi12223906-5 2024-09-15T03:50:33.836 INFO:tasks.rados.rados.0.smithi122.stdout:1056: finishing write tid 4 to smithi12223906-5 2024-09-15T03:50:33.836 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 190 (ObjNum 398 snap 118 seq_num 398) dirty exists 2024-09-15T03:50:33.836 INFO:tasks.rados.rados.0.smithi122.stdout:1056: left oid 5 (ObjNum 398 snap 118 seq_num 398) 2024-09-15T03:50:33.838 INFO:tasks.rados.rados.0.smithi122.stdout:1056: done (1 left) 2024-09-15T03:50:33.838 INFO:tasks.rados.rados.0.smithi122.stdout:1058: done (0 left) 2024-09-15T03:50:33.838 INFO:tasks.rados.rados.0.smithi122.stdout:1059: snap_remove snap 117 2024-09-15T03:50:34.809 INFO:tasks.rados.rados.0.smithi122.stdout:1059: done (0 left) 2024-09-15T03:50:34.809 INFO:tasks.rados.rados.0.smithi122.stdout:1060: snap_remove snap 116 2024-09-15T03:50:35.824 INFO:tasks.rados.rados.0.smithi122.stdout:1060: done (0 left) 2024-09-15T03:50:35.825 INFO:tasks.rados.rados.0.smithi122.stdout:1061: rollback oid 21 current snap is 118 2024-09-15T03:50:35.825 INFO:tasks.rados.rados.0.smithi122.stdout:1061: done (0 left) 2024-09-15T03:50:35.825 INFO:tasks.rados.rados.0.smithi122.stdout:1062: read oid 50 snap -1 2024-09-15T03:50:35.825 INFO:tasks.rados.rados.0.smithi122.stdout:1062: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:35.825 INFO:tasks.rados.rados.0.smithi122.stdout:1063: write oid 14 current snap is 118 2024-09-15T03:50:35.826 INFO:tasks.rados.rados.0.smithi122.stdout:1063: seq_num 401 ranges {739607=625205,1982691=499457} 2024-09-15T03:50:35.834 INFO:tasks.rados.rados.0.smithi122.stdout:1063: writing smithi12223906-14 from 739607 to 1364812 tid 1 2024-09-15T03:50:35.841 INFO:tasks.rados.rados.0.smithi122.stdout:1063: writing smithi12223906-14 from 1982691 to 2482148 tid 2 2024-09-15T03:50:35.841 INFO:tasks.rados.rados.0.smithi122.stdout:1064: write oid 11 current snap is 118 2024-09-15T03:50:35.841 INFO:tasks.rados.rados.0.smithi122.stdout:1064: seq_num 402 ranges {756414=700454,2041173=715241} 2024-09-15T03:50:35.849 INFO:tasks.rados.rados.0.smithi122.stdout:1064: writing smithi12223906-11 from 756414 to 1456868 tid 1 2024-09-15T03:50:35.857 INFO:tasks.rados.rados.0.smithi122.stdout:1064: writing smithi12223906-11 from 2041173 to 2756414 tid 2 2024-09-15T03:50:35.857 INFO:tasks.rados.rados.0.smithi122.stdout:1065: copy_from oid 7 from oid 38 current snap is 118 2024-09-15T03:50:35.857 INFO:tasks.rados.rados.0.smithi122.stdout:1062: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1063: finishing write tid 1 to smithi12223906-14 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1063: finishing write tid 2 to smithi12223906-14 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1063: finishing write tid 3 to smithi12223906-14 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1063: finishing write tid 4 to smithi12223906-14 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 521 (ObjNum 401 snap 118 seq_num 401) dirty exists 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1063: left oid 14 (ObjNum 401 snap 118 seq_num 401) 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1064: finishing write tid 1 to smithi12223906-11 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1064: finishing write tid 2 to smithi12223906-11 2024-09-15T03:50:36.112 INFO:tasks.rados.rados.0.smithi122.stdout:1062: done (3 left) 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1063: done (2 left) 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1066: rollback oid 21 current snap is 118 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1064: finishing write tid 3 to smithi12223906-11 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1064: finishing write tid 4 to smithi12223906-11 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 358 (ObjNum 402 snap 118 seq_num 402) dirty exists 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1064: left oid 11 (ObjNum 402 snap 118 seq_num 402) 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1065: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1065: finishing copy_from to smithi12223906-7 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 230 (ObjNum 318 snap 96 seq_num 318) dirty exists 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1064: done (2 left) 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1065: done (1 left) 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1066: done (0 left) 2024-09-15T03:50:36.113 INFO:tasks.rados.rados.0.smithi122.stdout:1067: snap_create 2024-09-15T03:50:36.832 INFO:tasks.rados.rados.0.smithi122.stdout:1067: done (0 left) 2024-09-15T03:50:36.833 INFO:tasks.rados.rados.0.smithi122.stdout:1068: read oid 21 snap -1 2024-09-15T03:50:36.833 INFO:tasks.rados.rados.0.smithi122.stdout:1068: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:36.833 INFO:tasks.rados.rados.0.smithi122.stdout:1069: read oid 39 snap -1 2024-09-15T03:50:36.833 INFO:tasks.rados.rados.0.smithi122.stdout:1069: expect deleted 2024-09-15T03:50:36.833 INFO:tasks.rados.rados.0.smithi122.stdout:1070: write (excl) oid 10 current snap is 119 2024-09-15T03:50:36.833 INFO:tasks.rados.rados.0.smithi122.stdout:1070: seq_num 403 ranges {773221=492056,1816008=503126} 2024-09-15T03:50:36.843 INFO:tasks.rados.rados.0.smithi122.stdout:1070: writing smithi12223906-10 from 773221 to 1265277 tid 1 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1070: writing smithi12223906-10 from 1816008 to 2319134 tid 2 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1071: read oid 32 snap -1 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1071: expect (ObjNum 358 snap 103 seq_num 358) 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1072: read oid 12 snap -1 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1072: expect (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1073: write (excl) oid 24 current snap is 119 2024-09-15T03:50:36.850 INFO:tasks.rados.rados.0.smithi122.stdout:1073: seq_num 404 ranges {790028=567305,1874490=763137} 2024-09-15T03:50:36.857 INFO:tasks.rados.rados.0.smithi122.stdout:1073: writing smithi12223906-24 from 790028 to 1357333 tid 1 2024-09-15T03:50:36.865 INFO:tasks.rados.rados.0.smithi122.stdout:1073: writing smithi12223906-24 from 1874490 to 2637627 tid 2 2024-09-15T03:50:36.865 INFO:tasks.rados.rados.0.smithi122.stdout:1074: rollback oid 19 current snap is 119 2024-09-15T03:50:36.865 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 118 2024-09-15T03:50:36.865 INFO:tasks.rados.rados.0.smithi122.stdout:1068: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:36.941 INFO:tasks.rados.rados.0.smithi122.stdout:1070: finishing write tid 1 to smithi12223906-10 2024-09-15T03:50:36.941 INFO:tasks.rados.rados.0.smithi122.stdout:1070: finishing write tid 2 to smithi12223906-10 2024-09-15T03:50:36.942 INFO:tasks.rados.rados.0.smithi122.stdout:1070: finishing write tid 3 to smithi12223906-10 2024-09-15T03:50:36.942 INFO:tasks.rados.rados.0.smithi122.stdout:1070: finishing write tid 4 to smithi12223906-10 2024-09-15T03:50:36.942 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 196 (ObjNum 403 snap 119 seq_num 403) dirty exists 2024-09-15T03:50:36.942 INFO:tasks.rados.rados.0.smithi122.stdout:1070: left oid 10 (ObjNum 403 snap 119 seq_num 403) 2024-09-15T03:50:36.942 INFO:tasks.rados.rados.0.smithi122.stdout:1071: expect (ObjNum 358 snap 103 seq_num 358) 2024-09-15T03:50:37.076 INFO:tasks.rados.rados.0.smithi122.stdout:1072: expect (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:37.255 INFO:tasks.rados.rados.0.smithi122.stdout:1073: finishing write tid 1 to smithi12223906-24 2024-09-15T03:50:37.255 INFO:tasks.rados.rados.0.smithi122.stdout:1073: finishing write tid 2 to smithi12223906-24 2024-09-15T03:50:37.255 INFO:tasks.rados.rados.0.smithi122.stdout:1073: finishing write tid 3 to smithi12223906-24 2024-09-15T03:50:37.255 INFO:tasks.rados.rados.0.smithi122.stdout:1073: finishing write tid 4 to smithi12223906-24 2024-09-15T03:50:37.255 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 525 (ObjNum 404 snap 119 seq_num 404) dirty exists 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1073: left oid 24 (ObjNum 404 snap 119 seq_num 404) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1074: finishing rollback tid 0 to smithi12223906-19 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1074: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1074: finishing rollback tid 2 to smithi12223906-19 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 363 (ObjNum 232 snap 65 seq_num 232) dirty exists 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1068: done (6 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1069: done (5 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1070: done (4 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1071: done (3 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1072: done (2 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1073: done (1 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1074: done (0 left) 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1075: copy_from oid 34 from oid 18 current snap is 119 2024-09-15T03:50:37.256 INFO:tasks.rados.rados.0.smithi122.stdout:1076: write (excl) oid 46 current snap is 119 2024-09-15T03:50:37.257 INFO:tasks.rados.rados.0.smithi122.stdout:1076: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2024-09-15T03:50:37.268 INFO:tasks.rados.rados.0.smithi122.stdout:1076: writing smithi12223906-46 from 406835 to 1049389 tid 1 2024-09-15T03:50:37.274 INFO:tasks.rados.rados.0.smithi122.stdout:1076: writing smithi12223906-46 from 1532972 to 2039767 tid 2 2024-09-15T03:50:37.278 INFO:tasks.rados.rados.0.smithi122.stdout:1076: writing smithi12223906-46 from 2772312 to 2806835 tid 3 2024-09-15T03:50:37.278 INFO:tasks.rados.rados.0.smithi122.stdout:1077: write (excl) oid 4 current snap is 119 2024-09-15T03:50:37.278 INFO:tasks.rados.rados.0.smithi122.stdout:1075: finishing copy_from to smithi12223906-34 2024-09-15T03:50:37.278 INFO:tasks.rados.rados.0.smithi122.stdout:1075: got expected ENOENT (src dne) 2024-09-15T03:50:37.278 INFO:tasks.rados.rados.0.smithi122.stdout:1077: seq_num 406 ranges {423642=717803,1875101=766806} 2024-09-15T03:50:37.284 INFO:tasks.rados.rados.0.smithi122.stdout:1077: writing smithi12223906-4 from 423642 to 1141445 tid 1 2024-09-15T03:50:37.293 INFO:tasks.rados.rados.0.smithi122.stdout:1077: writing smithi12223906-4 from 1875101 to 2641907 tid 2 2024-09-15T03:50:37.293 INFO:tasks.rados.rados.0.smithi122.stdout:1078: read oid 16 snap 118 2024-09-15T03:50:37.293 INFO:tasks.rados.rados.0.smithi122.stdout:1078: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:50:37.293 INFO:tasks.rados.rados.0.smithi122.stdout:1079: rollback oid 26 current snap is 119 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 118 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1075: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1075: done (4 left) 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1080: rollback oid 37 current snap is 119 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 37 to 118 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: finishing write tid 1 to smithi12223906-46 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: finishing write tid 2 to smithi12223906-46 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: finishing write tid 3 to smithi12223906-46 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: finishing write tid 4 to smithi12223906-46 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: finishing write tid 5 to smithi12223906-46 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 367 (ObjNum 405 snap 119 seq_num 405) dirty exists 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: left oid 46 (ObjNum 405 snap 119 seq_num 405) 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1076: done (4 left) 2024-09-15T03:50:37.294 INFO:tasks.rados.rados.0.smithi122.stdout:1081: delete oid 13 current snap is 119 2024-09-15T03:50:37.295 INFO:tasks.rados.rados.0.smithi122.stdout:1079: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:50:37.295 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 0 (ObjNum 657 snap 0 seq_num 1842315520) dirty dne 2024-09-15T03:50:37.295 INFO:tasks.rados.rados.0.smithi122.stdout:1077: finishing write tid 1 to smithi12223906-4 2024-09-15T03:50:37.299 INFO:tasks.rados.rados.0.smithi122.stdout:1079: done (4 left) 2024-09-15T03:50:37.299 INFO:tasks.rados.rados.0.smithi122.stdout:1081: done (3 left) 2024-09-15T03:50:37.300 INFO:tasks.rados.rados.0.smithi122.stdout:1082: delete oid 5 current snap is 119 2024-09-15T03:50:37.300 INFO:tasks.rados.rados.0.smithi122.stdout:1080: finishing rollback tid 0 to smithi12223906-37 2024-09-15T03:50:37.300 INFO:tasks.rados.rados.0.smithi122.stdout:1080: finishing rollback tid 1 to smithi12223906-37 2024-09-15T03:50:37.300 INFO:tasks.rados.rados.0.smithi122.stdout:1080: finishing rollback tid 2 to smithi12223906-37 2024-09-15T03:50:37.300 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 200 (ObjNum 369 snap 103 seq_num 369) dirty exists 2024-09-15T03:50:37.301 INFO:tasks.rados.rados.0.smithi122.stdout:1078: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:50:37.441 INFO:tasks.rados.rados.0.smithi122.stdout:1077: finishing write tid 2 to smithi12223906-4 2024-09-15T03:50:37.441 INFO:tasks.rados.rados.0.smithi122.stdout:1077: finishing write tid 3 to smithi12223906-4 2024-09-15T03:50:37.441 INFO:tasks.rados.rados.0.smithi122.stdout:1077: finishing write tid 4 to smithi12223906-4 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 197 (ObjNum 406 snap 119 seq_num 406) dirty exists 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:1077: left oid 4 (ObjNum 406 snap 119 seq_num 406) 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:1077: done (3 left) 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:1078: done (2 left) 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:1080: done (1 left) 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:1082: done (0 left) 2024-09-15T03:50:37.442 INFO:tasks.rados.rados.0.smithi122.stdout:1083: snap_create 2024-09-15T03:50:37.847 INFO:tasks.rados.rados.0.smithi122.stdout:1083: done (0 left) 2024-09-15T03:50:37.847 INFO:tasks.rados.rados.0.smithi122.stdout:1084: read oid 38 snap -1 2024-09-15T03:50:37.847 INFO:tasks.rados.rados.0.smithi122.stdout:1084: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:37.848 INFO:tasks.rados.rados.0.smithi122.stdout:1085: copy_from oid 15 from oid 21 current snap is 120 2024-09-15T03:50:37.848 INFO:tasks.rados.rados.0.smithi122.stdout:1086: read oid 22 snap -1 2024-09-15T03:50:37.848 INFO:tasks.rados.rados.0.smithi122.stdout:1086: expect (ObjNum 390 snap 113 seq_num 390) 2024-09-15T03:50:37.848 INFO:tasks.rados.rados.0.smithi122.stdout:1087: write (excl) oid 5 current snap is 120 2024-09-15T03:50:37.848 INFO:tasks.rados.rados.0.smithi122.stdout:1087: seq_num 409 ranges {474063=543550,1650547=514133,2765651=108412} 2024-09-15T03:50:37.857 INFO:tasks.rados.rados.0.smithi122.stdout:1087: writing smithi12223906-5 from 474063 to 1017613 tid 1 2024-09-15T03:50:37.864 INFO:tasks.rados.rados.0.smithi122.stdout:1087: writing smithi12223906-5 from 1650547 to 2164680 tid 2 2024-09-15T03:50:37.868 INFO:tasks.rados.rados.0.smithi122.stdout:1087: writing smithi12223906-5 from 2765651 to 2874063 tid 3 2024-09-15T03:50:37.868 INFO:tasks.rados.rados.0.smithi122.stdout:1088: copy_from oid 40 from oid 32 current snap is 120 2024-09-15T03:50:37.868 INFO:tasks.rados.rados.0.smithi122.stdout:1085: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:50:37.868 INFO:tasks.rados.rados.0.smithi122.stdout:1085: finishing copy_from to smithi12223906-15 2024-09-15T03:50:37.868 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 188 (ObjNum 369 snap 103 seq_num 369) dirty exists 2024-09-15T03:50:37.868 INFO:tasks.rados.rados.0.smithi122.stdout:1084: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:38.070 INFO:tasks.rados.rados.0.smithi122.stdout:1086: expect (ObjNum 390 snap 113 seq_num 390) 2024-09-15T03:50:38.278 INFO:tasks.rados.rados.0.smithi122.stdout:1087: finishing write tid 1 to smithi12223906-5 2024-09-15T03:50:38.278 INFO:tasks.rados.rados.0.smithi122.stdout:1087: finishing write tid 2 to smithi12223906-5 2024-09-15T03:50:38.278 INFO:tasks.rados.rados.0.smithi122.stdout:1087: finishing write tid 3 to smithi12223906-5 2024-09-15T03:50:38.278 INFO:tasks.rados.rados.0.smithi122.stdout:1087: finishing write tid 4 to smithi12223906-5 2024-09-15T03:50:38.278 INFO:tasks.rados.rados.0.smithi122.stdout:1087: finishing write tid 5 to smithi12223906-5 2024-09-15T03:50:38.278 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 195 (ObjNum 409 snap 120 seq_num 409) dirty exists 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1087: left oid 5 (ObjNum 409 snap 120 seq_num 409) 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1088: finishing copy_from racing read to smithi12223906-40 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1088: finishing copy_from to smithi12223906-40 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 499 (ObjNum 358 snap 103 seq_num 358) dirty exists 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1084: done (4 left) 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1085: done (3 left) 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1086: done (2 left) 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1087: done (1 left) 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1088: done (0 left) 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1089: write (excl) oid 18 current snap is 120 2024-09-15T03:50:38.279 INFO:tasks.rados.rados.0.smithi122.stdout:1089: seq_num 410 ranges {490870=618799,1592676=774144,2793074=97796} 2024-09-15T03:50:38.292 INFO:tasks.rados.rados.0.smithi122.stdout:1089: writing smithi12223906-18 from 490870 to 1109669 tid 1 2024-09-15T03:50:38.299 INFO:tasks.rados.rados.0.smithi122.stdout:1089: writing smithi12223906-18 from 1592676 to 2366820 tid 2 2024-09-15T03:50:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:1089: writing smithi12223906-18 from 2793074 to 2890870 tid 3 2024-09-15T03:50:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:1090: snap_remove snap 118 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1090: done (1 left) 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1091: delete oid 49 current snap is 120 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1089: finishing write tid 1 to smithi12223906-18 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1089: finishing write tid 2 to smithi12223906-18 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1089: finishing write tid 3 to smithi12223906-18 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1089: finishing write tid 4 to smithi12223906-18 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1089: finishing write tid 5 to smithi12223906-18 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 544 (ObjNum 410 snap 120 seq_num 410) dirty exists 2024-09-15T03:50:38.862 INFO:tasks.rados.rados.0.smithi122.stdout:1089: left oid 18 (ObjNum 410 snap 120 seq_num 410) 2024-09-15T03:50:38.873 INFO:tasks.rados.rados.0.smithi122.stdout:1089: done (1 left) 2024-09-15T03:50:38.873 INFO:tasks.rados.rados.0.smithi122.stdout:1091: done (0 left) 2024-09-15T03:50:38.873 INFO:tasks.rados.rados.0.smithi122.stdout:1092: copy_from oid 8 from oid 12 current snap is 120 2024-09-15T03:50:38.874 INFO:tasks.rados.rados.0.smithi122.stdout:1093: snap_create 2024-09-15T03:50:38.883 INFO:tasks.rados.rados.0.smithi122.stdout:1092: finishing copy_from racing read to smithi12223906-8 2024-09-15T03:50:38.891 INFO:tasks.rados.rados.0.smithi122.stdout:1092: finishing copy_from to smithi12223906-8 2024-09-15T03:50:38.891 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 348 (ObjNum 395 snap 116 seq_num 395) dirty exists 2024-09-15T03:50:39.870 INFO:tasks.rados.rados.0.smithi122.stdout:1092: done (1 left) 2024-09-15T03:50:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:1093: done (0 left) 2024-09-15T03:50:39.871 INFO:tasks.rados.rados.0.smithi122.stdout:1094: snap_create 2024-09-15T03:50:40.875 INFO:tasks.rados.rados.0.smithi122.stdout:1094: done (0 left) 2024-09-15T03:50:40.875 INFO:tasks.rados.rados.0.smithi122.stdout:1095: write (excl) oid 31 current snap is 122 2024-09-15T03:50:40.875 INFO:tasks.rados.rados.0.smithi122.stdout:1095: seq_num 412 ranges {524484=485650,1425993=661460,2847920=76564} 2024-09-15T03:50:40.885 INFO:tasks.rados.rados.0.smithi122.stdout:1095: writing smithi12223906-31 from 524484 to 1010134 tid 1 2024-09-15T03:50:40.892 INFO:tasks.rados.rados.0.smithi122.stdout:1095: writing smithi12223906-31 from 1425993 to 2087453 tid 2 2024-09-15T03:50:40.897 INFO:tasks.rados.rados.0.smithi122.stdout:1095: writing smithi12223906-31 from 2847920 to 2924484 tid 3 2024-09-15T03:50:40.897 INFO:tasks.rados.rados.0.smithi122.stdout:1096: write oid 20 current snap is 122 2024-09-15T03:50:40.897 INFO:tasks.rados.rados.0.smithi122.stdout:1096: seq_num 413 ranges {541291=560899,1884475=521471} 2024-09-15T03:50:40.903 INFO:tasks.rados.rados.0.smithi122.stdout:1096: writing smithi12223906-20 from 541291 to 1102190 tid 1 2024-09-15T03:50:40.911 INFO:tasks.rados.rados.0.smithi122.stdout:1096: writing smithi12223906-20 from 1884475 to 2405946 tid 2 2024-09-15T03:50:40.911 INFO:tasks.rados.rados.0.smithi122.stdout:1097: rollback oid 34 current snap is 122 2024-09-15T03:50:40.911 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 120 2024-09-15T03:50:40.911 INFO:tasks.rados.rados.0.smithi122.stdout:1095: finishing write tid 1 to smithi12223906-31 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1095: finishing write tid 2 to smithi12223906-31 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1095: finishing write tid 3 to smithi12223906-31 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1095: finishing write tid 4 to smithi12223906-31 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1095: finishing write tid 5 to smithi12223906-31 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 201 (ObjNum 412 snap 122 seq_num 412) dirty exists 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1095: left oid 31 (ObjNum 412 snap 122 seq_num 412) 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1095: done (2 left) 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1098: read oid 4 snap -1 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1098: expect (ObjNum 406 snap 119 seq_num 406) 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1099: read oid 35 snap 121 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1099: expect deleted 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1100: read oid 29 snap -1 2024-09-15T03:50:40.912 INFO:tasks.rados.rados.0.smithi122.stdout:1100: expect (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:50:40.913 INFO:tasks.rados.rados.0.smithi122.stdout:1101: snap_create 2024-09-15T03:50:40.918 INFO:tasks.rados.rados.0.smithi122.stdout:1096: finishing write tid 1 to smithi12223906-20 2024-09-15T03:50:40.919 INFO:tasks.rados.rados.0.smithi122.stdout:1096: finishing write tid 2 to smithi12223906-20 2024-09-15T03:50:40.919 INFO:tasks.rados.rados.0.smithi122.stdout:1096: finishing write tid 3 to smithi12223906-20 2024-09-15T03:50:40.919 INFO:tasks.rados.rados.0.smithi122.stdout:1096: finishing write tid 4 to smithi12223906-20 2024-09-15T03:50:40.919 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 80 (ObjNum 413 snap 122 seq_num 413) dirty exists 2024-09-15T03:50:40.920 INFO:tasks.rados.rados.0.smithi122.stdout:1096: left oid 20 (ObjNum 413 snap 122 seq_num 413) 2024-09-15T03:50:40.920 INFO:tasks.rados.rados.0.smithi122.stdout:1097: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:50:40.922 INFO:tasks.rados.rados.0.smithi122.stdout:1097: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:50:40.922 INFO:tasks.rados.rados.0.smithi122.stdout:1097: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:50:40.922 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 351 (ObjNum 385 snap 113 seq_num 385) dirty exists 2024-09-15T03:50:40.927 INFO:tasks.rados.rados.0.smithi122.stdout:1100: expect (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:50:41.146 INFO:tasks.rados.rados.0.smithi122.stdout:1098: expect (ObjNum 406 snap 119 seq_num 406) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1096: done (5 left) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1097: done (4 left) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1098: done (3 left) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1099: done (2 left) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1100: done (1 left) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1101: done (0 left) 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1102: write (excl) oid 35 current snap is 123 2024-09-15T03:50:41.795 INFO:tasks.rados.rados.0.smithi122.stdout:1102: seq_num 414 ranges {558098=636148,1826604=665129} 2024-09-15T03:50:41.806 INFO:tasks.rados.rados.0.smithi122.stdout:1102: writing smithi12223906-35 from 558098 to 1194246 tid 1 2024-09-15T03:50:41.814 INFO:tasks.rados.rados.0.smithi122.stdout:1102: writing smithi12223906-35 from 1826604 to 2491733 tid 2 2024-09-15T03:50:41.814 INFO:tasks.rados.rados.0.smithi122.stdout:1103: snap_remove snap 121 2024-09-15T03:50:42.810 INFO:tasks.rados.rados.0.smithi122.stdout:1103: done (1 left) 2024-09-15T03:50:42.810 INFO:tasks.rados.rados.0.smithi122.stdout:1104: read oid 31 snap 119 2024-09-15T03:50:42.810 INFO:tasks.rados.rados.0.smithi122.stdout:1104: expect (ObjNum 343 snap 102 seq_num 343) 2024-09-15T03:50:42.810 INFO:tasks.rados.rados.0.smithi122.stdout:1102: finishing write tid 1 to smithi12223906-35 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1102: finishing write tid 2 to smithi12223906-35 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1102: finishing write tid 3 to smithi12223906-35 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1102: finishing write tid 4 to smithi12223906-35 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 100 (ObjNum 414 snap 123 seq_num 414) dirty exists 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1102: left oid 35 (ObjNum 414 snap 123 seq_num 414) 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1102: done (1 left) 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1105: rollback oid 20 current snap is 123 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 120 2024-09-15T03:50:42.811 INFO:tasks.rados.rados.0.smithi122.stdout:1106: write oid 13 current snap is 123 2024-09-15T03:50:42.812 INFO:tasks.rados.rados.0.smithi122.stdout:1106: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2024-09-15T03:50:42.820 INFO:tasks.rados.rados.0.smithi122.stdout:1106: writing smithi12223906-13 from 574905 to 1286302 tid 1 2024-09-15T03:50:42.827 INFO:tasks.rados.rados.0.smithi122.stdout:1106: writing smithi12223906-13 from 1885086 to 2410226 tid 2 2024-09-15T03:50:42.830 INFO:tasks.rados.rados.0.smithi122.stdout:1106: writing smithi12223906-13 from 2930189 to 2974905 tid 3 2024-09-15T03:50:42.830 INFO:tasks.rados.rados.0.smithi122.stdout:1107: snap_create 2024-09-15T03:50:42.830 INFO:tasks.rados.rados.0.smithi122.stdout:1105: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:50:42.830 INFO:tasks.rados.rados.0.smithi122.stdout:1105: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:50:42.830 INFO:tasks.rados.rados.0.smithi122.stdout:1104: expect (ObjNum 343 snap 102 seq_num 343) 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1105: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 86 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1106: finishing write tid 1 to smithi12223906-13 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1106: finishing write tid 2 to smithi12223906-13 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1106: finishing write tid 3 to smithi12223906-13 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1106: finishing write tid 4 to smithi12223906-13 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1106: finishing write tid 5 to smithi12223906-13 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 372 (ObjNum 415 snap 123 seq_num 415) dirty exists 2024-09-15T03:50:42.955 INFO:tasks.rados.rados.0.smithi122.stdout:1106: left oid 13 (ObjNum 415 snap 123 seq_num 415) 2024-09-15T03:50:43.812 INFO:tasks.rados.rados.0.smithi122.stdout:1104: done (3 left) 2024-09-15T03:50:43.812 INFO:tasks.rados.rados.0.smithi122.stdout:1105: done (2 left) 2024-09-15T03:50:43.812 INFO:tasks.rados.rados.0.smithi122.stdout:1106: done (1 left) 2024-09-15T03:50:43.812 INFO:tasks.rados.rados.0.smithi122.stdout:1107: done (0 left) 2024-09-15T03:50:43.812 INFO:tasks.rados.rados.0.smithi122.stdout:1108: snap_remove snap 119 2024-09-15T03:50:44.816 INFO:tasks.rados.rados.0.smithi122.stdout:1108: done (0 left) 2024-09-15T03:50:44.816 INFO:tasks.rados.rados.0.smithi122.stdout:1109: read oid 44 snap -1 2024-09-15T03:50:44.816 INFO:tasks.rados.rados.0.smithi122.stdout:1109: expect deleted 2024-09-15T03:50:44.816 INFO:tasks.rados.rados.0.smithi122.stdout:1110: read oid 21 snap 123 2024-09-15T03:50:44.816 INFO:tasks.rados.rados.0.smithi122.stdout:1110: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:44.816 INFO:tasks.rados.rados.0.smithi122.stdout:1111: snap_create 2024-09-15T03:50:44.820 INFO:tasks.rados.rados.0.smithi122.stdout:1110: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1109: done (2 left) 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1110: done (1 left) 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1111: done (0 left) 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1112: read oid 46 snap -1 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1112: expect (ObjNum 405 snap 119 seq_num 405) 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1113: rollback oid 42 current snap is 125 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 120 2024-09-15T03:50:45.834 INFO:tasks.rados.rados.0.smithi122.stdout:1114: delete oid 32 current snap is 125 2024-09-15T03:50:45.835 INFO:tasks.rados.rados.0.smithi122.stdout:1113: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:50:45.836 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 0 (ObjNum 705 snap 0 seq_num 1842991568) dirty dne 2024-09-15T03:50:45.846 INFO:tasks.rados.rados.0.smithi122.stdout:1113: done (2 left) 2024-09-15T03:50:45.846 INFO:tasks.rados.rados.0.smithi122.stdout:1114: done (1 left) 2024-09-15T03:50:45.846 INFO:tasks.rados.rados.0.smithi122.stdout:1115: snap_create 2024-09-15T03:50:45.850 INFO:tasks.rados.rados.0.smithi122.stdout:1112: expect (ObjNum 405 snap 119 seq_num 405) 2024-09-15T03:50:46.797 INFO:tasks.rados.rados.0.smithi122.stdout:1112: done (1 left) 2024-09-15T03:50:46.797 INFO:tasks.rados.rados.0.smithi122.stdout:1115: done (0 left) 2024-09-15T03:50:46.797 INFO:tasks.rados.rados.0.smithi122.stdout:1116: read oid 9 snap -1 2024-09-15T03:50:46.798 INFO:tasks.rados.rados.0.smithi122.stdout:1116: expect deleted 2024-09-15T03:50:46.798 INFO:tasks.rados.rados.0.smithi122.stdout:1117: write oid 18 current snap is 126 2024-09-15T03:50:46.798 INFO:tasks.rados.rados.0.smithi122.stdout:1117: seq_num 417 ranges {608519=461895,1602050=528809,2585035=423484} 2024-09-15T03:50:46.804 INFO:tasks.rados.rados.0.smithi122.stdout:1117: writing smithi12223906-18 from 608519 to 1070414 tid 1 2024-09-15T03:50:46.810 INFO:tasks.rados.rados.0.smithi122.stdout:1117: writing smithi12223906-18 from 1602050 to 2130859 tid 2 2024-09-15T03:50:46.815 INFO:tasks.rados.rados.0.smithi122.stdout:1117: writing smithi12223906-18 from 2585035 to 3008519 tid 3 2024-09-15T03:50:46.815 INFO:tasks.rados.rados.0.smithi122.stdout:1118: delete oid 4 current snap is 126 2024-09-15T03:50:46.815 INFO:tasks.rados.rados.0.smithi122.stdout:1117: finishing write tid 1 to smithi12223906-18 2024-09-15T03:50:46.815 INFO:tasks.rados.rados.0.smithi122.stdout:1117: finishing write tid 2 to smithi12223906-18 2024-09-15T03:50:46.819 INFO:tasks.rados.rados.0.smithi122.stdout:1117: finishing write tid 3 to smithi12223906-18 2024-09-15T03:50:46.820 INFO:tasks.rados.rados.0.smithi122.stdout:1117: finishing write tid 4 to smithi12223906-18 2024-09-15T03:50:46.820 INFO:tasks.rados.rados.0.smithi122.stdout:1117: finishing write tid 5 to smithi12223906-18 2024-09-15T03:50:46.820 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 551 (ObjNum 417 snap 126 seq_num 417) dirty exists 2024-09-15T03:50:46.821 INFO:tasks.rados.rados.0.smithi122.stdout:1117: left oid 18 (ObjNum 417 snap 126 seq_num 417) 2024-09-15T03:50:46.822 INFO:tasks.rados.rados.0.smithi122.stdout:1116: done (2 left) 2024-09-15T03:50:46.822 INFO:tasks.rados.rados.0.smithi122.stdout:1117: done (1 left) 2024-09-15T03:50:46.822 INFO:tasks.rados.rados.0.smithi122.stdout:1118: done (0 left) 2024-09-15T03:50:46.822 INFO:tasks.rados.rados.0.smithi122.stdout:1119: snap_remove snap 123 2024-09-15T03:50:47.810 INFO:tasks.rados.rados.0.smithi122.stdout:1119: done (0 left) 2024-09-15T03:50:47.810 INFO:tasks.rados.rados.0.smithi122.stdout:1120: read oid 16 snap 124 2024-09-15T03:50:47.811 INFO:tasks.rados.rados.0.smithi122.stdout:1120: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:50:47.811 INFO:tasks.rados.rados.0.smithi122.stdout:1121: read oid 41 snap 120 2024-09-15T03:50:47.811 INFO:tasks.rados.rados.0.smithi122.stdout:1121: expect (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:50:47.811 INFO:tasks.rados.rados.0.smithi122.stdout:1122: copy_from oid 48 from oid 46 current snap is 126 2024-09-15T03:50:47.812 INFO:tasks.rados.rados.0.smithi122.stdout:1123: copy_from oid 44 from oid 39 current snap is 126 2024-09-15T03:50:47.812 INFO:tasks.rados.rados.0.smithi122.stdout:1124: write oid 28 current snap is 126 2024-09-15T03:50:47.812 INFO:tasks.rados.rados.0.smithi122.stdout:1124: seq_num 419 ranges {642133=728746,2119014=532478} 2024-09-15T03:50:47.821 INFO:tasks.rados.rados.0.smithi122.stdout:1124: writing smithi12223906-28 from 642133 to 1370879 tid 1 2024-09-15T03:50:47.828 INFO:tasks.rados.rados.0.smithi122.stdout:1124: writing smithi12223906-28 from 2119014 to 2651492 tid 2 2024-09-15T03:50:47.828 INFO:tasks.rados.rados.0.smithi122.stdout:1125: write (excl) oid 4 current snap is 126 2024-09-15T03:50:47.828 INFO:tasks.rados.rados.0.smithi122.stdout:1123: finishing copy_from to smithi12223906-44 2024-09-15T03:50:47.828 INFO:tasks.rados.rados.0.smithi122.stdout:1123: got expected ENOENT (src dne) 2024-09-15T03:50:47.829 INFO:tasks.rados.rados.0.smithi122.stdout:1125: seq_num 420 ranges {658940=403995,1777496=676136,2950951=107989} 2024-09-15T03:50:47.834 INFO:tasks.rados.rados.0.smithi122.stdout:1125: writing smithi12223906-4 from 658940 to 1062935 tid 1 2024-09-15T03:50:47.842 INFO:tasks.rados.rados.0.smithi122.stdout:1125: writing smithi12223906-4 from 1777496 to 2453632 tid 2 2024-09-15T03:50:47.846 INFO:tasks.rados.rados.0.smithi122.stdout:1125: writing smithi12223906-4 from 2950951 to 3058940 tid 3 2024-09-15T03:50:47.846 INFO:tasks.rados.rados.0.smithi122.stdout:1126: rollback oid 7 current snap is 126 2024-09-15T03:50:47.846 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 125 2024-09-15T03:50:47.846 INFO:tasks.rados.rados.0.smithi122.stdout:1123: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:50:47.846 INFO:tasks.rados.rados.0.smithi122.stdout:1121: expect (ObjNum 191 snap 53 seq_num 191) 2024-09-15T03:50:48.025 INFO:tasks.rados.rados.0.smithi122.stdout:1120: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:50:48.160 INFO:tasks.rados.rados.0.smithi122.stdout:1122: finishing copy_from racing read to smithi12223906-48 2024-09-15T03:50:48.160 INFO:tasks.rados.rados.0.smithi122.stdout:1122: finishing copy_from to smithi12223906-48 2024-09-15T03:50:48.160 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 367 (ObjNum 405 snap 119 seq_num 405) dirty exists 2024-09-15T03:50:48.160 INFO:tasks.rados.rados.0.smithi122.stdout:1124: finishing write tid 1 to smithi12223906-28 2024-09-15T03:50:48.160 INFO:tasks.rados.rados.0.smithi122.stdout:1124: finishing write tid 2 to smithi12223906-28 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1124: finishing write tid 3 to smithi12223906-28 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1124: finishing write tid 4 to smithi12223906-28 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 253 (ObjNum 419 snap 126 seq_num 419) dirty exists 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1124: left oid 28 (ObjNum 419 snap 126 seq_num 419) 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1125: finishing write tid 1 to smithi12223906-4 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1125: finishing write tid 2 to smithi12223906-4 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1120: done (6 left) 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1121: done (5 left) 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1122: done (4 left) 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1123: done (3 left) 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1124: done (2 left) 2024-09-15T03:50:48.161 INFO:tasks.rados.rados.0.smithi122.stdout:1127: snap_remove snap 124 2024-09-15T03:50:48.846 INFO:tasks.rados.rados.0.smithi122.stdout:1127: done (2 left) 2024-09-15T03:50:48.846 INFO:tasks.rados.rados.0.smithi122.stdout:1128: write (excl) oid 27 current snap is 126 2024-09-15T03:50:48.846 INFO:tasks.rados.rados.0.smithi122.stdout:1128: seq_num 421 ranges {675747=479244,1835978=536147} 2024-09-15T03:50:48.852 INFO:tasks.rados.rados.0.smithi122.stdout:1128: writing smithi12223906-27 from 675747 to 1154991 tid 1 2024-09-15T03:50:48.859 INFO:tasks.rados.rados.0.smithi122.stdout:1128: writing smithi12223906-27 from 1835978 to 2372125 tid 2 2024-09-15T03:50:48.859 INFO:tasks.rados.rados.0.smithi122.stdout:1129: write oid 15 current snap is 126 2024-09-15T03:50:48.859 INFO:tasks.rados.rados.0.smithi122.stdout:1129: seq_num 422 ranges {692554=554493,1778107=679805,2889444=203110} 2024-09-15T03:50:48.865 INFO:tasks.rados.rados.0.smithi122.stdout:1129: writing smithi12223906-15 from 692554 to 1247047 tid 1 2024-09-15T03:50:48.872 INFO:tasks.rados.rados.0.smithi122.stdout:1129: writing smithi12223906-15 from 1778107 to 2457912 tid 2 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1129: writing smithi12223906-15 from 2889444 to 3092554 tid 3 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1130: copy_from oid 10 from oid 12 current snap is 126 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1126: finishing rollback tid 0 to smithi12223906-7 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1125: finishing write tid 3 to smithi12223906-4 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1125: finishing write tid 4 to smithi12223906-4 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1126: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1125: finishing write tid 5 to smithi12223906-4 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 206 (ObjNum 420 snap 126 seq_num 420) dirty exists 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1125: left oid 4 (ObjNum 420 snap 126 seq_num 420) 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1126: finishing rollback tid 2 to smithi12223906-7 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 257 (ObjNum 318 snap 96 seq_num 318) dirty exists 2024-09-15T03:50:48.877 INFO:tasks.rados.rados.0.smithi122.stdout:1128: finishing write tid 1 to smithi12223906-27 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1128: finishing write tid 2 to smithi12223906-27 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1128: finishing write tid 3 to smithi12223906-27 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1128: finishing write tid 4 to smithi12223906-27 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 563 (ObjNum 421 snap 126 seq_num 421) dirty exists 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1128: left oid 27 (ObjNum 421 snap 126 seq_num 421) 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1129: finishing write tid 1 to smithi12223906-15 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1125: done (4 left) 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1126: done (3 left) 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1128: done (2 left) 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1131: read oid 6 snap -1 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1131: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1132: copy_from oid 28 from oid 9 current snap is 126 2024-09-15T03:50:48.878 INFO:tasks.rados.rados.0.smithi122.stdout:1133: delete oid 8 current snap is 126 2024-09-15T03:50:48.879 INFO:tasks.rados.rados.0.smithi122.stdout:1129: finishing write tid 2 to smithi12223906-15 2024-09-15T03:50:48.880 INFO:tasks.rados.rados.0.smithi122.stdout:1129: finishing write tid 3 to smithi12223906-15 2024-09-15T03:50:48.881 INFO:tasks.rados.rados.0.smithi122.stdout:1129: finishing write tid 4 to smithi12223906-15 2024-09-15T03:50:48.882 INFO:tasks.rados.rados.0.smithi122.stdout:1129: finishing write tid 5 to smithi12223906-15 2024-09-15T03:50:48.882 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 364 (ObjNum 422 snap 126 seq_num 422) dirty exists 2024-09-15T03:50:48.882 INFO:tasks.rados.rados.0.smithi122.stdout:1129: left oid 15 (ObjNum 422 snap 126 seq_num 422) 2024-09-15T03:50:48.885 INFO:tasks.rados.rados.0.smithi122.stdout:1129: done (4 left) 2024-09-15T03:50:48.885 INFO:tasks.rados.rados.0.smithi122.stdout:1133: done (3 left) 2024-09-15T03:50:48.885 INFO:tasks.rados.rados.0.smithi122.stdout:1134: write oid 35 current snap is 126 2024-09-15T03:50:48.885 INFO:tasks.rados.rados.0.smithi122.stdout:1134: seq_num 424 ranges {726168=704991,1895071=683474} 2024-09-15T03:50:48.894 INFO:tasks.rados.rados.0.smithi122.stdout:1134: writing smithi12223906-35 from 726168 to 1431159 tid 1 2024-09-15T03:50:48.901 INFO:tasks.rados.rados.0.smithi122.stdout:1134: writing smithi12223906-35 from 1895071 to 2578545 tid 2 2024-09-15T03:50:48.901 INFO:tasks.rados.rados.0.smithi122.stdout:1135: read oid 37 snap -1 2024-09-15T03:50:48.901 INFO:tasks.rados.rados.0.smithi122.stdout:1135: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:48.901 INFO:tasks.rados.rados.0.smithi122.stdout:1136: read oid 49 snap -1 2024-09-15T03:50:48.901 INFO:tasks.rados.rados.0.smithi122.stdout:1136: expect deleted 2024-09-15T03:50:48.902 INFO:tasks.rados.rados.0.smithi122.stdout:1137: copy_from oid 8 from oid 45 current snap is 126 2024-09-15T03:50:48.905 INFO:tasks.rados.rados.0.smithi122.stdout:1131: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:49.086 INFO:tasks.rados.rados.0.smithi122.stdout:1132: finishing copy_from to smithi12223906-28 2024-09-15T03:50:49.086 INFO:tasks.rados.rados.0.smithi122.stdout:1132: got expected ENOENT (src dne) 2024-09-15T03:50:49.086 INFO:tasks.rados.rados.0.smithi122.stdout:1132: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:50:49.086 INFO:tasks.rados.rados.0.smithi122.stdout:1130: finishing copy_from racing read to smithi12223906-10 2024-09-15T03:50:49.086 INFO:tasks.rados.rados.0.smithi122.stdout:1130: finishing copy_from to smithi12223906-10 2024-09-15T03:50:49.086 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 348 (ObjNum 395 snap 116 seq_num 395) dirty exists 2024-09-15T03:50:49.087 INFO:tasks.rados.rados.0.smithi122.stdout:1135: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:49.158 INFO:tasks.rados.rados.0.smithi122.stdout:1134: finishing write tid 1 to smithi12223906-35 2024-09-15T03:50:49.158 INFO:tasks.rados.rados.0.smithi122.stdout:1134: finishing write tid 2 to smithi12223906-35 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1134: finishing write tid 3 to smithi12223906-35 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1134: finishing write tid 4 to smithi12223906-35 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 104 (ObjNum 424 snap 126 seq_num 424) dirty exists 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1134: left oid 35 (ObjNum 424 snap 126 seq_num 424) 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1137: finishing copy_from racing read to smithi12223906-8 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1137: finishing copy_from to smithi12223906-8 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 360 (ObjNum 394 snap 116 seq_num 394) dirty exists 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1130: done (6 left) 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1131: done (5 left) 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1132: done (4 left) 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1134: done (3 left) 2024-09-15T03:50:49.159 INFO:tasks.rados.rados.0.smithi122.stdout:1135: done (2 left) 2024-09-15T03:50:49.160 INFO:tasks.rados.rados.0.smithi122.stdout:1136: done (1 left) 2024-09-15T03:50:49.160 INFO:tasks.rados.rados.0.smithi122.stdout:1137: done (0 left) 2024-09-15T03:50:49.160 INFO:tasks.rados.rados.0.smithi122.stdout:1138: snap_create 2024-09-15T03:50:49.850 INFO:tasks.rados.rados.0.smithi122.stdout:1138: done (0 left) 2024-09-15T03:50:49.850 INFO:tasks.rados.rados.0.smithi122.stdout:1139: delete oid 21 current snap is 127 2024-09-15T03:50:49.854 INFO:tasks.rados.rados.0.smithi122.stdout:1139: done (0 left) 2024-09-15T03:50:49.854 INFO:tasks.rados.rados.0.smithi122.stdout:1140: read oid 50 snap -1 2024-09-15T03:50:49.854 INFO:tasks.rados.rados.0.smithi122.stdout:1140: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:49.854 INFO:tasks.rados.rados.0.smithi122.stdout:1141: write oid 37 current snap is 127 2024-09-15T03:50:49.854 INFO:tasks.rados.rados.0.smithi122.stdout:1141: seq_num 426 ranges {759782=571842,2012035=687143} 2024-09-15T03:50:49.864 INFO:tasks.rados.rados.0.smithi122.stdout:1141: writing smithi12223906-37 from 759782 to 1331624 tid 1 2024-09-15T03:50:49.872 INFO:tasks.rados.rados.0.smithi122.stdout:1141: writing smithi12223906-37 from 2012035 to 2699178 tid 2 2024-09-15T03:50:49.872 INFO:tasks.rados.rados.0.smithi122.stdout:1142: snap_remove snap 126 2024-09-15T03:50:50.867 INFO:tasks.rados.rados.0.smithi122.stdout:1142: done (2 left) 2024-09-15T03:50:50.867 INFO:tasks.rados.rados.0.smithi122.stdout:1143: write oid 48 current snap is 127 2024-09-15T03:50:50.867 INFO:tasks.rados.rados.0.smithi122.stdout:1143: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2024-09-15T03:50:50.875 INFO:tasks.rados.rados.0.smithi122.stdout:1143: writing smithi12223906-48 from 776589 to 1423680 tid 1 2024-09-15T03:50:50.881 INFO:tasks.rados.rados.0.smithi122.stdout:1143: writing smithi12223906-48 from 2070517 to 2501318 tid 2 2024-09-15T03:50:50.885 INFO:tasks.rados.rados.0.smithi122.stdout:1143: writing smithi12223906-48 from 3026559 to 3176589 tid 3 2024-09-15T03:50:50.885 INFO:tasks.rados.rados.0.smithi122.stdout:1144: read oid 31 snap -1 2024-09-15T03:50:50.885 INFO:tasks.rados.rados.0.smithi122.stdout:1144: expect (ObjNum 412 snap 122 seq_num 412) 2024-09-15T03:50:50.886 INFO:tasks.rados.rados.0.smithi122.stdout:1145: delete oid 46 current snap is 127 2024-09-15T03:50:50.886 INFO:tasks.rados.rados.0.smithi122.stdout:1140: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:51.138 INFO:tasks.rados.rados.0.smithi122.stdout:1141: finishing write tid 1 to smithi12223906-37 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1141: finishing write tid 2 to smithi12223906-37 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1141: finishing write tid 3 to smithi12223906-37 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1141: finishing write tid 4 to smithi12223906-37 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 371 (ObjNum 426 snap 127 seq_num 426) dirty exists 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1141: left oid 37 (ObjNum 426 snap 127 seq_num 426) 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1143: finishing write tid 1 to smithi12223906-48 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1143: finishing write tid 2 to smithi12223906-48 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1143: finishing write tid 3 to smithi12223906-48 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1143: finishing write tid 4 to smithi12223906-48 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1143: finishing write tid 5 to smithi12223906-48 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 375 (ObjNum 427 snap 127 seq_num 427) dirty exists 2024-09-15T03:50:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:1143: left oid 48 (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:50:51.140 INFO:tasks.rados.rados.0.smithi122.stdout:1144: expect (ObjNum 412 snap 122 seq_num 412) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1140: done (4 left) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1141: done (3 left) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1143: done (2 left) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1144: done (1 left) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1145: done (0 left) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1146: rollback oid 33 current snap is 127 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 33 to 122 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1147: read oid 50 snap 120 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1147: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1148: write oid 47 current snap is 127 2024-09-15T03:50:51.259 INFO:tasks.rados.rados.0.smithi122.stdout:1148: seq_num 429 ranges {410203=797589,1787481=434470,2681405=528798} 2024-09-15T03:50:51.275 INFO:tasks.rados.rados.0.smithi122.stdout:1148: writing smithi12223906-47 from 410203 to 1207792 tid 1 2024-09-15T03:50:51.282 INFO:tasks.rados.rados.0.smithi122.stdout:1148: writing smithi12223906-47 from 1787481 to 2221951 tid 2 2024-09-15T03:50:51.287 INFO:tasks.rados.rados.0.smithi122.stdout:1148: writing smithi12223906-47 from 2681405 to 3210203 tid 3 2024-09-15T03:50:51.287 INFO:tasks.rados.rados.0.smithi122.stdout:1149: write (excl) oid 43 current snap is 127 2024-09-15T03:50:51.288 INFO:tasks.rados.rados.0.smithi122.stdout:1149: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2024-09-15T03:50:51.292 INFO:tasks.rados.rados.0.smithi122.stdout:1149: writing smithi12223906-43 from 427010 to 899848 tid 1 2024-09-15T03:50:51.298 INFO:tasks.rados.rados.0.smithi122.stdout:1149: writing smithi12223906-43 from 1329610 to 2024091 tid 2 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1149: writing smithi12223906-43 from 2708828 to 3227010 tid 3 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1150: read oid 12 snap 120 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1150: expect (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1151: read oid 42 snap 122 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1151: expect deleted 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1152: write (excl) oid 24 current snap is 127 2024-09-15T03:50:51.306 INFO:tasks.rados.rados.0.smithi122.stdout:1152: seq_num 431 ranges {443817=548087,1788092=438139,3019898=223919} 2024-09-15T03:50:51.314 INFO:tasks.rados.rados.0.smithi122.stdout:1152: writing smithi12223906-24 from 443817 to 991904 tid 1 2024-09-15T03:50:51.318 INFO:tasks.rados.rados.0.smithi122.stdout:1152: writing smithi12223906-24 from 1788092 to 2226231 tid 2 2024-09-15T03:50:51.324 INFO:tasks.rados.rados.0.smithi122.stdout:1152: writing smithi12223906-24 from 3019898 to 3243817 tid 3 2024-09-15T03:50:51.324 INFO:tasks.rados.rados.0.smithi122.stdout:1153: snap_remove snap 125 2024-09-15T03:50:51.804 INFO:tasks.rados.rados.0.smithi122.stdout:1153: done (7 left) 2024-09-15T03:50:51.805 INFO:tasks.rados.rados.0.smithi122.stdout:1154: write (excl) oid 17 current snap is 127 2024-09-15T03:50:51.805 INFO:tasks.rados.rados.0.smithi122.stdout:1154: seq_num 432 ranges {460624=623336,1846574=698150,3163674=96950} 2024-09-15T03:50:51.811 INFO:tasks.rados.rados.0.smithi122.stdout:1154: writing smithi12223906-17 from 460624 to 1083960 tid 1 2024-09-15T03:50:51.820 INFO:tasks.rados.rados.0.smithi122.stdout:1154: writing smithi12223906-17 from 1846574 to 2544724 tid 2 2024-09-15T03:50:51.824 INFO:tasks.rados.rados.0.smithi122.stdout:1154: writing smithi12223906-17 from 3163674 to 3260624 tid 3 2024-09-15T03:50:51.824 INFO:tasks.rados.rados.0.smithi122.stdout:1155: write oid 4 current snap is 127 2024-09-15T03:50:51.824 INFO:tasks.rados.rados.0.smithi122.stdout:1155: seq_num 433 ranges {477431=698585,1905056=441808,3074744=202687} 2024-09-15T03:50:51.830 INFO:tasks.rados.rados.0.smithi122.stdout:1155: writing smithi12223906-4 from 477431 to 1176016 tid 1 2024-09-15T03:50:51.837 INFO:tasks.rados.rados.0.smithi122.stdout:1155: writing smithi12223906-4 from 1905056 to 2346864 tid 2 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1155: writing smithi12223906-4 from 3074744 to 3277431 tid 3 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1156: read oid 21 snap 122 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1156: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1157: delete oid 25 current snap is 127 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1146: finishing rollback tid 0 to smithi12223906-33 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1146: finishing rollback tid 1 to smithi12223906-33 2024-09-15T03:50:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:1147: expect (ObjNum 201 snap 56 seq_num 201) 2024-09-15T03:50:52.091 INFO:tasks.rados.rados.0.smithi122.stdout:1146: finishing rollback tid 2 to smithi12223906-33 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 567 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1148: finishing write tid 1 to smithi12223906-47 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1148: finishing write tid 2 to smithi12223906-47 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1148: finishing write tid 3 to smithi12223906-47 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1148: finishing write tid 4 to smithi12223906-47 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1148: finishing write tid 5 to smithi12223906-47 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 199 (ObjNum 429 snap 127 seq_num 429) dirty exists 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1148: left oid 47 (ObjNum 429 snap 127 seq_num 429) 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1149: finishing write tid 1 to smithi12223906-43 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1149: finishing write tid 2 to smithi12223906-43 2024-09-15T03:50:52.092 INFO:tasks.rados.rados.0.smithi122.stdout:1146: done (10 left) 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1147: done (9 left) 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1148: done (8 left) 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1157: done (7 left) 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1158: read oid 22 snap -1 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1158: expect (ObjNum 390 snap 113 seq_num 390) 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1159: snap_create 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1149: finishing write tid 3 to smithi12223906-43 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1149: finishing write tid 4 to smithi12223906-43 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1149: finishing write tid 5 to smithi12223906-43 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 533 (ObjNum 430 snap 127 seq_num 430) dirty exists 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1149: left oid 43 (ObjNum 430 snap 127 seq_num 430) 2024-09-15T03:50:52.093 INFO:tasks.rados.rados.0.smithi122.stdout:1150: expect (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1152: finishing write tid 1 to smithi12223906-24 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1152: finishing write tid 2 to smithi12223906-24 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1152: finishing write tid 3 to smithi12223906-24 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1152: finishing write tid 4 to smithi12223906-24 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1152: finishing write tid 5 to smithi12223906-24 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 538 (ObjNum 431 snap 127 seq_num 431) dirty exists 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1152: left oid 24 (ObjNum 431 snap 127 seq_num 431) 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1154: finishing write tid 1 to smithi12223906-17 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1154: finishing write tid 2 to smithi12223906-17 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1154: finishing write tid 3 to smithi12223906-17 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1154: finishing write tid 4 to smithi12223906-17 2024-09-15T03:50:52.269 INFO:tasks.rados.rados.0.smithi122.stdout:1154: finishing write tid 5 to smithi12223906-17 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 322 (ObjNum 432 snap 127 seq_num 432) dirty exists 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1154: left oid 17 (ObjNum 432 snap 127 seq_num 432) 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1155: finishing write tid 1 to smithi12223906-4 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1155: finishing write tid 2 to smithi12223906-4 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1155: finishing write tid 3 to smithi12223906-4 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1155: finishing write tid 4 to smithi12223906-4 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1155: finishing write tid 5 to smithi12223906-4 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 210 (ObjNum 433 snap 127 seq_num 433) dirty exists 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1155: left oid 4 (ObjNum 433 snap 127 seq_num 433) 2024-09-15T03:50:52.270 INFO:tasks.rados.rados.0.smithi122.stdout:1156: expect (ObjNum 369 snap 103 seq_num 369) 2024-09-15T03:50:52.335 INFO:tasks.rados.rados.0.smithi122.stdout:1158: expect (ObjNum 390 snap 113 seq_num 390) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1149: done (8 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1150: done (7 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1151: done (6 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1152: done (5 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1154: done (4 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1155: done (3 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1156: done (2 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1158: done (1 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1159: done (0 left) 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1160: read oid 9 snap -1 2024-09-15T03:50:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1160: expect deleted 2024-09-15T03:50:52.880 INFO:tasks.rados.rados.0.smithi122.stdout:1161: write (excl) oid 21 current snap is 128 2024-09-15T03:50:52.880 INFO:tasks.rados.rados.0.smithi122.stdout:1161: seq_num 435 ranges {511045=565436,1622020=445477,2729590=556855} 2024-09-15T03:50:52.889 INFO:tasks.rados.rados.0.smithi122.stdout:1161: writing smithi12223906-21 from 511045 to 1076481 tid 1 2024-09-15T03:50:52.894 INFO:tasks.rados.rados.0.smithi122.stdout:1161: writing smithi12223906-21 from 1622020 to 2067497 tid 2 2024-09-15T03:50:52.901 INFO:tasks.rados.rados.0.smithi122.stdout:1161: writing smithi12223906-21 from 2729590 to 3286445 tid 3 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:1162: rollback oid 17 current snap is 128 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 122 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:1161: finishing write tid 1 to smithi12223906-21 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:1161: finishing write tid 2 to smithi12223906-21 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:1160: done (2 left) 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:1163: write oid 32 current snap is 128 2024-09-15T03:50:52.902 INFO:tasks.rados.rados.0.smithi122.stdout:1163: seq_num 436 ranges {527852=640685,1680502=589135,2757013=570839} 2024-09-15T03:50:52.908 INFO:tasks.rados.rados.0.smithi122.stdout:1163: writing smithi12223906-32 from 527852 to 1168537 tid 1 2024-09-15T03:50:52.915 INFO:tasks.rados.rados.0.smithi122.stdout:1163: writing smithi12223906-32 from 1680502 to 2269637 tid 2 2024-09-15T03:50:52.922 INFO:tasks.rados.rados.0.smithi122.stdout:1163: writing smithi12223906-32 from 2757013 to 3327852 tid 3 2024-09-15T03:50:52.922 INFO:tasks.rados.rados.0.smithi122.stdout:1164: read oid 47 snap -1 2024-09-15T03:50:52.922 INFO:tasks.rados.rados.0.smithi122.stdout:1164: expect (ObjNum 429 snap 127 seq_num 429) 2024-09-15T03:50:52.922 INFO:tasks.rados.rados.0.smithi122.stdout:1165: write oid 2 current snap is 128 2024-09-15T03:50:52.922 INFO:tasks.rados.rados.0.smithi122.stdout:1161: finishing write tid 3 to smithi12223906-21 2024-09-15T03:50:52.922 INFO:tasks.rados.rados.0.smithi122.stdout:1165: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2024-09-15T03:50:52.928 INFO:tasks.rados.rados.0.smithi122.stdout:1165: writing smithi12223906-2 from 544659 to 1260593 tid 1 2024-09-15T03:50:52.933 INFO:tasks.rados.rados.0.smithi122.stdout:1165: writing smithi12223906-2 from 1738984 to 2188130 tid 2 2024-09-15T03:50:52.941 INFO:tasks.rados.rados.0.smithi122.stdout:1165: writing smithi12223906-2 from 2784436 to 3344659 tid 3 2024-09-15T03:50:52.941 INFO:tasks.rados.rados.0.smithi122.stdout:1166: delete oid 19 current snap is 128 2024-09-15T03:50:52.941 INFO:tasks.rados.rados.0.smithi122.stdout:1161: finishing write tid 4 to smithi12223906-21 2024-09-15T03:50:52.941 INFO:tasks.rados.rados.0.smithi122.stdout:1161: finishing write tid 5 to smithi12223906-21 2024-09-15T03:50:52.942 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 379 (ObjNum 435 snap 128 seq_num 435) dirty exists 2024-09-15T03:50:52.942 INFO:tasks.rados.rados.0.smithi122.stdout:1161: left oid 21 (ObjNum 435 snap 128 seq_num 435) 2024-09-15T03:50:52.942 INFO:tasks.rados.rados.0.smithi122.stdout:1162: finishing rollback tid 0 to smithi12223906-17 2024-09-15T03:50:52.942 INFO:tasks.rados.rados.0.smithi122.stdout:1162: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:50:52.943 INFO:tasks.rados.rados.0.smithi122.stdout:1162: finishing rollback tid 2 to smithi12223906-17 2024-09-15T03:50:52.943 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 326 (ObjNum 371 snap 104 seq_num 371) dirty exists 2024-09-15T03:50:52.944 INFO:tasks.rados.rados.0.smithi122.stdout:1163: finishing write tid 1 to smithi12223906-32 2024-09-15T03:50:52.944 INFO:tasks.rados.rados.0.smithi122.stdout:1163: finishing write tid 2 to smithi12223906-32 2024-09-15T03:50:52.944 INFO:tasks.rados.rados.0.smithi122.stdout:1163: finishing write tid 3 to smithi12223906-32 2024-09-15T03:50:52.944 INFO:tasks.rados.rados.0.smithi122.stdout:1163: finishing write tid 4 to smithi12223906-32 2024-09-15T03:50:52.944 INFO:tasks.rados.rados.0.smithi122.stdout:1163: finishing write tid 5 to smithi12223906-32 2024-09-15T03:50:52.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 577 (ObjNum 436 snap 128 seq_num 436) dirty exists 2024-09-15T03:50:52.945 INFO:tasks.rados.rados.0.smithi122.stdout:1163: left oid 32 (ObjNum 436 snap 128 seq_num 436) 2024-09-15T03:50:52.945 INFO:tasks.rados.rados.0.smithi122.stdout:1164: expect (ObjNum 429 snap 127 seq_num 429) 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: finishing write tid 1 to smithi12223906-2 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: finishing write tid 2 to smithi12223906-2 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: finishing write tid 3 to smithi12223906-2 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: finishing write tid 4 to smithi12223906-2 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: finishing write tid 5 to smithi12223906-2 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 330 (ObjNum 437 snap 128 seq_num 437) dirty exists 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: left oid 2 (ObjNum 437 snap 128 seq_num 437) 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1161: done (5 left) 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1162: done (4 left) 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1163: done (3 left) 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1164: done (2 left) 2024-09-15T03:50:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:1165: done (1 left) 2024-09-15T03:50:53.138 INFO:tasks.rados.rados.0.smithi122.stdout:1166: done (0 left) 2024-09-15T03:50:53.138 INFO:tasks.rados.rados.0.smithi122.stdout:1167: write (excl) oid 40 current snap is 128 2024-09-15T03:50:53.138 INFO:tasks.rados.rados.0.smithi122.stdout:1167: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2024-09-15T03:50:53.151 INFO:tasks.rados.rados.0.smithi122.stdout:1167: writing smithi12223906-40 from 578273 to 1044705 tid 1 2024-09-15T03:50:53.158 INFO:tasks.rados.rados.0.smithi122.stdout:1167: writing smithi12223906-40 from 1739595 to 2192410 tid 2 2024-09-15T03:50:53.165 INFO:tasks.rados.rados.0.smithi122.stdout:1167: writing smithi12223906-40 from 2722929 to 3378273 tid 3 2024-09-15T03:50:53.165 INFO:tasks.rados.rados.0.smithi122.stdout:1168: read oid 7 snap -1 2024-09-15T03:50:53.165 INFO:tasks.rados.rados.0.smithi122.stdout:1168: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:53.165 INFO:tasks.rados.rados.0.smithi122.stdout:1169: read oid 14 snap -1 2024-09-15T03:50:53.165 INFO:tasks.rados.rados.0.smithi122.stdout:1169: expect (ObjNum 401 snap 118 seq_num 401) 2024-09-15T03:50:53.165 INFO:tasks.rados.rados.0.smithi122.stdout:1170: delete oid 49 current snap is 128 2024-09-15T03:50:53.166 INFO:tasks.rados.rados.0.smithi122.stdout:1167: finishing write tid 1 to smithi12223906-40 2024-09-15T03:50:53.166 INFO:tasks.rados.rados.0.smithi122.stdout:1167: finishing write tid 2 to smithi12223906-40 2024-09-15T03:50:53.168 INFO:tasks.rados.rados.0.smithi122.stdout:1170: done (3 left) 2024-09-15T03:50:53.168 INFO:tasks.rados.rados.0.smithi122.stdout:1171: snap_remove snap 127 2024-09-15T03:50:53.889 INFO:tasks.rados.rados.0.smithi122.stdout:1171: done (3 left) 2024-09-15T03:50:53.889 INFO:tasks.rados.rados.0.smithi122.stdout:1172: read oid 38 snap -1 2024-09-15T03:50:53.889 INFO:tasks.rados.rados.0.smithi122.stdout:1172: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:53.890 INFO:tasks.rados.rados.0.smithi122.stdout:1173: snap_remove snap 122 2024-09-15T03:50:54.902 INFO:tasks.rados.rados.0.smithi122.stdout:1173: done (4 left) 2024-09-15T03:50:54.902 INFO:tasks.rados.rados.0.smithi122.stdout:1174: write (excl) oid 4 current snap is 128 2024-09-15T03:50:54.902 INFO:tasks.rados.rados.0.smithi122.stdout:1167: finishing write tid 3 to smithi12223906-40 2024-09-15T03:50:54.902 INFO:tasks.rados.rados.0.smithi122.stdout:1174: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2024-09-15T03:50:54.910 INFO:tasks.rados.rados.0.smithi122.stdout:1174: writing smithi12223906-4 from 611887 to 1345170 tid 1 2024-09-15T03:50:54.917 INFO:tasks.rados.rados.0.smithi122.stdout:1174: writing smithi12223906-4 from 1972912 to 2429396 tid 2 2024-09-15T03:50:54.923 INFO:tasks.rados.rados.0.smithi122.stdout:1174: writing smithi12223906-4 from 2894128 to 3411887 tid 3 2024-09-15T03:50:54.923 INFO:tasks.rados.rados.0.smithi122.stdout:1175: write oid 13 current snap is 128 2024-09-15T03:50:54.923 INFO:tasks.rados.rados.0.smithi122.stdout:1175: seq_num 442 ranges {628694=408532,1515041=600142,2805198=552112} 2024-09-15T03:50:54.929 INFO:tasks.rados.rados.0.smithi122.stdout:1175: writing smithi12223906-13 from 628694 to 1037226 tid 1 2024-09-15T03:50:54.935 INFO:tasks.rados.rados.0.smithi122.stdout:1175: writing smithi12223906-13 from 1515041 to 2115183 tid 2 2024-09-15T03:50:54.942 INFO:tasks.rados.rados.0.smithi122.stdout:1175: writing smithi12223906-13 from 2805198 to 3357310 tid 3 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1176: read oid 17 snap -1 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1176: expect (ObjNum 371 snap 104 seq_num 371) 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1177: read oid 12 snap -1 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1177: expect (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1178: write (excl) oid 37 current snap is 128 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1167: finishing write tid 4 to smithi12223906-40 2024-09-15T03:50:54.943 INFO:tasks.rados.rados.0.smithi122.stdout:1178: seq_num 443 ranges {645501=483781,1573523=460153,2548974=763384} 2024-09-15T03:50:54.949 INFO:tasks.rados.rados.0.smithi122.stdout:1178: writing smithi12223906-37 from 645501 to 1129282 tid 1 2024-09-15T03:50:54.954 INFO:tasks.rados.rados.0.smithi122.stdout:1178: writing smithi12223906-37 from 1573523 to 2033676 tid 2 2024-09-15T03:50:54.961 INFO:tasks.rados.rados.0.smithi122.stdout:1178: writing smithi12223906-37 from 2548974 to 3312358 tid 3 2024-09-15T03:50:54.961 INFO:tasks.rados.rados.0.smithi122.stdout:1179: copy_from oid 5 from oid 25 current snap is 128 2024-09-15T03:50:54.961 INFO:tasks.rados.rados.0.smithi122.stdout:1167: finishing write tid 5 to smithi12223906-40 2024-09-15T03:50:54.962 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 503 (ObjNum 439 snap 128 seq_num 439) dirty exists 2024-09-15T03:50:54.962 INFO:tasks.rados.rados.0.smithi122.stdout:1167: left oid 40 (ObjNum 439 snap 128 seq_num 439) 2024-09-15T03:50:54.962 INFO:tasks.rados.rados.0.smithi122.stdout:1168: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:55.158 INFO:tasks.rados.rados.0.smithi122.stdout:1169: expect (ObjNum 401 snap 118 seq_num 401) 2024-09-15T03:50:55.361 INFO:tasks.rados.rados.0.smithi122.stdout:1172: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1174: finishing write tid 1 to smithi12223906-4 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1174: finishing write tid 2 to smithi12223906-4 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1174: finishing write tid 3 to smithi12223906-4 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1174: finishing write tid 4 to smithi12223906-4 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1174: finishing write tid 5 to smithi12223906-4 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 214 (ObjNum 441 snap 128 seq_num 441) dirty exists 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1174: left oid 4 (ObjNum 441 snap 128 seq_num 441) 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1175: finishing write tid 1 to smithi12223906-13 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1175: finishing write tid 2 to smithi12223906-13 2024-09-15T03:50:55.541 INFO:tasks.rados.rados.0.smithi122.stdout:1175: finishing write tid 3 to smithi12223906-13 2024-09-15T03:50:55.542 INFO:tasks.rados.rados.0.smithi122.stdout:1175: finishing write tid 4 to smithi12223906-13 2024-09-15T03:50:55.542 INFO:tasks.rados.rados.0.smithi122.stdout:1175: finishing write tid 5 to smithi12223906-13 2024-09-15T03:50:55.542 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 378 (ObjNum 442 snap 128 seq_num 442) dirty exists 2024-09-15T03:50:55.542 INFO:tasks.rados.rados.0.smithi122.stdout:1175: left oid 13 (ObjNum 442 snap 128 seq_num 442) 2024-09-15T03:50:55.542 INFO:tasks.rados.rados.0.smithi122.stdout:1177: expect (ObjNum 395 snap 116 seq_num 395) 2024-09-15T03:50:55.712 INFO:tasks.rados.rados.0.smithi122.stdout:1176: expect (ObjNum 371 snap 104 seq_num 371) 2024-09-15T03:50:55.779 INFO:tasks.rados.rados.0.smithi122.stdout:1178: finishing write tid 1 to smithi12223906-37 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1178: finishing write tid 2 to smithi12223906-37 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1179: finishing copy_from to smithi12223906-5 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1179: got expected ENOENT (src dne) 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1179: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1178: finishing write tid 3 to smithi12223906-37 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1178: finishing write tid 4 to smithi12223906-37 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1178: finishing write tid 5 to smithi12223906-37 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 383 (ObjNum 443 snap 128 seq_num 443) dirty exists 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1178: left oid 37 (ObjNum 443 snap 128 seq_num 443) 2024-09-15T03:50:55.780 INFO:tasks.rados.rados.0.smithi122.stdout:1167: done (9 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1168: done (8 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1169: done (7 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1172: done (6 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1174: done (5 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1175: done (4 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1176: done (3 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1177: done (2 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1178: done (1 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1179: done (0 left) 2024-09-15T03:50:55.781 INFO:tasks.rados.rados.0.smithi122.stdout:1180: snap_remove snap 120 2024-09-15T03:50:55.909 INFO:tasks.rados.rados.0.smithi122.stdout:1180: done (0 left) 2024-09-15T03:50:55.909 INFO:tasks.rados.rados.0.smithi122.stdout:1181: snap_create 2024-09-15T03:50:56.803 INFO:tasks.rados.rados.0.smithi122.stdout:1181: done (0 left) 2024-09-15T03:50:56.803 INFO:tasks.rados.rados.0.smithi122.stdout:1182: rollback oid 3 current snap is 129 2024-09-15T03:50:56.803 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 128 2024-09-15T03:50:56.803 INFO:tasks.rados.rados.0.smithi122.stdout:1183: copy_from oid 35 from oid 18 current snap is 129 2024-09-15T03:50:56.803 INFO:tasks.rados.rados.0.smithi122.stdout:1184: delete oid 12 current snap is 129 2024-09-15T03:50:56.807 INFO:tasks.rados.rados.0.smithi122.stdout:1184: done (2 left) 2024-09-15T03:50:56.808 INFO:tasks.rados.rados.0.smithi122.stdout:1185: copy_from oid 22 from oid 39 current snap is 129 2024-09-15T03:50:56.808 INFO:tasks.rados.rados.0.smithi122.stdout:1186: write oid 37 current snap is 129 2024-09-15T03:50:56.808 INFO:tasks.rados.rados.0.smithi122.stdout:1186: seq_num 445 ranges {679115=634279,2090487=463822,3003820=475295} 2024-09-15T03:50:56.817 INFO:tasks.rados.rados.0.smithi122.stdout:1186: writing smithi12223906-37 from 679115 to 1313394 tid 1 2024-09-15T03:50:56.824 INFO:tasks.rados.rados.0.smithi122.stdout:1186: writing smithi12223906-37 from 2090487 to 2554309 tid 2 2024-09-15T03:50:56.830 INFO:tasks.rados.rados.0.smithi122.stdout:1186: writing smithi12223906-37 from 3003820 to 3479115 tid 3 2024-09-15T03:50:56.830 INFO:tasks.rados.rados.0.smithi122.stdout:1187: read oid 33 snap -1 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1187: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1188: rollback oid 36 current snap is 129 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 36 to 128 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1185: finishing copy_from to smithi12223906-22 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1185: got expected ENOENT (src dne) 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1185: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1183: finishing copy_from racing read to smithi12223906-35 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1183: finishing copy_from to smithi12223906-35 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 551 (ObjNum 417 snap 126 seq_num 417) dirty exists 2024-09-15T03:50:56.831 INFO:tasks.rados.rados.0.smithi122.stdout:1182: finishing rollback tid 0 to smithi12223906-3 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1182: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1182: finishing rollback tid 2 to smithi12223906-3 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 555 (ObjNum 389 snap 113 seq_num 389) dirty exists 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1186: finishing write tid 1 to smithi12223906-37 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1186: finishing write tid 2 to smithi12223906-37 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1182: done (5 left) 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1183: done (4 left) 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1185: done (3 left) 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1189: read oid 5 snap -1 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1189: expect (ObjNum 409 snap 120 seq_num 409) 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1190: write (excl) oid 41 current snap is 129 2024-09-15T03:50:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1190: seq_num 446 ranges {695922=709528,2032616=607480,3198537=297385} 2024-09-15T03:50:56.837 INFO:tasks.rados.rados.0.smithi122.stdout:1190: writing smithi12223906-41 from 695922 to 1405450 tid 1 2024-09-15T03:50:56.844 INFO:tasks.rados.rados.0.smithi122.stdout:1190: writing smithi12223906-41 from 2032616 to 2640096 tid 2 2024-09-15T03:50:56.850 INFO:tasks.rados.rados.0.smithi122.stdout:1190: writing smithi12223906-41 from 3198537 to 3495922 tid 3 2024-09-15T03:50:56.850 INFO:tasks.rados.rados.0.smithi122.stdout:1191: write oid 50 current snap is 129 2024-09-15T03:50:56.850 INFO:tasks.rados.rados.0.smithi122.stdout:1186: finishing write tid 3 to smithi12223906-37 2024-09-15T03:50:56.850 INFO:tasks.rados.rados.0.smithi122.stdout:1191: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2024-09-15T03:50:56.858 INFO:tasks.rados.rados.0.smithi122.stdout:1191: writing smithi12223906-50 from 712729 to 1497506 tid 1 2024-09-15T03:50:56.864 INFO:tasks.rados.rados.0.smithi122.stdout:1191: writing smithi12223906-50 from 2091098 to 2558589 tid 2 2024-09-15T03:50:56.869 INFO:tasks.rados.rados.0.smithi122.stdout:1191: writing smithi12223906-50 from 3342313 to 3512729 tid 3 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:1192: snap_create 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:1186: finishing write tid 4 to smithi12223906-37 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:1186: finishing write tid 5 to smithi12223906-37 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 387 (ObjNum 445 snap 129 seq_num 445) dirty exists 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:1186: left oid 37 (ObjNum 445 snap 129 seq_num 445) 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:1188: finishing rollback tid 0 to smithi12223906-36 2024-09-15T03:50:56.870 INFO:tasks.rados.rados.0.smithi122.stdout:1189: expect (ObjNum 409 snap 120 seq_num 409) 2024-09-15T03:50:56.970 INFO:tasks.rados.rados.0.smithi122.stdout:1188: finishing rollback tid 1 to smithi12223906-36 2024-09-15T03:50:56.970 INFO:tasks.rados.rados.0.smithi122.stdout:1188: finishing rollback tid 2 to smithi12223906-36 2024-09-15T03:50:56.970 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 559 (ObjNum 397 snap 117 seq_num 397) dirty exists 2024-09-15T03:50:56.970 INFO:tasks.rados.rados.0.smithi122.stdout:1190: finishing write tid 1 to smithi12223906-41 2024-09-15T03:50:56.970 INFO:tasks.rados.rados.0.smithi122.stdout:1190: finishing write tid 2 to smithi12223906-41 2024-09-15T03:50:56.970 INFO:tasks.rados.rados.0.smithi122.stdout:1187: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:50:57.149 INFO:tasks.rados.rados.0.smithi122.stdout:1190: finishing write tid 3 to smithi12223906-41 2024-09-15T03:50:57.149 INFO:tasks.rados.rados.0.smithi122.stdout:1190: finishing write tid 4 to smithi12223906-41 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1190: finishing write tid 5 to smithi12223906-41 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 507 (ObjNum 446 snap 129 seq_num 446) dirty exists 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1190: left oid 41 (ObjNum 446 snap 129 seq_num 446) 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1191: finishing write tid 1 to smithi12223906-50 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1191: finishing write tid 2 to smithi12223906-50 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1191: finishing write tid 3 to smithi12223906-50 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1191: finishing write tid 4 to smithi12223906-50 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1191: finishing write tid 5 to smithi12223906-50 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 598 (ObjNum 447 snap 129 seq_num 447) dirty exists 2024-09-15T03:50:57.150 INFO:tasks.rados.rados.0.smithi122.stdout:1191: left oid 50 (ObjNum 447 snap 129 seq_num 447) 2024-09-15T03:50:57.815 INFO:tasks.rados.rados.0.smithi122.stdout:1186: done (6 left) 2024-09-15T03:50:57.815 INFO:tasks.rados.rados.0.smithi122.stdout:1187: done (5 left) 2024-09-15T03:50:57.815 INFO:tasks.rados.rados.0.smithi122.stdout:1188: done (4 left) 2024-09-15T03:50:57.815 INFO:tasks.rados.rados.0.smithi122.stdout:1189: done (3 left) 2024-09-15T03:50:57.816 INFO:tasks.rados.rados.0.smithi122.stdout:1190: done (2 left) 2024-09-15T03:50:57.816 INFO:tasks.rados.rados.0.smithi122.stdout:1191: done (1 left) 2024-09-15T03:50:57.816 INFO:tasks.rados.rados.0.smithi122.stdout:1192: done (0 left) 2024-09-15T03:50:57.816 INFO:tasks.rados.rados.0.smithi122.stdout:1193: delete oid 28 current snap is 130 2024-09-15T03:50:57.819 INFO:tasks.rados.rados.0.smithi122.stdout:1193: done (0 left) 2024-09-15T03:50:57.819 INFO:tasks.rados.rados.0.smithi122.stdout:1194: delete oid 5 current snap is 130 2024-09-15T03:50:57.823 INFO:tasks.rados.rados.0.smithi122.stdout:1194: done (0 left) 2024-09-15T03:50:57.823 INFO:tasks.rados.rados.0.smithi122.stdout:1195: write (excl) oid 4 current snap is 130 2024-09-15T03:50:57.823 INFO:tasks.rados.rados.0.smithi122.stdout:1195: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2024-09-15T03:50:57.831 INFO:tasks.rados.rados.0.smithi122.stdout:1195: writing smithi12223906-4 from 763150 to 1490027 tid 1 2024-09-15T03:50:57.838 INFO:tasks.rados.rados.0.smithi122.stdout:1195: writing smithi12223906-4 from 1982897 to 2597715 tid 2 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1195: writing smithi12223906-4 from 3024582 to 3499576 tid 3 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1196: read oid 49 snap -1 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1196: expect deleted 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1197: write oid 42 current snap is 130 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1195: finishing write tid 1 to smithi12223906-4 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1195: finishing write tid 2 to smithi12223906-4 2024-09-15T03:50:57.844 INFO:tasks.rados.rados.0.smithi122.stdout:1197: seq_num 451 ranges {779957=402126,1925026=758476,3335652=244305} 2024-09-15T03:50:57.850 INFO:tasks.rados.rados.0.smithi122.stdout:1197: writing smithi12223906-42 from 779957 to 1182083 tid 1 2024-09-15T03:50:57.858 INFO:tasks.rados.rados.0.smithi122.stdout:1197: writing smithi12223906-42 from 1925026 to 2683502 tid 2 2024-09-15T03:50:57.863 INFO:tasks.rados.rados.0.smithi122.stdout:1197: writing smithi12223906-42 from 3335652 to 3579957 tid 3 2024-09-15T03:50:57.863 INFO:tasks.rados.rados.0.smithi122.stdout:1198: write oid 26 current snap is 130 2024-09-15T03:50:57.864 INFO:tasks.rados.rados.0.smithi122.stdout:1198: seq_num 452 ranges {796764=477375,1983508=618487,3363075=233689} 2024-09-15T03:50:57.870 INFO:tasks.rados.rados.0.smithi122.stdout:1198: writing smithi12223906-26 from 796764 to 1274139 tid 1 2024-09-15T03:50:57.877 INFO:tasks.rados.rados.0.smithi122.stdout:1198: writing smithi12223906-26 from 1983508 to 2601995 tid 2 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1198: writing smithi12223906-26 from 3363075 to 3596764 tid 3 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1199: snap_create 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1195: finishing write tid 3 to smithi12223906-4 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1195: finishing write tid 4 to smithi12223906-4 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1195: finishing write tid 5 to smithi12223906-4 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 218 (ObjNum 450 snap 130 seq_num 450) dirty exists 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1195: left oid 4 (ObjNum 450 snap 130 seq_num 450) 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1197: finishing write tid 1 to smithi12223906-42 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1197: finishing write tid 2 to smithi12223906-42 2024-09-15T03:50:57.884 INFO:tasks.rados.rados.0.smithi122.stdout:1197: finishing write tid 3 to smithi12223906-42 2024-09-15T03:50:57.885 INFO:tasks.rados.rados.0.smithi122.stdout:1197: finishing write tid 4 to smithi12223906-42 2024-09-15T03:50:57.885 INFO:tasks.rados.rados.0.smithi122.stdout:1197: finishing write tid 5 to smithi12223906-42 2024-09-15T03:50:57.885 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 563 (ObjNum 451 snap 130 seq_num 451) dirty exists 2024-09-15T03:50:57.885 INFO:tasks.rados.rados.0.smithi122.stdout:1197: left oid 42 (ObjNum 451 snap 130 seq_num 451) 2024-09-15T03:50:57.885 INFO:tasks.rados.rados.0.smithi122.stdout:1198: finishing write tid 1 to smithi12223906-26 2024-09-15T03:50:57.885 INFO:tasks.rados.rados.0.smithi122.stdout:1198: finishing write tid 2 to smithi12223906-26 2024-09-15T03:50:57.888 INFO:tasks.rados.rados.0.smithi122.stdout:1198: finishing write tid 3 to smithi12223906-26 2024-09-15T03:50:57.889 INFO:tasks.rados.rados.0.smithi122.stdout:1198: finishing write tid 4 to smithi12223906-26 2024-09-15T03:50:57.890 INFO:tasks.rados.rados.0.smithi122.stdout:1198: finishing write tid 5 to smithi12223906-26 2024-09-15T03:50:57.890 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 364 (ObjNum 452 snap 130 seq_num 452) dirty exists 2024-09-15T03:50:57.890 INFO:tasks.rados.rados.0.smithi122.stdout:1198: left oid 26 (ObjNum 452 snap 130 seq_num 452) 2024-09-15T03:50:58.818 INFO:tasks.rados.rados.0.smithi122.stdout:1195: done (4 left) 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1196: done (3 left) 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1197: done (2 left) 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1198: done (1 left) 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1199: done (0 left) 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1200: rollback oid 5 current snap is 131 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 5 to 130 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1201: read oid 46 snap 128 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1201: expect deleted 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1202: rollback oid 20 current snap is 131 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 129 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1203: write (excl) oid 2 current snap is 131 2024-09-15T03:50:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:1203: seq_num 453 ranges {413571=552624,1641990=762145,2990498=425163} 2024-09-15T03:50:58.826 INFO:tasks.rados.rados.0.smithi122.stdout:1203: writing smithi12223906-2 from 413571 to 966195 tid 1 2024-09-15T03:50:58.835 INFO:tasks.rados.rados.0.smithi122.stdout:1203: writing smithi12223906-2 from 1641990 to 2404135 tid 2 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1203: writing smithi12223906-2 from 2990498 to 3415661 tid 3 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1204: read oid 45 snap -1 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1204: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1205: copy_from oid 27 from oid 6 current snap is 131 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1200: finishing rollback tid 1 to smithi12223906-5 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 200 (ObjNum 1457507296 snap 32654 seq_num 304) dirty dne 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1202: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1202: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1202: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 96 (ObjNum 201 snap 56 seq_num 201) dirty exists 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1203: finishing write tid 1 to smithi12223906-2 2024-09-15T03:50:58.842 INFO:tasks.rados.rados.0.smithi122.stdout:1203: finishing write tid 2 to smithi12223906-2 2024-09-15T03:50:58.843 INFO:tasks.rados.rados.0.smithi122.stdout:1200: done (5 left) 2024-09-15T03:50:58.843 INFO:tasks.rados.rados.0.smithi122.stdout:1201: done (4 left) 2024-09-15T03:50:58.843 INFO:tasks.rados.rados.0.smithi122.stdout:1202: done (3 left) 2024-09-15T03:50:58.843 INFO:tasks.rados.rados.0.smithi122.stdout:1206: snap_create 2024-09-15T03:50:58.847 INFO:tasks.rados.rados.0.smithi122.stdout:1203: finishing write tid 3 to smithi12223906-2 2024-09-15T03:50:58.847 INFO:tasks.rados.rados.0.smithi122.stdout:1203: finishing write tid 4 to smithi12223906-2 2024-09-15T03:50:58.847 INFO:tasks.rados.rados.0.smithi122.stdout:1203: finishing write tid 5 to smithi12223906-2 2024-09-15T03:50:58.847 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 342 (ObjNum 453 snap 131 seq_num 453) dirty exists 2024-09-15T03:50:58.847 INFO:tasks.rados.rados.0.smithi122.stdout:1203: left oid 2 (ObjNum 453 snap 131 seq_num 453) 2024-09-15T03:50:58.851 INFO:tasks.rados.rados.0.smithi122.stdout:1204: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:50:58.967 INFO:tasks.rados.rados.0.smithi122.stdout:1205: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:50:58.968 INFO:tasks.rados.rados.0.smithi122.stdout:1205: finishing copy_from to smithi12223906-27 2024-09-15T03:50:58.968 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 297 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:50:59.839 INFO:tasks.rados.rados.0.smithi122.stdout:1203: done (3 left) 2024-09-15T03:50:59.839 INFO:tasks.rados.rados.0.smithi122.stdout:1204: done (2 left) 2024-09-15T03:50:59.840 INFO:tasks.rados.rados.0.smithi122.stdout:1205: done (1 left) 2024-09-15T03:50:59.840 INFO:tasks.rados.rados.0.smithi122.stdout:1206: done (0 left) 2024-09-15T03:50:59.840 INFO:tasks.rados.rados.0.smithi122.stdout:1207: copy_from oid 3 from oid 2 current snap is 132 2024-09-15T03:50:59.840 INFO:tasks.rados.rados.0.smithi122.stdout:1208: delete oid 22 current snap is 132 2024-09-15T03:50:59.845 INFO:tasks.rados.rados.0.smithi122.stdout:1208: done (1 left) 2024-09-15T03:50:59.845 INFO:tasks.rados.rados.0.smithi122.stdout:1209: snap_remove snap 129 2024-09-15T03:51:00.848 INFO:tasks.rados.rados.0.smithi122.stdout:1209: done (1 left) 2024-09-15T03:51:00.848 INFO:tasks.rados.rados.0.smithi122.stdout:1210: rollback oid 9 current snap is 132 2024-09-15T03:51:00.848 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 128 2024-09-15T03:51:00.848 INFO:tasks.rados.rados.0.smithi122.stdout:1207: finishing copy_from racing read to smithi12223906-3 2024-09-15T03:51:00.849 INFO:tasks.rados.rados.0.smithi122.stdout:1211: copy_from oid 10 from oid 49 current snap is 132 2024-09-15T03:51:00.849 INFO:tasks.rados.rados.0.smithi122.stdout:1212: snap_remove snap 131 2024-09-15T03:51:01.808 INFO:tasks.rados.rados.0.smithi122.stdout:1212: done (3 left) 2024-09-15T03:51:01.880 INFO:tasks.rados.rados.0.smithi122.stdout:1213: snap_remove snap 130 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:1213: done (3 left) 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:1214: rollback oid 15 current snap is 132 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 128 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:1207: finishing copy_from to smithi12223906-3 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 342 (ObjNum 453 snap 131 seq_num 453) dirty exists 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:1210: finishing rollback tid 1 to smithi12223906-9 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 0 (ObjNum 177 snap 0 seq_num 1842591728) dirty dne 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:1211: finishing copy_from to smithi12223906-10 2024-09-15T03:51:02.808 INFO:tasks.rados.rados.0.smithi122.stdout:1211: got expected ENOENT (src dne) 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:1211: finishing copy_from racing read to smithi12223906-10 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:1207: done (3 left) 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:1210: done (2 left) 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:1211: done (1 left) 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:1215: rollback oid 45 current snap is 132 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 128 2024-09-15T03:51:02.809 INFO:tasks.rados.rados.0.smithi122.stdout:1216: snap_create 2024-09-15T03:51:02.813 INFO:tasks.rados.rados.0.smithi122.stdout:1214: finishing rollback tid 0 to smithi12223906-15 2024-09-15T03:51:02.815 INFO:tasks.rados.rados.0.smithi122.stdout:1215: finishing rollback tid 0 to smithi12223906-45 2024-09-15T03:51:02.816 INFO:tasks.rados.rados.0.smithi122.stdout:1215: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:51:02.816 INFO:tasks.rados.rados.0.smithi122.stdout:1215: finishing rollback tid 2 to smithi12223906-45 2024-09-15T03:51:02.816 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 381 (ObjNum 394 snap 116 seq_num 394) dirty exists 2024-09-15T03:51:02.816 INFO:tasks.rados.rados.0.smithi122.stdout:1214: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:51:02.816 INFO:tasks.rados.rados.0.smithi122.stdout:1214: finishing rollback tid 2 to smithi12223906-15 2024-09-15T03:51:02.816 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 374 (ObjNum 422 snap 126 seq_num 422) dirty exists 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1214: done (2 left) 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1215: done (1 left) 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1216: done (0 left) 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1217: rollback oid 36 current snap is 133 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 36 to 128 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1218: write oid 14 current snap is 133 2024-09-15T03:51:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1218: seq_num 455 ranges {447185=703122,1642601=765814,2928991=447707} 2024-09-15T03:51:03.818 INFO:tasks.rados.rados.0.smithi122.stdout:1218: writing smithi12223906-14 from 447185 to 1150307 tid 1 2024-09-15T03:51:03.825 INFO:tasks.rados.rados.0.smithi122.stdout:1218: writing smithi12223906-14 from 1642601 to 2408415 tid 2 2024-09-15T03:51:03.830 INFO:tasks.rados.rados.0.smithi122.stdout:1218: writing smithi12223906-14 from 2928991 to 3376698 tid 3 2024-09-15T03:51:03.830 INFO:tasks.rados.rados.0.smithi122.stdout:1219: write oid 38 current snap is 133 2024-09-15T03:51:03.830 INFO:tasks.rados.rados.0.smithi122.stdout:1217: finishing rollback tid 0 to smithi12223906-36 2024-09-15T03:51:03.831 INFO:tasks.rados.rados.0.smithi122.stdout:1219: seq_num 456 ranges {463992=778371,1701083=625825,2956414=658979} 2024-09-15T03:51:03.837 INFO:tasks.rados.rados.0.smithi122.stdout:1219: writing smithi12223906-38 from 463992 to 1242363 tid 1 2024-09-15T03:51:03.844 INFO:tasks.rados.rados.0.smithi122.stdout:1219: writing smithi12223906-38 from 1701083 to 2326908 tid 2 2024-09-15T03:51:03.852 INFO:tasks.rados.rados.0.smithi122.stdout:1219: writing smithi12223906-38 from 2956414 to 3615393 tid 3 2024-09-15T03:51:03.852 INFO:tasks.rados.rados.0.smithi122.stdout:1220: read oid 11 snap 128 2024-09-15T03:51:03.852 INFO:tasks.rados.rados.0.smithi122.stdout:1220: expect (ObjNum 402 snap 118 seq_num 402) 2024-09-15T03:51:03.852 INFO:tasks.rados.rados.0.smithi122.stdout:1221: snap_remove snap 132 2024-09-15T03:51:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:1221: done (4 left) 2024-09-15T03:51:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:1222: rollback oid 19 current snap is 133 2024-09-15T03:51:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 128 2024-09-15T03:51:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:1217: finishing rollback tid 1 to smithi12223906-36 2024-09-15T03:51:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:1217: finishing rollback tid 2 to smithi12223906-36 2024-09-15T03:51:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 575 (ObjNum 397 snap 117 seq_num 397) dirty exists 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1218: finishing write tid 1 to smithi12223906-14 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1218: finishing write tid 2 to smithi12223906-14 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1218: finishing write tid 3 to smithi12223906-14 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1218: finishing write tid 4 to smithi12223906-14 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1218: finishing write tid 5 to smithi12223906-14 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 580 (ObjNum 455 snap 133 seq_num 455) dirty exists 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1218: left oid 14 (ObjNum 455 snap 133 seq_num 455) 2024-09-15T03:51:04.866 INFO:tasks.rados.rados.0.smithi122.stdout:1220: expect (ObjNum 402 snap 118 seq_num 402) 2024-09-15T03:51:04.951 INFO:tasks.rados.rados.0.smithi122.stdout:1219: finishing write tid 1 to smithi12223906-38 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1219: finishing write tid 2 to smithi12223906-38 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1219: finishing write tid 3 to smithi12223906-38 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1219: finishing write tid 4 to smithi12223906-38 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1219: finishing write tid 5 to smithi12223906-38 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 368 (ObjNum 456 snap 133 seq_num 456) dirty exists 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1219: left oid 38 (ObjNum 456 snap 133 seq_num 456) 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1217: done (4 left) 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1218: done (3 left) 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1219: done (2 left) 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1220: done (1 left) 2024-09-15T03:51:04.952 INFO:tasks.rados.rados.0.smithi122.stdout:1223: write (excl) oid 13 current snap is 133 2024-09-15T03:51:04.953 INFO:tasks.rados.rados.0.smithi122.stdout:1222: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:51:04.953 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 0 (ObjNum 305 snap 0 seq_num 1842990304) dirty dne 2024-09-15T03:51:04.953 INFO:tasks.rados.rados.0.smithi122.stdout:1223: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2024-09-15T03:51:04.957 INFO:tasks.rados.rados.0.smithi122.stdout:1223: writing smithi12223906-13 from 480799 to 1050772 tid 1 2024-09-15T03:51:04.963 INFO:tasks.rados.rados.0.smithi122.stdout:1223: writing smithi12223906-13 from 1475918 to 2245401 tid 2 2024-09-15T03:51:04.970 INFO:tasks.rados.rados.0.smithi122.stdout:1223: writing smithi12223906-13 from 2700190 to 3286794 tid 3 2024-09-15T03:51:04.970 INFO:tasks.rados.rados.0.smithi122.stdout:1222: done (1 left) 2024-09-15T03:51:04.970 INFO:tasks.rados.rados.0.smithi122.stdout:1224: read oid 17 snap -1 2024-09-15T03:51:04.970 INFO:tasks.rados.rados.0.smithi122.stdout:1224: expect (ObjNum 371 snap 104 seq_num 371) 2024-09-15T03:51:04.970 INFO:tasks.rados.rados.0.smithi122.stdout:1225: snap_create 2024-09-15T03:51:04.975 INFO:tasks.rados.rados.0.smithi122.stdout:1223: finishing write tid 1 to smithi12223906-13 2024-09-15T03:51:04.975 INFO:tasks.rados.rados.0.smithi122.stdout:1223: finishing write tid 2 to smithi12223906-13 2024-09-15T03:51:04.975 INFO:tasks.rados.rados.0.smithi122.stdout:1223: finishing write tid 3 to smithi12223906-13 2024-09-15T03:51:04.975 INFO:tasks.rados.rados.0.smithi122.stdout:1223: finishing write tid 4 to smithi12223906-13 2024-09-15T03:51:04.976 INFO:tasks.rados.rados.0.smithi122.stdout:1223: finishing write tid 5 to smithi12223906-13 2024-09-15T03:51:04.976 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 385 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:51:04.976 INFO:tasks.rados.rados.0.smithi122.stdout:1223: left oid 13 (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:51:04.976 INFO:tasks.rados.rados.0.smithi122.stdout:1224: expect (ObjNum 371 snap 104 seq_num 371) 2024-09-15T03:51:05.872 INFO:tasks.rados.rados.0.smithi122.stdout:1223: done (2 left) 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1224: done (1 left) 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1225: done (0 left) 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1226: read oid 5 snap 133 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1226: expect deleted 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1227: read oid 25 snap -1 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1227: expect deleted 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1228: read oid 27 snap -1 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1228: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:05.873 INFO:tasks.rados.rados.0.smithi122.stdout:1229: snap_remove snap 128 2024-09-15T03:51:06.810 INFO:tasks.rados.rados.0.smithi122.stdout:1229: done (3 left) 2024-09-15T03:51:06.810 INFO:tasks.rados.rados.0.smithi122.stdout:1230: write oid 17 current snap is 134 2024-09-15T03:51:06.810 INFO:tasks.rados.rados.0.smithi122.stdout:1230: seq_num 458 ranges {497606=645222,1934400=629494,3243966=453640} 2024-09-15T03:51:06.819 INFO:tasks.rados.rados.0.smithi122.stdout:1230: writing smithi12223906-17 from 497606 to 1142828 tid 1 2024-09-15T03:51:06.828 INFO:tasks.rados.rados.0.smithi122.stdout:1230: writing smithi12223906-17 from 1934400 to 2563894 tid 2 2024-09-15T03:51:06.834 INFO:tasks.rados.rados.0.smithi122.stdout:1230: writing smithi12223906-17 from 3243966 to 3697606 tid 3 2024-09-15T03:51:06.834 INFO:tasks.rados.rados.0.smithi122.stdout:1231: snap_create 2024-09-15T03:51:06.835 INFO:tasks.rados.rados.0.smithi122.stdout:1228: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:1230: finishing write tid 1 to smithi12223906-17 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:1230: finishing write tid 2 to smithi12223906-17 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:1230: finishing write tid 3 to smithi12223906-17 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:1230: finishing write tid 4 to smithi12223906-17 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:1230: finishing write tid 5 to smithi12223906-17 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 353 (ObjNum 458 snap 134 seq_num 458) dirty exists 2024-09-15T03:51:07.019 INFO:tasks.rados.rados.0.smithi122.stdout:1230: left oid 17 (ObjNum 458 snap 134 seq_num 458) 2024-09-15T03:51:07.824 INFO:tasks.rados.rados.0.smithi122.stdout:1226: done (4 left) 2024-09-15T03:51:07.824 INFO:tasks.rados.rados.0.smithi122.stdout:1227: done (3 left) 2024-09-15T03:51:07.824 INFO:tasks.rados.rados.0.smithi122.stdout:1228: done (2 left) 2024-09-15T03:51:07.824 INFO:tasks.rados.rados.0.smithi122.stdout:1230: done (1 left) 2024-09-15T03:51:07.824 INFO:tasks.rados.rados.0.smithi122.stdout:1231: done (0 left) 2024-09-15T03:51:07.825 INFO:tasks.rados.rados.0.smithi122.stdout:1232: read oid 11 snap -1 2024-09-15T03:51:07.825 INFO:tasks.rados.rados.0.smithi122.stdout:1232: expect (ObjNum 402 snap 118 seq_num 402) 2024-09-15T03:51:07.825 INFO:tasks.rados.rados.0.smithi122.stdout:1233: snap_create 2024-09-15T03:51:07.827 INFO:tasks.rados.rados.0.smithi122.stdout:1232: expect (ObjNum 402 snap 118 seq_num 402) 2024-09-15T03:51:08.826 INFO:tasks.rados.rados.0.smithi122.stdout:1232: done (1 left) 2024-09-15T03:51:08.826 INFO:tasks.rados.rados.0.smithi122.stdout:1233: done (0 left) 2024-09-15T03:51:08.826 INFO:tasks.rados.rados.0.smithi122.stdout:1234: read oid 25 snap -1 2024-09-15T03:51:08.826 INFO:tasks.rados.rados.0.smithi122.stdout:1234: expect deleted 2024-09-15T03:51:08.827 INFO:tasks.rados.rados.0.smithi122.stdout:1235: rollback oid 5 current snap is 136 2024-09-15T03:51:08.827 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 5 to 134 2024-09-15T03:51:08.827 INFO:tasks.rados.rados.0.smithi122.stdout:1236: write oid 36 current snap is 136 2024-09-15T03:51:08.827 INFO:tasks.rados.rados.0.smithi122.stdout:1236: seq_num 459 ranges {514413=720471,1876529=773152,3438683=275730} 2024-09-15T03:51:08.835 INFO:tasks.rados.rados.0.smithi122.stdout:1236: writing smithi12223906-36 from 514413 to 1234884 tid 1 2024-09-15T03:51:08.843 INFO:tasks.rados.rados.0.smithi122.stdout:1236: writing smithi12223906-36 from 1876529 to 2649681 tid 2 2024-09-15T03:51:08.849 INFO:tasks.rados.rados.0.smithi122.stdout:1236: writing smithi12223906-36 from 3438683 to 3714413 tid 3 2024-09-15T03:51:08.849 INFO:tasks.rados.rados.0.smithi122.stdout:1237: read oid 39 snap -1 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1237: expect deleted 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1238: copy_from oid 28 from oid 14 current snap is 136 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1235: finishing rollback tid 1 to smithi12223906-5 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 0 (ObjNum 49 snap 0 seq_num 5) dirty dne 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1236: finishing write tid 1 to smithi12223906-36 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1236: finishing write tid 2 to smithi12223906-36 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1234: done (4 left) 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1235: done (3 left) 2024-09-15T03:51:08.850 INFO:tasks.rados.rados.0.smithi122.stdout:1239: snap_remove snap 133 2024-09-15T03:51:09.848 INFO:tasks.rados.rados.0.smithi122.stdout:1239: done (3 left) 2024-09-15T03:51:09.848 INFO:tasks.rados.rados.0.smithi122.stdout:1240: read oid 16 snap -1 2024-09-15T03:51:09.849 INFO:tasks.rados.rados.0.smithi122.stdout:1240: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:09.849 INFO:tasks.rados.rados.0.smithi122.stdout:1241: snap_remove snap 135 2024-09-15T03:51:10.862 INFO:tasks.rados.rados.0.smithi122.stdout:1241: done (4 left) 2024-09-15T03:51:10.862 INFO:tasks.rados.rados.0.smithi122.stdout:1242: write (excl) oid 8 current snap is 136 2024-09-15T03:51:10.862 INFO:tasks.rados.rados.0.smithi122.stdout:1242: seq_num 460 ranges {531220=795720,1935011=516810,3066106=420420} 2024-09-15T03:51:10.869 INFO:tasks.rados.rados.0.smithi122.stdout:1242: writing smithi12223906-8 from 531220 to 1326940 tid 1 2024-09-15T03:51:10.875 INFO:tasks.rados.rados.0.smithi122.stdout:1242: writing smithi12223906-8 from 1935011 to 2451821 tid 2 2024-09-15T03:51:10.882 INFO:tasks.rados.rados.0.smithi122.stdout:1242: writing smithi12223906-8 from 3066106 to 3486526 tid 3 2024-09-15T03:51:10.882 INFO:tasks.rados.rados.0.smithi122.stdout:1243: write (excl) oid 22 current snap is 136 2024-09-15T03:51:10.882 INFO:tasks.rados.rados.0.smithi122.stdout:1243: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2024-09-15T03:51:10.887 INFO:tasks.rados.rados.0.smithi122.stdout:1243: writing smithi12223906-22 from 548027 to 1018996 tid 1 2024-09-15T03:51:10.895 INFO:tasks.rados.rados.0.smithi122.stdout:1243: writing smithi12223906-22 from 1593493 to 2370314 tid 2 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:1243: writing smithi12223906-22 from 3093529 to 3725221 tid 3 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:1244: rollback oid 49 current snap is 136 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 134 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:1236: finishing write tid 3 to smithi12223906-36 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:1245: read oid 40 snap -1 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:1245: expect (ObjNum 439 snap 128 seq_num 439) 2024-09-15T03:51:10.903 INFO:tasks.rados.rados.0.smithi122.stdout:1246: write oid 46 current snap is 136 2024-09-15T03:51:10.904 INFO:tasks.rados.rados.0.smithi122.stdout:1246: seq_num 462 ranges {564834=546218,1651975=520479,2720952=559317} 2024-09-15T03:51:10.909 INFO:tasks.rados.rados.0.smithi122.stdout:1246: writing smithi12223906-46 from 564834 to 1111052 tid 1 2024-09-15T03:51:10.915 INFO:tasks.rados.rados.0.smithi122.stdout:1246: writing smithi12223906-46 from 1651975 to 2172454 tid 2 2024-09-15T03:51:10.922 INFO:tasks.rados.rados.0.smithi122.stdout:1246: writing smithi12223906-46 from 2720952 to 3280269 tid 3 2024-09-15T03:51:10.922 INFO:tasks.rados.rados.0.smithi122.stdout:1247: write (excl) oid 1 current snap is 136 2024-09-15T03:51:10.922 INFO:tasks.rados.rados.0.smithi122.stdout:1247: seq_num 463 ranges {581641=621467,1994104=780490,3432022=349619} 2024-09-15T03:51:10.928 INFO:tasks.rados.rados.0.smithi122.stdout:1247: writing smithi12223906-1 from 581641 to 1203108 tid 1 2024-09-15T03:51:10.937 INFO:tasks.rados.rados.0.smithi122.stdout:1247: writing smithi12223906-1 from 1994104 to 2774594 tid 2 2024-09-15T03:51:10.943 INFO:tasks.rados.rados.0.smithi122.stdout:1247: writing smithi12223906-1 from 3432022 to 3781641 tid 3 2024-09-15T03:51:10.943 INFO:tasks.rados.rados.0.smithi122.stdout:1248: read oid 17 snap -1 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1248: expect (ObjNum 458 snap 134 seq_num 458) 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1249: copy_from oid 32 from oid 3 current snap is 136 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1236: finishing write tid 4 to smithi12223906-36 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1236: finishing write tid 5 to smithi12223906-36 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 595 (ObjNum 459 snap 136 seq_num 459) dirty exists 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1236: left oid 36 (ObjNum 459 snap 136 seq_num 459) 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1236: done (11 left) 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1250: rollback oid 25 current snap is 136 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 134 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1238: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1238: finishing copy_from to smithi12223906-28 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 580 (ObjNum 455 snap 133 seq_num 455) dirty exists 2024-09-15T03:51:10.944 INFO:tasks.rados.rados.0.smithi122.stdout:1240: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:1242: finishing write tid 1 to smithi12223906-8 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:1242: finishing write tid 2 to smithi12223906-8 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:1242: finishing write tid 3 to smithi12223906-8 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:1242: finishing write tid 4 to smithi12223906-8 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:1242: finishing write tid 5 to smithi12223906-8 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 372 (ObjNum 460 snap 136 seq_num 460) dirty exists 2024-09-15T03:51:11.086 INFO:tasks.rados.rados.0.smithi122.stdout:1242: left oid 8 (ObjNum 460 snap 136 seq_num 460) 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1243: finishing write tid 1 to smithi12223906-22 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1243: finishing write tid 2 to smithi12223906-22 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1244: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1243: finishing write tid 3 to smithi12223906-22 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1243: finishing write tid 4 to smithi12223906-22 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1243: finishing write tid 5 to smithi12223906-22 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 622 (ObjNum 461 snap 136 seq_num 461) dirty exists 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1243: left oid 22 (ObjNum 461 snap 136 seq_num 461) 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1246: finishing write tid 1 to smithi12223906-46 2024-09-15T03:51:11.087 INFO:tasks.rados.rados.0.smithi122.stdout:1245: expect (ObjNum 439 snap 128 seq_num 439) 2024-09-15T03:51:11.264 INFO:tasks.rados.rados.0.smithi122.stdout:1246: finishing write tid 2 to smithi12223906-46 2024-09-15T03:51:11.264 INFO:tasks.rados.rados.0.smithi122.stdout:1246: finishing write tid 3 to smithi12223906-46 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1246: finishing write tid 4 to smithi12223906-46 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1246: finishing write tid 5 to smithi12223906-46 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 389 (ObjNum 462 snap 136 seq_num 462) dirty exists 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1246: left oid 46 (ObjNum 462 snap 136 seq_num 462) 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1247: finishing write tid 1 to smithi12223906-1 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1247: finishing write tid 2 to smithi12223906-1 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1250: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 0 (ObjNum 977 snap 0 seq_num 1842159840) dirty dne 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1247: finishing write tid 3 to smithi12223906-1 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1247: finishing write tid 4 to smithi12223906-1 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:1247: finishing write tid 5 to smithi12223906-1 2024-09-15T03:51:11.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 381 (ObjNum 463 snap 136 seq_num 463) dirty exists 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1247: left oid 1 (ObjNum 463 snap 136 seq_num 463) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1237: done (11 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1238: done (10 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1240: done (9 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1242: done (8 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1243: done (7 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1244: done (6 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1245: done (5 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1246: done (4 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1247: done (3 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1250: done (2 left) 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1251: write oid 6 current snap is 136 2024-09-15T03:51:11.266 INFO:tasks.rados.rados.0.smithi122.stdout:1248: expect (ObjNum 458 snap 134 seq_num 458) 2024-09-15T03:51:11.413 INFO:tasks.rados.rados.0.smithi122.stdout:1249: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:51:11.413 INFO:tasks.rados.rados.0.smithi122.stdout:1249: finishing copy_from to smithi12223906-32 2024-09-15T03:51:11.413 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 342 (ObjNum 453 snap 131 seq_num 453) dirty exists 2024-09-15T03:51:11.414 INFO:tasks.rados.rados.0.smithi122.stdout:1251: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2024-09-15T03:51:11.426 INFO:tasks.rados.rados.0.smithi122.stdout:1251: writing smithi12223906-6 from 598448 to 1011517 tid 1 2024-09-15T03:51:11.434 INFO:tasks.rados.rados.0.smithi122.stdout:1251: writing smithi12223906-6 from 1768939 to 2293087 tid 2 2024-09-15T03:51:11.439 INFO:tasks.rados.rados.0.smithi122.stdout:1251: writing smithi12223906-6 from 2775798 to 3357659 tid 3 2024-09-15T03:51:11.443 INFO:tasks.rados.rados.0.smithi122.stdout:1251: writing smithi12223906-6 from 3781488 to 3798448 tid 4 2024-09-15T03:51:11.443 INFO:tasks.rados.rados.0.smithi122.stdout:1248: done (2 left) 2024-09-15T03:51:11.443 INFO:tasks.rados.rados.0.smithi122.stdout:1249: done (1 left) 2024-09-15T03:51:11.443 INFO:tasks.rados.rados.0.smithi122.stdout:1252: read oid 16 snap -1 2024-09-15T03:51:11.443 INFO:tasks.rados.rados.0.smithi122.stdout:1252: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:11.443 INFO:tasks.rados.rados.0.smithi122.stdout:1253: read oid 2 snap -1 2024-09-15T03:51:11.444 INFO:tasks.rados.rados.0.smithi122.stdout:1253: expect (ObjNum 453 snap 131 seq_num 453) 2024-09-15T03:51:11.444 INFO:tasks.rados.rados.0.smithi122.stdout:1254: write oid 15 current snap is 136 2024-09-15T03:51:11.444 INFO:tasks.rados.rados.0.smithi122.stdout:1251: finishing write tid 1 to smithi12223906-6 2024-09-15T03:51:11.444 INFO:tasks.rados.rados.0.smithi122.stdout:1254: seq_num 465 ranges {615255=488318,1827421=784159,3203221=612034} 2024-09-15T03:51:11.449 INFO:tasks.rados.rados.0.smithi122.stdout:1254: writing smithi12223906-15 from 615255 to 1103573 tid 1 2024-09-15T03:51:11.457 INFO:tasks.rados.rados.0.smithi122.stdout:1254: writing smithi12223906-15 from 1827421 to 2611580 tid 2 2024-09-15T03:51:11.465 INFO:tasks.rados.rados.0.smithi122.stdout:1254: writing smithi12223906-15 from 3203221 to 3815255 tid 3 2024-09-15T03:51:11.465 INFO:tasks.rados.rados.0.smithi122.stdout:1255: write (excl) oid 43 current snap is 136 2024-09-15T03:51:11.466 INFO:tasks.rados.rados.0.smithi122.stdout:1255: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2024-09-15T03:51:11.472 INFO:tasks.rados.rados.0.smithi122.stdout:1255: writing smithi12223906-43 from 632062 to 1195629 tid 1 2024-09-15T03:51:11.479 INFO:tasks.rados.rados.0.smithi122.stdout:1255: writing smithi12223906-43 from 1885903 to 2413720 tid 2 2024-09-15T03:51:11.486 INFO:tasks.rados.rados.0.smithi122.stdout:1255: writing smithi12223906-43 from 2830644 to 3551402 tid 3 2024-09-15T03:51:11.486 INFO:tasks.rados.rados.0.smithi122.stdout:1256: write (excl) oid 49 current snap is 136 2024-09-15T03:51:11.486 INFO:tasks.rados.rados.0.smithi122.stdout:1251: finishing write tid 2 to smithi12223906-6 2024-09-15T03:51:11.486 INFO:tasks.rados.rados.0.smithi122.stdout:1256: seq_num 467 ranges {648869=638816,1828032=787828,3141714=532030} 2024-09-15T03:51:11.493 INFO:tasks.rados.rados.0.smithi122.stdout:1256: writing smithi12223906-49 from 648869 to 1287685 tid 1 2024-09-15T03:51:11.500 INFO:tasks.rados.rados.0.smithi122.stdout:1256: writing smithi12223906-49 from 1828032 to 2615860 tid 2 2024-09-15T03:51:11.506 INFO:tasks.rados.rados.0.smithi122.stdout:1256: writing smithi12223906-49 from 3141714 to 3673744 tid 3 2024-09-15T03:51:11.507 INFO:tasks.rados.rados.0.smithi122.stdout:1257: read oid 29 snap -1 2024-09-15T03:51:11.507 INFO:tasks.rados.rados.0.smithi122.stdout:1257: expect (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:51:11.507 INFO:tasks.rados.rados.0.smithi122.stdout:1258: snap_remove snap 134 2024-09-15T03:51:11.809 INFO:tasks.rados.rados.0.smithi122.stdout:1258: done (7 left) 2024-09-15T03:51:11.809 INFO:tasks.rados.rados.0.smithi122.stdout:1259: read oid 32 snap -1 2024-09-15T03:51:11.809 INFO:tasks.rados.rados.0.smithi122.stdout:1259: expect (ObjNum 453 snap 131 seq_num 453) 2024-09-15T03:51:11.809 INFO:tasks.rados.rados.0.smithi122.stdout:1260: rollback oid 9 current snap is 136 2024-09-15T03:51:11.809 INFO:tasks.rados.rados.0.smithi122.stdout:1260: done (8 left) 2024-09-15T03:51:11.810 INFO:tasks.rados.rados.0.smithi122.stdout:1261: snap_create 2024-09-15T03:51:11.810 INFO:tasks.rados.rados.0.smithi122.stdout:1251: finishing write tid 3 to smithi12223906-6 2024-09-15T03:51:11.810 INFO:tasks.rados.rados.0.smithi122.stdout:1252: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:11.944 INFO:tasks.rados.rados.0.smithi122.stdout:1251: finishing write tid 4 to smithi12223906-6 2024-09-15T03:51:11.944 INFO:tasks.rados.rados.0.smithi122.stdout:1251: finishing write tid 5 to smithi12223906-6 2024-09-15T03:51:11.944 INFO:tasks.rados.rados.0.smithi122.stdout:1251: finishing write tid 6 to smithi12223906-6 2024-09-15T03:51:11.944 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 303 (ObjNum 464 snap 136 seq_num 464) dirty exists 2024-09-15T03:51:11.945 INFO:tasks.rados.rados.0.smithi122.stdout:1251: left oid 6 (ObjNum 464 snap 136 seq_num 464) 2024-09-15T03:51:11.945 INFO:tasks.rados.rados.0.smithi122.stdout:1253: expect (ObjNum 453 snap 131 seq_num 453) 2024-09-15T03:51:12.098 INFO:tasks.rados.rados.0.smithi122.stdout:1254: finishing write tid 1 to smithi12223906-15 2024-09-15T03:51:12.098 INFO:tasks.rados.rados.0.smithi122.stdout:1254: finishing write tid 2 to smithi12223906-15 2024-09-15T03:51:12.098 INFO:tasks.rados.rados.0.smithi122.stdout:1254: finishing write tid 3 to smithi12223906-15 2024-09-15T03:51:12.098 INFO:tasks.rados.rados.0.smithi122.stdout:1254: finishing write tid 4 to smithi12223906-15 2024-09-15T03:51:12.098 INFO:tasks.rados.rados.0.smithi122.stdout:1254: finishing write tid 5 to smithi12223906-15 2024-09-15T03:51:12.098 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 386 (ObjNum 465 snap 136 seq_num 465) dirty exists 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1254: left oid 15 (ObjNum 465 snap 136 seq_num 465) 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1255: finishing write tid 1 to smithi12223906-43 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1255: finishing write tid 2 to smithi12223906-43 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1255: finishing write tid 3 to smithi12223906-43 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1255: finishing write tid 4 to smithi12223906-43 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1255: finishing write tid 5 to smithi12223906-43 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 604 (ObjNum 466 snap 136 seq_num 466) dirty exists 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1255: left oid 43 (ObjNum 466 snap 136 seq_num 466) 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1256: finishing write tid 1 to smithi12223906-49 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1256: finishing write tid 2 to smithi12223906-49 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1256: finishing write tid 3 to smithi12223906-49 2024-09-15T03:51:12.099 INFO:tasks.rados.rados.0.smithi122.stdout:1257: expect (ObjNum 341 snap 101 seq_num 341) 2024-09-15T03:51:12.293 INFO:tasks.rados.rados.0.smithi122.stdout:1256: finishing write tid 4 to smithi12223906-49 2024-09-15T03:51:12.293 INFO:tasks.rados.rados.0.smithi122.stdout:1256: finishing write tid 5 to smithi12223906-49 2024-09-15T03:51:12.293 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 359 (ObjNum 467 snap 136 seq_num 467) dirty exists 2024-09-15T03:51:12.294 INFO:tasks.rados.rados.0.smithi122.stdout:1256: left oid 49 (ObjNum 467 snap 136 seq_num 467) 2024-09-15T03:51:12.294 INFO:tasks.rados.rados.0.smithi122.stdout:1259: expect (ObjNum 453 snap 131 seq_num 453) 2024-09-15T03:51:12.812 INFO:tasks.rados.rados.0.smithi122.stdout:1251: done (8 left) 2024-09-15T03:51:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:1252: done (7 left) 2024-09-15T03:51:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:1253: done (6 left) 2024-09-15T03:51:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:1254: done (5 left) 2024-09-15T03:51:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:1255: done (4 left) 2024-09-15T03:51:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:1256: done (3 left) 2024-09-15T03:51:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:1257: done (2 left) 2024-09-15T03:51:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:1259: done (1 left) 2024-09-15T03:51:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:1261: done (0 left) 2024-09-15T03:51:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:1262: write oid 27 current snap is 137 2024-09-15T03:51:12.815 INFO:tasks.rados.rados.0.smithi122.stdout:1262: seq_num 468 ranges {665676=714065,1886514=531486,3169137=696539} 2024-09-15T03:51:12.822 INFO:tasks.rados.rados.0.smithi122.stdout:1262: writing smithi12223906-27 from 665676 to 1379741 tid 1 2024-09-15T03:51:12.829 INFO:tasks.rados.rados.0.smithi122.stdout:1262: writing smithi12223906-27 from 1886514 to 2418000 tid 2 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1262: writing smithi12223906-27 from 3169137 to 3865676 tid 3 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1263: copy_from oid 3 from oid 4 current snap is 137 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1262: finishing write tid 1 to smithi12223906-27 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1262: finishing write tid 2 to smithi12223906-27 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1264: rollback oid 34 current snap is 137 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 136 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1265: write (excl) oid 37 current snap is 137 2024-09-15T03:51:12.838 INFO:tasks.rados.rados.0.smithi122.stdout:1265: seq_num 469 ranges {682483=789314,1944996=791497,3196560=554574} 2024-09-15T03:51:12.846 INFO:tasks.rados.rados.0.smithi122.stdout:1265: writing smithi12223906-37 from 682483 to 1471797 tid 1 2024-09-15T03:51:12.853 INFO:tasks.rados.rados.0.smithi122.stdout:1265: writing smithi12223906-37 from 1944996 to 2736493 tid 2 2024-09-15T03:51:12.859 INFO:tasks.rados.rados.0.smithi122.stdout:1265: writing smithi12223906-37 from 3196560 to 3751134 tid 3 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1266: read oid 44 snap -1 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1266: expect deleted 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1267: read oid 45 snap -1 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1267: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1268: snap_create 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1262: finishing write tid 3 to smithi12223906-27 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1262: finishing write tid 4 to smithi12223906-27 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1262: finishing write tid 5 to smithi12223906-27 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 631 (ObjNum 468 snap 137 seq_num 468) dirty exists 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1262: left oid 27 (ObjNum 468 snap 137 seq_num 468) 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1264: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:51:12.860 INFO:tasks.rados.rados.0.smithi122.stdout:1264: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:1264: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 355 (ObjNum 385 snap 113 seq_num 385) dirty exists 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:1265: finishing write tid 1 to smithi12223906-37 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:1263: finishing copy_from racing read to smithi12223906-3 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:1265: finishing write tid 2 to smithi12223906-37 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:1263: finishing copy_from to smithi12223906-3 2024-09-15T03:51:12.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 218 (ObjNum 450 snap 130 seq_num 450) dirty exists 2024-09-15T03:51:12.864 INFO:tasks.rados.rados.0.smithi122.stdout:1265: finishing write tid 3 to smithi12223906-37 2024-09-15T03:51:12.866 INFO:tasks.rados.rados.0.smithi122.stdout:1265: finishing write tid 4 to smithi12223906-37 2024-09-15T03:51:12.866 INFO:tasks.rados.rados.0.smithi122.stdout:1265: finishing write tid 5 to smithi12223906-37 2024-09-15T03:51:12.866 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 391 (ObjNum 469 snap 137 seq_num 469) dirty exists 2024-09-15T03:51:12.866 INFO:tasks.rados.rados.0.smithi122.stdout:1265: left oid 37 (ObjNum 469 snap 137 seq_num 469) 2024-09-15T03:51:12.870 INFO:tasks.rados.rados.0.smithi122.stdout:1267: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:51:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:1262: done (6 left) 2024-09-15T03:51:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:1263: done (5 left) 2024-09-15T03:51:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:1264: done (4 left) 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1265: done (3 left) 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1266: done (2 left) 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1267: done (1 left) 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1268: done (0 left) 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1269: read oid 33 snap -1 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1269: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1270: write oid 27 current snap is 138 2024-09-15T03:51:13.815 INFO:tasks.rados.rados.0.smithi122.stdout:1270: seq_num 470 ranges {699290=464563,1603478=535155,2823983=765846} 2024-09-15T03:51:13.821 INFO:tasks.rados.rados.0.smithi122.stdout:1270: writing smithi12223906-27 from 699290 to 1163853 tid 1 2024-09-15T03:51:13.827 INFO:tasks.rados.rados.0.smithi122.stdout:1270: writing smithi12223906-27 from 1603478 to 2138633 tid 2 2024-09-15T03:51:13.835 INFO:tasks.rados.rados.0.smithi122.stdout:1270: writing smithi12223906-27 from 2823983 to 3589829 tid 3 2024-09-15T03:51:13.835 INFO:tasks.rados.rados.0.smithi122.stdout:1271: snap_create 2024-09-15T03:51:13.836 INFO:tasks.rados.rados.0.smithi122.stdout:1269: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:1270: finishing write tid 1 to smithi12223906-27 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:1270: finishing write tid 2 to smithi12223906-27 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:1270: finishing write tid 3 to smithi12223906-27 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:1270: finishing write tid 4 to smithi12223906-27 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:1270: finishing write tid 5 to smithi12223906-27 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 636 (ObjNum 470 snap 138 seq_num 470) dirty exists 2024-09-15T03:51:14.021 INFO:tasks.rados.rados.0.smithi122.stdout:1270: left oid 27 (ObjNum 470 snap 138 seq_num 470) 2024-09-15T03:51:14.817 INFO:tasks.rados.rados.0.smithi122.stdout:1269: done (2 left) 2024-09-15T03:51:14.817 INFO:tasks.rados.rados.0.smithi122.stdout:1270: done (1 left) 2024-09-15T03:51:14.817 INFO:tasks.rados.rados.0.smithi122.stdout:1271: done (0 left) 2024-09-15T03:51:14.817 INFO:tasks.rados.rados.0.smithi122.stdout:1272: copy_from oid 20 from oid 22 current snap is 139 2024-09-15T03:51:14.817 INFO:tasks.rados.rados.0.smithi122.stdout:1273: read oid 19 snap -1 2024-09-15T03:51:14.817 INFO:tasks.rados.rados.0.smithi122.stdout:1273: expect deleted 2024-09-15T03:51:14.818 INFO:tasks.rados.rados.0.smithi122.stdout:1274: write (excl) oid 9 current snap is 139 2024-09-15T03:51:14.818 INFO:tasks.rados.rados.0.smithi122.stdout:1274: seq_num 471 ranges {716097=539812,1945607=795166,3535053=381044} 2024-09-15T03:51:14.826 INFO:tasks.rados.rados.0.smithi122.stdout:1274: writing smithi12223906-9 from 716097 to 1255909 tid 1 2024-09-15T03:51:14.835 INFO:tasks.rados.rados.0.smithi122.stdout:1274: writing smithi12223906-9 from 1945607 to 2740773 tid 2 2024-09-15T03:51:14.842 INFO:tasks.rados.rados.0.smithi122.stdout:1274: writing smithi12223906-9 from 3535053 to 3916097 tid 3 2024-09-15T03:51:14.842 INFO:tasks.rados.rados.0.smithi122.stdout:1275: snap_remove snap 137 2024-09-15T03:51:15.838 INFO:tasks.rados.rados.0.smithi122.stdout:1275: done (3 left) 2024-09-15T03:51:15.838 INFO:tasks.rados.rados.0.smithi122.stdout:1276: read oid 4 snap 138 2024-09-15T03:51:15.838 INFO:tasks.rados.rados.0.smithi122.stdout:1276: expect (ObjNum 450 snap 130 seq_num 450) 2024-09-15T03:51:15.839 INFO:tasks.rados.rados.0.smithi122.stdout:1277: write oid 43 current snap is 139 2024-09-15T03:51:15.839 INFO:tasks.rados.rados.0.smithi122.stdout:1277: seq_num 472 ranges {732904=731414,2120442=538824,3278829=504743} 2024-09-15T03:51:15.848 INFO:tasks.rados.rados.0.smithi122.stdout:1277: writing smithi12223906-43 from 732904 to 1464318 tid 1 2024-09-15T03:51:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:1277: writing smithi12223906-43 from 2120442 to 2659266 tid 2 2024-09-15T03:51:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:1277: writing smithi12223906-43 from 3278829 to 3783572 tid 3 2024-09-15T03:51:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:1278: snap_create 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1274: finishing write tid 1 to smithi12223906-9 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1272: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1272: finishing copy_from to smithi12223906-20 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 622 (ObjNum 461 snap 136 seq_num 461) dirty exists 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1274: finishing write tid 2 to smithi12223906-9 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1274: finishing write tid 3 to smithi12223906-9 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1274: finishing write tid 4 to smithi12223906-9 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1274: finishing write tid 5 to smithi12223906-9 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 307 (ObjNum 471 snap 139 seq_num 471) dirty exists 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1274: left oid 9 (ObjNum 471 snap 139 seq_num 471) 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1277: finishing write tid 1 to smithi12223906-43 2024-09-15T03:51:15.864 INFO:tasks.rados.rados.0.smithi122.stdout:1276: expect (ObjNum 450 snap 130 seq_num 450) 2024-09-15T03:51:16.098 INFO:tasks.rados.rados.0.smithi122.stdout:1277: finishing write tid 2 to smithi12223906-43 2024-09-15T03:51:16.098 INFO:tasks.rados.rados.0.smithi122.stdout:1277: finishing write tid 3 to smithi12223906-43 2024-09-15T03:51:16.098 INFO:tasks.rados.rados.0.smithi122.stdout:1277: finishing write tid 4 to smithi12223906-43 2024-09-15T03:51:16.098 INFO:tasks.rados.rados.0.smithi122.stdout:1277: finishing write tid 5 to smithi12223906-43 2024-09-15T03:51:16.098 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 615 (ObjNum 472 snap 139 seq_num 472) dirty exists 2024-09-15T03:51:16.098 INFO:tasks.rados.rados.0.smithi122.stdout:1277: left oid 43 (ObjNum 472 snap 139 seq_num 472) 2024-09-15T03:51:16.813 INFO:tasks.rados.rados.0.smithi122.stdout:1272: done (5 left) 2024-09-15T03:51:16.814 INFO:tasks.rados.rados.0.smithi122.stdout:1273: done (4 left) 2024-09-15T03:51:16.814 INFO:tasks.rados.rados.0.smithi122.stdout:1274: done (3 left) 2024-09-15T03:51:16.814 INFO:tasks.rados.rados.0.smithi122.stdout:1276: done (2 left) 2024-09-15T03:51:16.814 INFO:tasks.rados.rados.0.smithi122.stdout:1277: done (1 left) 2024-09-15T03:51:16.814 INFO:tasks.rados.rados.0.smithi122.stdout:1278: done (0 left) 2024-09-15T03:51:16.814 INFO:tasks.rados.rados.0.smithi122.stdout:1279: delete oid 40 current snap is 140 2024-09-15T03:51:16.821 INFO:tasks.rados.rados.0.smithi122.stdout:1279: done (0 left) 2024-09-15T03:51:16.821 INFO:tasks.rados.rados.0.smithi122.stdout:1280: read oid 50 snap -1 2024-09-15T03:51:16.822 INFO:tasks.rados.rados.0.smithi122.stdout:1280: expect (ObjNum 447 snap 129 seq_num 447) 2024-09-15T03:51:16.822 INFO:tasks.rados.rados.0.smithi122.stdout:1281: snap_create 2024-09-15T03:51:16.834 INFO:tasks.rados.rados.0.smithi122.stdout:1280: expect (ObjNum 447 snap 129 seq_num 447) 2024-09-15T03:51:17.825 INFO:tasks.rados.rados.0.smithi122.stdout:1280: done (1 left) 2024-09-15T03:51:17.826 INFO:tasks.rados.rados.0.smithi122.stdout:1281: done (0 left) 2024-09-15T03:51:17.826 INFO:tasks.rados.rados.0.smithi122.stdout:1282: snap_remove snap 140 2024-09-15T03:51:18.835 INFO:tasks.rados.rados.0.smithi122.stdout:1282: done (0 left) 2024-09-15T03:51:18.835 INFO:tasks.rados.rados.0.smithi122.stdout:1283: snap_remove snap 136 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:1283: done (0 left) 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:1284: rollback oid 5 current snap is 141 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 5 to 138 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:1285: read oid 34 snap -1 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:1285: expect (ObjNum 385 snap 113 seq_num 385) 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:1286: read oid 47 snap -1 2024-09-15T03:51:19.849 INFO:tasks.rados.rados.0.smithi122.stdout:1286: expect (ObjNum 429 snap 127 seq_num 429) 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1287: rollback oid 43 current snap is 141 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 139 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1288: read oid 27 snap -1 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1288: expect (ObjNum 470 snap 138 seq_num 470) 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1289: read oid 41 snap -1 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1289: expect (ObjNum 446 snap 129 seq_num 446) 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1290: write oid 15 current snap is 141 2024-09-15T03:51:19.850 INFO:tasks.rados.rados.0.smithi122.stdout:1290: seq_num 474 ranges {766518=481912,1837406=542493,2933675=527287,3876584=89934} 2024-09-15T03:51:19.854 INFO:tasks.rados.rados.0.smithi122.stdout:1290: writing smithi12223906-15 from 766518 to 1248430 tid 1 2024-09-15T03:51:19.861 INFO:tasks.rados.rados.0.smithi122.stdout:1290: writing smithi12223906-15 from 1837406 to 2379899 tid 2 2024-09-15T03:51:19.868 INFO:tasks.rados.rados.0.smithi122.stdout:1290: writing smithi12223906-15 from 2933675 to 3460962 tid 3 2024-09-15T03:51:19.871 INFO:tasks.rados.rados.0.smithi122.stdout:1290: writing smithi12223906-15 from 3876584 to 3966518 tid 4 2024-09-15T03:51:19.871 INFO:tasks.rados.rados.0.smithi122.stdout:1291: delete oid 36 current snap is 141 2024-09-15T03:51:19.871 INFO:tasks.rados.rados.0.smithi122.stdout:1285: expect (ObjNum 385 snap 113 seq_num 385) 2024-09-15T03:51:19.951 INFO:tasks.rados.rados.0.smithi122.stdout:1286: expect (ObjNum 429 snap 127 seq_num 429) 2024-09-15T03:51:20.137 INFO:tasks.rados.rados.0.smithi122.stdout:1287: finishing rollback tid 0 to smithi12223906-43 2024-09-15T03:51:20.137 INFO:tasks.rados.rados.0.smithi122.stdout:1287: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:51:20.137 INFO:tasks.rados.rados.0.smithi122.stdout:1287: finishing rollback tid 2 to smithi12223906-43 2024-09-15T03:51:20.137 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 623 (ObjNum 472 snap 139 seq_num 472) dirty exists 2024-09-15T03:51:20.137 INFO:tasks.rados.rados.0.smithi122.stdout:1289: expect (ObjNum 446 snap 129 seq_num 446) 2024-09-15T03:51:20.353 INFO:tasks.rados.rados.0.smithi122.stdout:1284: finishing rollback tid 1 to smithi12223906-5 2024-09-15T03:51:20.353 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 0 (ObjNum 289 snap 0 seq_num 1842136416) dirty dne 2024-09-15T03:51:20.353 INFO:tasks.rados.rados.0.smithi122.stdout:1290: finishing write tid 1 to smithi12223906-15 2024-09-15T03:51:20.353 INFO:tasks.rados.rados.0.smithi122.stdout:1290: finishing write tid 2 to smithi12223906-15 2024-09-15T03:51:20.353 INFO:tasks.rados.rados.0.smithi122.stdout:1288: expect (ObjNum 470 snap 138 seq_num 470) 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1290: finishing write tid 3 to smithi12223906-15 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1290: finishing write tid 4 to smithi12223906-15 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1290: finishing write tid 5 to smithi12223906-15 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1290: finishing write tid 6 to smithi12223906-15 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 396 (ObjNum 474 snap 141 seq_num 474) dirty exists 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1290: left oid 15 (ObjNum 474 snap 141 seq_num 474) 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1284: done (7 left) 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1285: done (6 left) 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1286: done (5 left) 2024-09-15T03:51:20.566 INFO:tasks.rados.rados.0.smithi122.stdout:1287: done (4 left) 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1288: done (3 left) 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1289: done (2 left) 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1290: done (1 left) 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1291: done (0 left) 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1292: rollback oid 1 current snap is 141 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 1 to 138 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1293: read oid 20 snap -1 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1293: expect (ObjNum 461 snap 136 seq_num 461) 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1294: rollback oid 31 current snap is 141 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 31 to 138 2024-09-15T03:51:20.567 INFO:tasks.rados.rados.0.smithi122.stdout:1295: delete oid 26 current snap is 141 2024-09-15T03:51:20.576 INFO:tasks.rados.rados.0.smithi122.stdout:1295: done (3 left) 2024-09-15T03:51:20.576 INFO:tasks.rados.rados.0.smithi122.stdout:1296: write oid 34 current snap is 141 2024-09-15T03:51:20.576 INFO:tasks.rados.rados.0.smithi122.stdout:1296: seq_num 477 ranges {416939=707659,1896499=120440} 2024-09-15T03:51:20.585 INFO:tasks.rados.rados.0.smithi122.stdout:1296: writing smithi12223906-34 from 416939 to 1124598 tid 1 2024-09-15T03:51:20.592 INFO:tasks.rados.rados.0.smithi122.stdout:1296: writing smithi12223906-34 from 1896499 to 2016939 tid 2 2024-09-15T03:51:20.592 INFO:tasks.rados.rados.0.smithi122.stdout:1297: snap_remove snap 139 2024-09-15T03:51:20.853 INFO:tasks.rados.rados.0.smithi122.stdout:1297: done (4 left) 2024-09-15T03:51:20.853 INFO:tasks.rados.rados.0.smithi122.stdout:1298: delete oid 43 current snap is 141 2024-09-15T03:51:20.853 INFO:tasks.rados.rados.0.smithi122.stdout:1294: finishing rollback tid 0 to smithi12223906-31 2024-09-15T03:51:20.853 INFO:tasks.rados.rados.0.smithi122.stdout:1294: finishing rollback tid 1 to smithi12223906-31 2024-09-15T03:51:20.853 INFO:tasks.rados.rados.0.smithi122.stdout:1293: expect (ObjNum 461 snap 136 seq_num 461) 2024-09-15T03:51:20.973 INFO:tasks.rados.rados.0.smithi122.stdout:1292: finishing rollback tid 0 to smithi12223906-1 2024-09-15T03:51:20.973 INFO:tasks.rados.rados.0.smithi122.stdout:1294: finishing rollback tid 2 to smithi12223906-31 2024-09-15T03:51:20.973 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 221 (ObjNum 412 snap 122 seq_num 412) dirty exists 2024-09-15T03:51:20.973 INFO:tasks.rados.rados.0.smithi122.stdout:1292: finishing rollback tid 1 to smithi12223906-1 2024-09-15T03:51:20.973 INFO:tasks.rados.rados.0.smithi122.stdout:1292: finishing rollback tid 2 to smithi12223906-1 2024-09-15T03:51:20.973 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 400 (ObjNum 463 snap 136 seq_num 463) dirty exists 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1296: finishing write tid 1 to smithi12223906-34 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1296: finishing write tid 2 to smithi12223906-34 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1296: finishing write tid 3 to smithi12223906-34 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1296: finishing write tid 4 to smithi12223906-34 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 358 (ObjNum 477 snap 141 seq_num 477) dirty exists 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1296: left oid 34 (ObjNum 477 snap 141 seq_num 477) 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1292: done (4 left) 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1293: done (3 left) 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1294: done (2 left) 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1296: done (1 left) 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1298: done (0 left) 2024-09-15T03:51:20.974 INFO:tasks.rados.rados.0.smithi122.stdout:1299: snap_create 2024-09-15T03:51:21.870 INFO:tasks.rados.rados.0.smithi122.stdout:1299: done (0 left) 2024-09-15T03:51:21.871 INFO:tasks.rados.rados.0.smithi122.stdout:1300: read oid 24 snap 138 2024-09-15T03:51:21.871 INFO:tasks.rados.rados.0.smithi122.stdout:1300: expect (ObjNum 431 snap 127 seq_num 431) 2024-09-15T03:51:21.871 INFO:tasks.rados.rados.0.smithi122.stdout:1301: copy_from oid 20 from oid 27 current snap is 142 2024-09-15T03:51:21.871 INFO:tasks.rados.rados.0.smithi122.stdout:1302: write oid 47 current snap is 142 2024-09-15T03:51:21.871 INFO:tasks.rados.rados.0.smithi122.stdout:1302: seq_num 479 ranges {450553=574510,1613463=437090} 2024-09-15T03:51:21.881 INFO:tasks.rados.rados.0.smithi122.stdout:1302: writing smithi12223906-47 from 450553 to 1025063 tid 1 2024-09-15T03:51:21.888 INFO:tasks.rados.rados.0.smithi122.stdout:1302: writing smithi12223906-47 from 1613463 to 2050553 tid 2 2024-09-15T03:51:21.888 INFO:tasks.rados.rados.0.smithi122.stdout:1303: read oid 16 snap 138 2024-09-15T03:51:21.888 INFO:tasks.rados.rados.0.smithi122.stdout:1303: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:21.888 INFO:tasks.rados.rados.0.smithi122.stdout:1304: snap_create 2024-09-15T03:51:21.888 INFO:tasks.rados.rados.0.smithi122.stdout:1300: expect (ObjNum 431 snap 127 seq_num 431) 2024-09-15T03:51:22.139 INFO:tasks.rados.rados.0.smithi122.stdout:1301: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:51:22.140 INFO:tasks.rados.rados.0.smithi122.stdout:1303: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:22.267 INFO:tasks.rados.rados.0.smithi122.stdout:1302: finishing write tid 1 to smithi12223906-47 2024-09-15T03:51:22.267 INFO:tasks.rados.rados.0.smithi122.stdout:1302: finishing write tid 2 to smithi12223906-47 2024-09-15T03:51:22.267 INFO:tasks.rados.rados.0.smithi122.stdout:1302: finishing write tid 3 to smithi12223906-47 2024-09-15T03:51:22.268 INFO:tasks.rados.rados.0.smithi122.stdout:1301: finishing copy_from to smithi12223906-20 2024-09-15T03:51:22.268 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 636 (ObjNum 470 snap 138 seq_num 470) dirty exists 2024-09-15T03:51:22.268 INFO:tasks.rados.rados.0.smithi122.stdout:1302: finishing write tid 4 to smithi12223906-47 2024-09-15T03:51:22.268 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 203 (ObjNum 479 snap 142 seq_num 479) dirty exists 2024-09-15T03:51:22.268 INFO:tasks.rados.rados.0.smithi122.stdout:1302: left oid 47 (ObjNum 479 snap 142 seq_num 479) 2024-09-15T03:51:22.875 INFO:tasks.rados.rados.0.smithi122.stdout:1300: done (4 left) 2024-09-15T03:51:22.875 INFO:tasks.rados.rados.0.smithi122.stdout:1301: done (3 left) 2024-09-15T03:51:22.875 INFO:tasks.rados.rados.0.smithi122.stdout:1302: done (2 left) 2024-09-15T03:51:22.876 INFO:tasks.rados.rados.0.smithi122.stdout:1303: done (1 left) 2024-09-15T03:51:22.876 INFO:tasks.rados.rados.0.smithi122.stdout:1304: done (0 left) 2024-09-15T03:51:22.876 INFO:tasks.rados.rados.0.smithi122.stdout:1305: write oid 39 current snap is 143 2024-09-15T03:51:22.876 INFO:tasks.rados.rados.0.smithi122.stdout:1305: seq_num 480 ranges {467360=649759,1671945=395415} 2024-09-15T03:51:22.884 INFO:tasks.rados.rados.0.smithi122.stdout:1305: writing smithi12223906-39 from 467360 to 1117119 tid 1 2024-09-15T03:51:22.891 INFO:tasks.rados.rados.0.smithi122.stdout:1305: writing smithi12223906-39 from 1671945 to 2067360 tid 2 2024-09-15T03:51:22.891 INFO:tasks.rados.rados.0.smithi122.stdout:1306: snap_create 2024-09-15T03:51:22.891 INFO:tasks.rados.rados.0.smithi122.stdout:1305: finishing write tid 1 to smithi12223906-39 2024-09-15T03:51:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:1305: finishing write tid 2 to smithi12223906-39 2024-09-15T03:51:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:1305: finishing write tid 3 to smithi12223906-39 2024-09-15T03:51:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:1305: finishing write tid 4 to smithi12223906-39 2024-09-15T03:51:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 633 (ObjNum 480 snap 143 seq_num 480) dirty exists 2024-09-15T03:51:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:1305: left oid 39 (ObjNum 480 snap 143 seq_num 480) 2024-09-15T03:51:23.893 INFO:tasks.rados.rados.0.smithi122.stdout:1305: done (1 left) 2024-09-15T03:51:23.894 INFO:tasks.rados.rados.0.smithi122.stdout:1306: done (0 left) 2024-09-15T03:51:23.894 INFO:tasks.rados.rados.0.smithi122.stdout:1307: write oid 27 current snap is 144 2024-09-15T03:51:23.894 INFO:tasks.rados.rados.0.smithi122.stdout:1307: seq_num 481 ranges {484167=725008,1730427=353740} 2024-09-15T03:51:23.903 INFO:tasks.rados.rados.0.smithi122.stdout:1307: writing smithi12223906-27 from 484167 to 1209175 tid 1 2024-09-15T03:51:23.909 INFO:tasks.rados.rados.0.smithi122.stdout:1307: writing smithi12223906-27 from 1730427 to 2084167 tid 2 2024-09-15T03:51:23.909 INFO:tasks.rados.rados.0.smithi122.stdout:1308: read oid 39 snap -1 2024-09-15T03:51:23.909 INFO:tasks.rados.rados.0.smithi122.stdout:1308: expect (ObjNum 480 snap 143 seq_num 480) 2024-09-15T03:51:23.909 INFO:tasks.rados.rados.0.smithi122.stdout:1309: copy_from oid 1 from oid 44 current snap is 144 2024-09-15T03:51:23.909 INFO:tasks.rados.rados.0.smithi122.stdout:1310: snap_remove snap 138 2024-09-15T03:51:24.907 INFO:tasks.rados.rados.0.smithi122.stdout:1310: done (3 left) 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1311: snap_create 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1307: finishing write tid 1 to smithi12223906-27 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1309: finishing copy_from to smithi12223906-1 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1309: got expected ENOENT (src dne) 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1307: finishing write tid 2 to smithi12223906-27 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1309: finishing copy_from racing read to smithi12223906-1 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1307: finishing write tid 3 to smithi12223906-27 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1307: finishing write tid 4 to smithi12223906-27 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 644 (ObjNum 481 snap 144 seq_num 481) dirty exists 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1307: left oid 27 (ObjNum 481 snap 144 seq_num 481) 2024-09-15T03:51:24.908 INFO:tasks.rados.rados.0.smithi122.stdout:1308: expect (ObjNum 480 snap 143 seq_num 480) 2024-09-15T03:51:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:1307: done (3 left) 2024-09-15T03:51:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:1308: done (2 left) 2024-09-15T03:51:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:1309: done (1 left) 2024-09-15T03:51:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:1311: done (0 left) 2024-09-15T03:51:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:1312: delete oid 34 current snap is 145 2024-09-15T03:51:25.914 INFO:tasks.rados.rados.0.smithi122.stdout:1312: done (0 left) 2024-09-15T03:51:25.914 INFO:tasks.rados.rados.0.smithi122.stdout:1313: snap_remove snap 144 2024-09-15T03:51:26.821 INFO:tasks.rados.rados.0.smithi122.stdout:1313: done (0 left) 2024-09-15T03:51:26.821 INFO:tasks.rados.rados.0.smithi122.stdout:1314: write (excl) oid 29 current snap is 145 2024-09-15T03:51:26.821 INFO:tasks.rados.rados.0.smithi122.stdout:1314: seq_num 483 ranges {517781=475506,1731038=386743} 2024-09-15T03:51:26.827 INFO:tasks.rados.rados.0.smithi122.stdout:1314: writing smithi12223906-29 from 517781 to 993287 tid 1 2024-09-15T03:51:26.835 INFO:tasks.rados.rados.0.smithi122.stdout:1314: writing smithi12223906-29 from 1731038 to 2117781 tid 2 2024-09-15T03:51:26.835 INFO:tasks.rados.rados.0.smithi122.stdout:1315: write (excl) oid 35 current snap is 145 2024-09-15T03:51:26.835 INFO:tasks.rados.rados.0.smithi122.stdout:1314: finishing write tid 1 to smithi12223906-29 2024-09-15T03:51:26.835 INFO:tasks.rados.rados.0.smithi122.stdout:1315: seq_num 484 ranges {534588=550755,1789520=345068} 2024-09-15T03:51:26.841 INFO:tasks.rados.rados.0.smithi122.stdout:1315: writing smithi12223906-35 from 534588 to 1085343 tid 1 2024-09-15T03:51:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:1315: writing smithi12223906-35 from 1789520 to 2134588 tid 2 2024-09-15T03:51:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:1316: read oid 37 snap 143 2024-09-15T03:51:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:1316: expect (ObjNum 469 snap 137 seq_num 469) 2024-09-15T03:51:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:1317: write oid 38 current snap is 145 2024-09-15T03:51:26.847 INFO:tasks.rados.rados.0.smithi122.stdout:1314: finishing write tid 2 to smithi12223906-29 2024-09-15T03:51:26.848 INFO:tasks.rados.rados.0.smithi122.stdout:1317: seq_num 485 ranges {551395=626004,1848002=303393} 2024-09-15T03:51:26.854 INFO:tasks.rados.rados.0.smithi122.stdout:1317: writing smithi12223906-38 from 551395 to 1177399 tid 1 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:1317: writing smithi12223906-38 from 1848002 to 2151395 tid 2 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:1318: rollback oid 28 current snap is 145 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 28 to 143 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:1314: finishing write tid 3 to smithi12223906-29 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:1314: finishing write tid 4 to smithi12223906-29 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 583 (ObjNum 483 snap 145 seq_num 483) dirty exists 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:1314: left oid 29 (ObjNum 483 snap 145 seq_num 483) 2024-09-15T03:51:26.861 INFO:tasks.rados.rados.0.smithi122.stdout:1315: finishing write tid 1 to smithi12223906-35 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1314: done (4 left) 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1319: delete oid 42 current snap is 145 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1315: finishing write tid 2 to smithi12223906-35 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1315: finishing write tid 3 to smithi12223906-35 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1315: finishing write tid 4 to smithi12223906-35 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 554 (ObjNum 484 snap 145 seq_num 484) dirty exists 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1315: left oid 35 (ObjNum 484 snap 145 seq_num 484) 2024-09-15T03:51:26.862 INFO:tasks.rados.rados.0.smithi122.stdout:1316: expect (ObjNum 469 snap 137 seq_num 469) 2024-09-15T03:51:27.098 INFO:tasks.rados.rados.0.smithi122.stdout:1317: finishing write tid 1 to smithi12223906-38 2024-09-15T03:51:27.099 INFO:tasks.rados.rados.0.smithi122.stdout:1317: finishing write tid 2 to smithi12223906-38 2024-09-15T03:51:27.099 INFO:tasks.rados.rados.0.smithi122.stdout:1317: finishing write tid 3 to smithi12223906-38 2024-09-15T03:51:27.099 INFO:tasks.rados.rados.0.smithi122.stdout:1317: finishing write tid 4 to smithi12223906-38 2024-09-15T03:51:27.099 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 376 (ObjNum 485 snap 145 seq_num 485) dirty exists 2024-09-15T03:51:27.099 INFO:tasks.rados.rados.0.smithi122.stdout:1317: left oid 38 (ObjNum 485 snap 145 seq_num 485) 2024-09-15T03:51:27.099 INFO:tasks.rados.rados.0.smithi122.stdout:1318: finishing rollback tid 0 to smithi12223906-28 2024-09-15T03:51:27.100 INFO:tasks.rados.rados.0.smithi122.stdout:1318: finishing rollback tid 1 to smithi12223906-28 2024-09-15T03:51:27.100 INFO:tasks.rados.rados.0.smithi122.stdout:1318: finishing rollback tid 2 to smithi12223906-28 2024-09-15T03:51:27.100 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 586 (ObjNum 455 snap 133 seq_num 455) dirty exists 2024-09-15T03:51:27.100 INFO:tasks.rados.rados.0.smithi122.stdout:1315: done (4 left) 2024-09-15T03:51:27.100 INFO:tasks.rados.rados.0.smithi122.stdout:1316: done (3 left) 2024-09-15T03:51:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:1317: done (2 left) 2024-09-15T03:51:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:1318: done (1 left) 2024-09-15T03:51:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:1319: done (0 left) 2024-09-15T03:51:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:1320: read oid 38 snap -1 2024-09-15T03:51:27.102 INFO:tasks.rados.rados.0.smithi122.stdout:1320: expect (ObjNum 485 snap 145 seq_num 485) 2024-09-15T03:51:27.102 INFO:tasks.rados.rados.0.smithi122.stdout:1321: snap_create 2024-09-15T03:51:27.110 INFO:tasks.rados.rados.0.smithi122.stdout:1320: expect (ObjNum 485 snap 145 seq_num 485) 2024-09-15T03:51:27.930 INFO:tasks.rados.rados.0.smithi122.stdout:1320: done (1 left) 2024-09-15T03:51:27.930 INFO:tasks.rados.rados.0.smithi122.stdout:1321: done (0 left) 2024-09-15T03:51:27.930 INFO:tasks.rados.rados.0.smithi122.stdout:1322: read oid 16 snap -1 2024-09-15T03:51:27.931 INFO:tasks.rados.rados.0.smithi122.stdout:1322: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:27.931 INFO:tasks.rados.rados.0.smithi122.stdout:1323: copy_from oid 29 from oid 39 current snap is 146 2024-09-15T03:51:27.931 INFO:tasks.rados.rados.0.smithi122.stdout:1324: snap_create 2024-09-15T03:51:27.935 INFO:tasks.rados.rados.0.smithi122.stdout:1322: expect (ObjNum 383 snap 112 seq_num 383) 2024-09-15T03:51:28.078 INFO:tasks.rados.rados.0.smithi122.stdout:1323: finishing copy_from racing read to smithi12223906-29 2024-09-15T03:51:28.078 INFO:tasks.rados.rados.0.smithi122.stdout:1323: finishing copy_from to smithi12223906-29 2024-09-15T03:51:28.078 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 633 (ObjNum 480 snap 143 seq_num 480) dirty exists 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:1322: done (2 left) 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:1323: done (1 left) 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:1324: done (0 left) 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:1325: rollback oid 43 current snap is 147 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 146 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:1326: copy_from oid 39 from oid 32 current snap is 147 2024-09-15T03:51:28.933 INFO:tasks.rados.rados.0.smithi122.stdout:1327: snap_remove snap 143 2024-09-15T03:51:29.951 INFO:tasks.rados.rados.0.smithi122.stdout:1327: done (2 left) 2024-09-15T03:51:29.951 INFO:tasks.rados.rados.0.smithi122.stdout:1328: delete oid 16 current snap is 147 2024-09-15T03:51:29.952 INFO:tasks.rados.rados.0.smithi122.stdout:1325: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:51:29.952 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 0 (ObjNum 33 snap 0 seq_num 1817663720) dirty dne 2024-09-15T03:51:29.952 INFO:tasks.rados.rados.0.smithi122.stdout:1326: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:51:29.952 INFO:tasks.rados.rados.0.smithi122.stdout:1326: finishing copy_from to smithi12223906-39 2024-09-15T03:51:29.952 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 342 (ObjNum 453 snap 131 seq_num 453) dirty exists 2024-09-15T03:51:29.955 INFO:tasks.rados.rados.0.smithi122.stdout:1325: done (2 left) 2024-09-15T03:51:29.955 INFO:tasks.rados.rados.0.smithi122.stdout:1326: done (1 left) 2024-09-15T03:51:29.955 INFO:tasks.rados.rados.0.smithi122.stdout:1328: done (0 left) 2024-09-15T03:51:29.955 INFO:tasks.rados.rados.0.smithi122.stdout:1329: snap_remove snap 146 2024-09-15T03:51:30.959 INFO:tasks.rados.rados.0.smithi122.stdout:1329: done (0 left) 2024-09-15T03:51:30.959 INFO:tasks.rados.rados.0.smithi122.stdout:1330: read oid 1 snap 145 2024-09-15T03:51:30.959 INFO:tasks.rados.rados.0.smithi122.stdout:1330: expect (ObjNum 463 snap 136 seq_num 463) 2024-09-15T03:51:30.959 INFO:tasks.rados.rados.0.smithi122.stdout:1331: write (excl) oid 27 current snap is 147 2024-09-15T03:51:30.959 INFO:tasks.rados.rados.0.smithi122.stdout:1331: seq_num 488 ranges {601816=568104,1623448=451823} 2024-09-15T03:51:30.970 INFO:tasks.rados.rados.0.smithi122.stdout:1331: writing smithi12223906-27 from 601816 to 1169920 tid 1 2024-09-15T03:51:30.976 INFO:tasks.rados.rados.0.smithi122.stdout:1331: writing smithi12223906-27 from 1623448 to 2075271 tid 2 2024-09-15T03:51:30.976 INFO:tasks.rados.rados.0.smithi122.stdout:1332: read oid 20 snap -1 2024-09-15T03:51:30.976 INFO:tasks.rados.rados.0.smithi122.stdout:1332: expect (ObjNum 470 snap 138 seq_num 470) 2024-09-15T03:51:30.976 INFO:tasks.rados.rados.0.smithi122.stdout:1333: delete oid 39 current snap is 147 2024-09-15T03:51:30.976 INFO:tasks.rados.rados.0.smithi122.stdout:1331: finishing write tid 1 to smithi12223906-27 2024-09-15T03:51:30.976 INFO:tasks.rados.rados.0.smithi122.stdout:1330: expect (ObjNum 463 snap 136 seq_num 463) 2024-09-15T03:51:31.200 INFO:tasks.rados.rados.0.smithi122.stdout:1331: finishing write tid 2 to smithi12223906-27 2024-09-15T03:51:31.200 INFO:tasks.rados.rados.0.smithi122.stdout:1331: finishing write tid 3 to smithi12223906-27 2024-09-15T03:51:31.201 INFO:tasks.rados.rados.0.smithi122.stdout:1331: finishing write tid 4 to smithi12223906-27 2024-09-15T03:51:31.201 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 652 (ObjNum 488 snap 147 seq_num 488) dirty exists 2024-09-15T03:51:31.201 INFO:tasks.rados.rados.0.smithi122.stdout:1331: left oid 27 (ObjNum 488 snap 147 seq_num 488) 2024-09-15T03:51:31.201 INFO:tasks.rados.rados.0.smithi122.stdout:1332: expect (ObjNum 470 snap 138 seq_num 470) 2024-09-15T03:51:31.414 INFO:tasks.rados.rados.0.smithi122.stdout:1330: done (3 left) 2024-09-15T03:51:31.415 INFO:tasks.rados.rados.0.smithi122.stdout:1331: done (2 left) 2024-09-15T03:51:31.415 INFO:tasks.rados.rados.0.smithi122.stdout:1332: done (1 left) 2024-09-15T03:51:31.415 INFO:tasks.rados.rados.0.smithi122.stdout:1333: done (0 left) 2024-09-15T03:51:31.415 INFO:tasks.rados.rados.0.smithi122.stdout:1334: copy_from oid 1 from oid 8 current snap is 147 2024-09-15T03:51:31.415 INFO:tasks.rados.rados.0.smithi122.stdout:1335: write oid 20 current snap is 147 2024-09-15T03:51:31.415 INFO:tasks.rados.rados.0.smithi122.stdout:1335: seq_num 490 ranges {635430=718602,2140412=95018} 2024-09-15T03:51:31.433 INFO:tasks.rados.rados.0.smithi122.stdout:1335: writing smithi12223906-20 from 635430 to 1354032 tid 1 2024-09-15T03:51:31.438 INFO:tasks.rados.rados.0.smithi122.stdout:1335: writing smithi12223906-20 from 2140412 to 2235430 tid 2 2024-09-15T03:51:31.438 INFO:tasks.rados.rados.0.smithi122.stdout:1336: snap_remove snap 142 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:1336: done (2 left) 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:1337: snap_create 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:1334: finishing copy_from racing read to smithi12223906-1 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:1334: finishing copy_from to smithi12223906-1 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 372 (ObjNum 460 snap 136 seq_num 460) dirty exists 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:1335: finishing write tid 1 to smithi12223906-20 2024-09-15T03:51:31.971 INFO:tasks.rados.rados.0.smithi122.stdout:1335: finishing write tid 2 to smithi12223906-20 2024-09-15T03:51:31.972 INFO:tasks.rados.rados.0.smithi122.stdout:1335: finishing write tid 3 to smithi12223906-20 2024-09-15T03:51:31.972 INFO:tasks.rados.rados.0.smithi122.stdout:1335: finishing write tid 4 to smithi12223906-20 2024-09-15T03:51:31.972 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 639 (ObjNum 490 snap 147 seq_num 490) dirty exists 2024-09-15T03:51:31.972 INFO:tasks.rados.rados.0.smithi122.stdout:1335: left oid 20 (ObjNum 490 snap 147 seq_num 490) 2024-09-15T03:51:32.980 INFO:tasks.rados.rados.0.smithi122.stdout:1334: done (2 left) 2024-09-15T03:51:32.980 INFO:tasks.rados.rados.0.smithi122.stdout:1335: done (1 left) 2024-09-15T03:51:32.980 INFO:tasks.rados.rados.0.smithi122.stdout:1337: done (0 left) 2024-09-15T03:51:32.980 INFO:tasks.rados.rados.0.smithi122.stdout:1338: delete oid 50 current snap is 148 2024-09-15T03:51:32.984 INFO:tasks.rados.rados.0.smithi122.stdout:1338: done (0 left) 2024-09-15T03:51:32.985 INFO:tasks.rados.rados.0.smithi122.stdout:1339: snap_remove snap 141 2024-09-15T03:51:33.993 INFO:tasks.rados.rados.0.smithi122.stdout:1339: done (0 left) 2024-09-15T03:51:33.994 INFO:tasks.rados.rados.0.smithi122.stdout:1340: snap_remove snap 145 2024-09-15T03:51:34.997 INFO:tasks.rados.rados.0.smithi122.stdout:1340: done (0 left) 2024-09-15T03:51:34.997 INFO:tasks.rados.rados.0.smithi122.stdout:1341: delete oid 22 current snap is 148 2024-09-15T03:51:35.004 INFO:tasks.rados.rados.0.smithi122.stdout:1341: done (0 left) 2024-09-15T03:51:35.004 INFO:tasks.rados.rados.0.smithi122.stdout:1342: snap_create 2024-09-15T03:51:36.011 INFO:tasks.rados.rados.0.smithi122.stdout:1342: done (0 left) 2024-09-15T03:51:36.011 INFO:tasks.rados.rados.0.smithi122.stdout:1343: read oid 16 snap 148 2024-09-15T03:51:36.011 INFO:tasks.rados.rados.0.smithi122.stdout:1343: expect deleted 2024-09-15T03:51:36.011 INFO:tasks.rados.rados.0.smithi122.stdout:1344: write oid 10 current snap is 149 2024-09-15T03:51:36.011 INFO:tasks.rados.rados.0.smithi122.stdout:1344: seq_num 493 ranges {685851=544349,1799505=486346} 2024-09-15T03:51:36.019 INFO:tasks.rados.rados.0.smithi122.stdout:1344: writing smithi12223906-10 from 685851 to 1230200 tid 1 2024-09-15T03:51:36.026 INFO:tasks.rados.rados.0.smithi122.stdout:1344: writing smithi12223906-10 from 1799505 to 2285851 tid 2 2024-09-15T03:51:36.027 INFO:tasks.rados.rados.0.smithi122.stdout:1345: write oid 21 current snap is 149 2024-09-15T03:51:36.027 INFO:tasks.rados.rados.0.smithi122.stdout:1345: seq_num 494 ranges {702658=619598,1857987=444671} 2024-09-15T03:51:36.034 INFO:tasks.rados.rados.0.smithi122.stdout:1345: writing smithi12223906-21 from 702658 to 1322256 tid 1 2024-09-15T03:51:36.040 INFO:tasks.rados.rados.0.smithi122.stdout:1345: writing smithi12223906-21 from 1857987 to 2302658 tid 2 2024-09-15T03:51:36.040 INFO:tasks.rados.rados.0.smithi122.stdout:1346: write oid 31 current snap is 149 2024-09-15T03:51:36.040 INFO:tasks.rados.rados.0.smithi122.stdout:1346: seq_num 495 ranges {719465=411200,1632822=686643} 2024-09-15T03:51:36.046 INFO:tasks.rados.rados.0.smithi122.stdout:1346: writing smithi12223906-31 from 719465 to 1130665 tid 1 2024-09-15T03:51:36.053 INFO:tasks.rados.rados.0.smithi122.stdout:1346: writing smithi12223906-31 from 1632822 to 2319465 tid 2 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1347: snap_create 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1344: finishing write tid 1 to smithi12223906-10 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1344: finishing write tid 2 to smithi12223906-10 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1344: finishing write tid 3 to smithi12223906-10 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1344: finishing write tid 4 to smithi12223906-10 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 394 (ObjNum 493 snap 149 seq_num 493) dirty exists 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1344: left oid 10 (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1345: finishing write tid 1 to smithi12223906-21 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1345: finishing write tid 2 to smithi12223906-21 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1345: finishing write tid 3 to smithi12223906-21 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1345: finishing write tid 4 to smithi12223906-21 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 397 (ObjNum 494 snap 149 seq_num 494) dirty exists 2024-09-15T03:51:36.054 INFO:tasks.rados.rados.0.smithi122.stdout:1345: left oid 21 (ObjNum 494 snap 149 seq_num 494) 2024-09-15T03:51:36.055 INFO:tasks.rados.rados.0.smithi122.stdout:1346: finishing write tid 1 to smithi12223906-31 2024-09-15T03:51:36.060 INFO:tasks.rados.rados.0.smithi122.stdout:1346: finishing write tid 2 to smithi12223906-31 2024-09-15T03:51:36.063 INFO:tasks.rados.rados.0.smithi122.stdout:1346: finishing write tid 3 to smithi12223906-31 2024-09-15T03:51:36.063 INFO:tasks.rados.rados.0.smithi122.stdout:1346: finishing write tid 4 to smithi12223906-31 2024-09-15T03:51:36.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 224 (ObjNum 495 snap 149 seq_num 495) dirty exists 2024-09-15T03:51:36.064 INFO:tasks.rados.rados.0.smithi122.stdout:1346: left oid 31 (ObjNum 495 snap 149 seq_num 495) 2024-09-15T03:51:36.823 INFO:tasks.rados.rados.0.smithi122.stdout:1343: done (4 left) 2024-09-15T03:51:36.823 INFO:tasks.rados.rados.0.smithi122.stdout:1344: done (3 left) 2024-09-15T03:51:36.823 INFO:tasks.rados.rados.0.smithi122.stdout:1345: done (2 left) 2024-09-15T03:51:36.823 INFO:tasks.rados.rados.0.smithi122.stdout:1346: done (1 left) 2024-09-15T03:51:36.824 INFO:tasks.rados.rados.0.smithi122.stdout:1347: done (0 left) 2024-09-15T03:51:36.824 INFO:tasks.rados.rados.0.smithi122.stdout:1348: write oid 4 current snap is 150 2024-09-15T03:51:36.824 INFO:tasks.rados.rados.0.smithi122.stdout:1348: seq_num 496 ranges {736272=486449,1974951=361321} 2024-09-15T03:51:36.831 INFO:tasks.rados.rados.0.smithi122.stdout:1348: writing smithi12223906-4 from 736272 to 1222721 tid 1 2024-09-15T03:51:36.838 INFO:tasks.rados.rados.0.smithi122.stdout:1348: writing smithi12223906-4 from 1974951 to 2336272 tid 2 2024-09-15T03:51:36.838 INFO:tasks.rados.rados.0.smithi122.stdout:1349: read oid 33 snap 148 2024-09-15T03:51:36.838 INFO:tasks.rados.rados.0.smithi122.stdout:1349: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:36.838 INFO:tasks.rados.rados.0.smithi122.stdout:1350: write (excl) oid 16 current snap is 150 2024-09-15T03:51:36.838 INFO:tasks.rados.rados.0.smithi122.stdout:1348: finishing write tid 1 to smithi12223906-4 2024-09-15T03:51:36.839 INFO:tasks.rados.rados.0.smithi122.stdout:1350: seq_num 497 ranges {753079=561698,2033433=319646} 2024-09-15T03:51:36.846 INFO:tasks.rados.rados.0.smithi122.stdout:1350: writing smithi12223906-16 from 753079 to 1314777 tid 1 2024-09-15T03:51:36.853 INFO:tasks.rados.rados.0.smithi122.stdout:1350: writing smithi12223906-16 from 2033433 to 2353079 tid 2 2024-09-15T03:51:36.853 INFO:tasks.rados.rados.0.smithi122.stdout:1351: copy_from oid 44 from oid 42 current snap is 150 2024-09-15T03:51:36.853 INFO:tasks.rados.rados.0.smithi122.stdout:1348: finishing write tid 2 to smithi12223906-4 2024-09-15T03:51:36.853 INFO:tasks.rados.rados.0.smithi122.stdout:1348: finishing write tid 3 to smithi12223906-4 2024-09-15T03:51:36.854 INFO:tasks.rados.rados.0.smithi122.stdout:1348: finishing write tid 4 to smithi12223906-4 2024-09-15T03:51:36.854 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 227 (ObjNum 496 snap 150 seq_num 496) dirty exists 2024-09-15T03:51:36.854 INFO:tasks.rados.rados.0.smithi122.stdout:1348: left oid 4 (ObjNum 496 snap 150 seq_num 496) 2024-09-15T03:51:36.854 INFO:tasks.rados.rados.0.smithi122.stdout:1349: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:37.035 INFO:tasks.rados.rados.0.smithi122.stdout:1350: finishing write tid 1 to smithi12223906-16 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1351: finishing copy_from to smithi12223906-44 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1351: got expected ENOENT (src dne) 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1351: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1350: finishing write tid 2 to smithi12223906-16 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1350: finishing write tid 3 to smithi12223906-16 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1350: finishing write tid 4 to smithi12223906-16 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 429 (ObjNum 497 snap 150 seq_num 497) dirty exists 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1350: left oid 16 (ObjNum 497 snap 150 seq_num 497) 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1348: done (3 left) 2024-09-15T03:51:37.036 INFO:tasks.rados.rados.0.smithi122.stdout:1349: done (2 left) 2024-09-15T03:51:37.037 INFO:tasks.rados.rados.0.smithi122.stdout:1350: done (1 left) 2024-09-15T03:51:37.037 INFO:tasks.rados.rados.0.smithi122.stdout:1351: done (0 left) 2024-09-15T03:51:37.037 INFO:tasks.rados.rados.0.smithi122.stdout:1352: snap_remove snap 149 2024-09-15T03:51:38.033 INFO:tasks.rados.rados.0.smithi122.stdout:1352: done (0 left) 2024-09-15T03:51:38.034 INFO:tasks.rados.rados.0.smithi122.stdout:1353: copy_from oid 30 from oid 28 current snap is 150 2024-09-15T03:51:38.034 INFO:tasks.rados.rados.0.smithi122.stdout:1354: copy_from oid 43 from oid 4 current snap is 150 2024-09-15T03:51:38.034 INFO:tasks.rados.rados.0.smithi122.stdout:1355: rollback oid 49 current snap is 150 2024-09-15T03:51:38.034 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 148 2024-09-15T03:51:38.034 INFO:tasks.rados.rados.0.smithi122.stdout:1356: snap_remove snap 147 2024-09-15T03:51:39.046 INFO:tasks.rados.rados.0.smithi122.stdout:1356: done (3 left) 2024-09-15T03:51:39.046 INFO:tasks.rados.rados.0.smithi122.stdout:1357: write (excl) oid 16 current snap is 150 2024-09-15T03:51:39.046 INFO:tasks.rados.rados.0.smithi122.stdout:1357: seq_num 498 ranges {769886=636947,2091915=277971} 2024-09-15T03:51:39.055 INFO:tasks.rados.rados.0.smithi122.stdout:1357: writing smithi12223906-16 from 769886 to 1406833 tid 1 2024-09-15T03:51:39.062 INFO:tasks.rados.rados.0.smithi122.stdout:1357: writing smithi12223906-16 from 2091915 to 2369886 tid 2 2024-09-15T03:51:39.062 INFO:tasks.rados.rados.0.smithi122.stdout:1358: delete oid 26 current snap is 150 2024-09-15T03:51:39.062 INFO:tasks.rados.rados.0.smithi122.stdout:1355: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:51:39.062 INFO:tasks.rados.rados.0.smithi122.stdout:1355: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:51:39.062 INFO:tasks.rados.rados.0.smithi122.stdout:1354: finishing copy_from racing read to smithi12223906-43 2024-09-15T03:51:39.063 INFO:tasks.rados.rados.0.smithi122.stdout:1355: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:51:39.063 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 363 (ObjNum 467 snap 136 seq_num 467) dirty exists 2024-09-15T03:51:39.063 INFO:tasks.rados.rados.0.smithi122.stdout:1354: finishing copy_from to smithi12223906-43 2024-09-15T03:51:39.063 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 227 (ObjNum 496 snap 150 seq_num 496) dirty exists 2024-09-15T03:51:39.063 INFO:tasks.rados.rados.0.smithi122.stdout:1353: finishing copy_from racing read to smithi12223906-30 2024-09-15T03:51:39.063 INFO:tasks.rados.rados.0.smithi122.stdout:1353: finishing copy_from to smithi12223906-30 2024-09-15T03:51:39.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 586 (ObjNum 455 snap 133 seq_num 455) dirty exists 2024-09-15T03:51:39.064 INFO:tasks.rados.rados.0.smithi122.stdout:1353: done (4 left) 2024-09-15T03:51:39.065 INFO:tasks.rados.rados.0.smithi122.stdout:1354: done (3 left) 2024-09-15T03:51:39.065 INFO:tasks.rados.rados.0.smithi122.stdout:1355: done (2 left) 2024-09-15T03:51:39.065 INFO:tasks.rados.rados.0.smithi122.stdout:1358: done (1 left) 2024-09-15T03:51:39.065 INFO:tasks.rados.rados.0.smithi122.stdout:1359: write oid 14 current snap is 150 2024-09-15T03:51:39.065 INFO:tasks.rados.rados.0.smithi122.stdout:1359: seq_num 500 ranges {403500=787445,1692526=473837} 2024-09-15T03:51:39.072 INFO:tasks.rados.rados.0.smithi122.stdout:1359: writing smithi12223906-14 from 403500 to 1190945 tid 1 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:1359: writing smithi12223906-14 from 1692526 to 2166363 tid 2 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:1360: snap_create 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:1357: finishing write tid 1 to smithi12223906-16 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:1357: finishing write tid 2 to smithi12223906-16 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:1357: finishing write tid 3 to smithi12223906-16 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:1357: finishing write tid 4 to smithi12223906-16 2024-09-15T03:51:39.078 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 432 (ObjNum 498 snap 150 seq_num 498) dirty exists 2024-09-15T03:51:39.079 INFO:tasks.rados.rados.0.smithi122.stdout:1357: left oid 16 (ObjNum 498 snap 150 seq_num 498) 2024-09-15T03:51:39.086 INFO:tasks.rados.rados.0.smithi122.stdout:1359: finishing write tid 1 to smithi12223906-14 2024-09-15T03:51:39.086 INFO:tasks.rados.rados.0.smithi122.stdout:1359: finishing write tid 2 to smithi12223906-14 2024-09-15T03:51:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:1359: finishing write tid 3 to smithi12223906-14 2024-09-15T03:51:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:1359: finishing write tid 4 to smithi12223906-14 2024-09-15T03:51:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 659 (ObjNum 500 snap 150 seq_num 500) dirty exists 2024-09-15T03:51:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:1359: left oid 14 (ObjNum 500 snap 150 seq_num 500) 2024-09-15T03:51:40.059 INFO:tasks.rados.rados.0.smithi122.stdout:1357: done (2 left) 2024-09-15T03:51:40.059 INFO:tasks.rados.rados.0.smithi122.stdout:1359: done (1 left) 2024-09-15T03:51:40.059 INFO:tasks.rados.rados.0.smithi122.stdout:1360: done (0 left) 2024-09-15T03:51:40.059 INFO:tasks.rados.rados.0.smithi122.stdout:1361: snap_remove snap 150 2024-09-15T03:51:41.064 INFO:tasks.rados.rados.0.smithi122.stdout:1361: done (0 left) 2024-09-15T03:51:41.064 INFO:tasks.rados.rados.0.smithi122.stdout:1362: write (excl) oid 17 current snap is 151 2024-09-15T03:51:41.065 INFO:tasks.rados.rados.0.smithi122.stdout:1362: seq_num 501 ranges {420307=462694,1351008=617495} 2024-09-15T03:51:41.069 INFO:tasks.rados.rados.0.smithi122.stdout:1362: writing smithi12223906-17 from 420307 to 883001 tid 1 2024-09-15T03:51:41.076 INFO:tasks.rados.rados.0.smithi122.stdout:1362: writing smithi12223906-17 from 1351008 to 1968503 tid 2 2024-09-15T03:51:41.076 INFO:tasks.rados.rados.0.smithi122.stdout:1363: write (excl) oid 8 current snap is 151 2024-09-15T03:51:41.077 INFO:tasks.rados.rados.0.smithi122.stdout:1362: finishing write tid 1 to smithi12223906-17 2024-09-15T03:51:41.077 INFO:tasks.rados.rados.0.smithi122.stdout:1363: seq_num 502 ranges {437114=654296,1525843=477506} 2024-09-15T03:51:41.082 INFO:tasks.rados.rados.0.smithi122.stdout:1363: writing smithi12223906-8 from 437114 to 1091410 tid 1 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1363: writing smithi12223906-8 from 1525843 to 2003349 tid 2 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1364: rollback oid 50 current snap is 151 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 148 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1362: finishing write tid 2 to smithi12223906-17 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1362: finishing write tid 3 to smithi12223906-17 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1362: finishing write tid 4 to smithi12223906-17 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 369 (ObjNum 501 snap 151 seq_num 501) dirty exists 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1362: left oid 17 (ObjNum 501 snap 151 seq_num 501) 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1362: done (2 left) 2024-09-15T03:51:41.088 INFO:tasks.rados.rados.0.smithi122.stdout:1365: read oid 13 snap -1 2024-09-15T03:51:41.089 INFO:tasks.rados.rados.0.smithi122.stdout:1365: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:51:41.089 INFO:tasks.rados.rados.0.smithi122.stdout:1366: write oid 41 current snap is 151 2024-09-15T03:51:41.089 INFO:tasks.rados.rados.0.smithi122.stdout:1366: seq_num 503 ranges {453921=729545,1584325=621164} 2024-09-15T03:51:41.094 INFO:tasks.rados.rados.0.smithi122.stdout:1366: writing smithi12223906-41 from 453921 to 1183466 tid 1 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:1366: writing smithi12223906-41 from 1584325 to 2205489 tid 2 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:1367: copy_from oid 16 from oid 33 current snap is 151 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:1364: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:1363: finishing write tid 1 to smithi12223906-8 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:1363: finishing write tid 2 to smithi12223906-8 2024-09-15T03:51:41.101 INFO:tasks.rados.rados.0.smithi122.stdout:1363: finishing write tid 3 to smithi12223906-8 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:1363: finishing write tid 4 to smithi12223906-8 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 379 (ObjNum 502 snap 151 seq_num 502) dirty exists 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:1363: left oid 8 (ObjNum 502 snap 151 seq_num 502) 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:1363: done (4 left) 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:1364: done (3 left) 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:1368: write oid 31 current snap is 151 2024-09-15T03:51:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:1368: seq_num 504 ranges {470728=404794,1526454=481175} 2024-09-15T03:51:41.107 INFO:tasks.rados.rados.0.smithi122.stdout:1368: writing smithi12223906-31 from 470728 to 875522 tid 1 2024-09-15T03:51:41.113 INFO:tasks.rados.rados.0.smithi122.stdout:1368: writing smithi12223906-31 from 1526454 to 2007629 tid 2 2024-09-15T03:51:41.114 INFO:tasks.rados.rados.0.smithi122.stdout:1369: write (excl) oid 32 current snap is 151 2024-09-15T03:51:41.114 INFO:tasks.rados.rados.0.smithi122.stdout:1369: seq_num 505 ranges {487535=480043,1584936=624833} 2024-09-15T03:51:41.119 INFO:tasks.rados.rados.0.smithi122.stdout:1369: writing smithi12223906-32 from 487535 to 967578 tid 1 2024-09-15T03:51:41.126 INFO:tasks.rados.rados.0.smithi122.stdout:1369: writing smithi12223906-32 from 1584936 to 2209769 tid 2 2024-09-15T03:51:41.126 INFO:tasks.rados.rados.0.smithi122.stdout:1370: read oid 34 snap -1 2024-09-15T03:51:41.126 INFO:tasks.rados.rados.0.smithi122.stdout:1370: expect deleted 2024-09-15T03:51:41.127 INFO:tasks.rados.rados.0.smithi122.stdout:1371: read oid 1 snap -1 2024-09-15T03:51:41.127 INFO:tasks.rados.rados.0.smithi122.stdout:1371: expect (ObjNum 460 snap 136 seq_num 460) 2024-09-15T03:51:41.127 INFO:tasks.rados.rados.0.smithi122.stdout:1372: read oid 5 snap -1 2024-09-15T03:51:41.127 INFO:tasks.rados.rados.0.smithi122.stdout:1372: expect deleted 2024-09-15T03:51:41.127 INFO:tasks.rados.rados.0.smithi122.stdout:1373: write oid 47 current snap is 151 2024-09-15T03:51:41.127 INFO:tasks.rados.rados.0.smithi122.stdout:1365: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:1366: finishing write tid 1 to smithi12223906-41 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:1366: finishing write tid 2 to smithi12223906-41 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:1366: finishing write tid 3 to smithi12223906-41 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:1366: finishing write tid 4 to smithi12223906-41 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 589 (ObjNum 503 snap 151 seq_num 503) dirty exists 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:1366: left oid 41 (ObjNum 503 snap 151 seq_num 503) 2024-09-15T03:51:41.356 INFO:tasks.rados.rados.0.smithi122.stdout:1368: finishing write tid 1 to smithi12223906-31 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:1367: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:1367: finishing copy_from to smithi12223906-16 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 567 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:1368: finishing write tid 2 to smithi12223906-31 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:1368: finishing write tid 3 to smithi12223906-31 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:1368: finishing write tid 4 to smithi12223906-31 2024-09-15T03:51:41.357 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 230 (ObjNum 504 snap 151 seq_num 504) dirty exists 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1368: left oid 31 (ObjNum 504 snap 151 seq_num 504) 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1369: finishing write tid 1 to smithi12223906-32 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1369: finishing write tid 2 to smithi12223906-32 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1369: finishing write tid 3 to smithi12223906-32 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1369: finishing write tid 4 to smithi12223906-32 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 674 (ObjNum 505 snap 151 seq_num 505) dirty exists 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1369: left oid 32 (ObjNum 505 snap 151 seq_num 505) 2024-09-15T03:51:41.358 INFO:tasks.rados.rados.0.smithi122.stdout:1371: expect (ObjNum 460 snap 136 seq_num 460) 2024-09-15T03:51:41.554 INFO:tasks.rados.rados.0.smithi122.stdout:1373: seq_num 506 ranges {504342=555292,1643418=484844} 2024-09-15T03:51:41.569 INFO:tasks.rados.rados.0.smithi122.stdout:1373: writing smithi12223906-47 from 504342 to 1059634 tid 1 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1373: writing smithi12223906-47 from 1643418 to 2128262 tid 2 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1365: done (8 left) 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1366: done (7 left) 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1367: done (6 left) 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1368: done (5 left) 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1369: done (4 left) 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1370: done (3 left) 2024-09-15T03:51:41.576 INFO:tasks.rados.rados.0.smithi122.stdout:1371: done (2 left) 2024-09-15T03:51:41.577 INFO:tasks.rados.rados.0.smithi122.stdout:1372: done (1 left) 2024-09-15T03:51:41.577 INFO:tasks.rados.rados.0.smithi122.stdout:1374: rollback oid 25 current snap is 151 2024-09-15T03:51:41.577 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 148 2024-09-15T03:51:41.577 INFO:tasks.rados.rados.0.smithi122.stdout:1375: write oid 2 current snap is 151 2024-09-15T03:51:41.577 INFO:tasks.rados.rados.0.smithi122.stdout:1375: seq_num 507 ranges {521149=630541,1701900=628502} 2024-09-15T03:51:41.582 INFO:tasks.rados.rados.0.smithi122.stdout:1375: writing smithi12223906-2 from 521149 to 1151690 tid 1 2024-09-15T03:51:41.589 INFO:tasks.rados.rados.0.smithi122.stdout:1375: writing smithi12223906-2 from 1701900 to 2330402 tid 2 2024-09-15T03:51:41.589 INFO:tasks.rados.rados.0.smithi122.stdout:1376: write (excl) oid 35 current snap is 151 2024-09-15T03:51:41.589 INFO:tasks.rados.rados.0.smithi122.stdout:1374: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:51:41.589 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-15T03:51:41.589 INFO:tasks.rados.rados.0.smithi122.stdout:1376: seq_num 508 ranges {537956=705790,1644029=772160} 2024-09-15T03:51:41.596 INFO:tasks.rados.rados.0.smithi122.stdout:1376: writing smithi12223906-35 from 537956 to 1243746 tid 1 2024-09-15T03:51:41.603 INFO:tasks.rados.rados.0.smithi122.stdout:1376: writing smithi12223906-35 from 1644029 to 2416189 tid 2 2024-09-15T03:51:41.603 INFO:tasks.rados.rados.0.smithi122.stdout:1374: done (3 left) 2024-09-15T03:51:41.603 INFO:tasks.rados.rados.0.smithi122.stdout:1377: snap_remove snap 148 2024-09-15T03:51:41.623 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T03:51:41.624 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T03:51:41.624 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T03:51:41.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T03:51:41.627 INFO:tasks.ceph.mon.c:Started 2024-09-15T03:51:41.627 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T03:51:41.627 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T03:51:41.627 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T03:51:41.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T03:51:41.634 INFO:tasks.ceph.mon.f:Started 2024-09-15T03:51:41.634 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:51:41.634 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T03:51:41.634 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T03:51:41.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T03:51:41.637 INFO:tasks.ceph.mon.g:Started 2024-09-15T03:51:41.637 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-15T03:51:41.637 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-15T03:51:41.637 INFO:tasks.mon_thrash:Sending STOP to mon e 2024-09-15T03:51:41.637 INFO:tasks.ceph.mon.e:Sent signal 19 2024-09-15T03:51:41.637 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:51:52.809 INFO:tasks.rados.rados.0.smithi122.stdout:1377: done (3 left) 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1378: rollback oid 22 current snap is 151 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1378: done (3 left) 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1379: read oid 39 snap -1 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1379: expect deleted 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1380: read oid 48 snap -1 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1380: expect (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1381: write (excl) oid 42 current snap is 151 2024-09-15T03:51:52.810 INFO:tasks.rados.rados.0.smithi122.stdout:1381: seq_num 509 ranges {554763=781039,2102511=452252} 2024-09-15T03:51:52.818 INFO:tasks.rados.rados.0.smithi122.stdout:1381: writing smithi12223906-42 from 554763 to 1335802 tid 1 2024-09-15T03:51:52.826 INFO:tasks.rados.rados.0.smithi122.stdout:1381: writing smithi12223906-42 from 2102511 to 2554763 tid 2 2024-09-15T03:51:52.826 INFO:tasks.rados.rados.0.smithi122.stdout:1382: read oid 25 snap -1 2024-09-15T03:51:52.826 INFO:tasks.rados.rados.0.smithi122.stdout:1382: expect deleted 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1383: read oid 16 snap -1 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1383: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1384: copy_from oid 9 from oid 6 current snap is 151 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1373: finishing write tid 1 to smithi12223906-47 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1373: finishing write tid 2 to smithi12223906-47 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1373: finishing write tid 3 to smithi12223906-47 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1373: finishing write tid 4 to smithi12223906-47 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 206 (ObjNum 506 snap 151 seq_num 506) dirty exists 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1373: left oid 47 (ObjNum 506 snap 151 seq_num 506) 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1373: done (8 left) 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1385: copy_from oid 8 from oid 15 current snap is 151 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1375: finishing write tid 1 to smithi12223906-2 2024-09-15T03:51:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:1375: finishing write tid 2 to smithi12223906-2 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1375: finishing write tid 3 to smithi12223906-2 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1375: finishing write tid 4 to smithi12223906-2 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 373 (ObjNum 507 snap 151 seq_num 507) dirty exists 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1375: left oid 2 (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1376: finishing write tid 1 to smithi12223906-35 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1376: finishing write tid 2 to smithi12223906-35 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1376: finishing write tid 3 to smithi12223906-35 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1376: finishing write tid 4 to smithi12223906-35 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 557 (ObjNum 508 snap 151 seq_num 508) dirty exists 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1376: left oid 35 (ObjNum 508 snap 151 seq_num 508) 2024-09-15T03:51:52.828 INFO:tasks.rados.rados.0.smithi122.stdout:1380: expect (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:51:53.012 INFO:tasks.rados.rados.0.smithi122.stdout:1381: finishing write tid 1 to smithi12223906-42 2024-09-15T03:51:53.012 INFO:tasks.rados.rados.0.smithi122.stdout:1381: finishing write tid 2 to smithi12223906-42 2024-09-15T03:51:53.013 INFO:tasks.rados.rados.0.smithi122.stdout:1381: finishing write tid 3 to smithi12223906-42 2024-09-15T03:51:53.013 INFO:tasks.rados.rados.0.smithi122.stdout:1381: finishing write tid 4 to smithi12223906-42 2024-09-15T03:51:53.013 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 665 (ObjNum 509 snap 151 seq_num 509) dirty exists 2024-09-15T03:51:53.013 INFO:tasks.rados.rados.0.smithi122.stdout:1381: left oid 42 (ObjNum 509 snap 151 seq_num 509) 2024-09-15T03:51:53.013 INFO:tasks.rados.rados.0.smithi122.stdout:1383: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:51:53.185 INFO:tasks.rados.rados.0.smithi122.stdout:1384: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1384: finishing copy_from to smithi12223906-9 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 303 (ObjNum 464 snap 136 seq_num 464) dirty exists 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1385: finishing copy_from racing read to smithi12223906-8 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1385: finishing copy_from to smithi12223906-8 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 396 (ObjNum 474 snap 141 seq_num 474) dirty exists 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1375: done (8 left) 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1376: done (7 left) 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1379: done (6 left) 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1380: done (5 left) 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1381: done (4 left) 2024-09-15T03:51:53.186 INFO:tasks.rados.rados.0.smithi122.stdout:1382: done (3 left) 2024-09-15T03:51:53.187 INFO:tasks.rados.rados.0.smithi122.stdout:1383: done (2 left) 2024-09-15T03:51:53.187 INFO:tasks.rados.rados.0.smithi122.stdout:1384: done (1 left) 2024-09-15T03:51:53.187 INFO:tasks.rados.rados.0.smithi122.stdout:1385: done (0 left) 2024-09-15T03:51:53.187 INFO:tasks.rados.rados.0.smithi122.stdout:1386: snap_create 2024-09-15T03:51:53.821 INFO:tasks.rados.rados.0.smithi122.stdout:1386: done (0 left) 2024-09-15T03:51:53.822 INFO:tasks.rados.rados.0.smithi122.stdout:1387: read oid 14 snap -1 2024-09-15T03:51:53.822 INFO:tasks.rados.rados.0.smithi122.stdout:1387: expect (ObjNum 500 snap 150 seq_num 500) 2024-09-15T03:51:53.822 INFO:tasks.rados.rados.0.smithi122.stdout:1388: rollback oid 21 current snap is 152 2024-09-15T03:51:53.822 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 151 2024-09-15T03:51:53.822 INFO:tasks.rados.rados.0.smithi122.stdout:1389: delete oid 27 current snap is 152 2024-09-15T03:51:53.828 INFO:tasks.rados.rados.0.smithi122.stdout:1388: finishing rollback tid 0 to smithi12223906-21 2024-09-15T03:51:53.828 INFO:tasks.rados.rados.0.smithi122.stdout:1388: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:1389: done (2 left) 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:1390: rollback oid 45 current snap is 152 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 151 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:1391: copy_from oid 46 from oid 43 current snap is 152 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:1392: delete oid 17 current snap is 152 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:1388: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:51:53.829 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 400 (ObjNum 494 snap 149 seq_num 494) dirty exists 2024-09-15T03:51:53.834 INFO:tasks.rados.rados.0.smithi122.stdout:1388: done (4 left) 2024-09-15T03:51:53.834 INFO:tasks.rados.rados.0.smithi122.stdout:1392: done (3 left) 2024-09-15T03:51:53.834 INFO:tasks.rados.rados.0.smithi122.stdout:1393: write oid 42 current snap is 152 2024-09-15T03:51:53.834 INFO:tasks.rados.rados.0.smithi122.stdout:1393: seq_num 512 ranges {605184=723139,1877957=727227} 2024-09-15T03:51:53.846 INFO:tasks.rados.rados.0.smithi122.stdout:1393: writing smithi12223906-42 from 605184 to 1328323 tid 1 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1393: writing smithi12223906-42 from 1877957 to 2605184 tid 2 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1394: snap_create 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1390: finishing rollback tid 0 to smithi12223906-45 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1390: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1390: finishing rollback tid 2 to smithi12223906-45 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 392 (ObjNum 394 snap 116 seq_num 394) dirty exists 2024-09-15T03:51:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1387: expect (ObjNum 500 snap 150 seq_num 500) 2024-09-15T03:51:54.040 INFO:tasks.rados.rados.0.smithi122.stdout:1391: finishing copy_from racing read to smithi12223906-46 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:1391: finishing copy_from to smithi12223906-46 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 227 (ObjNum 496 snap 150 seq_num 496) dirty exists 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:1393: finishing write tid 1 to smithi12223906-42 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:1393: finishing write tid 2 to smithi12223906-42 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:1393: finishing write tid 3 to smithi12223906-42 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:1393: finishing write tid 4 to smithi12223906-42 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 669 (ObjNum 512 snap 152 seq_num 512) dirty exists 2024-09-15T03:51:54.041 INFO:tasks.rados.rados.0.smithi122.stdout:1393: left oid 42 (ObjNum 512 snap 152 seq_num 512) 2024-09-15T03:51:54.829 INFO:tasks.rados.rados.0.smithi122.stdout:1387: done (4 left) 2024-09-15T03:51:54.829 INFO:tasks.rados.rados.0.smithi122.stdout:1390: done (3 left) 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1391: done (2 left) 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1393: done (1 left) 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1394: done (0 left) 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1395: rollback oid 50 current snap is 153 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 151 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1396: read oid 3 snap 151 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1396: expect (ObjNum 450 snap 130 seq_num 450) 2024-09-15T03:51:54.830 INFO:tasks.rados.rados.0.smithi122.stdout:1397: snap_create 2024-09-15T03:51:54.831 INFO:tasks.rados.rados.0.smithi122.stdout:1395: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:51:54.831 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 0 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T03:51:54.842 INFO:tasks.rados.rados.0.smithi122.stdout:1396: expect (ObjNum 450 snap 130 seq_num 450) 2024-09-15T03:51:55.836 INFO:tasks.rados.rados.0.smithi122.stdout:1395: done (2 left) 2024-09-15T03:51:55.836 INFO:tasks.rados.rados.0.smithi122.stdout:1396: done (1 left) 2024-09-15T03:51:55.836 INFO:tasks.rados.rados.0.smithi122.stdout:1397: done (0 left) 2024-09-15T03:51:55.836 INFO:tasks.rados.rados.0.smithi122.stdout:1398: read oid 6 snap -1 2024-09-15T03:51:55.836 INFO:tasks.rados.rados.0.smithi122.stdout:1398: expect (ObjNum 464 snap 136 seq_num 464) 2024-09-15T03:51:55.836 INFO:tasks.rados.rados.0.smithi122.stdout:1399: delete oid 37 current snap is 154 2024-09-15T03:51:55.842 INFO:tasks.rados.rados.0.smithi122.stdout:1399: done (1 left) 2024-09-15T03:51:55.842 INFO:tasks.rados.rados.0.smithi122.stdout:1400: write oid 24 current snap is 154 2024-09-15T03:51:55.842 INFO:tasks.rados.rados.0.smithi122.stdout:1400: seq_num 514 ranges {638798=473637,1594921=783167} 2024-09-15T03:51:55.854 INFO:tasks.rados.rados.0.smithi122.stdout:1400: writing smithi12223906-24 from 638798 to 1112435 tid 1 2024-09-15T03:51:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:1400: writing smithi12223906-24 from 1594921 to 2378088 tid 2 2024-09-15T03:51:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:1401: read oid 48 snap -1 2024-09-15T03:51:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:1401: expect (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:51:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:1402: write oid 20 current snap is 154 2024-09-15T03:51:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:1402: seq_num 515 ranges {655605=548886,1653403=643178} 2024-09-15T03:51:55.869 INFO:tasks.rados.rados.0.smithi122.stdout:1402: writing smithi12223906-20 from 655605 to 1204491 tid 1 2024-09-15T03:51:55.875 INFO:tasks.rados.rados.0.smithi122.stdout:1402: writing smithi12223906-20 from 1653403 to 2296581 tid 2 2024-09-15T03:51:55.875 INFO:tasks.rados.rados.0.smithi122.stdout:1403: write oid 3 current snap is 154 2024-09-15T03:51:55.876 INFO:tasks.rados.rados.0.smithi122.stdout:1403: seq_num 516 ranges {672412=624135,1995532=676880} 2024-09-15T03:51:55.883 INFO:tasks.rados.rados.0.smithi122.stdout:1403: writing smithi12223906-3 from 672412 to 1296547 tid 1 2024-09-15T03:51:55.891 INFO:tasks.rados.rados.0.smithi122.stdout:1403: writing smithi12223906-3 from 1995532 to 2672412 tid 2 2024-09-15T03:51:55.891 INFO:tasks.rados.rados.0.smithi122.stdout:1404: write oid 11 current snap is 154 2024-09-15T03:51:55.891 INFO:tasks.rados.rados.0.smithi122.stdout:1404: seq_num 517 ranges {689219=415737,1770367=646847} 2024-09-15T03:51:55.897 INFO:tasks.rados.rados.0.smithi122.stdout:1404: writing smithi12223906-11 from 689219 to 1104956 tid 1 2024-09-15T03:51:55.905 INFO:tasks.rados.rados.0.smithi122.stdout:1404: writing smithi12223906-11 from 1770367 to 2417214 tid 2 2024-09-15T03:51:55.905 INFO:tasks.rados.rados.0.smithi122.stdout:1405: snap_create 2024-09-15T03:51:55.905 INFO:tasks.rados.rados.0.smithi122.stdout:1401: expect (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:51:56.100 INFO:tasks.rados.rados.0.smithi122.stdout:1398: expect (ObjNum 464 snap 136 seq_num 464) 2024-09-15T03:51:56.339 INFO:tasks.rados.rados.0.smithi122.stdout:1404: finishing write tid 1 to smithi12223906-11 2024-09-15T03:51:56.339 INFO:tasks.rados.rados.0.smithi122.stdout:1402: finishing write tid 1 to smithi12223906-20 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1402: finishing write tid 2 to smithi12223906-20 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1402: finishing write tid 3 to smithi12223906-20 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1402: finishing write tid 4 to smithi12223906-20 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 642 (ObjNum 515 snap 154 seq_num 515) dirty exists 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1402: left oid 20 (ObjNum 515 snap 154 seq_num 515) 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1404: finishing write tid 2 to smithi12223906-11 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1404: finishing write tid 3 to smithi12223906-11 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:1404: finishing write tid 4 to smithi12223906-11 2024-09-15T03:51:56.340 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 570 (ObjNum 517 snap 154 seq_num 517) dirty exists 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1404: left oid 11 (ObjNum 517 snap 154 seq_num 517) 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1400: finishing write tid 1 to smithi12223906-24 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1400: finishing write tid 2 to smithi12223906-24 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1400: finishing write tid 3 to smithi12223906-24 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1400: finishing write tid 4 to smithi12223906-24 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 673 (ObjNum 514 snap 154 seq_num 514) dirty exists 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1400: left oid 24 (ObjNum 514 snap 154 seq_num 514) 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1403: finishing write tid 1 to smithi12223906-3 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1403: finishing write tid 2 to smithi12223906-3 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1403: finishing write tid 3 to smithi12223906-3 2024-09-15T03:51:56.341 INFO:tasks.rados.rados.0.smithi122.stdout:1403: finishing write tid 4 to smithi12223906-3 2024-09-15T03:51:56.342 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 677 (ObjNum 516 snap 154 seq_num 516) dirty exists 2024-09-15T03:51:56.342 INFO:tasks.rados.rados.0.smithi122.stdout:1403: left oid 3 (ObjNum 516 snap 154 seq_num 516) 2024-09-15T03:51:56.638 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-15T03:51:56.638 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-15T03:51:56.638 INFO:tasks.mon_thrash:Sending CONT to mon e 2024-09-15T03:51:56.639 INFO:tasks.ceph.mon.e:Sent signal 18 2024-09-15T03:51:56.639 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:51:56.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:51:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: get_monmap_and_config 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: build_initial_monmap 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: monmap: 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:56.748334+0000 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:56.748334+0000 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:56.751 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 1 Processor -- start 2024-09-15T03:51:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 1 -- start start 2024-09-15T03:51:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: init 2024-09-15T03:51:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: _add_conns ranks=[4,8,5,0,6,7,3,1,2] 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient(hunting): picked mon.noname-c con 0x7f1cc80a6a80 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient(hunting): picked mon.noname-e con 0x7f1cc809fb80 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient(hunting): picked mon.noname-h con 0x7f1cc8138d30 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc804ca10 con 0x7f1cc80a6a80 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80a0f50 con 0x7f1cc8138d30 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc8133680 con 0x7f1cc809fb80 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient(hunting): authenticate will time out at 1261.178282s 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1ccdb33700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f1cc80a6a80 0x7f1cc8132f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55310/0 (socket says 172.21.15.26:55310) 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1ccdb33700 1 -- 172.21.15.26:0/3029967601 learned_addr learned my addr 172.21.15.26:0/3029967601 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:56.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198370272 0 0) 0x7f1cc804ca10 con 0x7f1cc80a6a80 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): my global_id is 184108 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb4003570 con 0x7f1cc80a6a80 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561924904 0 0) 0x7f1cc8133680 con 0x7f1cc809fb80 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): my global_id is 134576 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc804ca10 con 0x7f1cc809fb80 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187860751 0 0) 0x7f1cc80a0f50 con 0x7f1cc8138d30 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient(hunting): my global_id is 5090 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc8133680 con 0x7f1cc8138d30 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2927409085 0 0) 0x7f1cb4003570 con 0x7f1cc80a6a80 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 >> v1:172.21.15.26:6793/0 conn(0x7f1cc809fb80 legacy=0x7f1cc809ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 >> v1:172.21.15.122:6791/0 conn(0x7f1cc8138d30 legacy=0x7f1cc813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: _finish_hunting 0 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 monclient: found mon.noname-c 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80a6e50 con 0x7f1cc80a6a80 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: _finish_auth 0 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 5 monclient: authenticate success, global_id 184108 2024-09-15T03:51:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cc40017b0 con 0x7f1cc80a6a80 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: dump: 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1cc4001a90 con 0x7f1cc80a6a80 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cceb35700 4 set_mon_vals no callback set 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 1 -- 172.21.15.26:0/3029967601 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cc4004c30 con 0x7f1cc80a6a80 2024-09-15T03:51:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.744+0000 7f1cccb31700 10 monclient: dump: 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient: shutdown 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- 172.21.15.26:0/3029967601 >> v1:172.21.15.26:6791/0 conn(0x7f1cc80a6a80 legacy=0x7f1cc8132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- 172.21.15.26:0/3029967601 shutdown_connections 2024-09-15T03:51:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- 172.21.15.26:0/3029967601 >> 172.21.15.26:0/3029967601 conn(0x7f1cc806e380 msgr2=0x7f1cc80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- 172.21.15.26:0/3029967601 shutdown_connections 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- 172.21.15.26:0/3029967601 wait complete. 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient: build_initial_monmap 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient: monmap: 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 Processor -- start 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- start start 2024-09-15T03:51:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient: init 2024-09-15T03:51:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient: _add_conns ranks=[8,5,3,2,7,1,6,0,4] 2024-09-15T03:51:56.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient(hunting): picked mon.noname-e con 0x7f1cc809fb80 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient(hunting): picked mon.noname-h con 0x7f1cc80a6a80 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient(hunting): picked mon.noname-g con 0x7f1cc8138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80a1920 con 0x7f1cc8138d30 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc811c9e0 con 0x7f1cc80a6a80 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc809cfc0 con 0x7f1cc809fb80 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cceb35700 10 monclient(hunting): authenticate will time out at 1261.182969s 2024-09-15T03:51:56.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1ccdb33700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f1cc809fb80 0x7f1cc8142190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:55484/0 (socket says 172.21.15.26:55484) 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1ccdb33700 1 -- 172.21.15.26:0/2324561592 learned_addr learned my addr 172.21.15.26:0/2324561592 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656816770 0 0) 0x7f1cc809cfc0 con 0x7f1cc809fb80 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): my global_id is 134585 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cac003570 con 0x7f1cc809fb80 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256267392 0 0) 0x7f1cc811c9e0 con 0x7f1cc80a6a80 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): my global_id is 5099 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc809cfc0 con 0x7f1cc80a6a80 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756786640 0 0) 0x7f1cc80a1920 con 0x7f1cc8138d30 2024-09-15T03:51:56.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 10 monclient(hunting): my global_id is 184107 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.748+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc811c9e0 con 0x7f1cc8138d30 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4194318511 0 0) 0x7f1cac003570 con 0x7f1cc809fb80 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1cc80a1920 con 0x7f1cc809fb80 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cc4004b70 con 0x7f1cc809fb80 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): dump: 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.775 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:56.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3757705499 0 0) 0x7f1cc809cfc0 con 0x7f1cc80a6a80 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1cac003570 con 0x7f1cc80a6a80 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cb8001600 con 0x7f1cc80a6a80 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient(hunting): dump: 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:56.777 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 717123073 0 0) 0x7f1cc80a1920 con 0x7f1cc809fb80 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 >> v1:172.21.15.122:6791/0 conn(0x7f1cc80a6a80 legacy=0x7f1cc81428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 >> v1:172.21.15.122:6790/0 conn(0x7f1cc8138d30 legacy=0x7f1cc814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: _finish_hunting 0 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 monclient: found mon.e 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:56.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8142fb0 con 0x7f1cc809fb80 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: _finish_auth 0 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 5 monclient: authenticate success, global_id 134585 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 monclient: _renew_subs 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc811cfa0 con 0x7f1cc809fb80 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 monclient: _renew_subs 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc809b7a0 con 0x7f1cc809fb80 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1cc4004f00 con 0x7f1cc809fb80 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:56.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cc40017a0 con 0x7f1cc809fb80 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbffff700 4 set_mon_vals no callback set 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 10 monclient: dump: 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:56.780 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1cc400fc10 con 0x7f1cc809fb80 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca80050b0 con 0x7f1cc809fb80 2024-09-15T03:51:56.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(328..328 src has 224..328) v4 ==== 3959+0+0 (unknown 3140179720 0 0) 0x7f1cc4044130 con 0x7f1cc809fb80 2024-09-15T03:51:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.752+0000 7f1ccd332700 10 monclient: get_auth_request con 0x7f1cac037c40 auth_method 0 2024-09-15T03:51:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.760+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f1cc4015050 con 0x7f1cc809fb80 2024-09-15T03:51:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.760+0000 7f1cbdffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:56.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:56.760+0000 7f1cbdffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:56.826 INFO:tasks.rados.rados.0.smithi122.stdout:1398: done (6 left) 2024-09-15T03:51:56.826 INFO:tasks.rados.rados.0.smithi122.stdout:1400: done (5 left) 2024-09-15T03:51:56.826 INFO:tasks.rados.rados.0.smithi122.stdout:1401: done (4 left) 2024-09-15T03:51:56.826 INFO:tasks.rados.rados.0.smithi122.stdout:1402: done (3 left) 2024-09-15T03:51:56.826 INFO:tasks.rados.rados.0.smithi122.stdout:1403: done (2 left) 2024-09-15T03:51:56.826 INFO:tasks.rados.rados.0.smithi122.stdout:1404: done (1 left) 2024-09-15T03:51:56.827 INFO:tasks.rados.rados.0.smithi122.stdout:1405: done (0 left) 2024-09-15T03:51:56.827 INFO:tasks.rados.rados.0.smithi122.stdout:1406: rollback oid 35 current snap is 155 2024-09-15T03:51:56.827 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 152 2024-09-15T03:51:56.827 INFO:tasks.rados.rados.0.smithi122.stdout:1407: write (excl) oid 12 current snap is 155 2024-09-15T03:51:56.827 INFO:tasks.rados.rados.0.smithi122.stdout:1407: seq_num 518 ranges {706026=490986,1828849=790505} 2024-09-15T03:51:56.832 INFO:tasks.rados.rados.0.smithi122.stdout:1407: writing smithi12223906-12 from 706026 to 1197012 tid 1 2024-09-15T03:51:56.841 INFO:tasks.rados.rados.0.smithi122.stdout:1407: writing smithi12223906-12 from 1828849 to 2619354 tid 2 2024-09-15T03:51:56.841 INFO:tasks.rados.rados.0.smithi122.stdout:1408: snap_create 2024-09-15T03:51:56.841 INFO:tasks.rados.rados.0.smithi122.stdout:1406: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:51:56.841 INFO:tasks.rados.rados.0.smithi122.stdout:1406: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:51:56.842 INFO:tasks.rados.rados.0.smithi122.stdout:1406: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:51:56.842 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 560 (ObjNum 508 snap 151 seq_num 508) dirty exists 2024-09-15T03:51:56.842 INFO:tasks.rados.rados.0.smithi122.stdout:1407: finishing write tid 1 to smithi12223906-12 2024-09-15T03:51:56.845 INFO:tasks.rados.rados.0.smithi122.stdout:1407: finishing write tid 2 to smithi12223906-12 2024-09-15T03:51:56.845 INFO:tasks.rados.rados.0.smithi122.stdout:1407: finishing write tid 3 to smithi12223906-12 2024-09-15T03:51:56.846 INFO:tasks.rados.rados.0.smithi122.stdout:1407: finishing write tid 4 to smithi12223906-12 2024-09-15T03:51:56.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 362 (ObjNum 518 snap 155 seq_num 518) dirty exists 2024-09-15T03:51:56.846 INFO:tasks.rados.rados.0.smithi122.stdout:1407: left oid 12 (ObjNum 518 snap 155 seq_num 518) 2024-09-15T03:51:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.000+0000 7f1cceb35700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:51:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.000+0000 7f1cceb35700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:51:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.000+0000 7f1cceb35700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.000+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1ca80058e0 con 0x7f1cc809fb80 2024-09-15T03:51:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cbdffb700 1 -- 172.21.15.26:0/2324561592 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 1305240910) 0x7f1cc4014a50 con 0x7f1cc809fb80 2024-09-15T03:51:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cbdffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:51:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cbdffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:57.010 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:51:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 >> v1:172.21.15.122:6812/20895 conn(0x7f1cac037c40 legacy=0x7f1cac03a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 10 monclient: shutdown 2024-09-15T03:51:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 >> v1:172.21.15.26:6793/0 conn(0x7f1cc809fb80 legacy=0x7f1cc8142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 shutdown_connections 2024-09-15T03:51:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 >> 172.21.15.26:0/2324561592 conn(0x7f1cc806e380 msgr2=0x7f1cc80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 shutdown_connections 2024-09-15T03:51:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.004+0000 7f1cceb35700 1 -- 172.21.15.26:0/2324561592 wait complete. 2024-09-15T03:51:57.024 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:51:57.024 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:51:57.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:51:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient: get_monmap_and_config 2024-09-15T03:51:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient: monmap: 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:57.125107+0000 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:57.125107+0000 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.128 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.128 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.128 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.128 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 1 Processor -- start 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 1 -- start start 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient: init 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient: _add_conns ranks=[6,1,0,8,5,2,7,3,4] 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient(hunting): picked mon.noname-d con 0x7ff044063340 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient(hunting): picked mon.noname-f con 0x7ff044063a10 addr v1:172.21.15.122:6789/0 2024-09-15T03:51:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient(hunting): picked mon.noname-a con 0x7ff044133110 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04404ca10 con 0x7ff044133110 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0440a0ad0 con 0x7ff044063a10 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04409f4f0 con 0x7ff044063340 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff04b30d700 10 monclient(hunting): authenticate will time out at 1261.555422s 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff0498aa700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff044133110 0x7ff04413b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38180/0 (socket says 172.21.15.26:38180) 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff0498aa700 1 -- 172.21.15.26:0/3923752270 learned_addr learned my addr 172.21.15.26:0/3923752270 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.120+0000 7ff0490a9700 1 --1- 172.21.15.26:0/3923752270 >> v1:172.21.15.26:6792/0 conn(0x7ff044063340 0x7ff0440991e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53220/0 (socket says 172.21.15.26:53220) 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363361584 0 0) 0x7ff04404ca10 con 0x7ff044133110 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): my global_id is 4932 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff030003570 con 0x7ff044133110 2024-09-15T03:51:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188962079 0 0) 0x7ff04409f4f0 con 0x7ff044063340 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): my global_id is 5136 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04404ca10 con 0x7ff044063340 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011380950 0 0) 0x7ff0440a0ad0 con 0x7ff044063a10 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient(hunting): my global_id is 184105 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04409f4f0 con 0x7ff044063a10 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317561822 0 0) 0x7ff030003570 con 0x7ff044133110 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 >> v1:172.21.15.26:6792/0 conn(0x7ff044063340 legacy=0x7ff0440991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 >> v1:172.21.15.122:6789/0 conn(0x7ff044063a10 legacy=0x7ff044137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 monclient: found mon.noname-a 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044063710 con 0x7ff044133110 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 5 monclient: authenticate success, global_id 4932 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff03c001b30 con 0x7ff044133110 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: dump: 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff03c001d20 con 0x7ff044133110 2024-09-15T03:51:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 1 -- 172.21.15.26:0/3923752270 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff03c004c70 con 0x7ff044133110 2024-09-15T03:51:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 4 set_mon_vals no callback set 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff03bfff700 10 monclient: dump: 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: shutdown 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- 172.21.15.26:0/3923752270 >> v1:172.21.15.26:6789/0 conn(0x7ff044133110 legacy=0x7ff04413b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- 172.21.15.26:0/3923752270 shutdown_connections 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- 172.21.15.26:0/3923752270 >> 172.21.15.26:0/3923752270 conn(0x7ff0440a3cb0 msgr2=0x7ff04409fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- 172.21.15.26:0/3923752270 shutdown_connections 2024-09-15T03:51:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- 172.21.15.26:0/3923752270 wait complete. 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: monmap: 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 Processor -- start 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- start start 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: init 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient: _add_conns ranks=[2,0,5,4,6,7,8,3,1] 2024-09-15T03:51:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient(hunting): picked mon.noname-b con 0x7ff044063340 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient(hunting): picked mon.noname-a con 0x7ff044063a10 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient(hunting): picked mon.noname-h con 0x7ff044133110 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff044120cc0 con 0x7ff044063a10 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04409ecb0 con 0x7ff044063340 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff044099f20 con 0x7ff044133110 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.124+0000 7ff04b30d700 10 monclient(hunting): authenticate will time out at 1261.559923s 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0490a9700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ff044063340 0x7ff044141d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39500/0 (socket says 172.21.15.26:39500) 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0490a9700 1 -- 172.21.15.26:0/2404277905 learned_addr learned my addr 172.21.15.26:0/2404277905 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621681513 0 0) 0x7ff04409ecb0 con 0x7ff044063340 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): my global_id is 134588 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff028003570 con 0x7ff044063340 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507548863 0 0) 0x7ff044120cc0 con 0x7ff044063a10 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): my global_id is 4941 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04409ecb0 con 0x7ff044063a10 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 830916582 0 0) 0x7ff028003570 con 0x7ff044063340 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff044120cc0 con 0x7ff044063340 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124998225 0 0) 0x7ff044099f20 con 0x7ff044133110 2024-09-15T03:51:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): my global_id is 5108 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff028003570 con 0x7ff044133110 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff0400015d0 con 0x7ff044063340 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): dump: 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1147799401 0 0) 0x7ff04409ecb0 con 0x7ff044063a10 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff044099f20 con 0x7ff044063a10 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff034001a60 con 0x7ff044063a10 2024-09-15T03:51:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): dump: 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1300310498 0 0) 0x7ff044120cc0 con 0x7ff044063340 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.122:6791/0 conn(0x7ff044133110 legacy=0x7ff044149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.26:6789/0 conn(0x7ff044063a10 legacy=0x7ff044142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 monclient: found mon.b 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04411f9c0 con 0x7ff044063340 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 5 monclient: authenticate success, global_id 134588 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 10 monclient: _renew_subs 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff044142b80 con 0x7ff044063340 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 10 monclient: _renew_subs 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff044122030 con 0x7ff044063340 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff040004ae0 con 0x7ff044063340 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff040004f90 con 0x7ff044063340 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff03b7fe700 4 set_mon_vals no callback set 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: dump: 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff024006590 con 0x7ff024002840 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126683878 0 0) 0x7ff024006590 con 0x7ff024002840 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient(hunting): my global_id is 4950 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff044120cc0 con 0x7ff024002840 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff04000f950 con 0x7ff044063340 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(329..329 src has 224..329) v4 ==== 3987+0+0 (unknown 3959154610 0 0) 0x7ff040043fd0 con 0x7ff044063340 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2856834437 0 0) 0x7ff044120cc0 con 0x7ff024002840 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff024006590 con 0x7ff024002840 2024-09-15T03:51:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff0340021b0 con 0x7ff024002840 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.128+0000 7ff0397fa700 10 monclient: dump: 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0498aa700 10 monclient: get_auth_request con 0x7ff028037d20 auth_method 0 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3333858278 0 0) 0x7ff024006590 con 0x7ff024002840 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff02400a130 con 0x7ff024002840 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff034004450 con 0x7ff024002840 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.26:6789/0 conn(0x7ff024002840 legacy=0x7ff024004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.132+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.26:6789/0 conn(0x7ff024002840 legacy=0x7ff024004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff04b30d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff024009640 con 0x7ff0240090b0 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354941487 0 0) 0x7ff024009640 con 0x7ff0240090b0 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 10 monclient(hunting): my global_id is 4959 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff024006590 con 0x7ff0240090b0 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3597423203 0 0) 0x7ff024006590 con 0x7ff0240090b0 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff024009640 con 0x7ff0240090b0 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff034002c40 con 0x7ff0240090b0 2024-09-15T03:51:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 10 monclient: dump: 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4242929839 0 0) 0x7ff024009640 con 0x7ff0240090b0 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.140+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff0240098f0 con 0x7ff0240090b0 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1871628789) 0x7ff034004170 con 0x7ff0240090b0 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:57.161 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.162 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.163 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.165 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:57.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.167 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:57.170 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.173 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.174 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:57.175 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.176 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.177 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.178 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.181 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.182 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.183 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.184 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:57.186 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:57.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff0397fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:57.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff0397fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.26:6789/0 conn(0x7ff0240090b0 legacy=0x7ff024008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff0397fa700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.26:6789/0 conn(0x7ff0240090b0 legacy=0x7ff024008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.122:6812/20895 conn(0x7ff028037d20 legacy=0x7ff02803a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 10 monclient: shutdown 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 >> v1:172.21.15.26:6790/0 conn(0x7ff044063340 legacy=0x7ff044141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff0498aa700 1 -- 172.21.15.26:0/2404277905 reap_dead start 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 shutdown_connections 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 >> 172.21.15.26:0/2404277905 conn(0x7ff0440a3cb0 msgr2=0x7ff04409c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 shutdown_connections 2024-09-15T03:51:57.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.144+0000 7ff04b30d700 1 -- 172.21.15.26:0/2404277905 wait complete. 2024-09-15T03:51:57.190 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:57.190 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:51:57.190 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:51:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: get_monmap_and_config 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: monmap: 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:57.304367+0000 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:57.304367+0000 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.306 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.307 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 1 Processor -- start 2024-09-15T03:51:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 1 -- start start 2024-09-15T03:51:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: init 2024-09-15T03:51:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: _add_conns ranks=[5,2,0,7,6,1,4,3,8] 2024-09-15T03:51:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient(hunting): picked mon.noname-h con 0x7f0d700990c0 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient(hunting): picked mon.noname-b con 0x7f0d700a1310 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient(hunting): picked mon.noname-a con 0x7f0d701335a0 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7004ca10 con 0x7f0d701335a0 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700a1b20 con 0x7f0d700a1310 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700a10b0 con 0x7f0d700990c0 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient(hunting): authenticate will time out at 1261.734474s 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6effd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0d700a1310 0x7f0d70099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39504/0 (socket says 172.21.15.26:39504) 2024-09-15T03:51:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6effd700 1 -- 172.21.15.26:0/4265636923 learned_addr learned my addr 172.21.15.26:0/4265636923 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032949791 0 0) 0x7f0d700a1b20 con 0x7f0d700a1310 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): my global_id is 134597 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d5c003570 con 0x7f0d700a1310 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504214136 0 0) 0x7f0d700a10b0 con 0x7f0d700990c0 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): my global_id is 5117 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d700a1b20 con 0x7f0d700990c0 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226628278 0 0) 0x7f0d7004ca10 con 0x7f0d701335a0 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient(hunting): my global_id is 4968 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d700a10b0 con 0x7f0d701335a0 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036004200 0 0) 0x7f0d5c003570 con 0x7f0d700a1310 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 >> v1:172.21.15.122:6791/0 conn(0x7f0d700990c0 legacy=0x7f0d70132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 >> v1:172.21.15.26:6789/0 conn(0x7f0d701335a0 legacy=0x7f0d7013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 monclient: found mon.noname-b 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70099ba0 con 0x7f0d700a1310 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 5 monclient: authenticate success, global_id 134597 2024-09-15T03:51:57.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d600017b0 con 0x7f0d700a1310 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: dump: 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.313 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.314 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0d60001a90 con 0x7f0d700a1310 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d76238700 4 set_mon_vals no callback set 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 1 -- 172.21.15.26:0/4265636923 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d60004c30 con 0x7f0d700a1310 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.300+0000 7f0d6e7fc700 10 monclient: dump: 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.316 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.317 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.317 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient: shutdown 2024-09-15T03:51:57.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- 172.21.15.26:0/4265636923 >> v1:172.21.15.26:6790/0 conn(0x7f0d700a1310 legacy=0x7f0d70099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- 172.21.15.26:0/4265636923 shutdown_connections 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- 172.21.15.26:0/4265636923 >> 172.21.15.26:0/4265636923 conn(0x7f0d7006e370 msgr2=0x7f0d700a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- 172.21.15.26:0/4265636923 shutdown_connections 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- 172.21.15.26:0/4265636923 wait complete. 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient: monmap: 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:57.324 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 Processor -- start 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- start start 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient: init 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient: _add_conns ranks=[0,8,5,7,4,6,1,3,2] 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient(hunting): picked mon.noname-a con 0x7f0d700990c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient(hunting): picked mon.noname-e con 0x7f0d700a1310 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient(hunting): picked mon.noname-h con 0x7f0d701335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700a1ce0 con 0x7f0d700990c0 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7009a010 con 0x7f0d701335a0 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d70120ce0 con 0x7f0d700a1310 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d76238700 10 monclient(hunting): authenticate will time out at 1261.739292s 2024-09-15T03:51:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d6f7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0d700990c0 0x7f0d70131af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38230/0 (socket says 172.21.15.26:38230) 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d6f7fe700 1 -- 172.21.15.26:0/2595018206 learned_addr learned my addr 172.21.15.26:0/2595018206 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.304+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856786739 0 0) 0x7f0d700a1ce0 con 0x7f0d700990c0 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): my global_id is 4977 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d50003570 con 0x7f0d700990c0 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884092594 0 0) 0x7f0d70120ce0 con 0x7f0d700a1310 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): my global_id is 134594 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d700a1ce0 con 0x7f0d700a1310 2024-09-15T03:51:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042171439 0 0) 0x7f0d7009a010 con 0x7f0d701335a0 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): my global_id is 5126 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d70120ce0 con 0x7f0d701335a0 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2775309064 0 0) 0x7f0d50003570 con 0x7f0d700990c0 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d7009a010 con 0x7f0d700990c0 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d58002b50 con 0x7f0d700990c0 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): dump: 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.329 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4024614112 0 0) 0x7f0d700a1ce0 con 0x7f0d700a1310 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d50003570 con 0x7f0d700a1310 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d60004b70 con 0x7f0d700a1310 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): dump: 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3586407305 0 0) 0x7f0d70120ce0 con 0x7f0d701335a0 2024-09-15T03:51:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d700a1ce0 con 0x7f0d701335a0 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d68001cc0 con 0x7f0d701335a0 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): dump: 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 507422288 0 0) 0x7f0d7009a010 con 0x7f0d700990c0 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.26:6793/0 conn(0x7f0d700a1310 legacy=0x7f0d70147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.122:6791/0 conn(0x7f0d701335a0 legacy=0x7f0d7014a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 monclient: found mon.a 2024-09-15T03:51:57.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70134220 con 0x7f0d700990c0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 5 monclient: authenticate success, global_id 4977 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 10 monclient: _renew_subs 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d70120150 con 0x7f0d700990c0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 10 monclient: _renew_subs 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d70142bb0 con 0x7f0d700990c0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0d58001620 con 0x7f0d700990c0 2024-09-15T03:51:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d58001b50 con 0x7f0d700990c0 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d6dffb700 4 set_mon_vals no callback set 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient: dump: 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0d58001cb0 con 0x7f0d700990c0 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0d54006590 con 0x7f0d54002840 2024-09-15T03:51:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(329..329 src has 224..329) v4 ==== 3987+0+0 (unknown 3959154610 0 0) 0x7f0d58043f40 con 0x7f0d700990c0 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d6ffff700 10 monclient: get_auth_request con 0x7f0d50038000 auth_method 0 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375618685 0 0) 0x7f0d54006590 con 0x7f0d54002840 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 10 monclient(hunting): my global_id is 134606 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d7009a010 con 0x7f0d54002840 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3795796925 0 0) 0x7f0d7009a010 con 0x7f0d54002840 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.308+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d54006590 con 0x7f0d54002840 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d60001230 con 0x7f0d54002840 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 10 monclient: dump: 2024-09-15T03:51:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2934602530 0 0) 0x7f0d54006590 con 0x7f0d54002840 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d5400a130 con 0x7f0d54002840 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0d600075d0 con 0x7f0d54002840 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.26:6790/0 conn(0x7f0d54002840 legacy=0x7f0d54004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.312+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.26:6790/0 conn(0x7f0d54002840 legacy=0x7f0d54004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:57.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0d54009640 con 0x7f0d540090b0 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809882696 0 0) 0x7f0d54009640 con 0x7f0d540090b0 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient(hunting): my global_id is 134615 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d54006590 con 0x7f0d540090b0 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 540694278 0 0) 0x7f0d54006590 con 0x7f0d540090b0 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0d54009640 con 0x7f0d540090b0 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0d60005d50 con 0x7f0d540090b0 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient: dump: 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.343 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4219108210 0 0) 0x7f0d54009640 con 0x7f0d540090b0 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0d540098f0 con 0x7f0d540090b0 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1815058980) 0x7f0d600071a0 con 0x7f0d540090b0 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.26:6790/0 conn(0x7f0d540090b0 legacy=0x7f0d54008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d677fe700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.26:6790/0 conn(0x7f0d540090b0 legacy=0x7f0d54008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.345 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:57.345 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.346 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.347 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.348 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.349 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.350 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:57.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.352 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:57.353 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.354 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:57.355 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:57.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.357 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.358 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.359 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.360 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.362 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.363 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:51:57.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.367 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.368 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:57.368 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.122:6812/20895 conn(0x7f0d50038000 legacy=0x7f0d5003a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 10 monclient: shutdown 2024-09-15T03:51:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 >> v1:172.21.15.26:6789/0 conn(0x7f0d700990c0 legacy=0x7f0d70131af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d6ffff700 1 -- 172.21.15.26:0/2595018206 reap_dead start 2024-09-15T03:51:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d6ffff700 1 -- 172.21.15.26:0/2595018206 reap_dead start 2024-09-15T03:51:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 shutdown_connections 2024-09-15T03:51:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 >> 172.21.15.26:0/2595018206 conn(0x7f0d7006e370 msgr2=0x7f0d700a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 shutdown_connections 2024-09-15T03:51:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.320+0000 7f0d76238700 1 -- 172.21.15.26:0/2595018206 wait complete. 2024-09-15T03:51:57.370 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:57.371 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:51:57.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:51:57.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.480+0000 7efd23fff700 10 monclient: get_monmap_and_config 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.480+0000 7efd23fff700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.480+0000 7efd23fff700 10 monclient: monmap: 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:57.485349+0000 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:57.485349+0000 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.487 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.488 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.480+0000 7efd23fff700 1 Processor -- start 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.480+0000 7efd23fff700 1 -- start start 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.480+0000 7efd23fff700 10 monclient: init 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient: _add_conns ranks=[2,4,0,7,6,3,5,1,8] 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient(hunting): picked mon.noname-b con 0x7efd24070de0 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient(hunting): picked mon.noname-c con 0x7efd24099020 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient(hunting): picked mon.noname-a con 0x7efd24132f70 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2404ca10 con 0x7efd24132f70 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd24096140 con 0x7efd24070de0 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2406e620 con 0x7efd24099020 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd22ffd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7efd24070de0 0x7efd24098910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39526/0 (socket says 172.21.15.26:39526) 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd22ffd700 1 -- 172.21.15.26:0/161358541 learned_addr learned my addr 172.21.15.26:0/161358541 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient(hunting): authenticate will time out at 1261.916660s 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112271318 0 0) 0x7efd2404ca10 con 0x7efd24132f70 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): my global_id is 4986 2024-09-15T03:51:57.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd0c003570 con 0x7efd24132f70 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799036435 0 0) 0x7efd24096140 con 0x7efd24070de0 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): my global_id is 134624 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2404ca10 con 0x7efd24070de0 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712046155 0 0) 0x7efd2406e620 con 0x7efd24099020 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient(hunting): my global_id is 184117 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd24096140 con 0x7efd24099020 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 544131200 0 0) 0x7efd0c003570 con 0x7efd24132f70 2024-09-15T03:51:57.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 >> v1:172.21.15.26:6791/0 conn(0x7efd24099020 legacy=0x7efd24137d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 >> v1:172.21.15.26:6790/0 conn(0x7efd24070de0 legacy=0x7efd24098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 monclient: found mon.noname-a 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd24096440 con 0x7efd24132f70 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 5 monclient: authenticate success, global_id 4986 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd1c001b30 con 0x7efd24132f70 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: dump: 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.497 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd1c001da0 con 0x7efd24132f70 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 1 -- 172.21.15.26:0/161358541 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd1c0022d0 con 0x7efd24132f70 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 4 set_mon_vals no callback set 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd21ffb700 10 monclient: dump: 2024-09-15T03:51:57.498 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.499 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient: shutdown 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- 172.21.15.26:0/161358541 >> v1:172.21.15.26:6789/0 conn(0x7efd24132f70 legacy=0x7efd2413b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- 172.21.15.26:0/161358541 shutdown_connections 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- 172.21.15.26:0/161358541 >> 172.21.15.26:0/161358541 conn(0x7efd240a2e80 msgr2=0x7efd2409eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- 172.21.15.26:0/161358541 shutdown_connections 2024-09-15T03:51:57.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 1 -- 172.21.15.26:0/161358541 wait complete. 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.484+0000 7efd23fff700 10 monclient: monmap: 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 Processor -- start 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 -- start start 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: init 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: _add_conns ranks=[0,3,2,8,7,5,4,1,6] 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient(hunting): picked mon.noname-a con 0x7efd24070de0 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient(hunting): picked mon.noname-g con 0x7efd24132f70 addr v1:172.21.15.122:6790/0 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient(hunting): picked mon.noname-b con 0x7efd2412a550 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2412dcd0 con 0x7efd24070de0 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2409f8f0 con 0x7efd2412a550 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2409fb20 con 0x7efd24132f70 2024-09-15T03:51:57.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient(hunting): authenticate will time out at 1261.921535s 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd22ffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd24070de0 0x7efd24129730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38246/0 (socket says 172.21.15.26:38246) 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd22ffd700 1 -- 172.21.15.26:0/3303594989 learned_addr learned my addr 172.21.15.26:0/3303594989 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393724390 0 0) 0x7efd2412dcd0 con 0x7efd24070de0 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): my global_id is 4995 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd10003570 con 0x7efd24070de0 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793477528 0 0) 0x7efd2409f8f0 con 0x7efd2412a550 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): my global_id is 134633 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2412dcd0 con 0x7efd2412a550 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668437207 0 0) 0x7efd2409fb20 con 0x7efd24132f70 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): my global_id is 184116 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd2409f8f0 con 0x7efd24132f70 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4032350464 0 0) 0x7efd10003570 con 0x7efd24070de0 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd2409fb20 con 0x7efd24070de0 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd140015d0 con 0x7efd24070de0 2024-09-15T03:51:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): dump: 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 310573265 0 0) 0x7efd2412dcd0 con 0x7efd2412a550 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd10003570 con 0x7efd2412a550 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd1c001af0 con 0x7efd2412a550 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): dump: 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2350853289 0 0) 0x7efd2409f8f0 con 0x7efd24132f70 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd2412dcd0 con 0x7efd24132f70 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd180019e0 con 0x7efd24132f70 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient(hunting): dump: 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1595667456 0 0) 0x7efd10003570 con 0x7efd2412a550 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.122:6790/0 conn(0x7efd24132f70 legacy=0x7efd24129e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.26:6789/0 conn(0x7efd24070de0 legacy=0x7efd24129730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 monclient: found mon.b 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2412cd90 con 0x7efd2412a550 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 5 monclient: authenticate success, global_id 134633 2024-09-15T03:51:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: _renew_subs 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2412cb10 con 0x7efd2412a550 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: _renew_subs 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2412c7a0 con 0x7efd2412a550 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efd1c0022c0 con 0x7efd2412a550 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd1c002af0 con 0x7efd2412a550 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd217fa700 4 set_mon_vals no callback set 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.488+0000 7efd0affd700 10 monclient: dump: 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7efd1c00d4b0 con 0x7efd2412a550 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd23fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efd04006590 con 0x7efd04002840 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(329..329 src has 224..329) v4 ==== 3987+0+0 (unknown 3959154610 0 0) 0x7efd1c041920 con 0x7efd2412a550 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362068793 0 0) 0x7efd04006590 con 0x7efd04002840 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient(hunting): my global_id is 184126 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd10003570 con 0x7efd04002840 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd227fc700 10 monclient: get_auth_request con 0x7efd10038000 auth_method 0 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 780485504 0 0) 0x7efd10003570 con 0x7efd04002840 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd04006590 con 0x7efd04002840 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd14005090 con 0x7efd04002840 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient: dump: 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1761249812 0 0) 0x7efd04006590 con 0x7efd04002840 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd0400a130 con 0x7efd04002840 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7efd140073f0 con 0x7efd04002840 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.26:6791/0 conn(0x7efd04002840 legacy=0x7efd04004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.492+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.26:6791/0 conn(0x7efd04002840 legacy=0x7efd04004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd23fff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efd04007520 con 0x7efd040068d0 2024-09-15T03:51:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371394136 0 0) 0x7efd04007520 con 0x7efd040068d0 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd0affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd0affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd0affd700 10 monclient(hunting): my global_id is 184135 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd04006590 con 0x7efd040068d0 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.500+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2865609263 0 0) 0x7efd04006590 con 0x7efd040068d0 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd04007520 con 0x7efd040068d0 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efd14005ad0 con 0x7efd040068d0 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 10 monclient: dump: 2024-09-15T03:51:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 147642296 0 0) 0x7efd04007520 con 0x7efd040068d0 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efd04008760 con 0x7efd040068d0 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1984003) 0x7efd14007680 con 0x7efd040068d0 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.26:6791/0 conn(0x7efd040068d0 legacy=0x7efd04006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd0affd700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.26:6791/0 conn(0x7efd040068d0 legacy=0x7efd04006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:57.531 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:51:57.532 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.533 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:57.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:57.538 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.540 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.542 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.543 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:57.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.549 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:57.552 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:57.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.554 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.555 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:57.555 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:57.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.122:6812/20895 conn(0x7efd10038000 legacy=0x7efd1003a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 10 monclient: shutdown 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 >> v1:172.21.15.26:6790/0 conn(0x7efd2412a550 legacy=0x7efd24149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd237fe700 1 -- 172.21.15.26:0/3303594989 reap_dead start 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd237fe700 1 -- 172.21.15.26:0/3303594989 reap_dead start 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 shutdown_connections 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 >> 172.21.15.26:0/3303594989 conn(0x7efd240a2e80 msgr2=0x7efd2409e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 shutdown_connections 2024-09-15T03:51:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.504+0000 7efd23fff700 1 -- 172.21.15.26:0/3303594989 wait complete. 2024-09-15T03:51:57.557 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:57.558 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:51:57.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient: get_monmap_and_config 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient: monmap: 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:57.676841+0000 2024-09-15T03:51:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:57.676841+0000 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.679 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 1 Processor -- start 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 1 -- start start 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient: init 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient: _add_conns ranks=[0,8,4,3,2,5,1,7,6] 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient(hunting): picked mon.noname-a con 0x7f5ed00a6b20 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient(hunting): picked mon.noname-e con 0x7f5ed0099110 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient(hunting): picked mon.noname-c con 0x7f5ed0138da0 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:57.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed004ca10 con 0x7f5ed00a6b20 2024-09-15T03:51:57.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00a0d30 con 0x7f5ed0138da0 2024-09-15T03:51:57.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00a22f0 con 0x7f5ed0099110 2024-09-15T03:51:57.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ed53af700 10 monclient(hunting): authenticate will time out at 1262.107055s 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ecffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ed00a6b20 0x7f5ed01330e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38254/0 (socket says 172.21.15.26:38254) 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5ecffff700 1 -- 172.21.15.26:0/2141480962 learned_addr learned my addr 172.21.15.26:0/2141480962 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727276541 0 0) 0x7f5ed004ca10 con 0x7f5ed00a6b20 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): my global_id is 5004 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb0003570 con 0x7f5ed00a6b20 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713445854 0 0) 0x7f5ed00a22f0 con 0x7f5ed0099110 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): my global_id is 134603 2024-09-15T03:51:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed004ca10 con 0x7f5ed0099110 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045676327 0 0) 0x7f5ec0001110 con 0x7f5ed0138da0 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 10 monclient(hunting): my global_id is 184144 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.672+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed00a22f0 con 0x7f5ed0138da0 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1406927297 0 0) 0x7f5eb0003570 con 0x7f5ed00a6b20 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 >> v1:172.21.15.26:6793/0 conn(0x7f5ed0099110 legacy=0x7f5ed00994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 >> v1:172.21.15.26:6791/0 conn(0x7f5ed0138da0 legacy=0x7f5ed013b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 monclient: found mon.noname-a 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed01337f0 con 0x7f5ed00a6b20 2024-09-15T03:51:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 5 monclient: authenticate success, global_id 5004 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec80017b0 con 0x7f5ed00a6b20 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: dump: 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ec8001a90 con 0x7f5ed00a6b20 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 1 -- 172.21.15.26:0/2141480962 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec8004c30 con 0x7f5ed00a6b20 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 4 set_mon_vals no callback set 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5eceffd700 10 monclient: dump: 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: shutdown 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- 172.21.15.26:0/2141480962 >> v1:172.21.15.26:6789/0 conn(0x7f5ed00a6b20 legacy=0x7f5ed01330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- 172.21.15.26:0/2141480962 shutdown_connections 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- 172.21.15.26:0/2141480962 >> 172.21.15.26:0/2141480962 conn(0x7f5ed006e2d0 msgr2=0x7f5ed00a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- 172.21.15.26:0/2141480962 shutdown_connections 2024-09-15T03:51:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- 172.21.15.26:0/2141480962 wait complete. 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: monmap: 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 Processor -- start 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- start start 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: init 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient: _add_conns ranks=[2,6,0,1,3,8,4,5,7] 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient(hunting): picked mon.noname-b con 0x7f5ed0099110 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient(hunting): picked mon.noname-d con 0x7f5ed00a6b20 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient(hunting): picked mon.noname-a con 0x7f5ed0138da0 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00a1880 con 0x7f5ed0138da0 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0099e30 con 0x7f5ed0099110 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed0132d50 con 0x7f5ed00a6b20 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed53af700 10 monclient(hunting): authenticate will time out at 1262.111458s 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ecf7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5ed00a6b20 0x7f5ed0136080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53226/0 (socket says 172.21.15.26:53226) 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ecffff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f5ed0099110 0x7f5ed0135970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39546/0 (socket says 172.21.15.26:39546) 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ecf7fe700 1 -- 172.21.15.26:0/1648885544 learned_addr learned my addr 172.21.15.26:0/1648885544 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ed4bae700 1 --1- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6789/0 conn(0x7f5ed0138da0 0x7f5ed014a250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38266/0 (socket says 172.21.15.26:38266) 2024-09-15T03:51:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.676+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668134580 0 0) 0x7f5ed00a1880 con 0x7f5ed0138da0 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): my global_id is 5013 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb4003570 con 0x7f5ed0138da0 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966160954 0 0) 0x7f5ed004ca10 con 0x7f5ed00a6b20 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): my global_id is 5145 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed00a1880 con 0x7f5ed00a6b20 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230783848 0 0) 0x7f5ed0099e30 con 0x7f5ed0099110 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): my global_id is 134642 2024-09-15T03:51:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed004ca10 con 0x7f5ed0099110 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4183957049 0 0) 0x7f5eb4003570 con 0x7f5ed0138da0 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ed0099e30 con 0x7f5ed0138da0 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec0001b50 con 0x7f5ed0138da0 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): dump: 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1864491967 0 0) 0x7f5ed00a1880 con 0x7f5ed00a6b20 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5eb4003570 con 0x7f5ed00a6b20 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5eb8001ea0 con 0x7f5ed00a6b20 2024-09-15T03:51:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): dump: 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2837840758 0 0) 0x7f5ed004ca10 con 0x7f5ed0099110 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ed00a1880 con 0x7f5ed0099110 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec8004b70 con 0x7f5ed0099110 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): dump: 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 210478471 0 0) 0x7f5ed0099e30 con 0x7f5ed0138da0 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6792/0 conn(0x7f5ed00a6b20 legacy=0x7f5ed0136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6790/0 conn(0x7f5ed0099110 legacy=0x7f5ed0135970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 monclient: found mon.a 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0137e60 con 0x7f5ed0138da0 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 5 monclient: authenticate success, global_id 5013 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 10 monclient: _renew_subs 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed011c880 con 0x7f5ed0138da0 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 10 monclient: _renew_subs 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed011ce70 con 0x7f5ed0138da0 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ec0002070 con 0x7f5ed0138da0 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec0001180 con 0x7f5ed0138da0 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ece7fc700 4 set_mon_vals no callback set 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: dump: 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.705 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5ec00025a0 con 0x7f5ed0138da0 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5ea8006590 con 0x7f5ea8002840 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(329..329 src has 224..329) v4 ==== 3987+0+0 (unknown 3959154610 0 0) 0x7f5ec0043bb0 con 0x7f5ed0138da0 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074913282 0 0) 0x7f5ea8006590 con 0x7f5ea8002840 2024-09-15T03:51:57.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient(hunting): my global_id is 5154 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed0099e30 con 0x7f5ea8002840 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ecf7fe700 10 monclient: get_auth_request con 0x7f5eb4038150 auth_method 0 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3684500723 0 0) 0x7f5ed0099e30 con 0x7f5ea8002840 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ea8006590 con 0x7f5ea8002840 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec8001860 con 0x7f5ea8002840 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 10 monclient: dump: 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.708 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1279738862 0 0) 0x7f5ea8006590 con 0x7f5ea8002840 2024-09-15T03:51:57.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.680+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ea800a130 con 0x7f5ea8002840 2024-09-15T03:51:57.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.684+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5ec80052b0 con 0x7f5ea8002840 2024-09-15T03:51:57.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.684+0000 7f5ec7fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.684+0000 7f5ec7fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:57.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.684+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6792/0 conn(0x7f5ea8002840 legacy=0x7f5ea8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.684+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6792/0 conn(0x7f5ea8002840 legacy=0x7f5ea8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.688+0000 7f5ed53af700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.688+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5ea8009640 con 0x7f5ea80090b0 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490564703 0 0) 0x7f5ea8009640 con 0x7f5ea80090b0 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient(hunting): my global_id is 5163 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea8006590 con 0x7f5ea80090b0 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1908777704 0 0) 0x7f5ea8006590 con 0x7f5ea80090b0 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ea8009640 con 0x7f5ea80090b0 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ec8005b20 con 0x7f5ea80090b0 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:57.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient: dump: 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.713 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2987444469 0 0) 0x7f5ea8009640 con 0x7f5ea80090b0 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5ea80098f0 con 0x7f5ea80090b0 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 4045655022) 0x7f5ec80082a0 con 0x7f5ea80090b0 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6792/0 conn(0x7f5ea80090b0 legacy=0x7f5ea8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ec7fff700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6792/0 conn(0x7f5ea80090b0 legacy=0x7f5ea8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:57.715 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:57.716 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.717 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:57.718 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.719 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.721 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.722 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.724 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.725 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:57.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.729 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:57.731 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:57.732 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.733 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:57.734 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:57.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.122:6812/20895 conn(0x7f5eb4038150 legacy=0x7f5eb403a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 10 monclient: shutdown 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 >> v1:172.21.15.26:6789/0 conn(0x7f5ed0138da0 legacy=0x7f5ed014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed4bae700 1 -- 172.21.15.26:0/1648885544 reap_dead start 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed4bae700 1 -- 172.21.15.26:0/1648885544 reap_dead start 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 shutdown_connections 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 >> 172.21.15.26:0/1648885544 conn(0x7f5ed006e2d0 msgr2=0x7f5ed00a4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 shutdown_connections 2024-09-15T03:51:57.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.692+0000 7f5ed53af700 1 -- 172.21.15.26:0/1648885544 wait complete. 2024-09-15T03:51:57.736 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:57.737 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:51:57.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:51:57.839 INFO:tasks.rados.rados.0.smithi122.stdout:1406: done (2 left) 2024-09-15T03:51:57.839 INFO:tasks.rados.rados.0.smithi122.stdout:1407: done (1 left) 2024-09-15T03:51:57.839 INFO:tasks.rados.rados.0.smithi122.stdout:1408: done (0 left) 2024-09-15T03:51:57.839 INFO:tasks.rados.rados.0.smithi122.stdout:1409: rollback oid 35 current snap is 156 2024-09-15T03:51:57.839 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 155 2024-09-15T03:51:57.839 INFO:tasks.rados.rados.0.smithi122.stdout:1410: read oid 2 snap -1 2024-09-15T03:51:57.840 INFO:tasks.rados.rados.0.smithi122.stdout:1410: expect (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:51:57.840 INFO:tasks.rados.rados.0.smithi122.stdout:1411: read oid 42 snap -1 2024-09-15T03:51:57.840 INFO:tasks.rados.rados.0.smithi122.stdout:1411: expect (ObjNum 512 snap 152 seq_num 512) 2024-09-15T03:51:57.840 INFO:tasks.rados.rados.0.smithi122.stdout:1412: delete oid 46 current snap is 156 2024-09-15T03:51:57.845 INFO:tasks.rados.rados.0.smithi122.stdout:1412: done (3 left) 2024-09-15T03:51:57.845 INFO:tasks.rados.rados.0.smithi122.stdout:1413: read oid 19 snap -1 2024-09-15T03:51:57.845 INFO:tasks.rados.rados.0.smithi122.stdout:1413: expect deleted 2024-09-15T03:51:57.845 INFO:tasks.rados.rados.0.smithi122.stdout:1414: write oid 26 current snap is 156 2024-09-15T03:51:57.845 INFO:tasks.rados.rados.0.smithi122.stdout:1414: seq_num 520 ranges {739640=641484,1829460=794174} 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.844+0000 7f20b2c5d700 10 monclient: get_monmap_and_config 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.844+0000 7f20b2c5d700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.844+0000 7f20b2c5d700 10 monclient: monmap: 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:57.851781+0000 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:57.851781+0000 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.853 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.854 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 1 Processor -- start 2024-09-15T03:51:57.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 1 -- start start 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient: init 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient: _add_conns ranks=[4,3,7,8,5,1,0,6,2] 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient(hunting): picked mon.noname-c con 0x7f20ac130d70 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient(hunting): picked mon.noname-g con 0x7f20ac09e500 addr v1:172.21.15.122:6790/0 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient(hunting): picked mon.noname-i con 0x7f20ac137410 addr v1:172.21.15.122:6792/0 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac04ca10 con 0x7f20ac09e500 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac09d550 con 0x7f20ac130d70 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac09e8d0 con 0x7f20ac137410 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b2c5d700 10 monclient(hunting): authenticate will time out at 1262.281554s 2024-09-15T03:51:57.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b11fa700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f20ac137410 0x7f20ac1397f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:46272/0 (socket says 172.21.15.26:46272) 2024-09-15T03:51:57.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20b11fa700 1 -- 172.21.15.26:0/331329007 learned_addr learned my addr 172.21.15.26:0/331329007 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.857 INFO:tasks.rados.rados.0.smithi122.stdout:1414: writing smithi12223906-26 from 739640 to 1381124 tid 1 2024-09-15T03:51:57.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279748246 0 0) 0x7f20ac09d550 con 0x7f20ac130d70 2024-09-15T03:51:57.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20ab7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20ab7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20ab7fe700 10 monclient(hunting): my global_id is 184153 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.848+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2094003570 con 0x7f20ac130d70 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958407874 0 0) 0x7f2094003570 con 0x7f20ac130d70 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 >> v1:172.21.15.122:6792/0 conn(0x7f20ac137410 legacy=0x7f20ac1397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 >> v1:172.21.15.122:6790/0 conn(0x7f20ac09e500 legacy=0x7f20ac135cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 monclient: found mon.noname-c 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac131850 con 0x7f20ac130d70 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f209c0016e0 con 0x7f20ac130d70 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: dump: 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.860 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 5 monclient: authenticate success, global_id 184153 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f209c001990 con 0x7f20ac130d70 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 1 -- 172.21.15.26:0/331329007 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f209c002240 con 0x7f20ac130d70 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 4 set_mon_vals no callback set 2024-09-15T03:51:57.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20ab7fe700 10 monclient: dump: 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.862 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: shutdown 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- 172.21.15.26:0/331329007 >> v1:172.21.15.26:6791/0 conn(0x7f20ac130d70 legacy=0x7f20ac131140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- 172.21.15.26:0/331329007 shutdown_connections 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- 172.21.15.26:0/331329007 >> 172.21.15.26:0/331329007 conn(0x7f20ac063f30 msgr2=0x7f20ac0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- 172.21.15.26:0/331329007 shutdown_connections 2024-09-15T03:51:57.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- 172.21.15.26:0/331329007 wait complete. 2024-09-15T03:51:57.864 INFO:tasks.rados.rados.0.smithi122.stdout:1414: writing smithi12223906-26 from 1829460 to 2623634 tid 2 2024-09-15T03:51:57.864 INFO:tasks.rados.rados.0.smithi122.stdout:1415: snap_remove snap 154 2024-09-15T03:51:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: build_initial_monmap 2024-09-15T03:51:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: monmap: 2024-09-15T03:51:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:57.869 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 Processor -- start 2024-09-15T03:51:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- start start 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: init 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient: _add_conns ranks=[4,8,2,7,0,1,6,5,3] 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient(hunting): picked mon.noname-c con 0x7f20ac09e500 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient(hunting): picked mon.noname-e con 0x7f20ac130d70 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient(hunting): picked mon.noname-b con 0x7f20ac137410 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac06e140 con 0x7f20ac137410 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac1366d0 con 0x7f20ac09e500 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac09ccb0 con 0x7f20ac130d70 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b2c5d700 10 monclient(hunting): authenticate will time out at 1262.287306s 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b11fa700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f20ac137410 0x7f20ac14e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39556/0 (socket says 172.21.15.26:39556) 2024-09-15T03:51:57.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b11fa700 1 -- 172.21.15.26:0/3239094501 learned_addr learned my addr 172.21.15.26:0/3239094501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20b09f9700 1 --1- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6791/0 conn(0x7f20ac09e500 0x7f20ac134890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55384/0 (socket says 172.21.15.26:55384) 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634880651 0 0) 0x7f20ac06e140 con 0x7f20ac137410 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): my global_id is 134651 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2098003570 con 0x7f20ac137410 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712143796 0 0) 0x7f20ac09ccb0 con 0x7f20ac130d70 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): my global_id is 134612 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac06e140 con 0x7f20ac130d70 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749934570 0 0) 0x7f20ac1366d0 con 0x7f20ac09e500 2024-09-15T03:51:57.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 10 monclient(hunting): my global_id is 184162 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.852+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac09ccb0 con 0x7f20ac09e500 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3445444254 0 0) 0x7f2098003570 con 0x7f20ac137410 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20ac1366d0 con 0x7f20ac137410 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20a4001b90 con 0x7f20ac137410 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): dump: 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.873 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:57.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 272826504 0 0) 0x7f20ac09ccb0 con 0x7f20ac09e500 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2098003570 con 0x7f20ac09e500 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1109632392 0 0) 0x7f20ac06e140 con 0x7f20ac130d70 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20ac09ccb0 con 0x7f20ac130d70 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f209c001e40 con 0x7f20ac09e500 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): dump: 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.875 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20a00019e0 con 0x7f20ac130d70 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:57.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): dump: 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2452207141 0 0) 0x7f20ac1366d0 con 0x7f20ac137410 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6793/0 conn(0x7f20ac130d70 legacy=0x7f20ac134fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6791/0 conn(0x7f20ac09e500 legacy=0x7f20ac134890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 monclient: found mon.b 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac069320 con 0x7f20ac137410 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: _finish_auth 0 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:57.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 5 monclient: authenticate success, global_id 134651 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 10 monclient: _renew_subs 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac06a210 con 0x7f20ac137410 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 10 monclient: _renew_subs 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac1361c0 con 0x7f20ac137410 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f20a4004c30 con 0x7f20ac137410 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20a4005160 con 0x7f20ac137410 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20aaffd700 4 set_mon_vals no callback set 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:57.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: dump: 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f208c006590 con 0x7f208c002840 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f20a4005340 con 0x7f20ac137410 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(330..330 src has 224..330) v4 ==== 3895+0+0 (unknown 14320229 0 0) 0x7f20a4044250 con 0x7f20ac137410 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089340222 0 0) 0x7f208c006590 con 0x7f208c002840 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient(hunting): my global_id is 134621 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac1366d0 con 0x7f208c002840 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20abfff700 10 monclient: get_auth_request con 0x7f2098038060 auth_method 0 2024-09-15T03:51:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2830388091 0 0) 0x7f20ac1366d0 con 0x7f208c002840 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f208c006590 con 0x7f208c002840 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f209c001610 con 0x7f208c002840 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: dump: 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.882 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 356156937 0 0) 0x7f208c006590 con 0x7f208c002840 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f208c00a130 con 0x7f208c002840 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f209c0029a0 con 0x7f208c002840 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:57.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6793/0 conn(0x7f208c002840 legacy=0x7f208c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.856+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6793/0 conn(0x7f208c002840 legacy=0x7f208c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20b2c5d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:57.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f208c009640 con 0x7f208c0090b0 2024-09-15T03:51:57.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801920645 0 0) 0x7f208c009640 con 0x7f208c0090b0 2024-09-15T03:51:57.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient(hunting): my global_id is 134630 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f208c006590 con 0x7f208c0090b0 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4279644539 0 0) 0x7f208c006590 con 0x7f208c0090b0 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f208c009640 con 0x7f208c0090b0 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f209c001140 con 0x7f208c0090b0 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient: dump: 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:57.886 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3251465344 0 0) 0x7f208c009640 con 0x7f208c0090b0 2024-09-15T03:51:57.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f208c0098f0 con 0x7f208c0090b0 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2526339644) 0x7f209c007da0 con 0x7f208c0090b0 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6793/0 conn(0x7f208c0090b0 legacy=0x7f208c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.864+0000 7f20a8ff9700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6793/0 conn(0x7f208c0090b0 legacy=0x7f208c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:57.888 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:51:57.889 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:57.892 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:57.893 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:57.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.896 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.897 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.899 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:57.900 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.901 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:57.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.906 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.907 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:57.908 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.122:6812/20895 conn(0x7f2098038060 legacy=0x7f209803a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 10 monclient: shutdown 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b11fa700 1 -- 172.21.15.26:0/3239094501 reap_dead start 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 >> v1:172.21.15.26:6790/0 conn(0x7f20ac137410 legacy=0x7f20ac14e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 shutdown_connections 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 >> 172.21.15.26:0/3239094501 conn(0x7f20ac063f30 msgr2=0x7f20ac0a0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 shutdown_connections 2024-09-15T03:51:57.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:57.868+0000 7f20b2c5d700 1 -- 172.21.15.26:0/3239094501 wait complete. 2024-09-15T03:51:57.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:57.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:51:57.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:51:58.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.016+0000 7f50559a6700 10 monclient: get_monmap_and_config 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.016+0000 7f50559a6700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.016+0000 7f50559a6700 10 monclient: monmap: 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:58.022360+0000 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:58.022360+0000 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.025 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.016+0000 7f50559a6700 1 Processor -- start 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 1 -- start start 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient: init 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient: _add_conns ranks=[6,0,2,8,4,5,1,3,7] 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient(hunting): picked mon.noname-d con 0x7f505009f600 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:58.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient(hunting): picked mon.noname-a con 0x7f505009c640 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:58.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient(hunting): picked mon.noname-b con 0x7f505009b2c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:58.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505004ca10 con 0x7f505009c640 2024-09-15T03:51:58.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505009af10 con 0x7f505009b2c0 2024-09-15T03:51:58.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505009f9d0 con 0x7f505009f600 2024-09-15T03:51:58.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50549a4700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f505009f600 0x7f505009de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53258/0 (socket says 172.21.15.26:53258) 2024-09-15T03:51:58.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient(hunting): authenticate will time out at 1262.453415s 2024-09-15T03:51:58.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50549a4700 1 -- 172.21.15.26:0/3336028935 learned_addr learned my addr 172.21.15.26:0/3336028935 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779338858 0 0) 0x7f505004ca10 con 0x7f505009c640 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): my global_id is 5022 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5040003570 con 0x7f505009c640 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657398404 0 0) 0x7f505009f9d0 con 0x7f505009f600 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): my global_id is 5172 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505004ca10 con 0x7f505009f600 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596524380 0 0) 0x7f505009af10 con 0x7f505009b2c0 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient(hunting): my global_id is 134660 2024-09-15T03:51:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505009f9d0 con 0x7f505009b2c0 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2039576261 0 0) 0x7f5040003570 con 0x7f505009c640 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 >> v1:172.21.15.26:6792/0 conn(0x7f505009f600 legacy=0x7f505009de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 >> v1:172.21.15.26:6790/0 conn(0x7f505009b2c0 legacy=0x7f50501375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 monclient: found mon.noname-a 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505009ec20 con 0x7f505009c640 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 5 monclient: authenticate success, global_id 5022 2024-09-15T03:51:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f503c0017b0 con 0x7f505009c640 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: dump: 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f503c001a20 con 0x7f505009c640 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 4 set_mon_vals no callback set 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 1 -- 172.21.15.26:0/3336028935 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f503c001f50 con 0x7f505009c640 2024-09-15T03:51:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f504f7fe700 10 monclient: dump: 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 10 monclient: shutdown 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.020+0000 7f50559a6700 1 -- 172.21.15.26:0/3336028935 >> v1:172.21.15.26:6789/0 conn(0x7f505009c640 legacy=0x7f505009e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- 172.21.15.26:0/3336028935 shutdown_connections 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- 172.21.15.26:0/3336028935 >> 172.21.15.26:0/3336028935 conn(0x7f50500a2ea0 msgr2=0x7f505009eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- 172.21.15.26:0/3336028935 shutdown_connections 2024-09-15T03:51:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- 172.21.15.26:0/3336028935 wait complete. 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient: monmap: 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 Processor -- start 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- start start 2024-09-15T03:51:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient: init 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient: _add_conns ranks=[3,6,1,8,5,4,2,7,0] 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient(hunting): picked mon.noname-g con 0x7f505009b2c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient(hunting): picked mon.noname-d con 0x7f505009c640 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient(hunting): picked mon.noname-f con 0x7f505009f600 addr v1:172.21.15.122:6789/0 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5050127650 con 0x7f505009f600 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50501283f0 con 0x7f505009b2c0 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50501286d0 con 0x7f505009c640 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f50559a6700 10 monclient(hunting): authenticate will time out at 1262.459231s 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504ffff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f505009c640 0x7f505006cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53274/0 (socket says 172.21.15.26:53274) 2024-09-15T03:51:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504ffff700 1 -- 172.21.15.26:0/4280775238 learned_addr learned my addr 172.21.15.26:0/4280775238 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953309394 0 0) 0x7f50501286d0 con 0x7f505009c640 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504cff9700 10 monclient(hunting): my global_id is 5181 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.024+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5034003570 con 0x7f505009c640 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481197888 0 0) 0x7f5050127650 con 0x7f505009f600 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): my global_id is 184114 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50501286d0 con 0x7f505009f600 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015401252 0 0) 0x7f50501283f0 con 0x7f505009b2c0 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): my global_id is 184125 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5050127650 con 0x7f505009b2c0 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 833556783 0 0) 0x7f5034003570 con 0x7f505009c640 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50501283f0 con 0x7f505009c640 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f503c001770 con 0x7f505009c640 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): dump: 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2992653600 0 0) 0x7f5050127650 con 0x7f505009b2c0 2024-09-15T03:51:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5034003570 con 0x7f505009b2c0 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5044001bc0 con 0x7f505009b2c0 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): dump: 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.047 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1555895419 0 0) 0x7f50501286d0 con 0x7f505009f600 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5050127650 con 0x7f505009f600 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2003940611 0 0) 0x7f50501283f0 con 0x7f505009c640 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6790/0 conn(0x7f505009b2c0 legacy=0x7f505006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6789/0 conn(0x7f505009f600 legacy=0x7f505006d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 monclient: found mon.d 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50501302e0 con 0x7f505009c640 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 5 monclient: authenticate success, global_id 5181 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 10 monclient: _renew_subs 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5050146990 con 0x7f505009c640 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 10 monclient: _renew_subs 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5050146cc0 con 0x7f505009c640 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f503c002550 con 0x7f505009c640 2024-09-15T03:51:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f503c002a80 con 0x7f505009c640 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504effd700 4 set_mon_vals no callback set 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient: dump: 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5030006590 con 0x7f5030002840 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f503c002c60 con 0x7f505009c640 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(330..330 src has 224..330) v4 ==== 3895+0+0 (unknown 14320229 0 0) 0x7f503c044170 con 0x7f505009c640 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f50551a5700 10 monclient: get_auth_request con 0x7f5034037ee0 auth_method 0 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302652155 0 0) 0x7f5030006590 con 0x7f5030002840 2024-09-15T03:51:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 10 monclient(hunting): my global_id is 184123 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.028+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50501283f0 con 0x7f5030002840 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3001087514 0 0) 0x7f50501283f0 con 0x7f5030002840 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5030006590 con 0x7f5030002840 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f50440011d0 con 0x7f5030002840 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:51:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 10 monclient: dump: 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 315800899 0 0) 0x7f5030006590 con 0x7f5030002840 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f503000a130 con 0x7f5030002840 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f50440090e0 con 0x7f5030002840 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6789/0 conn(0x7f5030002840 legacy=0x7f5030004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.032+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6789/0 conn(0x7f5030002840 legacy=0x7f5030004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f50559a6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5030007520 con 0x7f50300068d0 2024-09-15T03:51:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914776669 0 0) 0x7f5030007520 con 0x7f50300068d0 2024-09-15T03:51:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 10 monclient(hunting): my global_id is 184132 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5030006590 con 0x7f50300068d0 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1065707687 0 0) 0x7f5030006590 con 0x7f50300068d0 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5030007520 con 0x7f50300068d0 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f50440078e0 con 0x7f50300068d0 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 10 monclient: dump: 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.057 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2008909355 0 0) 0x7f5030007520 con 0x7f50300068d0 2024-09-15T03:51:58.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.044+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5030008760 con 0x7f50300068d0 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 157904324) 0x7f504400c0d0 con 0x7f50300068d0 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f504cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f504cff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6789/0 conn(0x7f50300068d0 legacy=0x7f5030006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f504cff9700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6789/0 conn(0x7f50300068d0 legacy=0x7f5030006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:58.059 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.060 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.061 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.062 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:58.063 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.064 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.065 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.068 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.069 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.075 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:58.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.078 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:58.079 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.122:6812/20895 conn(0x7f5034037ee0 legacy=0x7f503403a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 10 monclient: shutdown 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 >> v1:172.21.15.26:6792/0 conn(0x7f505009c640 legacy=0x7f505006cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50551a5700 1 -- 172.21.15.26:0/4280775238 reap_dead start 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50551a5700 1 -- 172.21.15.26:0/4280775238 reap_dead start 2024-09-15T03:51:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 shutdown_connections 2024-09-15T03:51:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 >> 172.21.15.26:0/4280775238 conn(0x7f50500a2ea0 msgr2=0x7f5050094600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 shutdown_connections 2024-09-15T03:51:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.048+0000 7f50559a6700 1 -- 172.21.15.26:0/4280775238 wait complete. 2024-09-15T03:51:58.081 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:58.081 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:51:58.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient: get_monmap_and_config 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient: monmap: 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:58.193625+0000 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:58.193625+0000 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.196 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 1 Processor -- start 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 1 -- start start 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient: init 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient: _add_conns ranks=[5,6,0,1,2,4,3,8,7] 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient(hunting): picked mon.noname-h con 0x7fc5b80a4b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient(hunting): picked mon.noname-d con 0x7fc5b806b5c0 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 10 monclient(hunting): picked mon.noname-a con 0x7fc5b8139410 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b804ca10 con 0x7fc5b8139410 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b8068dc0 con 0x7fc5b80a4b40 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.188+0000 7fc5beddb700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b80a06e0 con 0x7fc5b806b5c0 2024-09-15T03:51:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 10 monclient(hunting): authenticate will time out at 1262.624071s 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b7fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fc5b806b5c0 0x7fc5b8137cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53288/0 (socket says 172.21.15.26:53288) 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b7fff700 1 -- 172.21.15.26:0/2492544827 learned_addr learned my addr 172.21.15.26:0/2492544827 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751759351 0 0) 0x7fc5b80a06e0 con 0x7fc5b806b5c0 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): my global_id is 5190 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a0003570 con 0x7fc5b806b5c0 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351996186 0 0) 0x7fc5b804ca10 con 0x7fc5b8139410 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): my global_id is 5031 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b80a06e0 con 0x7fc5b8139410 2024-09-15T03:51:58.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293467378 0 0) 0x7fc5b8068dc0 con 0x7fc5b80a4b40 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient(hunting): my global_id is 5135 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b804ca10 con 0x7fc5b80a4b40 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926855677 0 0) 0x7fc5a0003570 con 0x7fc5b806b5c0 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 >> v1:172.21.15.122:6791/0 conn(0x7fc5b80a4b40 legacy=0x7fc5b809aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 >> v1:172.21.15.26:6789/0 conn(0x7fc5b8139410 legacy=0x7fc5b813b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 monclient: found mon.noname-d 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80a4f10 con 0x7fc5b806b5c0 2024-09-15T03:51:58.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5ac0017b0 con 0x7fc5b806b5c0 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: dump: 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.202 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 5 monclient: authenticate success, global_id 5190 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc5ac001a20 con 0x7fc5b806b5c0 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 1 -- 172.21.15.26:0/2492544827 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5ac004c30 con 0x7fc5b806b5c0 2024-09-15T03:51:58.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 4 set_mon_vals no callback set 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5b77fe700 10 monclient: dump: 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.204 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.205 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.205 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 10 monclient: shutdown 2024-09-15T03:51:58.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 1 -- 172.21.15.26:0/2492544827 >> v1:172.21.15.26:6792/0 conn(0x7fc5b806b5c0 legacy=0x7fc5b8137cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 1 -- 172.21.15.26:0/2492544827 shutdown_connections 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 1 -- 172.21.15.26:0/2492544827 >> 172.21.15.26:0/2492544827 conn(0x7fc5b8063a10 msgr2=0x7fc5b80a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 1 -- 172.21.15.26:0/2492544827 shutdown_connections 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 1 -- 172.21.15.26:0/2492544827 wait complete. 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 10 monclient: monmap: 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.215 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.192+0000 7fc5beddb700 1 Processor -- start 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- start start 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: init 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: _add_conns ranks=[0,8,5,1,7,4,2,3,6] 2024-09-15T03:51:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient(hunting): picked mon.noname-a con 0x7fc5b80a4b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient(hunting): picked mon.noname-e con 0x7fc5b8139410 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient(hunting): picked mon.noname-h con 0x7fc5b8147d80 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b8099c20 con 0x7fc5b80a4b40 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b8142920 con 0x7fc5b8147d80 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b806e120 con 0x7fc5b8139410 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5bcb77700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc5b80a4b40 0x7fc5b8141a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38310/0 (socket says 172.21.15.26:38310) 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5bcb77700 1 -- 172.21.15.26:0/1769703468 learned_addr learned my addr 172.21.15.26:0/1769703468 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b7fff700 1 --1- 172.21.15.26:0/1769703468 >> v1:172.21.15.26:6793/0 conn(0x7fc5b8139410 0x7fc5b809d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:55540/0 (socket says 172.21.15.26:55540) 2024-09-15T03:51:58.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient(hunting): authenticate will time out at 1262.629258s 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848805851 0 0) 0x7fc5b806e120 con 0x7fc5b8139410 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): my global_id is 134639 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a4003570 con 0x7fc5b8139410 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971343162 0 0) 0x7fc5b8099c20 con 0x7fc5b80a4b40 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): my global_id is 5040 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b806e120 con 0x7fc5b80a4b40 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284608448 0 0) 0x7fc5b80a06e0 con 0x7fc5b8147d80 2024-09-15T03:51:58.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): my global_id is 5144 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b8099c20 con 0x7fc5b8147d80 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1834802769 0 0) 0x7fc5a4003570 con 0x7fc5b8139410 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5b80a06e0 con 0x7fc5b8139410 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5ac0017b0 con 0x7fc5b8139410 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): dump: 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1578843235 0 0) 0x7fc5b806e120 con 0x7fc5b80a4b40 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc5a4003570 con 0x7fc5b80a4b40 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5a8002ef0 con 0x7fc5b80a4b40 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient(hunting): dump: 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1029407948 0 0) 0x7fc5b80a06e0 con 0x7fc5b8139410 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.122:6791/0 conn(0x7fc5b8147d80 legacy=0x7fc5b814a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.26:6789/0 conn(0x7fc5b80a4b40 legacy=0x7fc5b8141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 monclient: found mon.e 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b806e5e0 con 0x7fc5b8139410 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 5 monclient: authenticate success, global_id 134639 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: _renew_subs 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b809a270 con 0x7fc5b8139410 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: _renew_subs 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b80996a0 con 0x7fc5b8139410 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc5ac004ba0 con 0x7fc5b8139410 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5ac005240 con 0x7fc5b8139410 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b67fc700 4 set_mon_vals no callback set 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5b4ff9700 10 monclient: dump: 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.224 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.196+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc598006590 con 0x7fc598002840 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc5ac00fc00 con 0x7fc5b8139410 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(330..330 src has 224..330) v4 ==== 3895+0+0 (unknown 14320229 0 0) 0x7fc5ac044000 con 0x7fc5b8139410 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494951528 0 0) 0x7fc598006590 con 0x7fc598002840 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient(hunting): my global_id is 184134 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b80a06e0 con 0x7fc598002840 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5bd378700 10 monclient: get_auth_request con 0x7fc5a4037cb0 auth_method 0 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2262625494 0 0) 0x7fc5b80a06e0 con 0x7fc598002840 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc598006590 con 0x7fc598002840 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5a8001ce0 con 0x7fc598002840 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient: dump: 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.226 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 17177153 0 0) 0x7fc598006590 con 0x7fc598002840 2024-09-15T03:51:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc59800a130 con 0x7fc598002840 2024-09-15T03:51:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc5a8005b90 con 0x7fc598002840 2024-09-15T03:51:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.122:6790/0 conn(0x7fc598002840 legacy=0x7fc598004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.200+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.122:6790/0 conn(0x7fc598002840 legacy=0x7fc598004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5beddb700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:58.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc598009640 con 0x7fc5980090b0 2024-09-15T03:51:58.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137654211 0 0) 0x7fc598009640 con 0x7fc5980090b0 2024-09-15T03:51:58.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5b4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5b4ff9700 10 monclient(hunting): my global_id is 184143 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.212+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc598006590 con 0x7fc5980090b0 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2562344476 0 0) 0x7fc598006590 con 0x7fc5980090b0 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc598009640 con 0x7fc5980090b0 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5a8005e40 con 0x7fc5980090b0 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 10 monclient: dump: 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2697568891 0 0) 0x7fc598009640 con 0x7fc5980090b0 2024-09-15T03:51:58.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc5980098f0 con 0x7fc5980090b0 2024-09-15T03:51:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 789284953) 0x7fc5a8008040 con 0x7fc5980090b0 2024-09-15T03:51:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.122:6790/0 conn(0x7fc5980090b0 legacy=0x7fc598008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5b4ff9700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.122:6790/0 conn(0x7fc5980090b0 legacy=0x7fc598008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:58.233 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.234 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.235 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:58.236 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.237 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.238 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.239 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:58.241 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:58.242 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.243 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.244 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.247 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.249 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.251 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:58.252 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.253 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:58.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.256 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:58.257 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.122:6812/20895 conn(0x7fc5a4037cb0 legacy=0x7fc5a403a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 10 monclient: shutdown 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 >> v1:172.21.15.26:6793/0 conn(0x7fc5b8139410 legacy=0x7fc5b809d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5bd378700 1 -- 172.21.15.26:0/1769703468 reap_dead start 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5bd378700 1 -- 172.21.15.26:0/1769703468 reap_dead start 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 shutdown_connections 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 >> 172.21.15.26:0/1769703468 conn(0x7fc5b8063a10 msgr2=0x7fc5b806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 shutdown_connections 2024-09-15T03:51:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.216+0000 7fc5beddb700 1 -- 172.21.15.26:0/1769703468 wait complete. 2024-09-15T03:51:58.260 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:58.260 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:51:58.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:51:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: get_monmap_and_config 2024-09-15T03:51:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: monmap: 2024-09-15T03:51:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:58.376469+0000 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:58.376469+0000 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.380 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 1 Processor -- start 2024-09-15T03:51:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 1 -- start start 2024-09-15T03:51:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: init 2024-09-15T03:51:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: _add_conns ranks=[1,4,2,6,5,8,0,3,7] 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient(hunting): picked mon.noname-f con 0x7f56f0132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient(hunting): picked mon.noname-c con 0x7f56f009faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient(hunting): picked mon.noname-b con 0x7f56f0138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f004ca10 con 0x7f56f0132b40 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f009d5d0 con 0x7f56f0138d40 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0133620 con 0x7f56f009faa0 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient(hunting): authenticate will time out at 1262.806152s 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f5d5d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f56f009faa0 0x7f56f009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55400/0 (socket says 172.21.15.26:55400) 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f5d5d700 1 -- 172.21.15.26:0/2015527001 learned_addr learned my addr 172.21.15.26:0/2015527001 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f6d5f700 1 --1- 172.21.15.26:0/2015527001 >> v1:172.21.15.26:6790/0 conn(0x7f56f0138d40 0x7f56f013b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39584/0 (socket says 172.21.15.26:39584) 2024-09-15T03:51:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884536814 0 0) 0x7f56f0133620 con 0x7f56f009faa0 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): my global_id is 184171 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56dc003570 con 0x7f56f009faa0 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505828174 0 0) 0x7f56f009d5d0 con 0x7f56f0138d40 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): my global_id is 134669 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f0133620 con 0x7f56f0138d40 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506606364 0 0) 0x7f56f004ca10 con 0x7f56f0132b40 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient(hunting): my global_id is 184141 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f009d5d0 con 0x7f56f0132b40 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142857768 0 0) 0x7f56f0133620 con 0x7f56f0138d40 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 >> v1:172.21.15.26:6791/0 conn(0x7f56f009faa0 legacy=0x7f56f009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 >> v1:172.21.15.122:6789/0 conn(0x7f56f0132b40 legacy=0x7f56f0132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 monclient: found mon.noname-b 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00a0cc0 con 0x7f56f0138d40 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56e8001b10 con 0x7f56f0138d40 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: dump: 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.386 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 5 monclient: authenticate success, global_id 134669 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f56e8001dc0 con 0x7f56f0138d40 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f7560700 4 set_mon_vals no callback set 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 1 -- 172.21.15.26:0/2015527001 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56e8004ef0 con 0x7f56f0138d40 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:51:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.372+0000 7f56f555c700 10 monclient: dump: 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 10 monclient: shutdown 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 -- 172.21.15.26:0/2015527001 >> v1:172.21.15.26:6790/0 conn(0x7f56f0138d40 legacy=0x7f56f013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 -- 172.21.15.26:0/2015527001 shutdown_connections 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 -- 172.21.15.26:0/2015527001 >> 172.21.15.26:0/2015527001 conn(0x7f56f006e370 msgr2=0x7f56f00a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 -- 172.21.15.26:0/2015527001 shutdown_connections 2024-09-15T03:51:58.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 -- 172.21.15.26:0/2015527001 wait complete. 2024-09-15T03:51:58.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 10 monclient: monmap: 2024-09-15T03:51:58.394 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.394 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:58.394 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.395 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 Processor -- start 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 1 -- start start 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.376+0000 7f56f7560700 10 monclient: init 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient: _add_conns ranks=[8,5,3,4,0,1,2,7,6] 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient(hunting): picked mon.noname-e con 0x7f56f009faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient(hunting): picked mon.noname-h con 0x7f56f0132b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient(hunting): picked mon.noname-g con 0x7f56f0138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f009cfa0 con 0x7f56f0138d40 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0137440 con 0x7f56f0132b40 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0138a40 con 0x7f56f009faa0 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient(hunting): authenticate will time out at 1262.812610s 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f655e700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f56f009faa0 0x7f56f0135550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:55552/0 (socket says 172.21.15.26:55552) 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f655e700 1 -- 172.21.15.26:0/3541875332 learned_addr learned my addr 172.21.15.26:0/3541875332 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305732912 0 0) 0x7f56f0138a40 con 0x7f56f009faa0 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): my global_id is 134648 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d4003570 con 0x7f56f009faa0 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3412781963 0 0) 0x7f56d4003570 con 0x7f56f009faa0 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56f0138a40 con 0x7f56f009faa0 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249180857 0 0) 0x7f56f009cfa0 con 0x7f56f0138d40 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): my global_id is 184152 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d4003570 con 0x7f56f0138d40 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56ec001b20 con 0x7f56f009faa0 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): dump: 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781818235 0 0) 0x7f56f0137440 con 0x7f56f0132b40 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient(hunting): my global_id is 5153 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f009cfa0 con 0x7f56f0132b40 2024-09-15T03:51:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2962574613 0 0) 0x7f56f0138a40 con 0x7f56f009faa0 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6791/0 conn(0x7f56f0132b40 legacy=0x7f56f0135c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6790/0 conn(0x7f56f0138d40 legacy=0x7f56f0136370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 monclient: found mon.e 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f009ccc0 con 0x7f56f009faa0 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 5 monclient: authenticate success, global_id 134648 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient: _renew_subs 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:58.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f0136f10 con 0x7f56f009faa0 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient: _renew_subs 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f0137c80 con 0x7f56f009faa0 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f56ec001ff0 con 0x7f56f009faa0 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f4d5b700 4 set_mon_vals no callback set 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56ec002520 con 0x7f56f009faa0 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 10 monclient: dump: 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.401 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f56ec002700 con 0x7f56f009faa0 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f56d8006590 con 0x7f56d8002840 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.380+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(330..330 src has 224..330) v4 ==== 3895+0+0 (unknown 14320229 0 0) 0x7f56ec043f80 con 0x7f56f009faa0 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56f6d5f700 10 monclient: get_auth_request con 0x7f56d4037740 auth_method 0 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285582626 0 0) 0x7f56d8006590 con 0x7f56d8002840 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient(hunting): my global_id is 5162 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f0138a40 con 0x7f56d8002840 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3742705582 0 0) 0x7f56f0138a40 con 0x7f56d8002840 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56d8006590 con 0x7f56d8002840 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56e0001900 con 0x7f56d8002840 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient: dump: 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.404 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.405 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 270269261 0 0) 0x7f56d8006590 con 0x7f56d8002840 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56d800a130 con 0x7f56d8002840 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f56e0005920 con 0x7f56d8002840 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6791/0 conn(0x7f56d8002840 legacy=0x7f56d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.384+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6791/0 conn(0x7f56d8002840 legacy=0x7f56d8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56f7560700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f56d8009640 con 0x7f56d80090b0 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210889917 0 0) 0x7f56d8009640 con 0x7f56d80090b0 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient(hunting): my global_id is 5171 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d8006590 con 0x7f56d80090b0 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1520160715 0 0) 0x7f56d8006590 con 0x7f56d80090b0 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56d8009640 con 0x7f56d80090b0 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f56e0004de0 con 0x7f56d80090b0 2024-09-15T03:51:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:51:58.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient: dump: 2024-09-15T03:51:58.409 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.413 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1378502823 0 0) 0x7f56d8009640 con 0x7f56d80090b0 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f56d80098f0 con 0x7f56d80090b0 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7738 (unknown 0 0 2860079035) 0x7f56e0005bb0 con 0x7f56d80090b0 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6791/0 conn(0x7f56d80090b0 legacy=0x7f56d8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.396+0000 7f56e67fc700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6791/0 conn(0x7f56d80090b0 legacy=0x7f56d8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:58.415 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.416 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.417 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:58.418 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:58.419 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.421 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.422 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.422 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.422 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.422 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.422 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:58.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:58.423 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:58.425 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:58.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:58.429 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:58.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.431 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.434 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:58.435 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.436 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.439 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.439 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:58.439 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:58.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.122:6812/20895 conn(0x7f56d4037740 legacy=0x7f56d4039bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 10 monclient: shutdown 2024-09-15T03:51:58.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 >> v1:172.21.15.26:6793/0 conn(0x7f56f009faa0 legacy=0x7f56f0135550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f6d5f700 1 -- 172.21.15.26:0/3541875332 reap_dead start 2024-09-15T03:51:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f6d5f700 1 -- 172.21.15.26:0/3541875332 reap_dead start 2024-09-15T03:51:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 shutdown_connections 2024-09-15T03:51:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 >> 172.21.15.26:0/3541875332 conn(0x7f56f006e370 msgr2=0x7f56f009e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 shutdown_connections 2024-09-15T03:51:58.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.400+0000 7f56f7560700 1 -- 172.21.15.26:0/3541875332 wait complete. 2024-09-15T03:51:58.441 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:58.442 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:51:58.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:51:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient: get_monmap_and_config 2024-09-15T03:51:58.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient: monmap: 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:58.556774+0000 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:58.556774+0000 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.560 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.561 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.562 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 1 Processor -- start 2024-09-15T03:51:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 1 -- start start 2024-09-15T03:51:58.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient: init 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient: _add_conns ranks=[5,3,6,7,1,0,4,2,8] 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient(hunting): picked mon.noname-h con 0x7fd00c063a10 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient(hunting): picked mon.noname-g con 0x7fd00c0a04b0 addr v1:172.21.15.122:6790/0 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient(hunting): picked mon.noname-d con 0x7fd00c138ac0 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c04ca10 con 0x7fd00c0a04b0 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c09bb30 con 0x7fd00c063a10 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c063de0 con 0x7fd00c138ac0 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 10 monclient(hunting): authenticate will time out at 1262.986990s 2024-09-15T03:51:58.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00ad9d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd00c138ac0 0x7fd00c13aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53292/0 (socket says 172.21.15.26:53292) 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00ad9d700 1 -- 172.21.15.26:0/2950658835 learned_addr learned my addr 172.21.15.26:0/2950658835 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080255441 0 0) 0x7fd00c063de0 con 0x7fd00c138ac0 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 10 monclient(hunting): my global_id is 5199 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcff8003570 con 0x7fd00c138ac0 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077646803 0 0) 0x7fcff8003570 con 0x7fd00c138ac0 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 >> v1:172.21.15.122:6791/0 conn(0x7fd00c063a10 legacy=0x7fd00c09dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 >> v1:172.21.15.122:6790/0 conn(0x7fd00c0a04b0 legacy=0x7fd00c09b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 monclient: found mon.noname-d 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c068510 con 0x7fd00c138ac0 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 5 monclient: authenticate success, global_id 5199 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd004001b30 con 0x7fd00c138ac0 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.552+0000 7fd00b59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:51:58.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: dump: 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.574 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd004001e50 con 0x7fd00c138ac0 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 1 -- 172.21.15.26:0/2950658835 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd004004fb0 con 0x7fd00c138ac0 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 4 set_mon_vals no callback set 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd009d9b700 10 monclient: dump: 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.576 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: shutdown 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- 172.21.15.26:0/2950658835 >> v1:172.21.15.26:6792/0 conn(0x7fd00c138ac0 legacy=0x7fd00c13aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- 172.21.15.26:0/2950658835 shutdown_connections 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- 172.21.15.26:0/2950658835 >> 172.21.15.26:0/2950658835 conn(0x7fd00c0a4410 msgr2=0x7fd00c0a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- 172.21.15.26:0/2950658835 shutdown_connections 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- 172.21.15.26:0/2950658835 wait complete. 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: monmap: 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 Processor -- start 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- start start 2024-09-15T03:51:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: init 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient: _add_conns ranks=[7,6,5,2,4,8,3,0,1] 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient(hunting): picked mon.noname-i con 0x7fd00c063a10 addr v1:172.21.15.122:6792/0 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient(hunting): picked mon.noname-d con 0x7fd00c0a04b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient(hunting): picked mon.noname-h con 0x7fd00c138ac0 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c09aed0 con 0x7fd00c138ac0 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c06e0a0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c139cb0 con 0x7fd00c063a10 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd00b59e700 10 monclient(hunting): authenticate will time out at 1262.991891s 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.556+0000 7fd003fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd00c0a04b0 0x7fd00c1395a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53298/0 (socket says 172.21.15.26:53298) 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd003fff700 1 -- 172.21.15.26:0/946998252 learned_addr learned my addr 172.21.15.26:0/946998252 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312733939 0 0) 0x7fd00c06e0a0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): my global_id is 5208 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfec003570 con 0x7fd00c0a04b0 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638585022 0 0) 0x7fd00c139cb0 con 0x7fd00c063a10 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): my global_id is 134557 2024-09-15T03:51:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c06e0a0 con 0x7fd00c063a10 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2894462973 0 0) 0x7fcfec003570 con 0x7fd00c0a04b0 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd00c139cb0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcff4001b20 con 0x7fd00c0a04b0 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): dump: 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369524986 0 0) 0x7fd00c09aed0 con 0x7fd00c138ac0 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): my global_id is 5189 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfec003570 con 0x7fd00c138ac0 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3399174193 0 0) 0x7fd00c06e0a0 con 0x7fd00c063a10 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd00c09aed0 con 0x7fd00c063a10 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcffc001a60 con 0x7fd00c063a10 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:51:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): dump: 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2142736567 0 0) 0x7fd00c139cb0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6792/0 conn(0x7fd00c063a10 legacy=0x7fd00c138e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6791/0 conn(0x7fd00c138ac0 legacy=0x7fd00c149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 monclient: found mon.d 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c13ad90 con 0x7fd00c0a04b0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 5 monclient: authenticate success, global_id 5208 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 10 monclient: _renew_subs 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c13a880 con 0x7fd00c0a04b0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcff40021b0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 10 monclient: _renew_subs 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:51:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c138570 con 0x7fd00c0a04b0 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcff40026e0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00959a700 4 set_mon_vals no callback set 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient: dump: 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.589 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd00c04bbf0 con 0x7fd00c09c380 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcff40028c0 con 0x7fd00c0a04b0 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(330..330 src has 224..330) v4 ==== 3895+0+0 (unknown 14320229 0 0) 0x7fcff4043f90 con 0x7fd00c0a04b0 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139225281 0 0) 0x7fd00c04bbf0 con 0x7fd00c09c380 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.560+0000 7fd0027fc700 10 monclient(hunting): my global_id is 134566 2024-09-15T03:51:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c139cb0 con 0x7fd00c09c380 2024-09-15T03:51:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd00ad9d700 10 monclient: get_auth_request con 0x7fcfec037d30 auth_method 0 2024-09-15T03:51:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 68825768 0 0) 0x7fd00c139cb0 con 0x7fd00c09c380 2024-09-15T03:51:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd00c04bbf0 con 0x7fd00c09c380 2024-09-15T03:51:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcffc0022c0 con 0x7fd00c09c380 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 10 monclient: dump: 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 128236231 0 0) 0x7fd00c04bbf0 con 0x7fd00c09c380 2024-09-15T03:51:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd00c133730 con 0x7fd00c09c380 2024-09-15T03:51:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fcffc004520 con 0x7fd00c09c380 2024-09-15T03:51:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6792/0 conn(0x7fd00c09c380 legacy=0x7fd00c09c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.564+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6792/0 conn(0x7fd00c09c380 legacy=0x7fd00c09c750 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.576+0000 7fd00b59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.576+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd00c06fac0 con 0x7fd00c09ce60 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197538098 0 0) 0x7fd00c06fac0 con 0x7fd00c09ce60 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient(hunting): my global_id is 134575 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c04bbf0 con 0x7fd00c09ce60 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1664466179 0 0) 0x7fd00c04bbf0 con 0x7fd00c09ce60 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd00c06fac0 con 0x7fd00c09ce60 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcffc002d10 con 0x7fd00c09ce60 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient: dump: 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3453880106 0 0) 0x7fd00c06fac0 con 0x7fd00c09ce60 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd00c067640 con 0x7fd00c09ce60 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2338538485) 0x7fcffc004210 con 0x7fd00c09ce60 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6792/0 conn(0x7fd00c09ce60 legacy=0x7fd00c02ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd0027fc700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6792/0 conn(0x7fd00c09ce60 legacy=0x7fd00c02ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:51:58.599 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.600 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.601 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:51:58.603 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:51:58.604 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.605 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.609 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.611 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:51:58.612 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.613 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:51:58.615 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:51:58.616 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.617 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:51:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.618 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:51:58.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:51:58.622 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:51:58.623 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:51:58.624 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:51:58.625 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:51:58.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.122:6812/20895 conn(0x7fcfec037d30 legacy=0x7fcfec03a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 10 monclient: shutdown 2024-09-15T03:51:58.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 >> v1:172.21.15.26:6792/0 conn(0x7fd00c0a04b0 legacy=0x7fd00c1395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00ad9d700 1 -- 172.21.15.26:0/946998252 reap_dead start 2024-09-15T03:51:58.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00ad9d700 1 -- 172.21.15.26:0/946998252 reap_dead start 2024-09-15T03:51:58.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 shutdown_connections 2024-09-15T03:51:58.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 >> 172.21.15.26:0/946998252 conn(0x7fd00c0a4410 msgr2=0x7fd00c0a09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 shutdown_connections 2024-09-15T03:51:58.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.580+0000 7fd00b59e700 1 -- 172.21.15.26:0/946998252 wait complete. 2024-09-15T03:51:58.627 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:51:58.627 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T03:51:58.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient: get_monmap_and_config 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient: monmap: 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:51:58.741615+0000 2024-09-15T03:51:58.743 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:51:58.741615+0000 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.744 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 1 Processor -- start 2024-09-15T03:51:58.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 1 -- start start 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient: init 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient: _add_conns ranks=[5,7,4,3,6,0,8,2,1] 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient(hunting): picked mon.noname-h con 0x7f88f80a6a80 addr v1:172.21.15.122:6791/0 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient(hunting): picked mon.noname-i con 0x7f88f809fb80 addr v1:172.21.15.122:6792/0 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient(hunting): picked mon.noname-c con 0x7f88f8138d30 addr v1:172.21.15.26:6791/0 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f804ca10 con 0x7f88f8138d30 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f80a0f50 con 0x7f88f80a6a80 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f8133680 con 0x7f88f809fb80 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fd442700 10 monclient(hunting): authenticate will time out at 1263.170997s 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fcc41700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f88f8138d30 0x7f88f813b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55406/0 (socket says 172.21.15.26:55406) 2024-09-15T03:51:58.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88fcc41700 1 -- 172.21.15.26:0/3986552399 learned_addr learned my addr 172.21.15.26:0/3986552399 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780494774 0 0) 0x7f88f804ca10 con 0x7f88f8138d30 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88f6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88f6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88f6ffd700 10 monclient(hunting): my global_id is 184180 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d8003570 con 0x7f88f8138d30 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.736+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628877646 0 0) 0x7f88f80a0f50 con 0x7f88f80a6a80 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient(hunting): my global_id is 5198 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f804ca10 con 0x7f88f80a6a80 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679783562 0 0) 0x7f88f8133680 con 0x7f88f809fb80 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient(hunting): my global_id is 134584 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f80a0f50 con 0x7f88f809fb80 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2934321553 0 0) 0x7f88d8003570 con 0x7f88f8138d30 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 >> v1:172.21.15.122:6792/0 conn(0x7f88f809fb80 legacy=0x7f88f809ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 >> v1:172.21.15.122:6791/0 conn(0x7f88f80a6a80 legacy=0x7f88f8132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 monclient: found mon.noname-c 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f80a6e50 con 0x7f88f8138d30 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 5 monclient: authenticate success, global_id 184180 2024-09-15T03:51:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f88f00017b0 con 0x7f88f8138d30 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: dump: 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.753 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f88f0001a90 con 0x7f88f8138d30 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: get_monmap_and_config success 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 1 -- 172.21.15.26:0/3986552399 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f88f0004c30 con 0x7f88f8138d30 2024-09-15T03:51:58.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 4 set_mon_vals no callback set 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88f6ffd700 10 monclient: dump: 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: shutdown 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 -- 172.21.15.26:0/3986552399 >> v1:172.21.15.26:6791/0 conn(0x7f88f8138d30 legacy=0x7f88f813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 -- 172.21.15.26:0/3986552399 shutdown_connections 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 -- 172.21.15.26:0/3986552399 >> 172.21.15.26:0/3986552399 conn(0x7f88f806e380 msgr2=0x7f88f80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 -- 172.21.15.26:0/3986552399 shutdown_connections 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 -- 172.21.15.26:0/3986552399 wait complete. 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: build_initial_monmap 2024-09-15T03:51:58.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: monmap: 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:51:58.757 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:51:58.758 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:51:58.758 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 Processor -- start 2024-09-15T03:51:58.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 1 -- start start 2024-09-15T03:51:58.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: init 2024-09-15T03:51:58.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: _reopen_session rank -1 2024-09-15T03:51:58.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.740+0000 7f88fd442700 10 monclient: _add_conns ranks=[7,1,0,6,3,5,8,4,2] 2024-09-15T03:51:58.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient(hunting): picked mon.noname-i con 0x7f88f809fb80 addr v1:172.21.15.122:6792/0 2024-09-15T03:51:58.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient(hunting): picked mon.noname-f con 0x7f88f80a6a80 addr v1:172.21.15.122:6789/0 2024-09-15T03:51:58.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient(hunting): picked mon.noname-a con 0x7f88f8138d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:51:58.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f809b830 con 0x7f88f8138d30 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f809cfc0 con 0x7f88f80a6a80 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f80a1920 con 0x7f88f809fb80 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient(hunting): _renew_subs 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient(hunting): authenticate will time out at 1263.176183s 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fcc41700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f88f8138d30 0x7f88f814a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38326/0 (socket says 172.21.15.26:38326) 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fcc41700 1 -- 172.21.15.26:0/2755257630 learned_addr learned my addr 172.21.15.26:0/2755257630 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837479978 0 0) 0x7f88f809b830 con 0x7f88f8138d30 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): my global_id is 5049 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88dc003570 con 0x7f88f8138d30 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837735627 0 0) 0x7f88f80a1920 con 0x7f88f809fb80 2024-09-15T03:51:58.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): my global_id is 134593 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f809b830 con 0x7f88f809fb80 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520966293 0 0) 0x7f88f809cfc0 con 0x7f88f80a6a80 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): my global_id is 184150 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f80a1920 con 0x7f88f80a6a80 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 653909636 0 0) 0x7f88dc003570 con 0x7f88f8138d30 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88f809cfc0 con 0x7f88f8138d30 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f88f0004b70 con 0x7f88f8138d30 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:51:58.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): dump: 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.764 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2874096572 0 0) 0x7f88f809cfc0 con 0x7f88f8138d30 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 >> v1:172.21.15.122:6792/0 conn(0x7f88f809fb80 legacy=0x7f88f8099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 >> v1:172.21.15.122:6789/0 conn(0x7f88f80a6a80 legacy=0x7f88f809a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: _finish_hunting 0 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 monclient: found mon.a 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f809b2b0 con 0x7f88f8138d30 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: _finish_auth 0 2024-09-15T03:51:58.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 5 monclient: authenticate success, global_id 5049 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient: _renew_subs 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f81329b0 con 0x7f88f8138d30 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient: _renew_subs 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f809c020 con 0x7f88f8138d30 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f88f0001100 con 0x7f88f8138d30 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f88f00051a0 con 0x7f88f8138d30 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88f5ffb700 4 set_mon_vals no callback set 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:51:58.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 10 monclient: dump: 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:51:58.767 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f88f000fb60 con 0x7f88f8138d30 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d00050b0 con 0x7f88f8138d30 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(330..330 src has 224..330) v4 ==== 3895+0+0 (unknown 14320229 0 0) 0x7f88f0043ff0 con 0x7f88f8138d30 2024-09-15T03:51:58.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.744+0000 7f88f7fff700 10 monclient: get_auth_request con 0x7f88dc0376f0 auth_method 0 2024-09-15T03:51:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.752+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f88f0014f50 con 0x7f88f8138d30 2024-09-15T03:51:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.752+0000 7f88effff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:51:58.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.752+0000 7f88effff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:51:58.875 INFO:tasks.rados.rados.0.smithi122.stdout:1415: done (5 left) 2024-09-15T03:51:58.876 INFO:tasks.rados.rados.0.smithi122.stdout:1416: read oid 45 snap 155 2024-09-15T03:51:58.876 INFO:tasks.rados.rados.0.smithi122.stdout:1416: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:51:58.876 INFO:tasks.rados.rados.0.smithi122.stdout:1417: delete oid 47 current snap is 156 2024-09-15T03:51:58.876 INFO:tasks.rados.rados.0.smithi122.stdout:1409: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:51:58.876 INFO:tasks.rados.rados.0.smithi122.stdout:1410: expect (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:51:59.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.996+0000 7f88fd442700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T03:51:59.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.996+0000 7f88fd442700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T03:51:59.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.996+0000 7f88fd442700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:51:59.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:58.996+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f88d0005b70 con 0x7f88f8138d30 2024-09-15T03:51:59.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88effff700 1 -- 172.21.15.26:0/2755257630 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f88f0014950 con 0x7f88f8138d30 2024-09-15T03:51:59.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88effff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T03:51:59.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88effff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:51:59.011 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.011 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=24,config=2,health=11,kv=7,logm=56} crc {auth=18636837,config=3140514009,health=1729694744,kv=1358559433,logm=3469656716}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=24,config=2,health=12,kv=8,logm=54} crc {auth=1080840657,config=3140514009,health=688882541,kv=2519098299,logm=711261984}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=23,config=2,health=12,kv=7,logm=56} crc {auth=4084983576,config=3140514009,health=1786782541,kv=3768028330,logm=36269128}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=24,config=2,health=12,kv=8,logm=54} crc {auth=288166306,config=3140514009,health=688882541,kv=2519098299,logm=166829848}) 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.012 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=24,config=2,health=12,kv=7,logm=55} crc {auth=3790575765,config=3140514009,health=1509387093,kv=777372615,logm=2080251374}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=12,kv=7,logm=55} crc {auth=2766716965,config=3140514009,health=1047516845,kv=1358559433,logm=1155783605}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=24,config=2,health=12,kv=8,logm=54} crc {auth=2907324433,config=3140514009,health=688882541,kv=2519098299,logm=2338334416}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=1,health=12,kv=7,logm=56} crc {auth=3133856025,config=1228700967,health=688882541,kv=1358559433,logm=1585095923}) 2024-09-15T03:51:59.013 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=24,config=2,health=12,kv=8,logm=54} crc {auth=4226638029,config=3140514009,health=688882541,kv=2519098299,logm=4139440137}) 2024-09-15T03:51:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 >> v1:172.21.15.122:6812/20895 conn(0x7f88dc0376f0 legacy=0x7f88dc039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 10 monclient: shutdown 2024-09-15T03:51:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:51:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 >> v1:172.21.15.26:6789/0 conn(0x7f88f8138d30 legacy=0x7f88f814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:51:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 shutdown_connections 2024-09-15T03:51:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 >> 172.21.15.26:0/2755257630 conn(0x7f88f806e380 msgr2=0x7f88f80a4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:51:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 shutdown_connections 2024-09-15T03:51:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:51:59.004+0000 7f88fd442700 1 -- 172.21.15.26:0/2755257630 wait complete. 2024-09-15T03:51:59.019 INFO:tasks.rados.rados.0.smithi122.stdout:1410: done (6 left) 2024-09-15T03:51:59.020 INFO:tasks.rados.rados.0.smithi122.stdout:1417: done (5 left) 2024-09-15T03:51:59.020 INFO:tasks.rados.rados.0.smithi122.stdout:1418: write (excl) oid 22 current snap is 156 2024-09-15T03:51:59.020 INFO:tasks.rados.rados.0.smithi122.stdout:1418: seq_num 522 ranges {773254=791982,2346424=426830} 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=983628883}) 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3325868498}) 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.030 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4259827571}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2996952577}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=329047985}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2196751793}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3492763290}) 2024-09-15T03:51:59.031 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568655263}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=873452158}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3160835694}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.008+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2654564140}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2237968752}) 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.032 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=304960070}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2330729539}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=429346454}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2501007324}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344198678}) 2024-09-15T03:51:59.033 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1211623766}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284002652}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3393655625}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1388240982}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.034 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3246476188}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3091889000}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2711440334}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1434089596}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2535361153}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3050726156}) 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.035 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=513077861}) 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4064414370}) 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2682639842}) 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.036 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3961571320}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.012+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1570785387}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=294182444}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3011892638}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=290835899}) 2024-09-15T03:51:59.037 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1496925469}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=484057280}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=738279597}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2502598922}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3614143077}) 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.038 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1186040482}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2202853596}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1111063330}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2225517748}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1164880359}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.039 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4036358166}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3738466543}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2344306339}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3006245799}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2774859466}) 2024-09-15T03:51:59.040 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3935584939}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2346391303}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=658640999}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.016+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3656056919}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=300984893}) 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.041 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=357342697}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3072357146}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2257391055}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4244763187}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=325712534}) 2024-09-15T03:51:59.042 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.043 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.043 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1501695730}) 2024-09-15T03:51:59.043 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.043 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=568513754}) 2024-09-15T03:51:59.043 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3300312871}) 2024-09-15T03:51:59.044 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T03:51:59.046 INFO:tasks.rados.rados.0.smithi122.stdout:1418: writing smithi12223906-22 from 773254 to 1565236 tid 1 2024-09-15T03:51:59.046 INFO:tasks.rados.rados.0.smithi122.stdout:1418: writing smithi12223906-22 from 2346424 to 2773254 tid 2 2024-09-15T03:51:59.047 INFO:tasks.rados.rados.0.smithi122.stdout:1413: done (5 left) 2024-09-15T03:51:59.047 INFO:tasks.rados.rados.0.smithi122.stdout:1419: copy_from oid 27 from oid 39 current snap is 156 2024-09-15T03:51:59.047 INFO:tasks.rados.rados.0.smithi122.stdout:1409: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:51:59.047 INFO:tasks.rados.rados.0.smithi122.stdout:1409: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:51:59.047 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 563 (ObjNum 508 snap 151 seq_num 508) dirty exists 2024-09-15T03:51:59.047 INFO:tasks.rados.rados.0.smithi122.stdout:1411: expect (ObjNum 512 snap 152 seq_num 512) 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=926802072}) 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2037269754}) 2024-09-15T03:51:59.074 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=7096480}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=661618831}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1410583534}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=421290196}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.020+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.075 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2586663283}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1458535918}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1829612109}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1566660720}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1587668882}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2324684106}) 2024-09-15T03:51:59.076 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=867610269}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1781861891}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=221799267}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2258192627}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4154692428}) 2024-09-15T03:51:59.077 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=202127454}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=432588733}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=913092372}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=768144131}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2387627357}) 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.078 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1410149697}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3686168860}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1994380606}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1365741188}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.024+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2696238374}) 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.079 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1056079032}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3531063665}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3405748289}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2033863611}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.080 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=116340083}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3957903803}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1288032279}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2344662633}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.028+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=462178290}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353629986}) 2024-09-15T03:51:59.081 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=632326495}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1277012358}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1770043973}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2331910954}) 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.082 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2403677137}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=481203534}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3362269564}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=932933511}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3319911774}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2062731085}) 2024-09-15T03:51:59.083 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2696925576}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2579522834}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=291999321}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4031194023}) 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.084 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1323574832}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=665052412}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.032+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1016613157}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3044610360}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1083728745}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.085 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1395822690}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1519298522}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1122945099}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1820192847}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3179660270}) 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.086 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177961873}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.036+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1136113760}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=543655990}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3482509464}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1600213428}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.087 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1800199088}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3792918885}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2160059643}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4221843988}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1712634801}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2619550121}) 2024-09-15T03:51:59.088 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4111865032}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1550045624}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1227978820}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1387312220}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.089 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2690208137}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1256833535}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=621996034}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2351737966}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2762736875}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4280434520}) 2024-09-15T03:51:59.090 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=597568771}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3640253802}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2369539290}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=415110069}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.091 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1339306618}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1103052316}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2480698359}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.040+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2616072226}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1336024059}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2952607790}) 2024-09-15T03:51:59.092 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1665829327}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1064671169}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=932569265}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2066925089}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.093 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3651750489}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=829858728}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3611483435}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3436465778}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=855802793}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=98,mdsmap=2} crc {logm=1118245559,mdsmap=3585761405}) 2024-09-15T03:51:59.094 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3413427832}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=99,mdsmap=1} crc {logm=1928177601,mdsmap=2826658659}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=97,mdsmap=3} crc {logm=1116148398,mdsmap=1531151339}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1508272376}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346860384}) 2024-09-15T03:51:59.095 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.044+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1232757975}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=51,osdmap=13} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2187687263,mon_config_key=733955277,monmap=4273209975,osd_metadata=2270045001,osd_pg_creating=2939340551,osd_snap=2477858800,osdmap=4154405090}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=1,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=53,osdmap=11} crc {logm=3040917445,mdsmap=1531151339,mgr=436553390,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=118667829,mon_config_key=733955277,monmap=4273209975,osd_metadata=2459566930,osd_pg_creating=2939340551,osd_snap=2750958187,osdmap=1841364643}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=54,osdmap=10} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2187687263,mon_config_key=733955277,monmap=2173287178,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2812659227,osdmap=355784262}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.096 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=1,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50,osdmap=14} crc {logm=3040917445,mdsmap=1531151339,mgr=991478979,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2037256730,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=1133804817,osdmap=4182272001}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=1,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=53,osdmap=10} crc {logm=3040917445,mdsmap=1531151339,mgr=436553390,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=118667829,mon_config_key=733955277,monmap=4273209975,osd_metadata=1529822416,osd_pg_creating=2939340551,osd_snap=801282818,osdmap=355784262}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50,osdmap=15} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=3431271968,mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=1177143022,osdmap=3452739888}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=1,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=1568706250,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=118667829,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=3659072479,osdmap=2002743283}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=51,osdmap=12} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1081971162,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2939340551,osd_snap=2698089462,osdmap=2002743283}) 2024-09-15T03:51:59.097 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.048+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=1,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=51,osdmap=13} crc {logm=3040917445,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=3389745721,mon_config_key=733955277,monmap=4273209975,osd_metadata=2459566930,osd_pg_creating=2939340551,osd_snap=1572254858,osdmap=1739423800}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=807850101}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4151342340}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=744536482}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3960705954}) 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.098 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2036337115}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=71613627}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4159503205}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2334033302}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1764285335}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.099 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=59} crc {osdmap=3790188456}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=61} crc {osdmap=292770603}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=60} crc {osdmap=4283044904}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=61} crc {osdmap=3040516424}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=60} crc {osdmap=2928040754}) 2024-09-15T03:51:59.100 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=61} crc {osdmap=10758662}) 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=1355311515}) 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=59} crc {osdmap=1891365234}) 2024-09-15T03:51:59.101 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T03:51:59.052+0000 7f2f95b1c700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=59} crc {osdmap=3048570867}) 2024-09-15T03:51:59.145 INFO:tasks.rados.rados.0.smithi122.stdout:1414: finishing write tid 1 to smithi12223906-26 2024-09-15T03:51:59.146 INFO:tasks.rados.rados.0.smithi122.stdout:1414: finishing write tid 2 to smithi12223906-26 2024-09-15T03:51:59.146 INFO:tasks.rados.rados.0.smithi122.stdout:1414: finishing write tid 3 to smithi12223906-26 2024-09-15T03:51:59.146 INFO:tasks.rados.rados.0.smithi122.stdout:1414: finishing write tid 4 to smithi12223906-26 2024-09-15T03:51:59.146 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 399 (ObjNum 520 snap 156 seq_num 520) dirty exists 2024-09-15T03:51:59.146 INFO:tasks.rados.rados.0.smithi122.stdout:1414: left oid 26 (ObjNum 520 snap 156 seq_num 520) 2024-09-15T03:51:59.146 INFO:tasks.rados.rados.0.smithi122.stdout:1416: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:51:59.259 INFO:tasks.rados.rados.0.smithi122.stdout:1418: finishing write tid 1 to smithi12223906-22 2024-09-15T03:51:59.259 INFO:tasks.rados.rados.0.smithi122.stdout:1419: finishing copy_from to smithi12223906-27 2024-09-15T03:51:59.259 INFO:tasks.rados.rados.0.smithi122.stdout:1419: got expected ENOENT (src dne) 2024-09-15T03:51:59.259 INFO:tasks.rados.rados.0.smithi122.stdout:1419: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:51:59.259 INFO:tasks.rados.rados.0.smithi122.stdout:1418: finishing write tid 2 to smithi12223906-22 2024-09-15T03:51:59.259 INFO:tasks.rados.rados.0.smithi122.stdout:1418: finishing write tid 3 to smithi12223906-22 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1418: finishing write tid 4 to smithi12223906-22 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 682 (ObjNum 522 snap 156 seq_num 522) dirty exists 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1418: left oid 22 (ObjNum 522 snap 156 seq_num 522) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1409: done (5 left) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1411: done (4 left) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1414: done (3 left) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1416: done (2 left) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1418: done (1 left) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1419: done (0 left) 2024-09-15T03:51:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:1420: delete oid 30 current snap is 156 2024-09-15T03:51:59.265 INFO:tasks.rados.rados.0.smithi122.stdout:1420: done (0 left) 2024-09-15T03:51:59.265 INFO:tasks.rados.rados.0.smithi122.stdout:1421: delete oid 28 current snap is 156 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:1421: done (0 left) 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:1422: rollback oid 19 current snap is 156 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 152 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:1423: read oid 18 snap -1 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:1423: expect (ObjNum 417 snap 126 seq_num 417) 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:1424: copy_from oid 43 from oid 48 current snap is 156 2024-09-15T03:51:59.273 INFO:tasks.rados.rados.0.smithi122.stdout:1425: snap_remove snap 155 2024-09-15T03:51:59.912 INFO:tasks.rados.rados.0.smithi122.stdout:1425: done (3 left) 2024-09-15T03:51:59.912 INFO:tasks.rados.rados.0.smithi122.stdout:1426: copy_from oid 24 from oid 36 current snap is 156 2024-09-15T03:51:59.912 INFO:tasks.rados.rados.0.smithi122.stdout:1422: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:51:59.912 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 0 (ObjNum 65 snap 0 seq_num 1842288896) dirty dne 2024-09-15T03:51:59.913 INFO:tasks.rados.rados.0.smithi122.stdout:1423: expect (ObjNum 417 snap 126 seq_num 417) 2024-09-15T03:52:00.045 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:52:00.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:52:00.046 INFO:tasks.rados.rados.0.smithi122.stdout:1424: finishing copy_from racing read to smithi12223906-43 2024-09-15T03:52:00.046 INFO:tasks.rados.rados.0.smithi122.stdout:1424: finishing copy_from to smithi12223906-43 2024-09-15T03:52:00.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 375 (ObjNum 427 snap 127 seq_num 427) dirty exists 2024-09-15T03:52:00.046 INFO:tasks.rados.rados.0.smithi122.stdout:1426: finishing copy_from to smithi12223906-24 2024-09-15T03:52:00.046 INFO:tasks.rados.rados.0.smithi122.stdout:1426: got expected ENOENT (src dne) 2024-09-15T03:52:00.046 INFO:tasks.rados.rados.0.smithi122.stdout:1426: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:52:00.047 INFO:tasks.rados.rados.0.smithi122.stdout:1422: done (3 left) 2024-09-15T03:52:00.047 INFO:tasks.rados.rados.0.smithi122.stdout:1423: done (2 left) 2024-09-15T03:52:00.047 INFO:tasks.rados.rados.0.smithi122.stdout:1424: done (1 left) 2024-09-15T03:52:00.047 INFO:tasks.rados.rados.0.smithi122.stdout:1426: done (0 left) 2024-09-15T03:52:00.047 INFO:tasks.rados.rados.0.smithi122.stdout:1427: snap_create 2024-09-15T03:52:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient: get_monmap_and_config 2024-09-15T03:52:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient: monmap: 2024-09-15T03:52:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:00.150044+0000 2024-09-15T03:52:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:00.150044+0000 2024-09-15T03:52:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.154 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.154 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.154 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.154 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.154 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.155 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.155 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.155 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.155 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.155 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 1 Processor -- start 2024-09-15T03:52:00.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 1 -- start start 2024-09-15T03:52:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient: init 2024-09-15T03:52:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient: _add_conns ranks=[0,8,1,4,5,2,6,7,3] 2024-09-15T03:52:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient(hunting): picked mon.noname-a con 0x7f0a08070f80 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:00.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient(hunting): picked mon.noname-e con 0x7f0a0809dee0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient(hunting): picked mon.noname-f con 0x7f0a081391e0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0804ca10 con 0x7f0a08070f80 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0809edc0 con 0x7f0a081391e0 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0809eae0 con 0x7f0a0809dee0 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.144+0000 7f0a0e7ee700 10 monclient(hunting): authenticate will time out at 1264.579927s 2024-09-15T03:52:00.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0d7ec700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a08070f80 0x7f0a080950b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38328/0 (socket says 172.21.15.26:38328) 2024-09-15T03:52:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0d7ec700 1 -- 172.21.15.26:0/4113637667 learned_addr learned my addr 172.21.15.26:0/4113637667 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614137997 0 0) 0x7f0a0804ca10 con 0x7f0a08070f80 2024-09-15T03:52:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient(hunting): my global_id is 5058 2024-09-15T03:52:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f0003570 con 0x7f0a08070f80 2024-09-15T03:52:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047424086 0 0) 0x7f0a0809eae0 con 0x7f0a0809dee0 2024-09-15T03:52:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient(hunting): my global_id is 134657 2024-09-15T03:52:00.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a0804ca10 con 0x7f0a0809dee0 2024-09-15T03:52:00.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4055644645 0 0) 0x7f09f0003570 con 0x7f0a08070f80 2024-09-15T03:52:00.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 >> v1:172.21.15.26:6793/0 conn(0x7f0a0809dee0 legacy=0x7f0a080957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 >> v1:172.21.15.122:6789/0 conn(0x7f0a081391e0 legacy=0x7f0a0813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 monclient: found mon.noname-a 2024-09-15T03:52:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:52:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0809e800 con 0x7f0a08070f80 2024-09-15T03:52:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 5 monclient: authenticate success, global_id 5058 2024-09-15T03:52:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f09f80017b0 con 0x7f0a08070f80 2024-09-15T03:52:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:52:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: dump: 2024-09-15T03:52:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f09f80019d0 con 0x7f0a08070f80 2024-09-15T03:52:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 4 set_mon_vals no callback set 2024-09-15T03:52:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 1 -- 172.21.15.26:0/4113637667 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f09f80048f0 con 0x7f0a08070f80 2024-09-15T03:52:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a07fff700 10 monclient: dump: 2024-09-15T03:52:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.181 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: shutdown 2024-09-15T03:52:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/4113637667 >> v1:172.21.15.26:6789/0 conn(0x7f0a08070f80 legacy=0x7f0a080950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/4113637667 shutdown_connections 2024-09-15T03:52:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/4113637667 >> 172.21.15.26:0/4113637667 conn(0x7f0a080a24e0 msgr2=0x7f0a080a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/4113637667 shutdown_connections 2024-09-15T03:52:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/4113637667 wait complete. 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: monmap: 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 Processor -- start 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- start start 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: init 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient: _add_conns ranks=[7,5,3,0,1,2,6,4,8] 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient(hunting): picked mon.noname-i con 0x7f0a08070f80 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient(hunting): picked mon.noname-h con 0x7f0a0809dee0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 10 monclient(hunting): picked mon.noname-g con 0x7f0a081391e0 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a08097410 con 0x7f0a081391e0 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a08128860 con 0x7f0a0809dee0 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.148+0000 7f0a0e7ee700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a08097130 con 0x7f0a08070f80 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient(hunting): authenticate will time out at 1264.584021s 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0cfeb700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f0a0809dee0 0x7f0a08128150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:46676/0 (socket says 172.21.15.26:46676) 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0d7ec700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f0a08070f80 0x7f0a08127a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:46330/0 (socket says 172.21.15.26:46330) 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0dfed700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f0a081391e0 0x7f0a0814a690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:54222/0 (socket says 172.21.15.26:54222) 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0cfeb700 1 -- 172.21.15.26:0/978181511 learned_addr learned my addr 172.21.15.26:0/978181511 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942200677 0 0) 0x7f0a08128860 con 0x7f0a0809dee0 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): my global_id is 5207 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f4003570 con 0x7f0a0809dee0 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518233509 0 0) 0x7f0a08097130 con 0x7f0a08070f80 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): my global_id is 134602 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a08128860 con 0x7f0a08070f80 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704043322 0 0) 0x7f0a08097410 con 0x7f0a081391e0 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): my global_id is 184170 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a08097130 con 0x7f0a081391e0 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2074221603 0 0) 0x7f09f4003570 con 0x7f0a0809dee0 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a08097410 con 0x7f0a0809dee0 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f09fc001d10 con 0x7f0a0809dee0 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): dump: 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3367154520 0 0) 0x7f0a08128860 con 0x7f0a08070f80 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09f4003570 con 0x7f0a08070f80 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f09f80016b0 con 0x7f0a08070f80 2024-09-15T03:52:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): dump: 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1443123279 0 0) 0x7f0a08097130 con 0x7f0a081391e0 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a08128860 con 0x7f0a081391e0 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a00002cc0 con 0x7f0a081391e0 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient(hunting): dump: 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3446089159 0 0) 0x7f0a08097410 con 0x7f0a0809dee0 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 >> v1:172.21.15.122:6792/0 conn(0x7f0a08070f80 legacy=0x7f0a08127a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 >> v1:172.21.15.122:6790/0 conn(0x7f0a081391e0 legacy=0x7f0a0814a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 monclient: found mon.h 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080976f0 con 0x7f0a0809dee0 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 5 monclient: authenticate success, global_id 5207 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient: _renew_subs 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0812a160 con 0x7f0a0809dee0 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient: _renew_subs 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a08098070 con 0x7f0a0809dee0 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f09fc0017a0 con 0x7f0a0809dee0 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f09fc002570 con 0x7f0a0809dee0 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a077fe700 4 set_mon_vals no callback set 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 10 monclient: dump: 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:00.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e80050b0 con 0x7f0a0809dee0 2024-09-15T03:52:00.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.152+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f09fc00fb20 con 0x7f0a0809dee0 2024-09-15T03:52:00.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.156+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(332..332 src has 224..332) v4 ==== 3939+0+0 (unknown 2925508691 0 0) 0x7f09fc043ea0 con 0x7f0a0809dee0 2024-09-15T03:52:00.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.156+0000 7f0a0d7ec700 10 monclient: get_auth_request con 0x7f09f4038100 auth_method 0 2024-09-15T03:52:00.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.164+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f09fc014d20 con 0x7f0a0809dee0 2024-09-15T03:52:00.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.164+0000 7f0a057fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.164+0000 7f0a057fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a0e7ee700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:52:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a0e7ee700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a0e7ee700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f09e80058e0 con 0x7f0a0809dee0 2024-09-15T03:52:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a057fa700 1 -- 172.21.15.26:0/978181511 <== mon.5 v1:172.21.15.122:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 3237905361) 0x7f09fc014720 con 0x7f0a0809dee0 2024-09-15T03:52:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a057fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.384+0000 7f0a057fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:00.393 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:52:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 >> v1:172.21.15.122:6812/20895 conn(0x7f09f4038100 legacy=0x7f09f403a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 10 monclient: shutdown 2024-09-15T03:52:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 >> v1:172.21.15.122:6791/0 conn(0x7f0a0809dee0 legacy=0x7f0a08128150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 shutdown_connections 2024-09-15T03:52:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 >> 172.21.15.26:0/978181511 conn(0x7f0a080a24e0 msgr2=0x7f0a08095470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 shutdown_connections 2024-09-15T03:52:00.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.388+0000 7f0a0e7ee700 1 -- 172.21.15.26:0/978181511 wait complete. 2024-09-15T03:52:00.407 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:52:00.407 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:52:00.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:52:00.407 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:52:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.508+0000 7fc51ab72700 10 monclient: get_monmap_and_config 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.508+0000 7fc51ab72700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.508+0000 7fc51ab72700 10 monclient: monmap: 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:00.515344+0000 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:00.515344+0000 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.518 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 Processor -- start 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- start start 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: init 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: _add_conns ranks=[0,2,1,6,3,5,8,4,7] 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient(hunting): picked mon.noname-a con 0x7fc514132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient(hunting): picked mon.noname-b con 0x7fc51409faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient(hunting): picked mon.noname-f con 0x7fc514138d40 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51404ca10 con 0x7fc514132b40 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51409d5d0 con 0x7fc514138d40 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc514133620 con 0x7fc51409faa0 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient(hunting): authenticate will time out at 1264.944713s 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51936f700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fc51409faa0 0x7fc51409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39594/0 (socket says 172.21.15.26:39594) 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51936f700 1 -- 172.21.15.26:0/41339402 learned_addr learned my addr 172.21.15.26:0/41339402 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385141579 0 0) 0x7fc514133620 con 0x7fc51409faa0 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient(hunting): my global_id is 134678 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500003570 con 0x7fc51409faa0 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885474161 0 0) 0x7fc51404ca10 con 0x7fc514132b40 2024-09-15T03:52:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient(hunting): my global_id is 5067 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc514133620 con 0x7fc514132b40 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179204887 0 0) 0x7fc500003570 con 0x7fc51409faa0 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 >> v1:172.21.15.122:6789/0 conn(0x7fc514138d40 legacy=0x7fc51413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 >> v1:172.21.15.26:6789/0 conn(0x7fc514132b40 legacy=0x7fc514132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 monclient: found mon.noname-b 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140a0cc0 con 0x7fc51409faa0 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5100017b0 con 0x7fc51409faa0 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 5 monclient: authenticate success, global_id 134678 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: dump: 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc510001a20 con 0x7fc51409faa0 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 1 -- 172.21.15.26:0/41339402 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc510004c30 con 0x7fc51409faa0 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 4 set_mon_vals no callback set 2024-09-15T03:52:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc518b6e700 10 monclient: dump: 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: shutdown 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- 172.21.15.26:0/41339402 >> v1:172.21.15.26:6790/0 conn(0x7fc51409faa0 legacy=0x7fc51409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- 172.21.15.26:0/41339402 shutdown_connections 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- 172.21.15.26:0/41339402 >> 172.21.15.26:0/41339402 conn(0x7fc51406e370 msgr2=0x7fc5140a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- 172.21.15.26:0/41339402 shutdown_connections 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- 172.21.15.26:0/41339402 wait complete. 2024-09-15T03:52:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: monmap: 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 Processor -- start 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 1 -- start start 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.512+0000 7fc51ab72700 10 monclient: init 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient: _add_conns ranks=[1,4,2,7,3,5,6,0,8] 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient(hunting): picked mon.noname-f con 0x7fc51409faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient(hunting): picked mon.noname-c con 0x7fc514132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient(hunting): picked mon.noname-b con 0x7fc514138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51409c6f0 con 0x7fc51409faa0 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51409ccc0 con 0x7fc514138d40 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc514098880 con 0x7fc514132b40 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient(hunting): authenticate will time out at 1264.948447s 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51936f700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fc514132b40 0x7fc514098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55420/0 (socket says 172.21.15.26:55420) 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51936f700 1 -- 172.21.15.26:0/1313200810 learned_addr learned my addr 172.21.15.26:0/1313200810 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095493601 0 0) 0x7fc514098880 con 0x7fc514132b40 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): my global_id is 184189 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f8003570 con 0x7fc514132b40 2024-09-15T03:52:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346592283 0 0) 0x7fc51409ccc0 con 0x7fc514138d40 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): my global_id is 134687 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc514098880 con 0x7fc514138d40 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 495081390 0 0) 0x7fc4f8003570 con 0x7fc514132b40 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc51409ccc0 con 0x7fc514132b40 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc510001830 con 0x7fc514132b40 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): dump: 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2889272804 0 0) 0x7fc514098880 con 0x7fc514138d40 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4f8003570 con 0x7fc514138d40 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593764370 0 0) 0x7fc51409c6f0 con 0x7fc51409faa0 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient(hunting): my global_id is 184177 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc514098880 con 0x7fc51409faa0 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 513163991 0 0) 0x7fc51409ccc0 con 0x7fc514132b40 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.26:6790/0 conn(0x7fc514138d40 legacy=0x7fc51414a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.122:6789/0 conn(0x7fc51409faa0 legacy=0x7fc514097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 monclient: found mon.c 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51409a8e0 con 0x7fc514132b40 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 5 monclient: authenticate success, global_id 184189 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient: _renew_subs 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51409abc0 con 0x7fc514132b40 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient: _renew_subs 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc514099b90 con 0x7fc514132b40 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc5100011d0 con 0x7fc514132b40 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc50b7fe700 4 set_mon_vals no callback set 2024-09-15T03:52:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc510004cb0 con 0x7fc514132b40 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 10 monclient: dump: 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc4f4006590 con 0x7fc4f4002840 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc51000fb20 con 0x7fc514132b40 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.516+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(332..332 src has 224..332) v4 ==== 3939+0+0 (unknown 2925508691 0 0) 0x7fc510044020 con 0x7fc514132b40 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071538239 0 0) 0x7fc4f4006590 con 0x7fc4f4002840 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient(hunting): my global_id is 5076 2024-09-15T03:52:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc51409ccc0 con 0x7fc4f4002840 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc51a371700 10 monclient: get_auth_request con 0x7fc4f80378e0 auth_method 0 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 488189355 0 0) 0x7fc51409ccc0 con 0x7fc4f4002840 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4f4006590 con 0x7fc4f4002840 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc504001970 con 0x7fc4f4002840 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient: dump: 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2144319019 0 0) 0x7fc4f4006590 con 0x7fc4f4002840 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4f400a130 con 0x7fc4f4002840 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc504001c50 con 0x7fc4f4002840 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.26:6789/0 conn(0x7fc4f4002840 legacy=0x7fc4f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.520+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.26:6789/0 conn(0x7fc4f4002840 legacy=0x7fc4f4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc4f4009640 con 0x7fc4f40090b0 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644402595 0 0) 0x7fc4f4009640 con 0x7fc4f40090b0 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient(hunting): my global_id is 5085 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4f4006590 con 0x7fc4f40090b0 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3789985634 0 0) 0x7fc4f4006590 con 0x7fc4f40090b0 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4f4009640 con 0x7fc4f40090b0 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc5040024b0 con 0x7fc4f40090b0 2024-09-15T03:52:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient: dump: 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2069006453 0 0) 0x7fc4f4009640 con 0x7fc4f40090b0 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc4f40098f0 con 0x7fc4f40090b0 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1710227903) 0x7fc5040028c0 con 0x7fc4f40090b0 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.26:6789/0 conn(0x7fc4f40090b0 legacy=0x7fc4f4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc509ffb700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.26:6789/0 conn(0x7fc4f40090b0 legacy=0x7fc4f4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:00.550 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.551 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.552 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:00.553 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:00.554 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.555 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:00.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.558 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.561 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:00.563 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:00.566 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.568 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.569 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.570 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.570 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.570 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.570 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:00.570 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.122:6812/20895 conn(0x7fc4f80378e0 legacy=0x7fc4f8039d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 10 monclient: shutdown 2024-09-15T03:52:00.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 >> v1:172.21.15.26:6791/0 conn(0x7fc514132b40 legacy=0x7fc514098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51a371700 1 -- 172.21.15.26:0/1313200810 reap_dead start 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51a371700 1 -- 172.21.15.26:0/1313200810 reap_dead start 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 shutdown_connections 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 >> 172.21.15.26:0/1313200810 conn(0x7fc51406e370 msgr2=0x7fc5140a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 shutdown_connections 2024-09-15T03:52:00.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.532+0000 7fc51ab72700 1 -- 172.21.15.26:0/1313200810 wait complete. 2024-09-15T03:52:00.572 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:00.572 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:52:00.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:52:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient: get_monmap_and_config 2024-09-15T03:52:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient: monmap: 2024-09-15T03:52:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.687 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:00.685858+0000 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:00.685858+0000 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.688 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 1 Processor -- start 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 1 -- start start 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient: init 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient: _add_conns ranks=[0,5,2,6,7,1,8,3,4] 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient(hunting): picked mon.noname-a con 0x7fe6700a6880 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient(hunting): picked mon.noname-h con 0x7fe67009faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient(hunting): picked mon.noname-b con 0x7fe670138d30 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67004ca10 con 0x7fe6700a6880 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700a0580 con 0x7fe670138d30 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe670133780 con 0x7fe67009faa0 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66ffff700 10 monclient(hunting): authenticate will time out at 1265.115699s 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66f7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe670138d30 0x7fe67013b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39610/0 (socket says 172.21.15.26:39610) 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.680+0000 7fe66f7fe700 1 -- 172.21.15.26:0/880065485 learned_addr learned my addr 172.21.15.26:0/880065485 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838286297 0 0) 0x7fe67004ca10 con 0x7fe6700a6880 2024-09-15T03:52:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient(hunting): my global_id is 5094 2024-09-15T03:52:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe658003570 con 0x7fe6700a6880 2024-09-15T03:52:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633184741 0 0) 0x7fe668001110 con 0x7fe670138d30 2024-09-15T03:52:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient(hunting): my global_id is 134696 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67004ca10 con 0x7fe670138d30 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1324662453 0 0) 0x7fe658003570 con 0x7fe6700a6880 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 >> v1:172.21.15.122:6791/0 conn(0x7fe67009faa0 legacy=0x7fe67009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 >> v1:172.21.15.26:6790/0 conn(0x7fe670138d30 legacy=0x7fe67013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 monclient: found mon.noname-a 2024-09-15T03:52:00.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700a6c50 con 0x7fe6700a6880 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe6640017b0 con 0x7fe6700a6880 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: dump: 2024-09-15T03:52:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.706 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 5 monclient: authenticate success, global_id 5094 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe664001a90 con 0x7fe6700a6880 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 1 -- 172.21.15.26:0/880065485 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe664001fc0 con 0x7fe6700a6880 2024-09-15T03:52:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 4 set_mon_vals no callback set 2024-09-15T03:52:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66dffb700 10 monclient: dump: 2024-09-15T03:52:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: shutdown 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- 172.21.15.26:0/880065485 >> v1:172.21.15.26:6789/0 conn(0x7fe6700a6880 legacy=0x7fe670133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- 172.21.15.26:0/880065485 shutdown_connections 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- 172.21.15.26:0/880065485 >> 172.21.15.26:0/880065485 conn(0x7fe67006e2a0 msgr2=0x7fe6700a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- 172.21.15.26:0/880065485 shutdown_connections 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- 172.21.15.26:0/880065485 wait complete. 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: monmap: 2024-09-15T03:52:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.713 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 Processor -- start 2024-09-15T03:52:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- start start 2024-09-15T03:52:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: init 2024-09-15T03:52:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient: _add_conns ranks=[7,1,5,0,4,2,8,3,6] 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient(hunting): picked mon.noname-i con 0x7fe67009faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient(hunting): picked mon.noname-f con 0x7fe6700a6880 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient(hunting): picked mon.noname-h con 0x7fe670138d30 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700a18c0 con 0x7fe6700a6880 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67009cf10 con 0x7fe670138d30 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67011d1a0 con 0x7fe67009faa0 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66ffff700 10 monclient(hunting): authenticate will time out at 1265.119492s 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66effd700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fe67009faa0 0x7fe670135530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:46342/0 (socket says 172.21.15.26:46342) 2024-09-15T03:52:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.684+0000 7fe66effd700 1 -- 172.21.15.26:0/1793025460 learned_addr learned my addr 172.21.15.26:0/1793025460 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163977415 0 0) 0x7fe67011d1a0 con 0x7fe67009faa0 2024-09-15T03:52:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): my global_id is 134611 2024-09-15T03:52:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe65c003570 con 0x7fe67009faa0 2024-09-15T03:52:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000949548 0 0) 0x7fe67009cf10 con 0x7fe670138d30 2024-09-15T03:52:00.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): my global_id is 5225 2024-09-15T03:52:00.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67011d1a0 con 0x7fe670138d30 2024-09-15T03:52:00.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296610486 0 0) 0x7fe6700a18c0 con 0x7fe6700a6880 2024-09-15T03:52:00.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): my global_id is 184186 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67009cf10 con 0x7fe6700a6880 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 768491857 0 0) 0x7fe65c003570 con 0x7fe67009faa0 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6700a18c0 con 0x7fe67009faa0 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe664001770 con 0x7fe67009faa0 2024-09-15T03:52:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): dump: 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.718 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.719 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.719 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.719 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.719 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.719 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.719 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3041445892 0 0) 0x7fe67011d1a0 con 0x7fe670138d30 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe65c003570 con 0x7fe670138d30 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe668001a90 con 0x7fe670138d30 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): dump: 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1271185575 0 0) 0x7fe67009cf10 con 0x7fe6700a6880 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe67011d1a0 con 0x7fe6700a6880 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe660001ce0 con 0x7fe6700a6880 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient(hunting): dump: 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3609932769 0 0) 0x7fe6700a18c0 con 0x7fe67009faa0 2024-09-15T03:52:00.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.122:6791/0 conn(0x7fe670138d30 legacy=0x7fe67014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.122:6789/0 conn(0x7fe6700a6880 legacy=0x7fe670135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 monclient: found mon.i 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670136350 con 0x7fe67009faa0 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 5 monclient: authenticate success, global_id 134611 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 10 monclient: _renew_subs 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:52:00.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe670131ac0 con 0x7fe67009faa0 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 10 monclient: _renew_subs 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe670137030 con 0x7fe67009faa0 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe6640011d0 con 0x7fe67009faa0 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe664002980 con 0x7fe67009faa0 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66d7fa700 4 set_mon_vals no callback set 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 10 monclient: dump: 2024-09-15T03:52:00.725 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.726 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe650006590 con 0x7fe650002840 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe664002b60 con 0x7fe67009faa0 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.688+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(332..332 src has 224..332) v4 ==== 3939+0+0 (unknown 2925508691 0 0) 0x7fe664044130 con 0x7fe67009faa0 2024-09-15T03:52:00.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915712738 0 0) 0x7fe650006590 con 0x7fe650002840 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient(hunting): my global_id is 134705 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6700a18c0 con 0x7fe650002840 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe66f7fe700 10 monclient: get_auth_request con 0x7fe65c0380f0 auth_method 0 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2593038239 0 0) 0x7fe6700a18c0 con 0x7fe650002840 2024-09-15T03:52:00.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe650006590 con 0x7fe650002840 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe6600024b0 con 0x7fe650002840 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient: dump: 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.729 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3937713854 0 0) 0x7fe650006590 con 0x7fe650002840 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe65000a130 con 0x7fe650002840 2024-09-15T03:52:00.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe6600028d0 con 0x7fe650002840 2024-09-15T03:52:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.26:6790/0 conn(0x7fe650002840 legacy=0x7fe650004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.692+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.26:6790/0 conn(0x7fe650002840 legacy=0x7fe650004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe66ffff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe650007500 con 0x7fe6500068b0 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622525815 0 0) 0x7fe650007500 con 0x7fe6500068b0 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 10 monclient(hunting): my global_id is 134714 2024-09-15T03:52:00.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe650006590 con 0x7fe6500068b0 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1552929760 0 0) 0x7fe650006590 con 0x7fe6500068b0 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe650007500 con 0x7fe6500068b0 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe660003090 con 0x7fe6500068b0 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 10 monclient: dump: 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.734 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2745023958 0 0) 0x7fe650007500 con 0x7fe6500068b0 2024-09-15T03:52:00.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.700+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe650008760 con 0x7fe6500068b0 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1954972988) 0x7fe660005660 con 0x7fe6500068b0 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe656ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe656ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.26:6790/0 conn(0x7fe6500068b0 legacy=0x7fe650006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe656ffd700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.26:6790/0 conn(0x7fe6500068b0 legacy=0x7fe650006d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:00.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:00.737 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:00.738 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:00.739 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.740 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:00.741 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.742 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.743 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:00.746 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.746 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.748 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:00.750 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.751 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.752 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.754 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:00.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:00.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:00.756 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.756 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:00.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.758 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.759 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.760 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.761 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:52:00.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.763 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:00.763 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:00.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.122:6812/20895 conn(0x7fe65c0380f0 legacy=0x7fe65c03a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 10 monclient: shutdown 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 >> v1:172.21.15.122:6792/0 conn(0x7fe67009faa0 legacy=0x7fe670135530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66f7fe700 1 -- 172.21.15.26:0/1793025460 reap_dead start 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66f7fe700 1 -- 172.21.15.26:0/1793025460 reap_dead start 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 shutdown_connections 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 >> 172.21.15.26:0/1793025460 conn(0x7fe67006e2a0 msgr2=0x7fe670067880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 shutdown_connections 2024-09-15T03:52:00.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.704+0000 7fe66ffff700 1 -- 172.21.15.26:0/1793025460 wait complete. 2024-09-15T03:52:00.765 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:00.766 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:52:00.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:52:00.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: get_monmap_and_config 2024-09-15T03:52:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: monmap: 2024-09-15T03:52:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:00.884359+0000 2024-09-15T03:52:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:00.884359+0000 2024-09-15T03:52:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.888 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 1 Processor -- start 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 1 -- start start 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: init 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: _add_conns ranks=[3,1,4,8,7,0,5,2,6] 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient(hunting): picked mon.noname-g con 0x7fa57c132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient(hunting): picked mon.noname-f con 0x7fa57c09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient(hunting): picked mon.noname-c con 0x7fa57c138d40 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c04ca10 con 0x7fa57c09faa0 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c09d5d0 con 0x7fa57c132b40 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c133620 con 0x7fa57c138d40 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient(hunting): authenticate will time out at 1265.314356s 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57bfff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fa57c138d40 0x7fa57c13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55430/0 (socket says 172.21.15.26:55430) 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57bfff700 1 -- 172.21.15.26:0/3500207909 learned_addr learned my addr 172.21.15.26:0/3500207909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590581159 0 0) 0x7fa57c133620 con 0x7fa57c138d40 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): my global_id is 184198 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa568003570 con 0x7fa57c138d40 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521126255 0 0) 0x7fa57c04ca10 con 0x7fa57c09faa0 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): my global_id is 184195 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c133620 con 0x7fa57c09faa0 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744401391 0 0) 0x7fa57c09d5d0 con 0x7fa57c132b40 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient(hunting): my global_id is 184179 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c04ca10 con 0x7fa57c132b40 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2815491582 0 0) 0x7fa568003570 con 0x7fa57c138d40 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 >> v1:172.21.15.122:6790/0 conn(0x7fa57c132b40 legacy=0x7fa57c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 >> v1:172.21.15.122:6789/0 conn(0x7fa57c09faa0 legacy=0x7fa57c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 monclient: found mon.noname-c 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c0a0cc0 con 0x7fa57c138d40 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa5740017b0 con 0x7fa57c138d40 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 5 monclient: authenticate success, global_id 184198 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: dump: 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa574001a20 con 0x7fa57c138d40 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 1 -- 172.21.15.26:0/3500207909 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa574004c30 con 0x7fa57c138d40 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa5808c2700 4 set_mon_vals no callback set 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.880+0000 7fa57a7fc700 10 monclient: dump: 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient: shutdown 2024-09-15T03:52:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- 172.21.15.26:0/3500207909 >> v1:172.21.15.26:6791/0 conn(0x7fa57c138d40 legacy=0x7fa57c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- 172.21.15.26:0/3500207909 shutdown_connections 2024-09-15T03:52:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- 172.21.15.26:0/3500207909 >> 172.21.15.26:0/3500207909 conn(0x7fa57c06e370 msgr2=0x7fa57c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- 172.21.15.26:0/3500207909 shutdown_connections 2024-09-15T03:52:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- 172.21.15.26:0/3500207909 wait complete. 2024-09-15T03:52:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient: build_initial_monmap 2024-09-15T03:52:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient: monmap: 2024-09-15T03:52:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 Processor -- start 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- start start 2024-09-15T03:52:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient: init 2024-09-15T03:52:00.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:00.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient: _add_conns ranks=[0,2,8,6,4,3,7,1,5] 2024-09-15T03:52:00.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient(hunting): picked mon.noname-a con 0x7fa57c09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:00.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient(hunting): picked mon.noname-b con 0x7fa57c132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:00.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient(hunting): picked mon.noname-e con 0x7fa57c138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:00.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c11cb70 con 0x7fa57c09faa0 2024-09-15T03:52:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c09ccc0 con 0x7fa57c132b40 2024-09-15T03:52:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57c09cfa0 con 0x7fa57c138d40 2024-09-15T03:52:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5808c2700 10 monclient(hunting): authenticate will time out at 1265.318912s 2024-09-15T03:52:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa57b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa57c09faa0 0x7fa57c099570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38372/0 (socket says 172.21.15.26:38372) 2024-09-15T03:52:00.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa57b7fe700 1 -- 172.21.15.26:0/3355942666 learned_addr learned my addr 172.21.15.26:0/3355942666 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101113355 0 0) 0x7fa57c11cb70 con 0x7fa57c09faa0 2024-09-15T03:52:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): my global_id is 5103 2024-09-15T03:52:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa55c003570 con 0x7fa57c09faa0 2024-09-15T03:52:00.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443671063 0 0) 0x7fa57c09cfa0 con 0x7fa57c138d40 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): my global_id is 134666 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c11cb70 con 0x7fa57c138d40 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064165535 0 0) 0x7fa57c09ccc0 con 0x7fa57c132b40 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 10 monclient(hunting): my global_id is 134723 2024-09-15T03:52:00.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.884+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c09cfa0 con 0x7fa57c132b40 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1412413207 0 0) 0x7fa55c003570 con 0x7fa57c09faa0 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa57c09ccc0 con 0x7fa57c09faa0 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa564002b70 con 0x7fa57c09faa0 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): dump: 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.919 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 775171763 0 0) 0x7fa57c11cb70 con 0x7fa57c138d40 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa55c003570 con 0x7fa57c138d40 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa574001770 con 0x7fa57c138d40 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): dump: 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 917789308 0 0) 0x7fa57c09ccc0 con 0x7fa57c09faa0 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6793/0 conn(0x7fa57c138d40 legacy=0x7fa57c09a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6790/0 conn(0x7fa57c132b40 legacy=0x7fa57c099c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: _finish_hunting 0 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 monclient: found mon.a 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c09b5b0 con 0x7fa57c09faa0 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: _finish_auth 0 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 5 monclient: authenticate success, global_id 5103 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 10 monclient: _renew_subs 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c09bcd0 con 0x7fa57c09faa0 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 10 monclient: _renew_subs 2024-09-15T03:52:00.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa5640016e0 con 0x7fa57c09faa0 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5808c2700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c09b0a0 con 0x7fa57c09faa0 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa564001c10 con 0x7fa57c09faa0 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: dump: 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.924 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa579ffb700 4 set_mon_vals no callback set 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5717fa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa554006590 con 0x7fa554002840 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa564001e40 con 0x7fa57c09faa0 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(332..332 src has 224..332) v4 ==== 3939+0+0 (unknown 2925508691 0 0) 0x7fa564044040 con 0x7fa57c09faa0 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747272571 0 0) 0x7fa554006590 con 0x7fa554002840 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient(hunting): my global_id is 184207 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c09ccc0 con 0x7fa554002840 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa57bfff700 10 monclient: get_auth_request con 0x7fa55c037cd0 auth_method 0 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3831534732 0 0) 0x7fa57c09ccc0 con 0x7fa554002840 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa554006590 con 0x7fa554002840 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa56c0025d0 con 0x7fa554002840 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 10 monclient: dump: 2024-09-15T03:52:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.928 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3768962501 0 0) 0x7fa554006590 con 0x7fa554002840 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.888+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa55400a130 con 0x7fa554002840 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.892+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa56c0049c0 con 0x7fa554002840 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.892+0000 7fa5737fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:00.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.892+0000 7fa5737fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:00.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.892+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6791/0 conn(0x7fa554002840 legacy=0x7fa554004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.892+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6791/0 conn(0x7fa554002840 legacy=0x7fa554004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:00.931 INFO:tasks.rados.rados.0.smithi122.stdout:1427: done (0 left) 2024-09-15T03:52:00.931 INFO:tasks.rados.rados.0.smithi122.stdout:1428: snap_create 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.900+0000 7fa5717fa700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.900+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa554007520 con 0x7fa5540068d0 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438922350 0 0) 0x7fa554007520 con 0x7fa5540068d0 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 10 monclient(hunting): my global_id is 184216 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa554006590 con 0x7fa5540068d0 2024-09-15T03:52:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 622962375 0 0) 0x7fa554006590 con 0x7fa5540068d0 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa554007520 con 0x7fa5540068d0 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa56c0031b0 con 0x7fa5540068d0 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.908+0000 7fa5737fe700 10 monclient: dump: 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:00.934 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1206027060 0 0) 0x7fa554007520 con 0x7fa5540068d0 2024-09-15T03:52:00.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa554008760 con 0x7fa5540068d0 2024-09-15T03:52:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 2434365416) 0x7fa56c0046b0 con 0x7fa5540068d0 2024-09-15T03:52:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6791/0 conn(0x7fa5540068d0 legacy=0x7fa554006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5737fe700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6791/0 conn(0x7fa5540068d0 legacy=0x7fa554006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:00.937 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:00.938 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.939 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.940 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:00.941 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.942 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:00.943 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:00.944 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:00.945 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.946 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:00.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.947 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:00.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.947 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:00.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:00.947 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.948 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.948 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:00.948 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.949 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.950 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:00.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:00.950 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:00.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:00.952 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.952 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.952 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.953 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:00.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:00.953 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.954 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:00.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:00.956 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:00.957 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.957 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.962 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.965 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:00.965 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.965 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.967 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:00.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:00.967 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:00.967 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:00.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.968 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:00.968 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:00.968 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:00.968 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.968 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.969 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:00.969 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:00.969 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:00.969 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:00.969 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:00.969 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.970 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.971 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.971 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.971 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.971 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:00.971 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:00.971 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:00.972 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:00.972 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:00.972 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:00.972 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:00.972 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:00.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:00.973 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:00.973 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:00.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.122:6812/20895 conn(0x7fa55c037cd0 legacy=0x7fa55c03a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5717fa700 10 monclient: shutdown 2024-09-15T03:52:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5717fa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 >> v1:172.21.15.26:6789/0 conn(0x7fa57c09faa0 legacy=0x7fa57c099570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa57bfff700 1 -- 172.21.15.26:0/3355942666 reap_dead start 2024-09-15T03:52:00.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa57bfff700 1 -- 172.21.15.26:0/3355942666 reap_dead start 2024-09-15T03:52:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.920+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 shutdown_connections 2024-09-15T03:52:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.924+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 >> 172.21.15.26:0/3355942666 conn(0x7fa57c06e370 msgr2=0x7fa57c09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.924+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 shutdown_connections 2024-09-15T03:52:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:00.924+0000 7fa5717fa700 1 -- 172.21.15.26:0/3355942666 wait complete. 2024-09-15T03:52:00.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:00.978 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:52:00.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:52:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.084+0000 7f80ca930700 10 monclient: get_monmap_and_config 2024-09-15T03:52:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.084+0000 7f80ca930700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.084+0000 7f80ca930700 10 monclient: monmap: 2024-09-15T03:52:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.093 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:01.091661+0000 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:01.091661+0000 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.094 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 1 Processor -- start 2024-09-15T03:52:01.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 1 -- start start 2024-09-15T03:52:01.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient: init 2024-09-15T03:52:01.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient: _add_conns ranks=[0,1,6,4,5,2,8,7,3] 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient(hunting): picked mon.noname-a con 0x7f80c4063340 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient(hunting): picked mon.noname-f con 0x7f80c4063a10 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient(hunting): picked mon.noname-d con 0x7f80c4133110 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c404ca10 con 0x7f80c4063340 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c40a0ad0 con 0x7f80c4063a10 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c409f4f0 con 0x7f80c4133110 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient(hunting): authenticate will time out at 1265.521912s 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c3fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f80c4063340 0x7f80c40991e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38382/0 (socket says 172.21.15.26:38382) 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c8ecd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f80c4133110 0x7f80c413b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:53314/0 (socket says 172.21.15.26:53314) 2024-09-15T03:52:01.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c3fff700 1 -- 172.21.15.26:0/3808957024 learned_addr learned my addr 172.21.15.26:0/3808957024 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798317640 0 0) 0x7f80c409f4f0 con 0x7f80c4133110 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): my global_id is 5217 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80ac003570 con 0x7f80c4133110 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751836050 0 0) 0x7f80c404ca10 con 0x7f80c4063340 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): my global_id is 5112 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c409f4f0 con 0x7f80c4063340 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056134209 0 0) 0x7f80c40a0ad0 con 0x7f80c4063a10 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient(hunting): my global_id is 184204 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c404ca10 con 0x7f80c4063a10 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969938174 0 0) 0x7f80c404ca10 con 0x7f80c4063a10 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 >> v1:172.21.15.26:6792/0 conn(0x7f80c4133110 legacy=0x7f80c413b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 >> v1:172.21.15.26:6789/0 conn(0x7f80c4063340 legacy=0x7f80c40991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 monclient: found mon.noname-f 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c4063710 con 0x7f80c4063a10 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 5 monclient: authenticate success, global_id 184204 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b8001750 con 0x7f80c4063a10 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: dump: 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f80b80019c0 con 0x7f80c4063a10 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80ca930700 4 set_mon_vals no callback set 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 1 -- 172.21.15.26:0/3808957024 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b8001ef0 con 0x7f80c4063a10 2024-09-15T03:52:01.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.088+0000 7f80c2ffd700 10 monclient: dump: 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.107 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: shutdown 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/3808957024 >> v1:172.21.15.122:6789/0 conn(0x7f80c4063a10 legacy=0x7f80c4137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/3808957024 shutdown_connections 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/3808957024 >> 172.21.15.26:0/3808957024 conn(0x7f80c40a3cb0 msgr2=0x7f80c409fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/3808957024 shutdown_connections 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/3808957024 wait complete. 2024-09-15T03:52:01.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: monmap: 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 Processor -- start 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- start start 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: init 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: _add_conns ranks=[4,8,5,3,2,1,7,6,0] 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient(hunting): picked mon.noname-c con 0x7f80c4063340 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient(hunting): picked mon.noname-e con 0x7f80c4063a10 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient(hunting): picked mon.noname-h con 0x7f80c4133110 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c409ecb0 con 0x7f80c4063340 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c4099f20 con 0x7f80c4133110 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c411c700 con 0x7f80c4063a10 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient(hunting): authenticate will time out at 1265.526054s 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80c3fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f80c4063340 0x7f80c4119f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55448/0 (socket says 172.21.15.26:55448) 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80c3fff700 1 -- 172.21.15.26:0/2270572372 learned_addr learned my addr 172.21.15.26:0/2270572372 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304918423 0 0) 0x7f80c409ecb0 con 0x7f80c4063340 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): my global_id is 184225 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b0003570 con 0x7f80c4063340 2024-09-15T03:52:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249657458 0 0) 0x7f80c411c700 con 0x7f80c4063a10 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): my global_id is 134675 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c409ecb0 con 0x7f80c4063a10 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 29293888 0 0) 0x7f80b0003570 con 0x7f80c4063340 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80c411c700 con 0x7f80c4063340 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b4001510 con 0x7f80c4063340 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): dump: 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1889003266 0 0) 0x7f80c409ecb0 con 0x7f80c4063a10 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80b0003570 con 0x7f80c4063a10 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b8001e70 con 0x7f80c4063a10 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): dump: 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262251605 0 0) 0x7f80c4099f20 con 0x7f80c4133110 2024-09-15T03:52:01.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient(hunting): my global_id is 5234 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c409ecb0 con 0x7f80c4133110 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2588714588 0 0) 0x7f80b0003570 con 0x7f80c4063a10 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.122:6791/0 conn(0x7f80c4133110 legacy=0x7f80c411ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.26:6791/0 conn(0x7f80c4063340 legacy=0x7f80c4119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 monclient: found mon.e 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c411d550 con 0x7f80c4063a10 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 5 monclient: authenticate success, global_id 134675 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: _renew_subs 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c412e580 con 0x7f80c4063a10 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f80b8002470 con 0x7f80c4063a10 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b8002920 con 0x7f80c4063a10 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80abfff700 10 monclient: dump: 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.117 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80c1ffb700 4 set_mon_vals no callback set 2024-09-15T03:52:01.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: _renew_subs 2024-09-15T03:52:01.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.092+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c414af60 con 0x7f80c4063a10 2024-09-15T03:52:01.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80ca930700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f80c411d790 con 0x7f80c40a19d0 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396992990 0 0) 0x7f80c411c700 con 0x7f80c40a19d0 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient(hunting): my global_id is 5226 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b0003570 con 0x7f80c40a19d0 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f80b800d330 con 0x7f80c4063a10 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(333..333 src has 224..333) v4 ==== 3911+0+0 (unknown 2717171964 0 0) 0x7f80b8041730 con 0x7f80c4063a10 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3715738856 0 0) 0x7f80b0003570 con 0x7f80c40a19d0 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80c411c700 con 0x7f80c40a19d0 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b4004d60 con 0x7f80c40a19d0 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:52:01.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient: dump: 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.123 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80c8ecd700 10 monclient: get_auth_request con 0x7f80b0037be0 auth_method 0 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2683453835 0 0) 0x7f80c411c700 con 0x7f80c40a19d0 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80c411ba90 con 0x7f80c40a19d0 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f80b4005060 con 0x7f80c40a19d0 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.26:6792/0 conn(0x7f80c40a19d0 legacy=0x7f80c40a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.096+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.26:6792/0 conn(0x7f80c40a19d0 legacy=0x7f80c40a1da0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80ca930700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:01.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f80c411d950 con 0x7f80c40a24b0 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078630179 0 0) 0x7f80c411d950 con 0x7f80c40a24b0 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80abfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80abfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80abfff700 10 monclient(hunting): my global_id is 5235 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.104+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c411c700 con 0x7f80c40a24b0 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 21530080 0 0) 0x7f80c411c700 con 0x7f80c40a24b0 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80c411d950 con 0x7f80c40a24b0 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f80b4005910 con 0x7f80c40a24b0 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 10 monclient: dump: 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.125 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.126 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3385830472 0 0) 0x7f80c411d950 con 0x7f80c40a24b0 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f80c411dba0 con 0x7f80c40a24b0 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 4020278064) 0x7f80b4008050 con 0x7f80c40a24b0 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.26:6792/0 conn(0x7f80c40a24b0 legacy=0x7f80c402ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80abfff700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.26:6792/0 conn(0x7f80c40a24b0 legacy=0x7f80c402ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:01.127 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:01.128 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.129 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.130 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:01.131 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:01.132 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:01.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.134 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.135 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.139 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.141 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.142 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.145 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.146 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:01.147 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.148 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:01.149 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.122:6812/20895 conn(0x7f80b0037be0 legacy=0x7f80b003a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 10 monclient: shutdown 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 >> v1:172.21.15.26:6793/0 conn(0x7f80c4063a10 legacy=0x7f80c411a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80c8ecd700 1 -- 172.21.15.26:0/2270572372 reap_dead start 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 shutdown_connections 2024-09-15T03:52:01.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 >> 172.21.15.26:0/2270572372 conn(0x7f80c40a3cb0 msgr2=0x7f80c40989f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 shutdown_connections 2024-09-15T03:52:01.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.108+0000 7f80ca930700 1 -- 172.21.15.26:0/2270572372 wait complete. 2024-09-15T03:52:01.151 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:01.151 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:52:01.151 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:52:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient: get_monmap_and_config 2024-09-15T03:52:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient: monmap: 2024-09-15T03:52:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.276 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:01.268880+0000 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:01.268880+0000 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 1 Processor -- start 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 1 -- start start 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient: init 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient: _add_conns ranks=[3,7,8,0,5,1,6,2,4] 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient(hunting): picked mon.noname-g con 0x7f8cbc070f80 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient(hunting): picked mon.noname-i con 0x7f8cbc098910 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient(hunting): picked mon.noname-e con 0x7f8cbc098d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc04ca10 con 0x7f8cbc070f80 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc096440 con 0x7f8cbc098910 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc09f700 con 0x7f8cbc098d40 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbb59e700 10 monclient(hunting): authenticate will time out at 1265.699046s 2024-09-15T03:52:01.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbad9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8cbc098d40 0x7f8cbc13b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:55610/0 (socket says 172.21.15.26:55610) 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cbad9d700 1 -- 172.21.15.26:0/531306930 learned_addr learned my addr 172.21.15.26:0/531306930 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730020456 0 0) 0x7f8cbc09f700 con 0x7f8cbc098d40 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): my global_id is 134684 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca4003570 con 0x7f8cbc098d40 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718992874 0 0) 0x7f8cbc04ca10 con 0x7f8cbc070f80 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): my global_id is 184188 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc09f700 con 0x7f8cbc070f80 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401077001 0 0) 0x7f8cbc096440 con 0x7f8cbc098910 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 10 monclient(hunting): my global_id is 134620 2024-09-15T03:52:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.264+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc04ca10 con 0x7f8cbc098910 2024-09-15T03:52:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1335387677 0 0) 0x7f8ca4003570 con 0x7f8cbc098d40 2024-09-15T03:52:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 >> v1:172.21.15.122:6792/0 conn(0x7f8cbc098910 legacy=0x7f8cbc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 >> v1:172.21.15.122:6790/0 conn(0x7f8cbc070f80 legacy=0x7f8cbc132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 monclient: found mon.noname-e 2024-09-15T03:52:01.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc06e620 con 0x7f8cbc098d40 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 5 monclient: authenticate success, global_id 134684 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb40017b0 con 0x7f8cbc098d40 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: dump: 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.282 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8cb4001a90 con 0x7f8cbc098d40 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 1 -- 172.21.15.26:0/531306930 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb4004c30 con 0x7f8cbc098d40 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 4 set_mon_vals no callback set 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cb959a700 10 monclient: dump: 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 10 monclient: shutdown 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 -- 172.21.15.26:0/531306930 >> v1:172.21.15.26:6793/0 conn(0x7f8cbc098d40 legacy=0x7f8cbc13b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 -- 172.21.15.26:0/531306930 shutdown_connections 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 -- 172.21.15.26:0/531306930 >> 172.21.15.26:0/531306930 conn(0x7f8cbc0a2ea0 msgr2=0x7f8cbc09eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 -- 172.21.15.26:0/531306930 shutdown_connections 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 -- 172.21.15.26:0/531306930 wait complete. 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 10 monclient: monmap: 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 Processor -- start 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 1 -- start start 2024-09-15T03:52:01.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 10 monclient: init 2024-09-15T03:52:01.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.268+0000 7f8cbb59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient: _add_conns ranks=[5,7,2,8,4,0,1,3,6] 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient(hunting): picked mon.noname-h con 0x7f8cbc070f80 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient(hunting): picked mon.noname-i con 0x7f8cbc098910 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient(hunting): picked mon.noname-b con 0x7f8cbc098d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc09fb40 con 0x7f8cbc098d40 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc09f910 con 0x7f8cbc070f80 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc142b10 con 0x7f8cbc098910 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient(hunting): authenticate will time out at 1265.704238s 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbad9d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f8cbc098d40 0x7f8cbc14a010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39640/0 (socket says 172.21.15.26:39640) 2024-09-15T03:52:01.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbad9d700 1 -- 172.21.15.26:0/1011573779 learned_addr learned my addr 172.21.15.26:0/1011573779 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008568857 0 0) 0x7f8cbc09fb40 con 0x7f8cbc098d40 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): my global_id is 134732 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c9c003570 con 0x7f8cbc098d40 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021594034 0 0) 0x7f8cbc142b10 con 0x7f8cbc098910 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): my global_id is 134629 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc09fb40 con 0x7f8cbc098910 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414235806 0 0) 0x7f8cbc09f910 con 0x7f8cbc070f80 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): my global_id is 5243 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc142b10 con 0x7f8cbc070f80 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4057630830 0 0) 0x7f8c9c003570 con 0x7f8cbc098d40 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8cbc09f910 con 0x7f8cbc098d40 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb4001c00 con 0x7f8cbc098d40 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.297 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4011005785 0 0) 0x7f8cbc142b10 con 0x7f8cbc070f80 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8c9c003570 con 0x7f8cbc070f80 2024-09-15T03:52:01.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1865157522 0 0) 0x7f8cbc09fb40 con 0x7f8cbc098910 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8cbc142b10 con 0x7f8cbc098910 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cac001de0 con 0x7f8cbc098910 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.299 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb00019d0 con 0x7f8cbc070f80 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.300 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.301 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1571185397 0 0) 0x7f8cbc09f910 con 0x7f8cbc098d40 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.122:6792/0 conn(0x7f8cbc098910 legacy=0x7f8cbc131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.122:6791/0 conn(0x7f8cbc070f80 legacy=0x7f8cbc141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 monclient: found mon.b 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc12d180 con 0x7f8cbc098d40 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 5 monclient: authenticate success, global_id 134732 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient: _renew_subs 2024-09-15T03:52:01.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc12d8d0 con 0x7f8cbc098d40 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient: _renew_subs 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc1321a0 con 0x7f8cbc098d40 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8cb4004e20 con 0x7f8cbc098d40 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb40052d0 con 0x7f8cbc098d40 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cb8d99700 4 set_mon_vals no callback set 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient: dump: 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.303 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.304 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8ca0006590 con 0x7f8ca0002840 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76207560 0 0) 0x7f8ca0006590 con 0x7f8ca0002840 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 10 monclient(hunting): my global_id is 134693 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc09f910 con 0x7f8ca0002840 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.272+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8cb400fc90 con 0x7f8cbc098d40 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(333..333 src has 224..333) v4 ==== 3911+0+0 (unknown 2717171964 0 0) 0x7f8cb4044040 con 0x7f8cbc098d40 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3080500988 0 0) 0x7f8cbc09f910 con 0x7f8ca0002840 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ca0006590 con 0x7f8ca0002840 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb0002130 con 0x7f8ca0002840 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 10 monclient: dump: 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8cb9d9b700 10 monclient: get_auth_request con 0x7f8c9c038060 auth_method 0 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3907475552 0 0) 0x7f8ca0006590 con 0x7f8ca0002840 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ca000a130 con 0x7f8ca0002840 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8cb0002500 con 0x7f8ca0002840 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.26:6793/0 conn(0x7f8ca0002840 legacy=0x7f8ca0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.276+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.26:6793/0 conn(0x7f8ca0002840 legacy=0x7f8ca0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8ca0009640 con 0x7f8ca00090b0 2024-09-15T03:52:01.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536992948 0 0) 0x7f8ca0009640 con 0x7f8ca00090b0 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient(hunting): my global_id is 134702 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca0006590 con 0x7f8ca00090b0 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2698098907 0 0) 0x7f8ca0006590 con 0x7f8ca00090b0 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ca0009640 con 0x7f8ca00090b0 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8cb0002d60 con 0x7f8ca00090b0 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient: dump: 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 601435200 0 0) 0x7f8ca0009640 con 0x7f8ca00090b0 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8ca00098f0 con 0x7f8ca00090b0 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2283256546) 0x7f8cb0005120 con 0x7f8ca00090b0 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.26:6793/0 conn(0x7f8ca00090b0 legacy=0x7f8ca0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8caa7fc700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.26:6793/0 conn(0x7f8ca00090b0 legacy=0x7f8ca0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:01.310 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:01.311 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.312 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.313 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:01.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.315 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.317 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.318 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.319 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.321 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.323 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:01.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.325 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:01.327 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:01.328 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.329 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:01.330 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:01.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.122:6812/20895 conn(0x7f8c9c038060 legacy=0x7f8c9c03a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 10 monclient: shutdown 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 >> v1:172.21.15.26:6790/0 conn(0x7f8cbc098d40 legacy=0x7f8cbc14a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbad9d700 1 -- 172.21.15.26:0/1011573779 reap_dead start 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbad9d700 1 -- 172.21.15.26:0/1011573779 reap_dead start 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 shutdown_connections 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 >> 172.21.15.26:0/1011573779 conn(0x7f8cbc0a2ea0 msgr2=0x7f8cbc09dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 shutdown_connections 2024-09-15T03:52:01.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.284+0000 7f8cbb59e700 1 -- 172.21.15.26:0/1011573779 wait complete. 2024-09-15T03:52:01.333 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:01.333 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:52:01.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.440+0000 7f75b9627700 10 monclient: get_monmap_and_config 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.440+0000 7f75b9627700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.440+0000 7f75b9627700 10 monclient: monmap: 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:01.447501+0000 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:01.447501+0000 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.449 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.450 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 1 Processor -- start 2024-09-15T03:52:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 1 -- start start 2024-09-15T03:52:01.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient: init 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient: _add_conns ranks=[0,8,6,4,2,7,3,1,5] 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient(hunting): picked mon.noname-a con 0x7f75b4070de0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient(hunting): picked mon.noname-e con 0x7f75b406cdc0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient(hunting): picked mon.noname-d con 0x7f75b4137060 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b404ca10 con 0x7f75b4070de0 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b406c3e0 con 0x7f75b4137060 2024-09-15T03:52:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b2ffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f75b4070de0 0x7f75b409d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38390/0 (socket says 172.21.15.26:38390) 2024-09-15T03:52:01.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b2ffd700 1 -- 172.21.15.26:0/3578703281 learned_addr learned my addr 172.21.15.26:0/3578703281 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 1 -- 172.21.15.26:0/3578703281 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40711b0 con 0x7f75b406cdc0 2024-09-15T03:52:01.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b27fc700 1 -- 172.21.15.26:0/3578703281 >> v1:172.21.15.26:6793/0 conn(0x7f75b406cdc0 legacy=0x7f75b409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-15T03:52:01.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b27fc700 1 -- 172.21.15.26:0/3578703281 >> v1:172.21.15.26:6793/0 conn(0x7f75b406cdc0 legacy=0x7f75b409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b27fc700 1 --1- 172.21.15.26:0/3578703281 >> v1:172.21.15.26:6793/0 conn(0x7f75b406cdc0 0x7f75b409d940 :-1 s=CONNECTING_SEND_CONNECT_MSG pgs=0 cs=0 l=0).handle_connect_reply_1 read connect reply failed 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient(hunting): authenticate will time out at 1265.879044s 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837229135 0 0) 0x7f75b404ca10 con 0x7f75b4070de0 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient(hunting): my global_id is 5121 2024-09-15T03:52:01.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f759c003570 con 0x7f75b4070de0 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484924641 0 0) 0x7f75b406c3e0 con 0x7f75b4137060 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient(hunting): my global_id is 5244 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b404ca10 con 0x7f75b4137060 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944596247 0 0) 0x7f759c003570 con 0x7f75b4070de0 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 >> v1:172.21.15.26:6793/0 conn(0x7f75b406cdc0 legacy=0x7f75b409d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 >> v1:172.21.15.26:6792/0 conn(0x7f75b4137060 legacy=0x7f75b4139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 monclient: found mon.noname-a 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b406d190 con 0x7f75b4070de0 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75a40017b0 con 0x7f75b4070de0 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: dump: 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.458 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 5 monclient: authenticate success, global_id 5121 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75a4001a90 con 0x7f75b4070de0 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 1 -- 172.21.15.26:0/3578703281 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75a4001c80 con 0x7f75b4070de0 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b9627700 4 set_mon_vals no callback set 2024-09-15T03:52:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.444+0000 7f75b1ffb700 10 monclient: dump: 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.461 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient: shutdown 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- 172.21.15.26:0/3578703281 >> v1:172.21.15.26:6789/0 conn(0x7f75b4070de0 legacy=0x7f75b409d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- 172.21.15.26:0/3578703281 shutdown_connections 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- 172.21.15.26:0/3578703281 >> 172.21.15.26:0/3578703281 conn(0x7f75b4063e10 msgr2=0x7f75b40a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- 172.21.15.26:0/3578703281 shutdown_connections 2024-09-15T03:52:01.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- 172.21.15.26:0/3578703281 wait complete. 2024-09-15T03:52:01.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient: monmap: 2024-09-15T03:52:01.469 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.469 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:01.469 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 Processor -- start 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- start start 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient: init 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient: _add_conns ranks=[3,2,1,4,7,0,6,5,8] 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient(hunting): picked mon.noname-g con 0x7f75b406cdc0 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient(hunting): picked mon.noname-b con 0x7f75b4070de0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient(hunting): picked mon.noname-f con 0x7f75b4137060 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:01.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b409cce0 con 0x7f75b4137060 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b406bc60 con 0x7f75b4070de0 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b411baf0 con 0x7f75b406cdc0 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b9627700 10 monclient(hunting): authenticate will time out at 1265.883196s 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b27fc700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f75b4070de0 0x7f75b4140110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39652/0 (socket says 172.21.15.26:39652) 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f75b27fc700 1 -- 172.21.15.26:0/3017573133 learned_addr learned my addr 172.21.15.26:0/3017573133 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630266755 0 0) 0x7f75b406bc60 con 0x7f75b4070de0 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 10 monclient(hunting): my global_id is 134741 2024-09-15T03:52:01.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a0003570 con 0x7f75b4070de0 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5310789 0 0) 0x7f75b411baf0 con 0x7f75b406cdc0 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 10 monclient(hunting): my global_id is 184197 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.448+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b406bc60 con 0x7f75b406cdc0 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019530267 0 0) 0x7f75b409cce0 con 0x7f75b4137060 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): my global_id is 184213 2024-09-15T03:52:01.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b411baf0 con 0x7f75b4137060 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3777973896 0 0) 0x7f75a0003570 con 0x7f75b4070de0 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f75b409cce0 con 0x7f75b4070de0 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75a8002aa0 con 0x7f75b4070de0 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): dump: 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.474 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3553060017 0 0) 0x7f75b409cce0 con 0x7f75b4070de0 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6790/0 conn(0x7f75b406cdc0 legacy=0x7f75b413fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6789/0 conn(0x7f75b4137060 legacy=0x7f75b4147b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 monclient: found mon.b 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b411d5b0 con 0x7f75b4070de0 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 5 monclient: authenticate success, global_id 134741 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 10 monclient: _renew_subs 2024-09-15T03:52:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b411c6c0 con 0x7f75b4070de0 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 10 monclient: _renew_subs 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b4133930 con 0x7f75b4070de0 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f75a8002d80 con 0x7f75b4070de0 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75a8001580 con 0x7f75b4070de0 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b17fa700 4 set_mon_vals no callback set 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: dump: 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7594006590 con 0x7f7594002840 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f75a8001760 con 0x7f75b4070de0 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(333..333 src has 224..333) v4 ==== 3911+0+0 (unknown 2717171964 0 0) 0x7f75a8043ec0 con 0x7f75b4070de0 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853144796 0 0) 0x7f7594006590 con 0x7f7594002840 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient(hunting): my global_id is 184222 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b409cce0 con 0x7f7594002840 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f75b37fe700 10 monclient: get_auth_request con 0x7f75a0037680 auth_method 0 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3919574893 0 0) 0x7f75b409cce0 con 0x7f7594002840 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7594006590 con 0x7f7594002840 2024-09-15T03:52:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75a4005010 con 0x7f7594002840 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.452+0000 7f759affd700 10 monclient: dump: 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2858309990 0 0) 0x7f7594006590 con 0x7f7594002840 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f759400a130 con 0x7f7594002840 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f75a4007210 con 0x7f7594002840 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6789/0 conn(0x7f7594002840 legacy=0x7f7594004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.456+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6789/0 conn(0x7f7594002840 legacy=0x7f7594004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f75b9627700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7594007520 con 0x7f75940068d0 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950646248 0 0) 0x7f7594007520 con 0x7f75940068d0 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 10 monclient(hunting): my global_id is 184231 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7594006590 con 0x7f75940068d0 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2375572876 0 0) 0x7f7594006590 con 0x7f75940068d0 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7594007520 con 0x7f75940068d0 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f75a4005aa0 con 0x7f75940068d0 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 10 monclient: dump: 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1768000749 0 0) 0x7f7594007520 con 0x7f75940068d0 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.468+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7594008760 con 0x7f75940068d0 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 2141370001) 0x7f75a4006f40 con 0x7f75940068d0 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f759affd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f759affd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6789/0 conn(0x7f75940068d0 legacy=0x7f7594006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f759affd700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6789/0 conn(0x7f75940068d0 legacy=0x7f7594006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:01.491 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:01.492 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.494 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.495 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.496 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.497 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.498 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:01.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.501 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.505 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:01.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:01.507 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.508 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.510 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.511 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.513 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.514 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.514 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.514 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.514 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:01.514 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:01.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.122:6812/20895 conn(0x7f75a0037680 legacy=0x7f75a0039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 10 monclient: shutdown 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 >> v1:172.21.15.26:6790/0 conn(0x7f75b4070de0 legacy=0x7f75b4140110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b37fe700 1 -- 172.21.15.26:0/3017573133 reap_dead start 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b37fe700 1 -- 172.21.15.26:0/3017573133 reap_dead start 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 shutdown_connections 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 >> 172.21.15.26:0/3017573133 conn(0x7f75b4063e10 msgr2=0x7f75b40a13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 shutdown_connections 2024-09-15T03:52:01.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.472+0000 7f75b9627700 1 -- 172.21.15.26:0/3017573133 wait complete. 2024-09-15T03:52:01.517 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:01.517 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:52:01.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: get_monmap_and_config 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: monmap: 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:01.636442+0000 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:01.636442+0000 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.639 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 1 Processor -- start 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 1 -- start start 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: init 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: _add_conns ranks=[2,1,7,4,5,6,8,3,0] 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient(hunting): picked mon.noname-b con 0x7f68580a48e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient(hunting): picked mon.noname-f con 0x7f685809aca0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient(hunting): picked mon.noname-i con 0x7f685809a5e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685804ca10 con 0x7f685809aca0 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68580696a0 con 0x7f68580a48e0 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6858068d20 con 0x7f685809a5e0 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient(hunting): authenticate will time out at 1266.065881s 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685ccd9700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f68580a48e0 0x7f6858099ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39662/0 (socket says 172.21.15.26:39662) 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685ccd9700 1 -- 172.21.15.26:0/4246507419 learned_addr learned my addr 172.21.15.26:0/4246507419 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813863664 0 0) 0x7f68580696a0 con 0x7f68580a48e0 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): my global_id is 134750 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6840003570 con 0x7f68580a48e0 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317111025 0 0) 0x7f6858068d20 con 0x7f685809a5e0 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): my global_id is 134638 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68580696a0 con 0x7f685809a5e0 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028654950 0 0) 0x7f685804ca10 con 0x7f685809aca0 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient(hunting): my global_id is 184240 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6858068d20 con 0x7f685809aca0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386891787 0 0) 0x7f6840003570 con 0x7f68580a48e0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 >> v1:172.21.15.122:6792/0 conn(0x7f685809a5e0 legacy=0x7f685813b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 >> v1:172.21.15.122:6789/0 conn(0x7f685809aca0 legacy=0x7f6858137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 monclient: found mon.noname-b 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858067640 con 0x7f68580a48e0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f684c0017b0 con 0x7f68580a48e0 2024-09-15T03:52:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 5 monclient: authenticate success, global_id 134750 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: dump: 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.646 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f684c001a20 con 0x7f68580a48e0 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 1 -- 172.21.15.26:0/4246507419 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f684c004c30 con 0x7f68580a48e0 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f68577fe700 10 monclient: dump: 2024-09-15T03:52:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.632+0000 7f685dcdb700 4 set_mon_vals no callback set 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient: shutdown 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 -- 172.21.15.26:0/4246507419 >> v1:172.21.15.26:6790/0 conn(0x7f68580a48e0 legacy=0x7f6858099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 -- 172.21.15.26:0/4246507419 shutdown_connections 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 -- 172.21.15.26:0/4246507419 >> 172.21.15.26:0/4246507419 conn(0x7f68580638b0 msgr2=0x7f68580a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 -- 172.21.15.26:0/4246507419 shutdown_connections 2024-09-15T03:52:01.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 -- 172.21.15.26:0/4246507419 wait complete. 2024-09-15T03:52:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient: monmap: 2024-09-15T03:52:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 Processor -- start 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 1 -- start start 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient: init 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient: _add_conns ranks=[5,8,4,2,3,0,6,1,7] 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient(hunting): picked mon.noname-h con 0x7f685809a5e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient(hunting): picked mon.noname-e con 0x7f685809aca0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.636+0000 7f685dcdb700 10 monclient(hunting): picked mon.noname-c con 0x7f68580a48e0 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6858122e90 con 0x7f68580a48e0 2024-09-15T03:52:01.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685813e220 con 0x7f685809a5e0 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685813eee0 con 0x7f685809aca0 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 monclient(hunting): authenticate will time out at 1266.072084s 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685d4da700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f68580a48e0 0x7f685814a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55458/0 (socket says 172.21.15.26:55458) 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685d4da700 1 -- 172.21.15.26:0/1444417363 learned_addr learned my addr 172.21.15.26:0/1444417363 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754758413 0 0) 0x7f685813e220 con 0x7f685809a5e0 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): my global_id is 5252 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6844003570 con 0x7f685809a5e0 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547781853 0 0) 0x7f6858122e90 con 0x7f68580a48e0 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): my global_id is 184234 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685813e220 con 0x7f68580a48e0 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 399771882 0 0) 0x7f6844003570 con 0x7f685809a5e0 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6858122e90 con 0x7f685809a5e0 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f684c001770 con 0x7f685809a5e0 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172753536 0 0) 0x7f685813eee0 con 0x7f685809aca0 2024-09-15T03:52:01.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): my global_id is 134711 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6844003570 con 0x7f685809aca0 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4123089759 0 0) 0x7f685813e220 con 0x7f68580a48e0 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f685813eee0 con 0x7f68580a48e0 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6850001da0 con 0x7f68580a48e0 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:01.660 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.661 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1362637356 0 0) 0x7f6858122e90 con 0x7f685809a5e0 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.26:6793/0 conn(0x7f685809aca0 legacy=0x7f6858146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.26:6791/0 conn(0x7f68580a48e0 legacy=0x7f685814a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 monclient: found mon.h 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858122bb0 con 0x7f685809a5e0 2024-09-15T03:52:01.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 5 monclient: authenticate success, global_id 5252 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 monclient: _renew_subs 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6858123170 con 0x7f685809a5e0 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 monclient: _renew_subs 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f685813e630 con 0x7f685809a5e0 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f684c004e60 con 0x7f685809a5e0 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f684c005310 con 0x7f685809a5e0 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6856ffd700 4 set_mon_vals no callback set 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f6854ff9700 10 monclient: dump: 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.664 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.665 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.640+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6838006590 con 0x7f6838002840 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f684c00fcd0 con 0x7f685809a5e0 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(333..333 src has 224..333) v4 ==== 3911+0+0 (unknown 2717171964 0 0) 0x7f684c044000 con 0x7f685809a5e0 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668711415 0 0) 0x7f6838006590 con 0x7f6838002840 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient(hunting): my global_id is 184206 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6858122e90 con 0x7f6838002840 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f685d4da700 10 monclient: get_auth_request con 0x7f6844037d90 auth_method 0 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 757124577 0 0) 0x7f6858122e90 con 0x7f6838002840 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6838006590 con 0x7f6838002840 2024-09-15T03:52:01.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f68480018e0 con 0x7f6838002840 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient: dump: 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.667 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 576925502 0 0) 0x7f6838006590 con 0x7f6838002840 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f683800a130 con 0x7f6838002840 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6848001f20 con 0x7f6838002840 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.122:6790/0 conn(0x7f6838002840 legacy=0x7f6838004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.644+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.122:6790/0 conn(0x7f6838002840 legacy=0x7f6838004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f685dcdb700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6838009640 con 0x7f68380090b0 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589659255 0 0) 0x7f6838009640 con 0x7f68380090b0 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 10 monclient(hunting): my global_id is 184215 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6838006590 con 0x7f68380090b0 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1956400179 0 0) 0x7f6838006590 con 0x7f68380090b0 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6838009640 con 0x7f68380090b0 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f68480027b0 con 0x7f68380090b0 2024-09-15T03:52:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.652+0000 7f6854ff9700 10 monclient: dump: 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1745729946 0 0) 0x7f6838009640 con 0x7f68380090b0 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f68380098f0 con 0x7f68380090b0 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1505776396) 0x7f6848002910 con 0x7f68380090b0 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.122:6790/0 conn(0x7f68380090b0 legacy=0x7f6838008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f6854ff9700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.122:6790/0 conn(0x7f68380090b0 legacy=0x7f6838008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.674 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:01.675 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:01.676 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:01.677 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:01.677 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:01.677 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:01.677 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:01.677 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.677 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.681 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.682 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:01.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.689 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.693 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.694 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.695 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.696 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:01.697 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.122:6812/20895 conn(0x7f6844037d90 legacy=0x7f684403a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 10 monclient: shutdown 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 >> v1:172.21.15.122:6791/0 conn(0x7f685809a5e0 legacy=0x7f685813db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685d4da700 1 -- 172.21.15.26:0/1444417363 reap_dead start 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685d4da700 1 -- 172.21.15.26:0/1444417363 reap_dead start 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 shutdown_connections 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 >> 172.21.15.26:0/1444417363 conn(0x7f68580638b0 msgr2=0x7f6858098200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 shutdown_connections 2024-09-15T03:52:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.656+0000 7f685dcdb700 1 -- 172.21.15.26:0/1444417363 wait complete. 2024-09-15T03:52:01.699 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:01.699 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:52:01.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:52:01.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.812+0000 7f73e302e700 10 monclient: get_monmap_and_config 2024-09-15T03:52:01.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.812+0000 7f73e302e700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.812+0000 7f73e302e700 10 monclient: monmap: 2024-09-15T03:52:01.819 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.819 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:01.817789+0000 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:01.817789+0000 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.820 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.821 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.821 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 Processor -- start 2024-09-15T03:52:01.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- start start 2024-09-15T03:52:01.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient: init 2024-09-15T03:52:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient: _add_conns ranks=[3,2,7,8,5,6,4,0,1] 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient(hunting): picked mon.noname-g con 0x7f73dc0a2a70 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient(hunting): picked mon.noname-b con 0x7f73dc09e3d0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient(hunting): picked mon.noname-i con 0x7f73dc09c060 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc04ca10 con 0x7f73dc09e3d0 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc099ef0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc099300 con 0x7f73dc09c060 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient(hunting): authenticate will time out at 1266.253476s 2024-09-15T03:52:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e182b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f73dc09e3d0 0x7f73dc1349b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:39664/0 (socket says 172.21.15.26:39664) 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e182b700 1 -- 172.21.15.26:0/3454911566 learned_addr learned my addr 172.21.15.26:0/3454911566 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782052613 0 0) 0x7f73dc04ca10 con 0x7f73dc09e3d0 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient(hunting): my global_id is 134759 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c8003570 con 0x7f73dc09e3d0 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307744892 0 0) 0x7f73dc099ef0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient(hunting): my global_id is 184224 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc04ca10 con 0x7f73dc0a2a70 2024-09-15T03:52:01.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 341959754 0 0) 0x7f73c8003570 con 0x7f73dc09e3d0 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 >> v1:172.21.15.122:6792/0 conn(0x7f73dc09c060 legacy=0x7f73dc1380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 >> v1:172.21.15.122:6790/0 conn(0x7f73dc0a2a70 legacy=0x7f73dc09b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 monclient: found mon.noname-b 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc09dd40 con 0x7f73dc09e3d0 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73d4001b30 con 0x7f73dc09e3d0 2024-09-15T03:52:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: dump: 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.832 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 5 monclient: authenticate success, global_id 134759 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f73d4001e10 con 0x7f73dc09e3d0 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 1 -- 172.21.15.26:0/3454911566 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73d4002000 con 0x7f73dc09e3d0 2024-09-15T03:52:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 4 set_mon_vals no callback set 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e102a700 10 monclient: dump: 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 10 monclient: shutdown 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- 172.21.15.26:0/3454911566 >> v1:172.21.15.26:6790/0 conn(0x7f73dc09e3d0 legacy=0x7f73dc1349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- 172.21.15.26:0/3454911566 shutdown_connections 2024-09-15T03:52:01.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- 172.21.15.26:0/3454911566 >> 172.21.15.26:0/3454911566 conn(0x7f73dc063cd0 msgr2=0x7f73dc0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- 172.21.15.26:0/3454911566 shutdown_connections 2024-09-15T03:52:01.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.820+0000 7f73e302e700 1 -- 172.21.15.26:0/3454911566 wait complete. 2024-09-15T03:52:01.837 INFO:tasks.rados.rados.0.smithi122.stdout:1428: done (0 left) 2024-09-15T03:52:01.837 INFO:tasks.rados.rados.0.smithi122.stdout:1429: read oid 22 snap -1 2024-09-15T03:52:01.838 INFO:tasks.rados.rados.0.smithi122.stdout:1429: expect (ObjNum 522 snap 156 seq_num 522) 2024-09-15T03:52:01.838 INFO:tasks.rados.rados.0.smithi122.stdout:1430: write oid 29 current snap is 158 2024-09-15T03:52:01.838 INFO:tasks.rados.rados.0.smithi122.stdout:1430: seq_num 525 ranges {423675=734082,1721870=545170,2717777=105898} 2024-09-15T03:52:01.838 INFO:tasks.rados.rados.0.smithi122.stdout:1430: writing smithi12223906-29 from 423675 to 1157757 tid 1 2024-09-15T03:52:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient: build_initial_monmap 2024-09-15T03:52:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient: monmap: 2024-09-15T03:52:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:01.842 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:01.843 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 1 Processor -- start 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 1 -- start start 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient: init 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient: _add_conns ranks=[4,0,8,1,2,6,5,3,7] 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient(hunting): picked mon.noname-c con 0x7f73dc09c060 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient(hunting): picked mon.noname-a con 0x7f73dc09e3d0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient(hunting): picked mon.noname-e con 0x7f73dc0a2a70 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:01.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc06de90 con 0x7f73dc09e3d0 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc099530 con 0x7f73dc09c060 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc11e180 con 0x7f73dc0a2a70 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e302e700 10 monclient(hunting): authenticate will time out at 1266.258357s 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e202c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f73dc09c060 0x7f73dc11d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55466/0 (socket says 172.21.15.26:55466) 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73e202c700 1 -- 172.21.15.26:0/1441215496 learned_addr learned my addr 172.21.15.26:0/1441215496 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497893106 0 0) 0x7f73dc11e180 con 0x7f73dc0a2a70 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): my global_id is 134720 2024-09-15T03:52:01.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c0003570 con 0x7f73dc0a2a70 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920676836 0 0) 0x7f73dc06de90 con 0x7f73dc09e3d0 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): my global_id is 5130 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc11e180 con 0x7f73dc09e3d0 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 479934573 0 0) 0x7f73c0003570 con 0x7f73dc0a2a70 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73dc06de90 con 0x7f73dc0a2a70 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73cc0019a0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): dump: 2024-09-15T03:52:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.847 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2970088904 0 0) 0x7f73dc11e180 con 0x7f73dc09e3d0 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73c0003570 con 0x7f73dc09e3d0 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73d4001af0 con 0x7f73dc09e3d0 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient(hunting): dump: 2024-09-15T03:52:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2871120277 0 0) 0x7f73dc06de90 con 0x7f73dc0a2a70 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.26:6791/0 conn(0x7f73dc09c060 legacy=0x7f73dc11d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.26:6789/0 conn(0x7f73dc09e3d0 legacy=0x7f73dc11da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 1 monclient: found mon.e 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.824+0000 7f73d27fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc09d310 con 0x7f73dc0a2a70 2024-09-15T03:52:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 5 monclient: authenticate success, global_id 134720 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 10 monclient: _renew_subs 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc11f8e0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 10 monclient: _renew_subs 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc09caf0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f73dc09cd40 con 0x7f73dc120ac0 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f73cc002c70 con 0x7f73dc0a2a70 2024-09-15T03:52:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73cc002270 con 0x7f73dc0a2a70 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: dump: 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d3fff700 4 set_mon_vals no callback set 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f73cc0023d0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(334..334 src has 224..334) v4 ==== 3911+0+0 (unknown 4244863055 0 0) 0x7f73cc0413f0 con 0x7f73dc0a2a70 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984215294 0 0) 0x7f73dc099530 con 0x7f73dc120ac0 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient(hunting): my global_id is 5261 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc06de90 con 0x7f73dc120ac0 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73e182b700 10 monclient: get_auth_request con 0x7f73c0037b00 auth_method 0 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2481126571 0 0) 0x7f73dc06de90 con 0x7f73dc120ac0 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73dc099530 con 0x7f73dc120ac0 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73d80028d0 con 0x7f73dc120ac0 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 10 monclient: dump: 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 504530725 0 0) 0x7f73dc099530 con 0x7f73dc120ac0 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.828+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73dc11eb20 con 0x7f73dc120ac0 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.832+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f73d8005810 con 0x7f73dc120ac0 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.832+0000 7f73d27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.832+0000 7f73d27fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.832+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.122:6791/0 conn(0x7f73dc120ac0 legacy=0x7f73dc06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.832+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.122:6791/0 conn(0x7f73dc120ac0 legacy=0x7f73dc06be60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.858 INFO:tasks.rados.rados.0.smithi122.stdout:1430: writing smithi12223906-29 from 1721870 to 2267040 tid 2 2024-09-15T03:52:01.858 INFO:tasks.rados.rados.0.smithi122.stdout:1430: writing smithi12223906-29 from 2717777 to 2823675 tid 3 2024-09-15T03:52:01.858 INFO:tasks.rados.rados.0.smithi122.stdout:1431: write oid 21 current snap is 158 2024-09-15T03:52:01.859 INFO:tasks.rados.rados.0.smithi122.stdout:1431: seq_num 526 ranges {440482=409331,1380352=405181,2461553=378929} 2024-09-15T03:52:01.859 INFO:tasks.rados.rados.0.smithi122.stdout:1431: writing smithi12223906-21 from 440482 to 849813 tid 1 2024-09-15T03:52:01.859 INFO:tasks.rados.rados.0.smithi122.stdout:1431: writing smithi12223906-21 from 1380352 to 1785533 tid 2 2024-09-15T03:52:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73e302e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f73dc11fc00 con 0x7f73dc02ad10 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934215596 0 0) 0x7f73dc11fc00 con 0x7f73dc02ad10 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 10 monclient(hunting): my global_id is 5270 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73dc099530 con 0x7f73dc02ad10 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2759127905 0 0) 0x7f73dc099530 con 0x7f73dc02ad10 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73dc11fc00 con 0x7f73dc02ad10 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f73d8001880 con 0x7f73dc02ad10 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.836+0000 7f73d27fc700 10 monclient: dump: 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3810854923 0 0) 0x7f73dc11fc00 con 0x7f73dc02ad10 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f73dc11fa70 con 0x7f73dc02ad10 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7738 (unknown 0 0 890576080) 0x7f73d8005ac0 con 0x7f73dc02ad10 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.122:6791/0 conn(0x7f73dc02ad10 legacy=0x7f73dc06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73d27fc700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.122:6791/0 conn(0x7f73dc02ad10 legacy=0x7f73dc06c570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:01.863 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:01.864 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.865 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.865 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.865 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.865 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.865 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.866 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:01.867 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:01.868 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:01.869 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:01.870 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.871 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.872 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.875 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.878 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:01.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:01.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:01.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:52:01.885 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:01.886 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:01.888 INFO:tasks.rados.rados.0.smithi122.stdout:1431: writing smithi12223906-21 from 2461553 to 2840482 tid 3 2024-09-15T03:52:01.888 INFO:tasks.rados.rados.0.smithi122.stdout:1432: copy_from oid 33 from oid 35 current snap is 158 2024-09-15T03:52:01.888 INFO:tasks.rados.rados.0.smithi122.stdout:1430: finishing write tid 1 to smithi12223906-29 2024-09-15T03:52:01.888 INFO:tasks.rados.rados.0.smithi122.stdout:1429: expect (ObjNum 522 snap 156 seq_num 522) 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.122:6812/20895 conn(0x7f73c0037b00 legacy=0x7f73c0039fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 10 monclient: shutdown 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 >> v1:172.21.15.26:6793/0 conn(0x7f73dc0a2a70 legacy=0x7f73dc147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e282d700 1 -- 172.21.15.26:0/1441215496 reap_dead start 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e282d700 1 -- 172.21.15.26:0/1441215496 reap_dead start 2024-09-15T03:52:01.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 shutdown_connections 2024-09-15T03:52:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 >> 172.21.15.26:0/1441215496 conn(0x7f73dc063cd0 msgr2=0x7f73dc09e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 shutdown_connections 2024-09-15T03:52:01.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.840+0000 7f73e302e700 1 -- 172.21.15.26:0/1441215496 wait complete. 2024-09-15T03:52:01.890 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:01.890 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:52:01.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:1430: finishing write tid 2 to smithi12223906-29 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:1430: finishing write tid 3 to smithi12223906-29 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:1430: finishing write tid 4 to smithi12223906-29 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:1430: finishing write tid 5 to smithi12223906-29 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 638 (ObjNum 525 snap 158 seq_num 525) dirty exists 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:1430: left oid 29 (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:01.960 INFO:tasks.rados.rados.0.smithi122.stdout:1431: finishing write tid 1 to smithi12223906-21 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1431: finishing write tid 2 to smithi12223906-21 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1431: finishing write tid 3 to smithi12223906-21 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1429: done (3 left) 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1430: done (2 left) 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1433: snap_create 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1431: finishing write tid 4 to smithi12223906-21 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1431: finishing write tid 5 to smithi12223906-21 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 405 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1431: left oid 21 (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1432: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:1432: finishing copy_from to smithi12223906-33 2024-09-15T03:52:01.961 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 563 (ObjNum 508 snap 151 seq_num 508) dirty exists 2024-09-15T03:52:02.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.996+0000 7ff7db0e1700 10 monclient: get_monmap_and_config 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.996+0000 7ff7db0e1700 10 monclient: build_initial_monmap 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.996+0000 7ff7db0e1700 10 monclient: monmap: 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:02.001533+0000 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:02.001533+0000 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:02.003 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.996+0000 7ff7db0e1700 1 Processor -- start 2024-09-15T03:52:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.996+0000 7ff7db0e1700 1 -- start start 2024-09-15T03:52:02.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:01.996+0000 7ff7db0e1700 10 monclient: init 2024-09-15T03:52:02.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient: _add_conns ranks=[5,1,7,0,8,3,6,2,4] 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient(hunting): picked mon.noname-h con 0x7ff7d409cf50 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient(hunting): picked mon.noname-f con 0x7ff7d409d320 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient(hunting): picked mon.noname-i con 0x7ff7d41356f0 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d404ca10 con 0x7ff7d409d320 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d4068a20 con 0x7ff7d409cf50 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d409eb40 con 0x7ff7d41356f0 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient(hunting): authenticate will time out at 1266.432466s 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d8e7d700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7ff7d409cf50 0x7ff7d409b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:46730/0 (socket says 172.21.15.26:46730) 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d8e7d700 1 -- 172.21.15.26:0/1482690421 learned_addr learned my addr 172.21.15.26:0/1482690421 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:02.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683870199 0 0) 0x7ff7d4068a20 con 0x7ff7d409cf50 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): my global_id is 5279 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c4003570 con 0x7ff7d409cf50 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060592289 0 0) 0x7ff7d404ca10 con 0x7ff7d409d320 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): my global_id is 184249 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d4068a20 con 0x7ff7d409d320 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263579979 0 0) 0x7ff7d409eb40 con 0x7ff7d41356f0 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient(hunting): my global_id is 134647 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d404ca10 con 0x7ff7d41356f0 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108601210 0 0) 0x7ff7d4068a20 con 0x7ff7d409d320 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 >> v1:172.21.15.122:6792/0 conn(0x7ff7d41356f0 legacy=0x7ff7d4137ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 >> v1:172.21.15.122:6791/0 conn(0x7ff7d409cf50 legacy=0x7ff7d409b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 monclient: found mon.noname-f 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d409a9d0 con 0x7ff7d409d320 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: _finish_auth 0 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 5 monclient: authenticate success, global_id 184249 2024-09-15T03:52:02.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0001790 con 0x7ff7d409d320 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: dump: 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.012 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7c0001a40 con 0x7ff7d409d320 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 4 set_mon_vals no callback set 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 1 -- 172.21.15.26:0/1482690421 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0001f70 con 0x7ff7d409d320 2024-09-15T03:52:02.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7d37fe700 10 monclient: dump: 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.014 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 10 monclient: shutdown 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- 172.21.15.26:0/1482690421 >> v1:172.21.15.122:6789/0 conn(0x7ff7d409d320 legacy=0x7ff7d409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- 172.21.15.26:0/1482690421 shutdown_connections 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- 172.21.15.26:0/1482690421 >> 172.21.15.26:0/1482690421 conn(0x7ff7d40a2560 msgr2=0x7ff7d40a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- 172.21.15.26:0/1482690421 shutdown_connections 2024-09-15T03:52:02.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.000+0000 7ff7db0e1700 1 -- 172.21.15.26:0/1482690421 wait complete. 2024-09-15T03:52:02.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient: build_initial_monmap 2024-09-15T03:52:02.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient: monmap: 2024-09-15T03:52:02.021 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 1 Processor -- start 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 1 -- start start 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient: init 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient: _add_conns ranks=[4,5,7,2,0,8,6,1,3] 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient(hunting): picked mon.noname-c con 0x7ff7d409cf50 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient(hunting): picked mon.noname-h con 0x7ff7d409d320 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient(hunting): picked mon.noname-i con 0x7ff7d41356f0 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d412dbb0 con 0x7ff7d409cf50 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d412c970 con 0x7ff7d409d320 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d412d8d0 con 0x7ff7d41356f0 2024-09-15T03:52:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7db0e1700 10 monclient(hunting): authenticate will time out at 1266.438311s 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d8e7d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7ff7d409cf50 0x7ff7d4095310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55468/0 (socket says 172.21.15.26:55468) 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d8e7d700 1 -- 172.21.15.26:0/2008658324 learned_addr learned my addr 172.21.15.26:0/2008658324 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779163845 0 0) 0x7ff7d412dbb0 con 0x7ff7d409cf50 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): my global_id is 184252 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b8003570 con 0x7ff7d409cf50 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365187810 0 0) 0x7ff7d412d8d0 con 0x7ff7d41356f0 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): my global_id is 134656 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d412dbb0 con 0x7ff7d41356f0 2024-09-15T03:52:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719096831 0 0) 0x7ff7d412c970 con 0x7ff7d409d320 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): my global_id is 5288 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d412d8d0 con 0x7ff7d409d320 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 579281330 0 0) 0x7ff7b8003570 con 0x7ff7d409cf50 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7d412c970 con 0x7ff7d409cf50 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c80015b0 con 0x7ff7d409cf50 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.026 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3097114018 0 0) 0x7ff7d412dbb0 con 0x7ff7d41356f0 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7b8003570 con 0x7ff7d41356f0 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7cc001cc0 con 0x7ff7d41356f0 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.027 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1404529105 0 0) 0x7ff7d412d8d0 con 0x7ff7d409d320 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7d412dbb0 con 0x7ff7d409d320 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0001750 con 0x7ff7d409d320 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.029 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2439247820 0 0) 0x7ff7d412c970 con 0x7ff7d409cf50 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6792/0 conn(0x7ff7d41356f0 legacy=0x7ff7d414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6791/0 conn(0x7ff7d409d320 legacy=0x7ff7d4146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:52:02.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 monclient: found mon.c 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.004+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d412de90 con 0x7ff7d409cf50 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: _finish_auth 0 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 5 monclient: authenticate success, global_id 184252 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 10 monclient: _renew_subs 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d412bda0 con 0x7ff7d409cf50 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 10 monclient: _renew_subs 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d406a350 con 0x7ff7d409cf50 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7c8004ae0 con 0x7ff7d409cf50 2024-09-15T03:52:02.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c8004f90 con 0x7ff7d409cf50 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d2ffd700 4 set_mon_vals no callback set 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: dump: 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.032 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff7b4006590 con 0x7ff7b4002840 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff7c800f950 con 0x7ff7d409cf50 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(334..334 src has 224..334) v4 ==== 3911+0+0 (unknown 4244863055 0 0) 0x7ff7c8043ff0 con 0x7ff7d409cf50 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258705119 0 0) 0x7ff7b4006590 con 0x7ff7b4002840 2024-09-15T03:52:02.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient(hunting): my global_id is 134665 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d412c970 con 0x7ff7b4002840 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d967e700 10 monclient: get_auth_request con 0x7ff7b8038030 auth_method 0 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1845305076 0 0) 0x7ff7d412c970 con 0x7ff7b4002840 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7b4006590 con 0x7ff7b4002840 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0002580 con 0x7ff7b4002840 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: dump: 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.034 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1253460487 0 0) 0x7ff7b4006590 con 0x7ff7b4002840 2024-09-15T03:52:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7b400a130 con 0x7ff7b4002840 2024-09-15T03:52:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff7c00047d0 con 0x7ff7b4002840 2024-09-15T03:52:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6792/0 conn(0x7ff7b4002840 legacy=0x7ff7b4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.008+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6792/0 conn(0x7ff7b4002840 legacy=0x7ff7b4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7db0e1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff7b4007520 con 0x7ff7b40068d0 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810153044 0 0) 0x7ff7b4007520 con 0x7ff7b40068d0 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 10 monclient(hunting): my global_id is 134674 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b4006590 con 0x7ff7b40068d0 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 156098572 0 0) 0x7ff7b4006590 con 0x7ff7b40068d0 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7b4007520 con 0x7ff7b40068d0 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0003010 con 0x7ff7b40068d0 2024-09-15T03:52:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 10 monclient: dump: 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3790915310 0 0) 0x7ff7b4007520 con 0x7ff7b40068d0 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.020+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff7b4008760 con 0x7ff7b40068d0 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2479512813) 0x7ff7c0004a00 con 0x7ff7b40068d0 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d0ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d0ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6792/0 conn(0x7ff7b40068d0 legacy=0x7ff7b4006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d0ff9700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6792/0 conn(0x7ff7b40068d0 legacy=0x7ff7b4006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 58, 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:52:02.041 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:02.042 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:02.043 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:02.044 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:02.045 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:02.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.047 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:02.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.051 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:02.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.055 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.058 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:02.059 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:02.060 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.122:6812/20895 conn(0x7ff7b8038030 legacy=0x7ff7b803a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 10 monclient: shutdown 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 >> v1:172.21.15.26:6791/0 conn(0x7ff7d409cf50 legacy=0x7ff7d4095310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d967e700 1 -- 172.21.15.26:0/2008658324 reap_dead start 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7d967e700 1 -- 172.21.15.26:0/2008658324 reap_dead start 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 shutdown_connections 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 >> 172.21.15.26:0/2008658324 conn(0x7ff7d40a2560 msgr2=0x7ff7d409b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 shutdown_connections 2024-09-15T03:52:02.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.024+0000 7ff7db0e1700 1 -- 172.21.15.26:0/2008658324 wait complete. 2024-09-15T03:52:02.063 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:52:02.063 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['d', 'g', 'a'] 2024-09-15T03:52:02.063 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T03:52:02.063 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-09-15T03:52:02.063 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-15T03:52:02.063 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-15T03:52:02.064 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:52:02.081 INFO:tasks.ceph.mon.d:Stopped 2024-09-15T03:52:02.082 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T03:52:02.082 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:52:02.082 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T03:52:02.082 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:52:02.099 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T03:52:02.099 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-15T03:52:02.099 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T03:52:02.099 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T03:52:02.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:52:02.122 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T03:52:02.122 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-15T03:52:02.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:52:02.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.220+0000 7fe8151c6700 10 monclient: get_monmap_and_config 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.220+0000 7fe8151c6700 10 monclient: build_initial_monmap 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.220+0000 7fe8151c6700 10 monclient: monmap: 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:02.227683+0000 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:02.227683+0000 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:02.230 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 Processor -- start 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- start start 2024-09-15T03:52:02.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient: init 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient: _add_conns ranks=[4,8,0,6,1,2,7,5,3] 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient(hunting): picked mon.noname-c con 0x7fe8100a6a80 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient(hunting): picked mon.noname-e con 0x7fe81009fb80 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient(hunting): picked mon.noname-a con 0x7fe810138d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81004ca10 con 0x7fe810138d30 2024-09-15T03:52:02.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8100a0f50 con 0x7fe8100a6a80 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe810133680 con 0x7fe81009fb80 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient(hunting): authenticate will time out at 1266.657114s 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8149c5700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7fe810138d30 legacy=0x7fe81013b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80ffff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fe8100a6a80 0x7fe810132f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55480/0 (socket says 172.21.15.26:55480) 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80ffff700 1 -- 172.21.15.26:0/2546984557 learned_addr learned my addr 172.21.15.26:0/2546984557 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792222559 0 0) 0x7fe8100a0f50 con 0x7fe8100a6a80 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient(hunting): my global_id is 184261 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f0003570 con 0x7fe8100a6a80 2024-09-15T03:52:02.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259681163 0 0) 0x7fe810133680 con 0x7fe81009fb80 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient(hunting): my global_id is 134729 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8100a0f50 con 0x7fe81009fb80 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4294449958 0 0) 0x7fe8100a0f50 con 0x7fe81009fb80 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 >> v1:172.21.15.26:6791/0 conn(0x7fe8100a6a80 legacy=0x7fe810132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 >> v1:172.21.15.26:6789/0 conn(0x7fe810138d30 legacy=0x7fe81013b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: _finish_hunting 0 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 monclient: found mon.noname-e 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100a6e50 con 0x7fe81009fb80 2024-09-15T03:52:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: _finish_auth 0 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe7f8001790 con 0x7fe81009fb80 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 5 monclient: authenticate success, global_id 134729 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: dump: 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe7f8001a30 con 0x7fe81009fb80 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 1 -- 172.21.15.26:0/2546984557 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe7f8001f60 con 0x7fe81009fb80 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 4 set_mon_vals no callback set 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe80effd700 10 monclient: dump: 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 10 monclient: shutdown 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- 172.21.15.26:0/2546984557 >> v1:172.21.15.26:6793/0 conn(0x7fe81009fb80 legacy=0x7fe81009ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- 172.21.15.26:0/2546984557 shutdown_connections 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- 172.21.15.26:0/2546984557 >> 172.21.15.26:0/2546984557 conn(0x7fe81006e380 msgr2=0x7fe8100a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- 172.21.15.26:0/2546984557 shutdown_connections 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.224+0000 7fe8151c6700 1 -- 172.21.15.26:0/2546984557 wait complete. 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient: build_initial_monmap 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient: monmap: 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 1 Processor -- start 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 1 -- start start 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient: init 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient: _add_conns ranks=[4,8,5,2,3,0,1,7,6] 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient(hunting): picked mon.noname-c con 0x7fe81009fb80 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient(hunting): picked mon.noname-e con 0x7fe8100a6a80 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient(hunting): picked mon.noname-h con 0x7fe810138d30 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe810136c60 con 0x7fe81009fb80 2024-09-15T03:52:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe810136980 con 0x7fe810138d30 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8101348b0 con 0x7fe8100a6a80 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe8151c6700 10 monclient(hunting): authenticate will time out at 1266.663038s 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe80ffff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fe81009fb80 0x7fe810133a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55488/0 (socket says 172.21.15.26:55488) 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe80ffff700 1 -- 172.21.15.26:0/1038693048 learned_addr learned my addr 172.21.15.26:0/1038693048 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087130119 0 0) 0x7fe810136c60 con 0x7fe81009fb80 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 10 monclient(hunting): my global_id is 184270 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f4003570 con 0x7fe81009fb80 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361318827 0 0) 0x7fe8101348b0 con 0x7fe8100a6a80 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 10 monclient(hunting): my global_id is 134738 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.228+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe810136c60 con 0x7fe8100a6a80 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281550479 0 0) 0x7fe810136980 con 0x7fe810138d30 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): my global_id is 5297 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8101348b0 con 0x7fe810138d30 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1371849902 0 0) 0x7fe7f4003570 con 0x7fe81009fb80 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe810136980 con 0x7fe81009fb80 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe800001cc0 con 0x7fe81009fb80 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): dump: 2024-09-15T03:52:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.255 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1032770453 0 0) 0x7fe810136c60 con 0x7fe8100a6a80 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7f4003570 con 0x7fe8100a6a80 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe7f8001ee0 con 0x7fe8100a6a80 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): dump: 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.256 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2505019166 0 0) 0x7fe8101348b0 con 0x7fe810138d30 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe810136c60 con 0x7fe810138d30 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe808004530 con 0x7fe810138d30 2024-09-15T03:52:02.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient(hunting): dump: 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.258 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2477814208 0 0) 0x7fe810136980 con 0x7fe81009fb80 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 >> v1:172.21.15.26:6793/0 conn(0x7fe8100a6a80 legacy=0x7fe810134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 >> v1:172.21.15.122:6791/0 conn(0x7fe810138d30 legacy=0x7fe81014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: _finish_hunting 0 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 monclient: found mon.c 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810137b80 con 0x7fe81009fb80 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: _finish_auth 0 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 5 monclient: authenticate success, global_id 184270 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 monclient: _renew_subs 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe810136700 con 0x7fe81009fb80 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 monclient: _renew_subs 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8101357b0 con 0x7fe81009fb80 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe800004ae0 con 0x7fe81009fb80 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:02.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe800005010 con 0x7fe81009fb80 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 10 monclient: dump: 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:02.261 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe80e7fc700 4 set_mon_vals no callback set 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81004bbf0 con 0x7fe81009fb80 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe80000f9d0 con 0x7fe81009fb80 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(334..334 src has 224..334) v4 ==== 3911+0+0 (unknown 4244863055 0 0) 0x7fe800043f00 con 0x7fe81009fb80 2024-09-15T03:52:02.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.232+0000 7fe80f7fe700 10 monclient: get_auth_request con 0x7fe7f40380c0 auth_method 0 2024-09-15T03:52:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.240+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fe8000113c0 con 0x7fe81009fb80 2024-09-15T03:52:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.240+0000 7fe807fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:02.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.240+0000 7fe807fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe8151c6700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:52:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe8151c6700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe8151c6700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe81006fac0 con 0x7fe81009fb80 2024-09-15T03:52:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe807fff700 1 -- 172.21.15.26:0/1038693048 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 3749132318) 0x7fe800014790 con 0x7fe81009fb80 2024-09-15T03:52:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe807fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:02.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.464+0000 7fe807fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:02.473 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 >> v1:172.21.15.122:6812/20895 conn(0x7fe7f40380c0 legacy=0x7fe7f403a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 10 monclient: shutdown 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 >> v1:172.21.15.26:6791/0 conn(0x7fe81009fb80 legacy=0x7fe810133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 shutdown_connections 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 >> 172.21.15.26:0/1038693048 conn(0x7fe81006e380 msgr2=0x7fe8100678f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 shutdown_connections 2024-09-15T03:52:02.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:02.468+0000 7fe8151c6700 1 -- 172.21.15.26:0/1038693048 wait complete. 2024-09-15T03:52:05.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:52:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.592+0000 7fce6f09e700 10 monclient: get_monmap_and_config 2024-09-15T03:52:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.592+0000 7fce6f09e700 10 monclient: build_initial_monmap 2024-09-15T03:52:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.592+0000 7fce6f09e700 10 monclient: monmap: 2024-09-15T03:52:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:05.599812+0000 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:05.599812+0000 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:05.603 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 1 Processor -- start 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 1 -- start start 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient: init 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient: _add_conns ranks=[1,5,4,3,8,0,6,7,2] 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient(hunting): picked mon.noname-f con 0x7fce68070de0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient(hunting): picked mon.noname-h con 0x7fce680950b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient(hunting): picked mon.noname-c con 0x7fce68095b90 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6804ca10 con 0x7fce68070de0 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce68068980 con 0x7fce68095b90 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce680711b0 con 0x7fce680950b0 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6f09e700 10 monclient(hunting): authenticate will time out at 1270.030985s 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6e89d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fce68095b90 0x7fce68137f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55494/0 (socket says 172.21.15.26:55494) 2024-09-15T03:52:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6e89d700 1 -- 172.21.15.26:0/827236174 learned_addr learned my addr 172.21.15.26:0/827236174 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711066399 0 0) 0x7fce68068980 con 0x7fce68095b90 2024-09-15T03:52:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): my global_id is 184279 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce54003570 con 0x7fce68095b90 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332806752 0 0) 0x7fce6804ca10 con 0x7fce68070de0 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): my global_id is 184258 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce68068980 con 0x7fce68070de0 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307256129 0 0) 0x7fce680711b0 con 0x7fce680950b0 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 10 monclient(hunting): my global_id is 5306 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.596+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6804ca10 con 0x7fce680950b0 2024-09-15T03:52:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1260879798 0 0) 0x7fce54003570 con 0x7fce68095b90 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 >> v1:172.21.15.122:6791/0 conn(0x7fce680950b0 legacy=0x7fce68095480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 >> v1:172.21.15.122:6789/0 conn(0x7fce68070de0 legacy=0x7fce6809bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: _finish_hunting 0 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 monclient: found mon.noname-c 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6809aea0 con 0x7fce68095b90 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: _finish_auth 0 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce60001b30 con 0x7fce68095b90 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: dump: 2024-09-15T03:52:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 5 monclient: authenticate success, global_id 184279 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce60001e10 con 0x7fce68095b90 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 1 -- 172.21.15.26:0/827236174 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce60002000 con 0x7fce68095b90 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 4 set_mon_vals no callback set 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6d09a700 10 monclient: dump: 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: shutdown 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- 172.21.15.26:0/827236174 >> v1:172.21.15.26:6791/0 conn(0x7fce68095b90 legacy=0x7fce68137f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- 172.21.15.26:0/827236174 shutdown_connections 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- 172.21.15.26:0/827236174 >> 172.21.15.26:0/827236174 conn(0x7fce680a24e0 msgr2=0x7fce680a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- 172.21.15.26:0/827236174 shutdown_connections 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- 172.21.15.26:0/827236174 wait complete. 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: build_initial_monmap 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: monmap: 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:05.613 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:05.614 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 Processor -- start 2024-09-15T03:52:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- start start 2024-09-15T03:52:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: init 2024-09-15T03:52:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient: _add_conns ranks=[0,8,4,7,5,6,3,1,2] 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient(hunting): picked mon.noname-a con 0x7fce68070de0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient(hunting): picked mon.noname-e con 0x7fce680950b0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 10 monclient(hunting): picked mon.noname-c con 0x7fce68095b90 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6809c720 con 0x7fce68070de0 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6f09e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6806ea00 con 0x7fce68095b90 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6812aa10 con 0x7fce680950b0 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.600+0000 7fce6e09c700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7fce68070de0 legacy=0x7fce6806ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient(hunting): authenticate will time out at 1270.036078s 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6d89b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fce680950b0 0x7fce6806d120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:55688/0 (socket says 172.21.15.26:55688) 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6d89b700 1 -- 172.21.15.26:0/2682105965 learned_addr learned my addr 172.21.15.26:0/2682105965 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399276242 0 0) 0x7fce6806ea00 con 0x7fce68095b90 2024-09-15T03:52:05.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): my global_id is 184288 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4c003570 con 0x7fce68095b90 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759108360 0 0) 0x7fce6812aa10 con 0x7fce680950b0 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): my global_id is 134747 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce6806ea00 con 0x7fce680950b0 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3509006900 0 0) 0x7fce4c003570 con 0x7fce68095b90 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce6812aa10 con 0x7fce68095b90 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce60001ba0 con 0x7fce68095b90 2024-09-15T03:52:05.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 651763384 0 0) 0x7fce6812aa10 con 0x7fce68095b90 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 >> v1:172.21.15.26:6793/0 conn(0x7fce680950b0 legacy=0x7fce6806d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 >> v1:172.21.15.26:6789/0 conn(0x7fce68070de0 legacy=0x7fce6806ca10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 monclient: found mon.c 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6812a5e0 con 0x7fce68095b90 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 5 monclient: authenticate success, global_id 184288 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient: _renew_subs 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6812bcf0 con 0x7fce68095b90 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient: _renew_subs 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce68131690 con 0x7fce68095b90 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce60001f80 con 0x7fce68095b90 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:05.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce600054e0 con 0x7fce68095b90 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6c899700 4 set_mon_vals no callback set 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 10 monclient: dump: 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:05.624 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fce6000ff20 con 0x7fce68095b90 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce500050b0 con 0x7fce68095b90 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(334..334 src has 224..334) v4 ==== 3911+0+0 (unknown 4244863055 0 0) 0x7fce60044210 con 0x7fce68095b90 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.604+0000 7fce6e09c700 10 monclient: get_auth_request con 0x7fce4c037470 auth_method 0 2024-09-15T03:52:05.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.612+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fce60014ce0 con 0x7fce68095b90 2024-09-15T03:52:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.612+0000 7fce5e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:05.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.612+0000 7fce5e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:05.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce6f09e700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:52:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce6f09e700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce6f09e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fce500058e0 con 0x7fce68095b90 2024-09-15T03:52:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce5e7fc700 1 -- 172.21.15.26:0/2682105965 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 3764670336) 0x7fce60014ce0 con 0x7fce68095b90 2024-09-15T03:52:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce5e7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.844+0000 7fce5e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:05.852 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":58,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":9,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 >> v1:172.21.15.122:6812/20895 conn(0x7fce4c037470 legacy=0x7fce4c039920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 10 monclient: shutdown 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 >> v1:172.21.15.26:6791/0 conn(0x7fce68095b90 legacy=0x7fce68146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 shutdown_connections 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 >> 172.21.15.26:0/2682105965 conn(0x7fce680a24e0 msgr2=0x7fce68129e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 shutdown_connections 2024-09-15T03:52:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:05.848+0000 7fce6f09e700 1 -- 172.21.15.26:0/2682105965 wait complete. 2024-09-15T03:52:08.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient: get_monmap_and_config 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient: build_initial_monmap 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient: monmap: 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:08.984825+0000 2024-09-15T03:52:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:08.984825+0000 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:08.987 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 1 Processor -- start 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 1 -- start start 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient: init 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient: _add_conns ranks=[5,7,4,2,3,6,8,0,1] 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient(hunting): picked mon.noname-h con 0x7fd1a009bee0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient(hunting): picked mon.noname-i con 0x7fd1a009eca0 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient(hunting): picked mon.noname-c con 0x7fd1a0137030 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a004ca10 con 0x7fd1a0137030 2024-09-15T03:52:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a009ca30 con 0x7fd1a009bee0 2024-09-15T03:52:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a009f090 con 0x7fd1a009eca0 2024-09-15T03:52:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a6942700 10 monclient(hunting): authenticate will time out at 1273.414357s 2024-09-15T03:52:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a4edf700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd1a0137030 0x7fd1a0139410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:60094/0 (socket says 172.21.15.26:60094) 2024-09-15T03:52:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:08.980+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 learned_addr learned my addr 172.21.15.26:0/525774027 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient(hunting): tick 2024-09-15T03:52:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:52:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:52:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6792/0 conn(0x7fd1a009eca0 legacy=0x7fd1a0131470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6791/0 conn(0x7fd1a009bee0 legacy=0x7fd1a0130d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6791/0 conn(0x7fd1a0137030 legacy=0x7fd1a0139410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient: _add_conns ranks=[1,6,3,8,2,0] 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient(hunting): picked mon.noname-f con 0x7fd18c003830 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient(hunting): picked mon.noname-d con 0x7fd18c007490 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient(hunting): picked mon.noname-g con 0x7fd18c00b1f0 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0131b80 con 0x7fd18c003830 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c00fff0 con 0x7fd18c00b1f0 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6792/0 conn(0x7fd18c007490 legacy=0x7fd18c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c011240 con 0x7fd18c007490 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6790/0 conn(0x7fd18c00b1f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:11.980+0000 7fd19dffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:12.184+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6792/0 conn(0x7fd18c007490 legacy=0x7fd18c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:12.184+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6790/0 conn(0x7fd18c00b1f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:12.588+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6792/0 conn(0x7fd18c007490 legacy=0x7fd18c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:12.588+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6790/0 conn(0x7fd18c00b1f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:13.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:13.392+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6792/0 conn(0x7fd18c007490 legacy=0x7fd18c009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:13.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:13.392+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6790/0 conn(0x7fd18c00b1f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient(hunting): tick 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6792/0 conn(0x7fd18c007490 legacy=0x7fd18c009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6790/0 conn(0x7fd18c00b1f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.122:6789/0 conn(0x7fd18c003830 legacy=0x7fd18c005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient: _add_conns ranks=[2,0,8] 2024-09-15T03:52:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 reap_dead start 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient(hunting): picked mon.noname-b con 0x7fd18c00f110 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd1a4edf700 1 -- 172.21.15.26:0/525774027 reap_dead start 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient(hunting): picked mon.noname-a con 0x7fd18c00a0f0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient(hunting): picked mon.noname-e con 0x7fd18c00a4c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c012490 con 0x7fd18c00a0f0 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c00fff0 con 0x7fd18c00f110 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c011240 con 0x7fd18c00a4c0 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19dffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:14.980+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6789/0 conn(0x7fd18c00a0f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:15.184+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6789/0 conn(0x7fd18c00a0f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:15.588+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6789/0 conn(0x7fd18c00a0f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:16.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.392+0000 7fd19f7fe700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6789/0 conn(0x7fd18c00a0f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:16.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904391127 0 0) 0x7fd18c011240 con 0x7fd18c00a4c0 2024-09-15T03:52:16.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:16.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:16.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient(hunting): my global_id is 134756 2024-09-15T03:52:16.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd188003570 con 0x7fd18c00a4c0 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795154052 0 0) 0x7fd188003570 con 0x7fd18c00a4c0 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6790/0 conn(0x7fd18c00f110 legacy=0x7fd18c00f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6789/0 conn(0x7fd18c00a0f0 legacy=0x7fd18c00d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient: _finish_hunting 0 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 monclient: found mon.noname-e 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c005cd0 con 0x7fd18c00a4c0 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient: _finish_auth 0 2024-09-15T03:52:16.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd1980017b0 con 0x7fd18c00a4c0 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd19effd700 10 monclient: dump: 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:16.955 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd1a6942700 5 monclient: authenticate success, global_id 134756 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.944+0000 7fd1a6942700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd1980019e0 con 0x7fd18c00a4c0 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd19effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd19effd700 1 -- 172.21.15.26:0/525774027 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd198004c30 con 0x7fd18c00a4c0 2024-09-15T03:52:16.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 4 set_mon_vals no callback set 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd19effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd19effd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd19effd700 10 monclient: dump: 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:16.957 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:16.958 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:16.958 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:16.958 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:16.958 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:16.958 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:16.958 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 10 monclient: shutdown 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 1 -- 172.21.15.26:0/525774027 >> v1:172.21.15.26:6793/0 conn(0x7fd18c00a4c0 legacy=0x7fd18c009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 1 -- 172.21.15.26:0/525774027 shutdown_connections 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 1 -- 172.21.15.26:0/525774027 >> 172.21.15.26:0/525774027 conn(0x7fd1a0063d30 msgr2=0x7fd1a00a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 1 -- 172.21.15.26:0/525774027 shutdown_connections 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 1 -- 172.21.15.26:0/525774027 wait complete. 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 10 monclient: build_initial_monmap 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.948+0000 7fd1a6942700 10 monclient: monmap: 2024-09-15T03:52:16.960 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:16.961 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 1 Processor -- start 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 1 -- start start 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient: init 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient: _add_conns ranks=[4,6,2,8,0,1,5,7,3] 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient(hunting): picked mon.noname-c con 0x7fd18c00a0f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient(hunting): picked mon.noname-d con 0x7fd18c00a4c0 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient(hunting): picked mon.noname-b con 0x7fd18c00f110 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a0131df0 con 0x7fd18c00f110 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a011a910 con 0x7fd18c00a0f0 2024-09-15T03:52:16.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd19f7fe700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7fd18c00a4c0 legacy=0x7fd1a01456d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a011ac40 con 0x7fd18c00a4c0 2024-09-15T03:52:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a6942700 10 monclient(hunting): authenticate will time out at 1281.385669s 2024-09-15T03:52:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a4edf700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fd18c00f110 0x7fd1a0148e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48836/0 (socket says 172.21.15.26:48836) 2024-09-15T03:52:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd1a4edf700 1 -- 172.21.15.26:0/3209716238 learned_addr learned my addr 172.21.15.26:0/3209716238 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.952+0000 7fd19ffff700 1 --1- 172.21.15.26:0/3209716238 >> v1:172.21.15.26:6791/0 conn(0x7fd18c00a0f0 0x7fd1a012f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49028/0 (socket says 172.21.15.26:49028) 2024-09-15T03:52:16.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718520746 0 0) 0x7fd1a011a910 con 0x7fd18c00a0f0 2024-09-15T03:52:16.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): my global_id is 184297 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd18c00ef30 con 0x7fd18c00a0f0 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245096279 0 0) 0x7fd1a0131df0 con 0x7fd18c00f110 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): my global_id is 134768 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a011a910 con 0x7fd18c00f110 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2036673828 0 0) 0x7fd18c00ef30 con 0x7fd18c00a0f0 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd1a0131df0 con 0x7fd18c00a0f0 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd194002980 con 0x7fd18c00a0f0 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): dump: 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1828845570 0 0) 0x7fd1a0131df0 con 0x7fd18c00a0f0 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 >> v1:172.21.15.26:6792/0 conn(0x7fd18c00a4c0 legacy=0x7fd1a01456d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 >> v1:172.21.15.26:6790/0 conn(0x7fd18c00f110 legacy=0x7fd1a0148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: _finish_hunting 0 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 monclient: found mon.c 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:16.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a011af20 con 0x7fd18c00a0f0 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: _finish_auth 0 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 5 monclient: authenticate success, global_id 184297 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 10 monclient: _renew_subs 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a0134840 con 0x7fd18c00a0f0 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 10 monclient: _renew_subs 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a0133cf0 con 0x7fd18c00a0f0 2024-09-15T03:52:16.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd194002d30 con 0x7fd18c00a0f0 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd194001850 con 0x7fd18c00a0f0 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd19e7fc700 4 set_mon_vals no callback set 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.972+0000 7fd187fff700 10 monclient: dump: 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:16.987 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:16.988 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:16.988 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:16.988 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:16.988 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:16.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd1a6942700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:16.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd1a6942700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:16.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd1a6942700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:16.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1800050b0 con 0x7fd18c00a0f0 2024-09-15T03:52:16.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd194001a30 con 0x7fd18c00a0f0 2024-09-15T03:52:16.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(334..334 src has 224..334) v4 ==== 3911+0+0 (unknown 4244863055 0 0) 0x7fd194043d80 con 0x7fd18c00a0f0 2024-09-15T03:52:16.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.976+0000 7fd19f7fe700 10 monclient: get_auth_request con 0x7fd18c039710 auth_method 0 2024-09-15T03:52:16.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.988+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fd194070480 con 0x7fd18c00a0f0 2024-09-15T03:52:16.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.988+0000 7fd187fff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:16.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:16.988+0000 7fd187fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:17.028 INFO:tasks.rados.rados.0.smithi122.stdout:1431: done (2 left) 2024-09-15T03:52:17.028 INFO:tasks.rados.rados.0.smithi122.stdout:1432: done (1 left) 2024-09-15T03:52:17.028 INFO:tasks.rados.rados.0.smithi122.stdout:1433: done (0 left) 2024-09-15T03:52:17.028 INFO:tasks.rados.rados.0.smithi122.stdout:1434: write oid 40 current snap is 159 2024-09-15T03:52:17.028 INFO:tasks.rados.rados.0.smithi122.stdout:1434: seq_num 527 ranges {457289=484580,1438834=548839,2772623=84666} 2024-09-15T03:52:17.036 INFO:tasks.rados.rados.0.smithi122.stdout:1434: writing smithi12223906-40 from 457289 to 941869 tid 1 2024-09-15T03:52:17.044 INFO:tasks.rados.rados.0.smithi122.stdout:1434: writing smithi12223906-40 from 1438834 to 1987673 tid 2 2024-09-15T03:52:17.049 INFO:tasks.rados.rados.0.smithi122.stdout:1434: writing smithi12223906-40 from 2772623 to 2857289 tid 3 2024-09-15T03:52:17.050 INFO:tasks.rados.rados.0.smithi122.stdout:1435: write oid 11 current snap is 159 2024-09-15T03:52:17.050 INFO:tasks.rados.rados.0.smithi122.stdout:1434: finishing write tid 1 to smithi12223906-40 2024-09-15T03:52:17.050 INFO:tasks.rados.rados.0.smithi122.stdout:1435: seq_num 528 ranges {474096=559829,1780963=408850,2800046=74050} 2024-09-15T03:52:17.054 INFO:tasks.rados.rados.0.smithi122.stdout:1435: writing smithi12223906-11 from 474096 to 1033925 tid 1 2024-09-15T03:52:17.061 INFO:tasks.rados.rados.0.smithi122.stdout:1435: writing smithi12223906-11 from 1780963 to 2189813 tid 2 2024-09-15T03:52:17.066 INFO:tasks.rados.rados.0.smithi122.stdout:1435: writing smithi12223906-11 from 2800046 to 2874096 tid 3 2024-09-15T03:52:17.066 INFO:tasks.rados.rados.0.smithi122.stdout:1436: snap_remove snap 151 2024-09-15T03:52:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd1a6942700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:52:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd1a6942700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd1a6942700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:52:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd1800058e0 con 0x7fd18c00a0f0 2024-09-15T03:52:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd187fff700 1 -- 172.21.15.26:0/3209716238 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2682 (unknown 254746657 0 1773863567) 0x7fd194014800 con 0x7fd18c00a0f0 2024-09-15T03:52:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd187fff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:52:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.224+0000 7fd187fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:17.233 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":60,"quorum":[1,2,4,5,7,8],"quorum_names":["f","b","c","h","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:52:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 >> v1:172.21.15.122:6812/20895 conn(0x7fd18c039710 legacy=0x7fd18c03bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 10 monclient: shutdown 2024-09-15T03:52:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:17.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 >> v1:172.21.15.26:6791/0 conn(0x7fd18c00a0f0 legacy=0x7fd1a012f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 shutdown_connections 2024-09-15T03:52:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 >> 172.21.15.26:0/3209716238 conn(0x7fd1a0063d30 msgr2=0x7fd1a009ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 shutdown_connections 2024-09-15T03:52:17.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.228+0000 7fd1a6942700 1 -- 172.21.15.26:0/3209716238 wait complete. 2024-09-15T03:52:17.250 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-15T03:52:17.250 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-09-15T03:52:17.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:52:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.348+0000 7f0619f4e700 10 monclient: get_monmap_and_config 2024-09-15T03:52:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.348+0000 7f0619f4e700 10 monclient: build_initial_monmap 2024-09-15T03:52:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.348+0000 7f0619f4e700 10 monclient: monmap: 2024-09-15T03:52:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:17.355825+0000 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:17.355825+0000 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:17.358 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 1 Processor -- start 2024-09-15T03:52:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 1 -- start start 2024-09-15T03:52:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient: init 2024-09-15T03:52:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient: _add_conns ranks=[6,0,3,5,7,8,2,4,1] 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient(hunting): picked mon.noname-d con 0x7f06140a6880 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient(hunting): picked mon.noname-a con 0x7f061409faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient(hunting): picked mon.noname-g con 0x7f0614138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061404ca10 con 0x7f061409faa0 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06140a0580 con 0x7f0614138d30 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0614133780 con 0x7f06140a6880 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0613fff700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f061409faa0 legacy=0x7f061409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0618f4c700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f06140a6880 legacy=0x7f0614133070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f0619f4e700 10 monclient(hunting): authenticate will time out at 1281.785684s 2024-09-15T03:52:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.352+0000 7f061974d700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f0614138d30 legacy=0x7f061413b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:17.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.556+0000 7f0613fff700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f061409faa0 legacy=0x7f061409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:17.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.556+0000 7f0618f4c700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f06140a6880 legacy=0x7f0614133070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:17.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.556+0000 7f061974d700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f0614138d30 legacy=0x7f061413b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:17.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.960+0000 7f0618f4c700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f06140a6880 legacy=0x7f0614133070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:17.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.960+0000 7f0613fff700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f061409faa0 legacy=0x7f061409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:17.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:17.960+0000 7f061974d700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f0614138d30 legacy=0x7f061413b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1436: done (2 left) 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1437: delete oid 1 current snap is 159 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1434: finishing write tid 2 to smithi12223906-40 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1434: finishing write tid 3 to smithi12223906-40 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1434: finishing write tid 4 to smithi12223906-40 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1434: finishing write tid 5 to smithi12223906-40 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 594 (ObjNum 527 snap 159 seq_num 527) dirty exists 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1434: left oid 40 (ObjNum 527 snap 159 seq_num 527) 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1435: finishing write tid 1 to smithi12223906-11 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1435: finishing write tid 2 to smithi12223906-11 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1435: finishing write tid 3 to smithi12223906-11 2024-09-15T03:52:18.046 INFO:tasks.rados.rados.0.smithi122.stdout:1435: finishing write tid 4 to smithi12223906-11 2024-09-15T03:52:18.047 INFO:tasks.rados.rados.0.smithi122.stdout:1435: finishing write tid 5 to smithi12223906-11 2024-09-15T03:52:18.047 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 574 (ObjNum 528 snap 159 seq_num 528) dirty exists 2024-09-15T03:52:18.047 INFO:tasks.rados.rados.0.smithi122.stdout:1435: left oid 11 (ObjNum 528 snap 159 seq_num 528) 2024-09-15T03:52:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:1434: done (2 left) 2024-09-15T03:52:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:1435: done (1 left) 2024-09-15T03:52:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:1437: done (0 left) 2024-09-15T03:52:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:1438: snap_create 2024-09-15T03:52:18.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:18.764+0000 7f0613fff700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f061409faa0 legacy=0x7f061409fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:18.764+0000 7f0618f4c700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f06140a6880 legacy=0x7f0614133070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:18.764+0000 7f061974d700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f0614138d30 legacy=0x7f061413b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:19.047 INFO:tasks.rados.rados.0.smithi122.stdout:1438: done (0 left) 2024-09-15T03:52:19.047 INFO:tasks.rados.rados.0.smithi122.stdout:1439: write (excl) oid 35 current snap is 160 2024-09-15T03:52:19.048 INFO:tasks.rados.rados.0.smithi122.stdout:1439: seq_num 530 ranges {507710=710327,1897927=412519,2854892=52818} 2024-09-15T03:52:19.056 INFO:tasks.rados.rados.0.smithi122.stdout:1439: writing smithi12223906-35 from 507710 to 1218037 tid 1 2024-09-15T03:52:19.062 INFO:tasks.rados.rados.0.smithi122.stdout:1439: writing smithi12223906-35 from 1897927 to 2310446 tid 2 2024-09-15T03:52:19.066 INFO:tasks.rados.rados.0.smithi122.stdout:1439: writing smithi12223906-35 from 2854892 to 2907710 tid 3 2024-09-15T03:52:19.066 INFO:tasks.rados.rados.0.smithi122.stdout:1440: delete oid 18 current snap is 160 2024-09-15T03:52:19.066 INFO:tasks.rados.rados.0.smithi122.stdout:1439: finishing write tid 1 to smithi12223906-35 2024-09-15T03:52:19.069 INFO:tasks.rados.rados.0.smithi122.stdout:1439: finishing write tid 2 to smithi12223906-35 2024-09-15T03:52:19.069 INFO:tasks.rados.rados.0.smithi122.stdout:1439: finishing write tid 3 to smithi12223906-35 2024-09-15T03:52:19.070 INFO:tasks.rados.rados.0.smithi122.stdout:1439: finishing write tid 4 to smithi12223906-35 2024-09-15T03:52:19.071 INFO:tasks.rados.rados.0.smithi122.stdout:1439: finishing write tid 5 to smithi12223906-35 2024-09-15T03:52:19.071 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 567 (ObjNum 530 snap 160 seq_num 530) dirty exists 2024-09-15T03:52:19.071 INFO:tasks.rados.rados.0.smithi122.stdout:1439: left oid 35 (ObjNum 530 snap 160 seq_num 530) 2024-09-15T03:52:19.071 INFO:tasks.rados.rados.0.smithi122.stdout:1439: done (1 left) 2024-09-15T03:52:19.072 INFO:tasks.rados.rados.0.smithi122.stdout:1440: done (0 left) 2024-09-15T03:52:19.072 INFO:tasks.rados.rados.0.smithi122.stdout:1441: rollback oid 46 current snap is 160 2024-09-15T03:52:19.072 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 46 to 157 2024-09-15T03:52:19.072 INFO:tasks.rados.rados.0.smithi122.stdout:1442: write (excl) oid 43 current snap is 160 2024-09-15T03:52:19.072 INFO:tasks.rados.rados.0.smithi122.stdout:1442: seq_num 532 ranges {541324=460825,1498538=416188,2393385=547939} 2024-09-15T03:52:19.077 INFO:tasks.rados.rados.0.smithi122.stdout:1442: writing smithi12223906-43 from 541324 to 1002149 tid 1 2024-09-15T03:52:19.084 INFO:tasks.rados.rados.0.smithi122.stdout:1442: writing smithi12223906-43 from 1498538 to 1914726 tid 2 2024-09-15T03:52:19.090 INFO:tasks.rados.rados.0.smithi122.stdout:1442: writing smithi12223906-43 from 2393385 to 2941324 tid 3 2024-09-15T03:52:19.090 INFO:tasks.rados.rados.0.smithi122.stdout:1443: delete oid 41 current snap is 160 2024-09-15T03:52:19.090 INFO:tasks.rados.rados.0.smithi122.stdout:1441: finishing rollback tid 1 to smithi12223906-46 2024-09-15T03:52:19.090 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 393 (ObjNum 705 snap 0 seq_num 1842175872) dirty dne 2024-09-15T03:52:19.090 INFO:tasks.rados.rados.0.smithi122.stdout:1442: finishing write tid 1 to smithi12223906-43 2024-09-15T03:52:19.090 INFO:tasks.rados.rados.0.smithi122.stdout:1442: finishing write tid 2 to smithi12223906-43 2024-09-15T03:52:19.095 INFO:tasks.rados.rados.0.smithi122.stdout:1442: finishing write tid 3 to smithi12223906-43 2024-09-15T03:52:19.096 INFO:tasks.rados.rados.0.smithi122.stdout:1442: finishing write tid 4 to smithi12223906-43 2024-09-15T03:52:19.096 INFO:tasks.rados.rados.0.smithi122.stdout:1442: finishing write tid 5 to smithi12223906-43 2024-09-15T03:52:19.096 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 694 (ObjNum 532 snap 160 seq_num 532) dirty exists 2024-09-15T03:52:19.096 INFO:tasks.rados.rados.0.smithi122.stdout:1442: left oid 43 (ObjNum 532 snap 160 seq_num 532) 2024-09-15T03:52:19.097 INFO:tasks.rados.rados.0.smithi122.stdout:1441: done (2 left) 2024-09-15T03:52:19.097 INFO:tasks.rados.rados.0.smithi122.stdout:1442: done (1 left) 2024-09-15T03:52:19.097 INFO:tasks.rados.rados.0.smithi122.stdout:1443: done (0 left) 2024-09-15T03:52:19.097 INFO:tasks.rados.rados.0.smithi122.stdout:1444: write (excl) oid 8 current snap is 160 2024-09-15T03:52:19.097 INFO:tasks.rados.rados.0.smithi122.stdout:1444: seq_num 534 ranges {574938=727676,1731855=703504,2848231=126707} 2024-09-15T03:52:19.110 INFO:tasks.rados.rados.0.smithi122.stdout:1444: writing smithi12223906-8 from 574938 to 1302614 tid 1 2024-09-15T03:52:19.116 INFO:tasks.rados.rados.0.smithi122.stdout:1444: writing smithi12223906-8 from 1731855 to 2435359 tid 2 2024-09-15T03:52:19.120 INFO:tasks.rados.rados.0.smithi122.stdout:1444: writing smithi12223906-8 from 2848231 to 2974938 tid 3 2024-09-15T03:52:19.120 INFO:tasks.rados.rados.0.smithi122.stdout:1445: write (excl) oid 49 current snap is 160 2024-09-15T03:52:19.120 INFO:tasks.rados.rados.0.smithi122.stdout:1445: seq_num 535 ranges {591745=402925,1790337=563515} 2024-09-15T03:52:19.126 INFO:tasks.rados.rados.0.smithi122.stdout:1445: writing smithi12223906-49 from 591745 to 994670 tid 1 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1445: writing smithi12223906-49 from 1790337 to 2353852 tid 2 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1446: copy_from oid 42 from oid 10 current snap is 160 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1444: finishing write tid 1 to smithi12223906-8 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1444: finishing write tid 2 to smithi12223906-8 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1444: finishing write tid 3 to smithi12223906-8 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1444: finishing write tid 4 to smithi12223906-8 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1444: finishing write tid 5 to smithi12223906-8 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 403 (ObjNum 534 snap 160 seq_num 534) dirty exists 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1444: left oid 8 (ObjNum 534 snap 160 seq_num 534) 2024-09-15T03:52:19.135 INFO:tasks.rados.rados.0.smithi122.stdout:1445: finishing write tid 1 to smithi12223906-49 2024-09-15T03:52:19.136 INFO:tasks.rados.rados.0.smithi122.stdout:1444: done (2 left) 2024-09-15T03:52:19.136 INFO:tasks.rados.rados.0.smithi122.stdout:1447: delete oid 28 current snap is 160 2024-09-15T03:52:19.137 INFO:tasks.rados.rados.0.smithi122.stdout:1447: done (2 left) 2024-09-15T03:52:19.137 INFO:tasks.rados.rados.0.smithi122.stdout:1448: snap_remove snap 158 2024-09-15T03:52:20.073 INFO:tasks.rados.rados.0.smithi122.stdout:1448: done (2 left) 2024-09-15T03:52:20.073 INFO:tasks.rados.rados.0.smithi122.stdout:1449: write oid 47 current snap is 160 2024-09-15T03:52:20.073 INFO:tasks.rados.rados.0.smithi122.stdout:1445: finishing write tid 2 to smithi12223906-49 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:1445: finishing write tid 3 to smithi12223906-49 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:1445: finishing write tid 4 to smithi12223906-49 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 387 (ObjNum 535 snap 160 seq_num 535) dirty exists 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:1445: left oid 49 (ObjNum 535 snap 160 seq_num 535) 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:1446: finishing copy_from racing read to smithi12223906-42 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:1446: finishing copy_from to smithi12223906-42 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 394 (ObjNum 493 snap 149 seq_num 493) dirty exists 2024-09-15T03:52:20.074 INFO:tasks.rados.rados.0.smithi122.stdout:1449: seq_num 537 ranges {625359=553423,1790948=567184,2930500=94859} 2024-09-15T03:52:20.081 INFO:tasks.rados.rados.0.smithi122.stdout:1449: writing smithi12223906-47 from 625359 to 1178782 tid 1 2024-09-15T03:52:20.087 INFO:tasks.rados.rados.0.smithi122.stdout:1449: writing smithi12223906-47 from 1790948 to 2358132 tid 2 2024-09-15T03:52:20.091 INFO:tasks.rados.rados.0.smithi122.stdout:1449: writing smithi12223906-47 from 2930500 to 3025359 tid 3 2024-09-15T03:52:20.092 INFO:tasks.rados.rados.0.smithi122.stdout:1445: done (2 left) 2024-09-15T03:52:20.092 INFO:tasks.rados.rados.0.smithi122.stdout:1446: done (1 left) 2024-09-15T03:52:20.092 INFO:tasks.rados.rados.0.smithi122.stdout:1450: write oid 35 current snap is 160 2024-09-15T03:52:20.092 INFO:tasks.rados.rados.0.smithi122.stdout:1450: seq_num 538 ranges {642166=628672,1849430=710842} 2024-09-15T03:52:20.098 INFO:tasks.rados.rados.0.smithi122.stdout:1450: writing smithi12223906-35 from 642166 to 1270838 tid 1 2024-09-15T03:52:20.106 INFO:tasks.rados.rados.0.smithi122.stdout:1450: writing smithi12223906-35 from 1849430 to 2560272 tid 2 2024-09-15T03:52:20.106 INFO:tasks.rados.rados.0.smithi122.stdout:1451: read oid 10 snap -1 2024-09-15T03:52:20.106 INFO:tasks.rados.rados.0.smithi122.stdout:1451: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:20.106 INFO:tasks.rados.rados.0.smithi122.stdout:1452: read oid 46 snap -1 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1452: expect deleted 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1453: read oid 21 snap -1 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1453: expect (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1454: read oid 27 snap -1 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1454: expect deleted 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1455: copy_from oid 44 from oid 2 current snap is 160 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1449: finishing write tid 1 to smithi12223906-47 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1449: finishing write tid 2 to smithi12223906-47 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1449: finishing write tid 3 to smithi12223906-47 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1449: finishing write tid 4 to smithi12223906-47 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1449: finishing write tid 5 to smithi12223906-47 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 215 (ObjNum 537 snap 160 seq_num 537) dirty exists 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1449: left oid 47 (ObjNum 537 snap 160 seq_num 537) 2024-09-15T03:52:20.107 INFO:tasks.rados.rados.0.smithi122.stdout:1450: finishing write tid 1 to smithi12223906-35 2024-09-15T03:52:20.108 INFO:tasks.rados.rados.0.smithi122.stdout:1449: done (6 left) 2024-09-15T03:52:20.108 INFO:tasks.rados.rados.0.smithi122.stdout:1456: delete oid 39 current snap is 160 2024-09-15T03:52:20.109 INFO:tasks.rados.rados.0.smithi122.stdout:1452: done (6 left) 2024-09-15T03:52:20.109 INFO:tasks.rados.rados.0.smithi122.stdout:1454: done (5 left) 2024-09-15T03:52:20.109 INFO:tasks.rados.rados.0.smithi122.stdout:1456: done (4 left) 2024-09-15T03:52:20.109 INFO:tasks.rados.rados.0.smithi122.stdout:1457: read oid 45 snap -1 2024-09-15T03:52:20.109 INFO:tasks.rados.rados.0.smithi122.stdout:1457: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:52:20.109 INFO:tasks.rados.rados.0.smithi122.stdout:1458: copy_from oid 47 from oid 48 current snap is 160 2024-09-15T03:52:20.110 INFO:tasks.rados.rados.0.smithi122.stdout:1459: delete oid 39 current snap is 160 2024-09-15T03:52:20.112 INFO:tasks.rados.rados.0.smithi122.stdout:1451: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:20.293 INFO:tasks.rados.rados.0.smithi122.stdout:1453: expect (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:52:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient(hunting): tick 2024-09-15T03:52:20.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 monclient(hunting): continuing hunt 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f06140a6880 legacy=0x7f0614133070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f0614138d30 legacy=0x7f061413b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f061409faa0 legacy=0x7f061409fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient: _add_conns ranks=[7,2,5,4,8,1] 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient(hunting): picked mon.noname-i con 0x7f05f40037d0 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:20.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient(hunting): picked mon.noname-b con 0x7f05f4007450 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient(hunting): picked mon.noname-h con 0x7f05f400b1d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06140a6c50 con 0x7f05f4007450 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061404ca10 con 0x7f05f400b1d0 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0614133780 con 0x7f05f40037d0 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06127fc700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0613fff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f05f4007450 0x7f05f4009900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48838/0 (socket says 172.21.15.26:48838) 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0613fff700 1 -- 172.21.15.26:0/4281848409 learned_addr learned my addr 172.21.15.26:0/4281848409 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:20.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724596862 0 0) 0x7f06140a6c50 con 0x7f05f4007450 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): my global_id is 134777 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0600003570 con 0x7f05f4007450 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836438305 0 0) 0x7f061404ca10 con 0x7f05f400b1d0 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): my global_id is 5315 2024-09-15T03:52:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06140a6c50 con 0x7f05f400b1d0 2024-09-15T03:52:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001601904 0 0) 0x7f0614133780 con 0x7f05f40037d0 2024-09-15T03:52:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient(hunting): my global_id is 134683 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061404ca10 con 0x7f05f40037d0 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 418339528 0 0) 0x7f0600003570 con 0x7f05f4007450 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 >> v1:172.21.15.122:6792/0 conn(0x7f05f40037d0 legacy=0x7f05f4005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 >> v1:172.21.15.122:6791/0 conn(0x7f05f400b1d0 legacy=0x7f05f400d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: _finish_hunting 0 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 monclient: found mon.noname-b 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f061974d700 1 -- 172.21.15.26:0/4281848409 reap_dead start 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4011fc0 con 0x7f05f4007450 2024-09-15T03:52:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: _finish_auth 0 2024-09-15T03:52:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 5 monclient: authenticate success, global_id 134777 2024-09-15T03:52:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05fc004230 con 0x7f05f4007450 2024-09-15T03:52:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:20.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:20.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: dump: 2024-09-15T03:52:20.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.370 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.370 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.370 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.370 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.370 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.370 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.371 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.372 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.372 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:20.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f05fc0044d0 con 0x7f05f4007450 2024-09-15T03:52:20.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:20.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 1 -- 172.21.15.26:0/4281848409 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05fc004a00 con 0x7f05f4007450 2024-09-15T03:52:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 4 set_mon_vals no callback set 2024-09-15T03:52:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f06137fe700 10 monclient: dump: 2024-09-15T03:52:20.373 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.374 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.375 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.376 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.376 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 10 monclient: shutdown 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 1 -- 172.21.15.26:0/4281848409 >> v1:172.21.15.26:6790/0 conn(0x7f05f4007450 legacy=0x7f05f4009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 1 -- 172.21.15.26:0/4281848409 shutdown_connections 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 1 -- 172.21.15.26:0/4281848409 >> 172.21.15.26:0/4281848409 conn(0x7f061406e2a0 msgr2=0x7f06140a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 1 -- 172.21.15.26:0/4281848409 shutdown_connections 2024-09-15T03:52:20.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 1 -- 172.21.15.26:0/4281848409 wait complete. 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 10 monclient: build_initial_monmap 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.356+0000 7f0619f4e700 10 monclient: monmap: 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:20.391 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 Processor -- start 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 -- start start 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: init 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: _add_conns ranks=[2,0,1,6,4,7,3,8,5] 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient(hunting): picked mon.noname-b con 0x7f05f4007450 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:20.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient(hunting): picked mon.noname-a con 0x7f061412b810 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient(hunting): picked mon.noname-f con 0x7f061412c310 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06140a18c0 con 0x7f061412b810 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061409cf10 con 0x7f061412c310 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061409b3d0 con 0x7f05f4007450 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0613fff700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f061412b810 legacy=0x7f061412bc00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient(hunting): authenticate will time out at 1284.793788s 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0618f4c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f05f4007450 0x7f061409c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48854/0 (socket says 172.21.15.26:48854) 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0618f4c700 1 -- 172.21.15.26:0/1909356701 learned_addr learned my addr 172.21.15.26:0/1909356701 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60728123 0 0) 0x7f061409b3d0 con 0x7f05f4007450 2024-09-15T03:52:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): my global_id is 134786 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f4009130 con 0x7f05f4007450 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568853384 0 0) 0x7f061409cf10 con 0x7f061412c310 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): my global_id is 184267 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061409b3d0 con 0x7f061412c310 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1317526498 0 0) 0x7f05f4009130 con 0x7f05f4007450 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f061409cf10 con 0x7f05f4007450 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f06040047c0 con 0x7f05f4007450 2024-09-15T03:52:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.395 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4035805806 0 0) 0x7f061409b3d0 con 0x7f061412c310 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05f4009130 con 0x7f061412c310 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f060c004370 con 0x7f061412c310 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient(hunting): dump: 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2779538437 0 0) 0x7f061409cf10 con 0x7f05f4007450 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.122:6789/0 conn(0x7f061412c310 legacy=0x7f0614149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.26:6789/0 conn(0x7f061412b810 legacy=0x7f061412bc00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 monclient: found mon.b 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061409a4e0 con 0x7f05f4007450 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 10 monclient: _finish_auth 0 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0610ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 5 monclient: authenticate success, global_id 134786 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: _renew_subs 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061414b2b0 con 0x7f05f4007450 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: _renew_subs 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.360+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f061414b5e0 con 0x7f05f4007450 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0604004af0 con 0x7f05f4007450 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0604005020 con 0x7f05f4007450 2024-09-15T03:52:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0612ffd700 4 set_mon_vals no callback set 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: dump: 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0604005200 con 0x7f05f4007450 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0619f4e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f05f8006590 con 0x7f05f8002840 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(338..338 src has 224..338) v4 ==== 3971+0+0 (unknown 3565202667 0 0) 0x7f0604044390 con 0x7f05f4007450 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029815736 0 0) 0x7f05f8006590 con 0x7f05f8002840 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient(hunting): my global_id is 134795 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061409cf10 con 0x7f05f8002840 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f061974d700 10 monclient: get_auth_request con 0x7f05f4038c50 auth_method 0 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4134711355 0 0) 0x7f061409cf10 con 0x7f05f8002840 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05f8006590 con 0x7f05f8002840 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05fc004230 con 0x7f05f8002840 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 10 monclient: dump: 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.402 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.403 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2677124312 0 0) 0x7f05f8006590 con 0x7f05f8002840 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.364+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05f800a130 con 0x7f05f8002840 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.368+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f05fc004920 con 0x7f05f8002840 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.368+0000 7f0610ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.368+0000 7f0610ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.368+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.26:6790/0 conn(0x7f05f8002840 legacy=0x7f05f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.368+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.26:6790/0 conn(0x7f05f8002840 legacy=0x7f05f8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0619f4e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f05f8009640 con 0x7f05f80090b0 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110925037 0 0) 0x7f05f8009640 con 0x7f05f80090b0 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient(hunting): my global_id is 134804 2024-09-15T03:52:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f8006590 con 0x7f05f80090b0 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1365804718 0 0) 0x7f05f8006590 con 0x7f05f80090b0 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05f8009640 con 0x7f05f80090b0 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05fc005530 con 0x7f05f80090b0 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient: dump: 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2818120080 0 0) 0x7f05f8009640 con 0x7f05f80090b0 2024-09-15T03:52:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f05f80098f0 con 0x7f05f80090b0 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 2467315320) 0x7f05fc007770 con 0x7f05f80090b0 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.26:6790/0 conn(0x7f05f80090b0 legacy=0x7f05f8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.380+0000 7f0610ff9700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.26:6790/0 conn(0x7f05f80090b0 legacy=0x7f05f8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:20.407 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 60, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:20.408 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.409 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.410 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:20.411 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:20.412 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.414 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.417 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:20.419 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.420 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:20.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.423 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.424 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.425 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.427 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:20.428 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:20.429 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:20.430 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 5 2024-09-15T03:52:20.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.430 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:20.430 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.122:6812/20895 conn(0x7f05f4038c50 legacy=0x7f05f403b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 10 monclient: shutdown 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 >> v1:172.21.15.26:6790/0 conn(0x7f05f4007450 legacy=0x7f061409c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f061974d700 1 -- 172.21.15.26:0/1909356701 reap_dead start 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f061974d700 1 -- 172.21.15.26:0/1909356701 reap_dead start 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 shutdown_connections 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 >> 172.21.15.26:0/1909356701 conn(0x7f061406e2a0 msgr2=0x7f061406f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 shutdown_connections 2024-09-15T03:52:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.384+0000 7f0619f4e700 1 -- 172.21.15.26:0/1909356701 wait complete. 2024-09-15T03:52:20.432 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:52:20.433 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-09-15T03:52:20.433 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:52:20.441 INFO:tasks.rados.rados.0.smithi122.stdout:1450: finishing write tid 2 to smithi12223906-35 2024-09-15T03:52:20.441 INFO:tasks.rados.rados.0.smithi122.stdout:1450: finishing write tid 3 to smithi12223906-35 2024-09-15T03:52:20.441 INFO:tasks.rados.rados.0.smithi122.stdout:1450: finishing write tid 4 to smithi12223906-35 2024-09-15T03:52:20.441 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 570 (ObjNum 538 snap 160 seq_num 538) dirty exists 2024-09-15T03:52:20.441 INFO:tasks.rados.rados.0.smithi122.stdout:1450: left oid 35 (ObjNum 538 snap 160 seq_num 538) 2024-09-15T03:52:20.441 INFO:tasks.rados.rados.0.smithi122.stdout:1457: expect (ObjNum 394 snap 116 seq_num 394) 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1455: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1455: finishing copy_from to smithi12223906-44 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 373 (ObjNum 507 snap 151 seq_num 507) dirty exists 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1458: finishing copy_from racing read to smithi12223906-47 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1458: finishing copy_from to smithi12223906-47 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 375 (ObjNum 427 snap 127 seq_num 427) dirty exists 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1450: done (6 left) 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1451: done (5 left) 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1453: done (4 left) 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1455: done (3 left) 2024-09-15T03:52:20.545 INFO:tasks.rados.rados.0.smithi122.stdout:1457: done (2 left) 2024-09-15T03:52:20.546 INFO:tasks.rados.rados.0.smithi122.stdout:1458: done (1 left) 2024-09-15T03:52:20.546 INFO:tasks.rados.rados.0.smithi122.stdout:1459: done (0 left) 2024-09-15T03:52:20.546 INFO:tasks.rados.rados.0.smithi122.stdout:1460: snap_remove snap 152 2024-09-15T03:52:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient: get_monmap_and_config 2024-09-15T03:52:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient: build_initial_monmap 2024-09-15T03:52:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient: monmap: 2024-09-15T03:52:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:20.549505+0000 2024-09-15T03:52:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:20.549505+0000 2024-09-15T03:52:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:20.552 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:20.553 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:20.553 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:20.553 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:20.553 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:20.554 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:20.554 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:20.554 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:20.554 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 1 Processor -- start 2024-09-15T03:52:20.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 1 -- start start 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient: init 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient: _add_conns ranks=[6,5,4,0,7,3,8,2,1] 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient(hunting): picked mon.noname-d con 0x7f12980a48e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient(hunting): picked mon.noname-h con 0x7f129809aca0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient(hunting): picked mon.noname-c con 0x7f129809a5e0 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129804ca10 con 0x7f129809a5e0 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12980696a0 con 0x7f129809aca0 2024-09-15T03:52:20.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1298068d20 con 0x7f12980a48e0 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129759e700 10 monclient(hunting): authenticate will time out at 1284.979250s 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f129659c700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f12980a48e0 legacy=0x7f1298099ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f1296d9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f129809a5e0 0x7f129813b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49042/0 (socket says 172.21.15.26:49042) 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.544+0000 7f1296d9d700 1 -- 172.21.15.26:0/684732765 learned_addr learned my addr 172.21.15.26:0/684732765 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217619758 0 0) 0x7f129804ca10 con 0x7f129809a5e0 2024-09-15T03:52:20.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient(hunting): my global_id is 184306 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1284003570 con 0x7f129809a5e0 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114155570 0 0) 0x7f12980696a0 con 0x7f129809aca0 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient(hunting): my global_id is 5324 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f129804ca10 con 0x7f129809aca0 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2923198083 0 0) 0x7f1284003570 con 0x7f129809a5e0 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 >> v1:172.21.15.26:6792/0 conn(0x7f12980a48e0 legacy=0x7f1298099ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 >> v1:172.21.15.122:6791/0 conn(0x7f129809aca0 legacy=0x7f1298137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 monclient: found mon.noname-c 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:52:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298067640 con 0x7f129809a5e0 2024-09-15T03:52:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: _finish_auth 0 2024-09-15T03:52:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 5 monclient: authenticate success, global_id 184306 2024-09-15T03:52:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1290001b30 con 0x7f129809a5e0 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: dump: 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.576 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1290001da0 con 0x7f129809a5e0 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 1 -- 172.21.15.26:0/684732765 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1290004fb0 con 0x7f129809a5e0 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 4 set_mon_vals no callback set 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f1295d9b700 10 monclient: dump: 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.580 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.580 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.580 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: shutdown 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- 172.21.15.26:0/684732765 >> v1:172.21.15.26:6791/0 conn(0x7f129809a5e0 legacy=0x7f129813b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- 172.21.15.26:0/684732765 shutdown_connections 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- 172.21.15.26:0/684732765 >> 172.21.15.26:0/684732765 conn(0x7f12980638b0 msgr2=0x7f12980a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- 172.21.15.26:0/684732765 shutdown_connections 2024-09-15T03:52:20.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- 172.21.15.26:0/684732765 wait complete. 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: build_initial_monmap 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: monmap: 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:20.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 Processor -- start 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- start start 2024-09-15T03:52:20.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: init 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient: _add_conns ranks=[1,5,2,4,6,8,7,3,0] 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient(hunting): picked mon.noname-f con 0x7f129809a5e0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient(hunting): picked mon.noname-h con 0x7f129809aca0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient(hunting): picked mon.noname-b con 0x7f12980a48e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1298123230 con 0x7f129809a5e0 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129813e000 con 0x7f12980a48e0 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1298122c70 con 0x7f129809aca0 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:20.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.548+0000 7f129759e700 10 monclient(hunting): authenticate will time out at 1284.983934s 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f1296d9d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f12980a48e0 0x7f129814a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48902/0 (socket says 172.21.15.26:48902) 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f1296d9d700 1 -- 172.21.15.26:0/3730811235 learned_addr learned my addr 172.21.15.26:0/3730811235 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085646768 0 0) 0x7f129813e000 con 0x7f12980a48e0 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): my global_id is 134813 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1278003570 con 0x7f12980a48e0 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991561275 0 0) 0x7f1298122c70 con 0x7f129809aca0 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): my global_id is 5333 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f129813e000 con 0x7f129809aca0 2024-09-15T03:52:20.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361830737 0 0) 0x7f1298123230 con 0x7f129809a5e0 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): my global_id is 184276 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1298122c70 con 0x7f129809a5e0 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 362664332 0 0) 0x7f1278003570 con 0x7f12980a48e0 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1298123230 con 0x7f12980a48e0 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1290001bb0 con 0x7f12980a48e0 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): dump: 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.587 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2004168831 0 0) 0x7f129813e000 con 0x7f129809aca0 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1278003570 con 0x7f129809aca0 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1726258491 0 0) 0x7f1298122c70 con 0x7f129809a5e0 2024-09-15T03:52:20.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f129813e000 con 0x7f129809a5e0 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1280001bb0 con 0x7f129809aca0 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): dump: 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.589 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2872503609 0 0) 0x7f1298123230 con 0x7f12980a48e0 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.122:6791/0 conn(0x7f129809aca0 legacy=0x7f1298146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.122:6789/0 conn(0x7f129809a5e0 legacy=0x7f129813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: _finish_hunting 0 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 monclient: found mon.b 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298122990 con 0x7f12980a48e0 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: _finish_auth 0 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 5 monclient: authenticate success, global_id 134813 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 10 monclient: _renew_subs 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1298122f50 con 0x7f12980a48e0 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 10 monclient: _renew_subs 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1298121ac0 con 0x7f12980a48e0 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1290005280 con 0x7f12980a48e0 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1290005730 con 0x7f12980a48e0 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129559a700 4 set_mon_vals no callback set 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient: dump: 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f12900100f0 con 0x7f12980a48e0 2024-09-15T03:52:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(338..338 src has 224..338) v4 ==== 3971+0+0 (unknown 3565202667 0 0) 0x7f12900445c0 con 0x7f12980a48e0 2024-09-15T03:52:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f127c006590 con 0x7f127c002840 2024-09-15T03:52:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f129659c700 10 monclient: get_auth_request con 0x7f1278037f90 auth_method 0 2024-09-15T03:52:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426473574 0 0) 0x7f127c006590 con 0x7f127c002840 2024-09-15T03:52:20.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 10 monclient(hunting): my global_id is 184315 2024-09-15T03:52:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.552+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1298123230 con 0x7f127c002840 2024-09-15T03:52:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4195339921 0 0) 0x7f1298123230 con 0x7f127c002840 2024-09-15T03:52:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f127c006590 con 0x7f127c002840 2024-09-15T03:52:20.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1280002380 con 0x7f127c002840 2024-09-15T03:52:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 10 monclient: dump: 2024-09-15T03:52:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.597 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1965817847 0 0) 0x7f127c006590 con 0x7f127c002840 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f127c00a130 con 0x7f127c002840 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1280002830 con 0x7f127c002840 2024-09-15T03:52:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.26:6791/0 conn(0x7f127c002840 legacy=0x7f127c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.556+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.26:6791/0 conn(0x7f127c002840 legacy=0x7f127c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f129759e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f127c009640 con 0x7f127c0090b0 2024-09-15T03:52:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450311740 0 0) 0x7f127c009640 con 0x7f127c0090b0 2024-09-15T03:52:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 10 monclient(hunting): my global_id is 184324 2024-09-15T03:52:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f127c006590 con 0x7f127c0090b0 2024-09-15T03:52:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2864288356 0 0) 0x7f127c006590 con 0x7f127c0090b0 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f127c009640 con 0x7f127c0090b0 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1280002cb0 con 0x7f127c0090b0 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 10 monclient: dump: 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.564+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1624325205 0 0) 0x7f127c009640 con 0x7f127c0090b0 2024-09-15T03:52:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f127c0098f0 con 0x7f127c0090b0 2024-09-15T03:52:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 407660994) 0x7f1280003480 con 0x7f127c0090b0 2024-09-15T03:52:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f128effd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f128effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.26:6791/0 conn(0x7f127c0090b0 legacy=0x7f127c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f128effd700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.26:6791/0 conn(0x7f127c0090b0 legacy=0x7f127c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:20.608 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 60, 2024-09-15T03:52:20.608 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:20.609 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:20.610 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:20.610 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.610 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:52:20.610 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:20.610 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:20.611 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:20.611 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:20.611 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:20.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:20.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:20.612 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:20.612 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:20.612 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:20.612 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:20.612 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:20.612 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:20.613 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.615 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:20.615 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:20.615 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:20.615 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:20.615 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:20.616 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.617 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.618 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:20.619 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.620 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.621 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.625 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.626 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.627 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:20.630 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:20.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.634 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:20.635 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:20.636 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:20.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:20.636 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:20.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:20.636 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:20.636 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.122:6812/20895 conn(0x7f1278037f90 legacy=0x7f127803a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 10 monclient: shutdown 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 >> v1:172.21.15.26:6790/0 conn(0x7f12980a48e0 legacy=0x7f129814a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f1296d9d700 1 -- 172.21.15.26:0/3730811235 reap_dead start 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f1296d9d700 1 -- 172.21.15.26:0/3730811235 reap_dead start 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 shutdown_connections 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 >> 172.21.15.26:0/3730811235 conn(0x7f12980638b0 msgr2=0x7f129806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 shutdown_connections 2024-09-15T03:52:20.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.568+0000 7f129759e700 1 -- 172.21.15.26:0/3730811235 wait complete. 2024-09-15T03:52:20.639 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:52:20.639 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon e 2024-09-15T03:52:20.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.748+0000 7f6cd3215700 10 monclient: get_monmap_and_config 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.748+0000 7f6cd3215700 10 monclient: build_initial_monmap 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.748+0000 7f6cd3215700 10 monclient: monmap: 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:20.754479+0000 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:20.754479+0000 2024-09-15T03:52:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:20.757 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.748+0000 7f6cd3215700 1 Processor -- start 2024-09-15T03:52:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.748+0000 7f6cd3215700 1 -- start start 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.748+0000 7f6cd3215700 10 monclient: init 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient: _add_conns ranks=[5,3,7,4,6,2,8,0,1] 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient(hunting): picked mon.noname-h con 0x7f6ccc132b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient(hunting): picked mon.noname-g con 0x7f6ccc09faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient(hunting): picked mon.noname-i con 0x7f6ccc138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc04ca10 con 0x7f6ccc09faa0 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc09d5d0 con 0x7f6ccc132b40 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc133620 con 0x7f6ccc138d40 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient(hunting): authenticate will time out at 1285.184529s 2024-09-15T03:52:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1a12700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc09fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd2a14700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f6ccc138d40 0x7f6ccc13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:41354/0 (socket says 172.21.15.26:41354) 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd2213700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f6ccc132b40 0x7f6ccc132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:39026/0 (socket says 172.21.15.26:39026) 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd2a14700 1 -- 172.21.15.26:0/4005268413 learned_addr learned my addr 172.21.15.26:0/4005268413 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058029569 0 0) 0x7f6ccc133620 con 0x7f6ccc138d40 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient(hunting): my global_id is 134692 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb8003570 con 0x7f6ccc138d40 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398133505 0 0) 0x7f6ccc09d5d0 con 0x7f6ccc132b40 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient(hunting): my global_id is 5342 2024-09-15T03:52:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ccc133620 con 0x7f6ccc132b40 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3713006214 0 0) 0x7f6cb8003570 con 0x7f6ccc138d40 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 >> v1:172.21.15.122:6791/0 conn(0x7f6ccc132b40 legacy=0x7f6ccc132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 >> v1:172.21.15.122:6790/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: _finish_hunting 0 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 monclient: found mon.noname-i 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0a0cc0 con 0x7f6ccc138d40 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: _finish_auth 0 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc4001b30 con 0x7f6ccc138d40 2024-09-15T03:52:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 5 monclient: authenticate success, global_id 134692 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: dump: 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.765 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6cc4001d50 con 0x7f6ccc138d40 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 4 set_mon_vals no callback set 2024-09-15T03:52:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 1 -- 172.21.15.26:0/4005268413 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc4004c70 con 0x7f6ccc138d40 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd1211700 10 monclient: dump: 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient: shutdown 2024-09-15T03:52:20.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- 172.21.15.26:0/4005268413 >> v1:172.21.15.122:6792/0 conn(0x7f6ccc138d40 legacy=0x7f6ccc13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- 172.21.15.26:0/4005268413 shutdown_connections 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- 172.21.15.26:0/4005268413 >> 172.21.15.26:0/4005268413 conn(0x7f6ccc06e370 msgr2=0x7f6ccc0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- 172.21.15.26:0/4005268413 shutdown_connections 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 1 -- 172.21.15.26:0/4005268413 wait complete. 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient: build_initial_monmap 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.752+0000 7f6cd3215700 10 monclient: monmap: 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:20.769 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 1 Processor -- start 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 1 -- start start 2024-09-15T03:52:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient: init 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient: _add_conns ranks=[0,3,6,7,8,2,4,5,1] 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient(hunting): picked mon.noname-a con 0x7f6ccc09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient(hunting): picked mon.noname-g con 0x7f6ccc132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient(hunting): picked mon.noname-d con 0x7f6ccc138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc09cfa0 con 0x7f6ccc09faa0 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc11c6e0 con 0x7f6ccc132b40 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd2213700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc1354d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc09ccc0 con 0x7f6ccc138d40 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd3215700 10 monclient(hunting): authenticate will time out at 1285.189271s 2024-09-15T03:52:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd2a14700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f6ccc138d40 legacy=0x7f6ccc1362f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.756+0000 7f6cd1a12700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f6ccc132b40 legacy=0x7f6ccc135be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.960+0000 7f6cd2213700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc1354d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.960+0000 7f6cd2a14700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f6ccc138d40 legacy=0x7f6ccc1362f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:20.960+0000 7f6cd1a12700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f6ccc132b40 legacy=0x7f6ccc135be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:21.082 INFO:tasks.rados.rados.0.smithi122.stdout:1460: done (0 left) 2024-09-15T03:52:21.082 INFO:tasks.rados.rados.0.smithi122.stdout:1461: write oid 11 current snap is 160 2024-09-15T03:52:21.083 INFO:tasks.rados.rados.0.smithi122.stdout:1461: seq_num 541 ranges {692587=570772,2024876=574522,3040192=52395} 2024-09-15T03:52:21.089 INFO:tasks.rados.rados.0.smithi122.stdout:1461: writing smithi12223906-11 from 692587 to 1263359 tid 1 2024-09-15T03:52:21.097 INFO:tasks.rados.rados.0.smithi122.stdout:1461: writing smithi12223906-11 from 2024876 to 2599398 tid 2 2024-09-15T03:52:21.100 INFO:tasks.rados.rados.0.smithi122.stdout:1461: writing smithi12223906-11 from 3040192 to 3092587 tid 3 2024-09-15T03:52:21.101 INFO:tasks.rados.rados.0.smithi122.stdout:1462: snap_create 2024-09-15T03:52:21.101 INFO:tasks.rados.rados.0.smithi122.stdout:1461: finishing write tid 1 to smithi12223906-11 2024-09-15T03:52:21.103 INFO:tasks.rados.rados.0.smithi122.stdout:1461: finishing write tid 2 to smithi12223906-11 2024-09-15T03:52:21.103 INFO:tasks.rados.rados.0.smithi122.stdout:1461: finishing write tid 3 to smithi12223906-11 2024-09-15T03:52:21.104 INFO:tasks.rados.rados.0.smithi122.stdout:1461: finishing write tid 4 to smithi12223906-11 2024-09-15T03:52:21.104 INFO:tasks.rados.rados.0.smithi122.stdout:1461: finishing write tid 5 to smithi12223906-11 2024-09-15T03:52:21.105 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 579 (ObjNum 541 snap 160 seq_num 541) dirty exists 2024-09-15T03:52:21.105 INFO:tasks.rados.rados.0.smithi122.stdout:1461: left oid 11 (ObjNum 541 snap 160 seq_num 541) 2024-09-15T03:52:21.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:21.364+0000 7f6cd2213700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc1354d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:21.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:21.364+0000 7f6cd2a14700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f6ccc138d40 legacy=0x7f6ccc1362f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:21.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:21.364+0000 7f6cd1a12700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f6ccc132b40 legacy=0x7f6ccc135be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:21.714 INFO:tasks.rados.rados.0.smithi122.stdout:1461: done (1 left) 2024-09-15T03:52:21.715 INFO:tasks.rados.rados.0.smithi122.stdout:1462: done (0 left) 2024-09-15T03:52:21.715 INFO:tasks.rados.rados.0.smithi122.stdout:1463: read oid 17 snap -1 2024-09-15T03:52:21.715 INFO:tasks.rados.rados.0.smithi122.stdout:1463: expect deleted 2024-09-15T03:52:21.715 INFO:tasks.rados.rados.0.smithi122.stdout:1464: delete oid 22 current snap is 161 2024-09-15T03:52:21.729 INFO:tasks.rados.rados.0.smithi122.stdout:1463: done (1 left) 2024-09-15T03:52:21.729 INFO:tasks.rados.rados.0.smithi122.stdout:1464: done (0 left) 2024-09-15T03:52:21.729 INFO:tasks.rados.rados.0.smithi122.stdout:1465: read oid 44 snap -1 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1465: expect (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1466: read oid 50 snap -1 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1466: expect deleted 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1467: copy_from oid 5 from oid 49 current snap is 161 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1468: copy_from oid 9 from oid 8 current snap is 161 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1469: write (excl) oid 14 current snap is 161 2024-09-15T03:52:21.730 INFO:tasks.rados.rados.0.smithi122.stdout:1469: seq_num 543 ranges {726201=721270,2141840=578191} 2024-09-15T03:52:21.739 INFO:tasks.rados.rados.0.smithi122.stdout:1469: writing smithi12223906-14 from 726201 to 1447471 tid 1 2024-09-15T03:52:21.748 INFO:tasks.rados.rados.0.smithi122.stdout:1469: writing smithi12223906-14 from 2141840 to 2720031 tid 2 2024-09-15T03:52:21.748 INFO:tasks.rados.rados.0.smithi122.stdout:1470: snap_remove snap 156 2024-09-15T03:52:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:22.168+0000 7f6cd2213700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc1354d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:22.168+0000 7f6cd2a14700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f6ccc138d40 legacy=0x7f6ccc1362f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:22.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:22.168+0000 7f6cd1a12700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f6ccc132b40 legacy=0x7f6ccc135be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:22.734 INFO:tasks.rados.rados.0.smithi122.stdout:1465: expect (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:52:22.869 INFO:tasks.rados.rados.0.smithi122.stdout:1467: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:52:22.869 INFO:tasks.rados.rados.0.smithi122.stdout:1468: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:52:22.869 INFO:tasks.rados.rados.0.smithi122.stdout:1467: finishing copy_from to smithi12223906-5 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 387 (ObjNum 535 snap 160 seq_num 535) dirty exists 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:1469: finishing write tid 1 to smithi12223906-14 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:1469: finishing write tid 2 to smithi12223906-14 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:1469: finishing write tid 3 to smithi12223906-14 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:1468: finishing copy_from to smithi12223906-9 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 403 (ObjNum 534 snap 160 seq_num 534) dirty exists 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:1469: finishing write tid 4 to smithi12223906-14 2024-09-15T03:52:22.870 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 714 (ObjNum 543 snap 161 seq_num 543) dirty exists 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1469: left oid 14 (ObjNum 543 snap 161 seq_num 543) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1465: done (5 left) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1466: done (4 left) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1467: done (3 left) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1468: done (2 left) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1469: done (1 left) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1470: done (0 left) 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1471: rollback oid 18 current snap is 161 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 157 2024-09-15T03:52:22.871 INFO:tasks.rados.rados.0.smithi122.stdout:1472: write (excl) oid 12 current snap is 161 2024-09-15T03:52:22.872 INFO:tasks.rados.rados.0.smithi122.stdout:1472: seq_num 544 ranges {743008=796519,2200322=721849} 2024-09-15T03:52:22.886 INFO:tasks.rados.rados.0.smithi122.stdout:1472: writing smithi12223906-12 from 743008 to 1539527 tid 1 2024-09-15T03:52:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:1472: writing smithi12223906-12 from 2200322 to 2922171 tid 2 2024-09-15T03:52:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:1473: read oid 42 snap 160 2024-09-15T03:52:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:1473: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:1471: finishing rollback tid 1 to smithi12223906-18 2024-09-15T03:52:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:1471: finishing rollback tid 2 to smithi12223906-18 2024-09-15T03:52:22.894 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 707 (ObjNum 417 snap 126 seq_num 417) dirty exists 2024-09-15T03:52:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:1471: done (2 left) 2024-09-15T03:52:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:1474: write oid 47 current snap is 161 2024-09-15T03:52:22.895 INFO:tasks.rados.rados.0.smithi122.stdout:1474: seq_num 545 ranges {759815=471768,1742451=465507,2917178=242637} 2024-09-15T03:52:22.901 INFO:tasks.rados.rados.0.smithi122.stdout:1474: writing smithi12223906-47 from 759815 to 1231583 tid 1 2024-09-15T03:52:22.907 INFO:tasks.rados.rados.0.smithi122.stdout:1474: writing smithi12223906-47 from 1742451 to 2207958 tid 2 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1474: writing smithi12223906-47 from 2917178 to 3159815 tid 3 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1475: copy_from oid 3 from oid 21 current snap is 161 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1472: finishing write tid 1 to smithi12223906-12 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1476: read oid 13 snap -1 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1476: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1477: write oid 20 current snap is 161 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1472: finishing write tid 2 to smithi12223906-12 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1472: finishing write tid 3 to smithi12223906-12 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1472: finishing write tid 4 to smithi12223906-12 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 365 (ObjNum 544 snap 161 seq_num 544) dirty exists 2024-09-15T03:52:22.913 INFO:tasks.rados.rados.0.smithi122.stdout:1472: left oid 12 (ObjNum 544 snap 161 seq_num 544) 2024-09-15T03:52:22.914 INFO:tasks.rados.rados.0.smithi122.stdout:1473: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:1474: finishing write tid 1 to smithi12223906-47 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:1474: finishing write tid 2 to smithi12223906-47 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:1474: finishing write tid 3 to smithi12223906-47 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:1474: finishing write tid 4 to smithi12223906-47 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:1474: finishing write tid 5 to smithi12223906-47 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 391 (ObjNum 545 snap 161 seq_num 545) dirty exists 2024-09-15T03:52:23.095 INFO:tasks.rados.rados.0.smithi122.stdout:1474: left oid 47 (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:52:23.096 INFO:tasks.rados.rados.0.smithi122.stdout:1476: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:52:23.302 INFO:tasks.rados.rados.0.smithi122.stdout:1475: finishing copy_from racing read to smithi12223906-3 2024-09-15T03:52:23.302 INFO:tasks.rados.rados.0.smithi122.stdout:1475: finishing copy_from to smithi12223906-3 2024-09-15T03:52:23.302 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 405 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:23.302 INFO:tasks.rados.rados.0.smithi122.stdout:1477: seq_num 546 ranges {776622=547017,1800933=725518,2944601=232021} 2024-09-15T03:52:23.317 INFO:tasks.rados.rados.0.smithi122.stdout:1477: writing smithi12223906-20 from 776622 to 1323639 tid 1 2024-09-15T03:52:23.325 INFO:tasks.rados.rados.0.smithi122.stdout:1477: writing smithi12223906-20 from 1800933 to 2526451 tid 2 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1477: writing smithi12223906-20 from 2944601 to 3176622 tid 3 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1472: done (5 left) 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1473: done (4 left) 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1474: done (3 left) 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1475: done (2 left) 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1476: done (1 left) 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:1478: rollback oid 17 current snap is 161 2024-09-15T03:52:23.329 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 160 2024-09-15T03:52:23.330 INFO:tasks.rados.rados.0.smithi122.stdout:1479: write (excl) oid 44 current snap is 161 2024-09-15T03:52:23.330 INFO:tasks.rados.rados.0.smithi122.stdout:1479: seq_num 547 ranges {793429=622266,1859415=469176,2972024=221405} 2024-09-15T03:52:23.337 INFO:tasks.rados.rados.0.smithi122.stdout:1479: writing smithi12223906-44 from 793429 to 1415695 tid 1 2024-09-15T03:52:23.342 INFO:tasks.rados.rados.0.smithi122.stdout:1479: writing smithi12223906-44 from 1859415 to 2328591 tid 2 2024-09-15T03:52:23.347 INFO:tasks.rados.rados.0.smithi122.stdout:1479: writing smithi12223906-44 from 2972024 to 3193429 tid 3 2024-09-15T03:52:23.347 INFO:tasks.rados.rados.0.smithi122.stdout:1480: read oid 41 snap -1 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1480: expect deleted 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1481: copy_from oid 5 from oid 29 current snap is 161 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: finishing write tid 1 to smithi12223906-20 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1478: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 0 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: finishing write tid 2 to smithi12223906-20 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: finishing write tid 3 to smithi12223906-20 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: finishing write tid 4 to smithi12223906-20 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: finishing write tid 5 to smithi12223906-20 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 646 (ObjNum 546 snap 161 seq_num 546) dirty exists 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: left oid 20 (ObjNum 546 snap 161 seq_num 546) 2024-09-15T03:52:23.348 INFO:tasks.rados.rados.0.smithi122.stdout:1477: done (4 left) 2024-09-15T03:52:23.349 INFO:tasks.rados.rados.0.smithi122.stdout:1478: done (3 left) 2024-09-15T03:52:23.349 INFO:tasks.rados.rados.0.smithi122.stdout:1482: snap_create 2024-09-15T03:52:23.352 INFO:tasks.rados.rados.0.smithi122.stdout:1479: finishing write tid 1 to smithi12223906-44 2024-09-15T03:52:23.353 INFO:tasks.rados.rados.0.smithi122.stdout:1479: finishing write tid 2 to smithi12223906-44 2024-09-15T03:52:23.353 INFO:tasks.rados.rados.0.smithi122.stdout:1479: finishing write tid 3 to smithi12223906-44 2024-09-15T03:52:23.353 INFO:tasks.rados.rados.0.smithi122.stdout:1479: finishing write tid 4 to smithi12223906-44 2024-09-15T03:52:23.354 INFO:tasks.rados.rados.0.smithi122.stdout:1479: finishing write tid 5 to smithi12223906-44 2024-09-15T03:52:23.354 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 712 (ObjNum 547 snap 161 seq_num 547) dirty exists 2024-09-15T03:52:23.354 INFO:tasks.rados.rados.0.smithi122.stdout:1479: left oid 44 (ObjNum 547 snap 161 seq_num 547) 2024-09-15T03:52:23.359 INFO:tasks.rados.rados.0.smithi122.stdout:1481: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:52:23.361 INFO:tasks.rados.rados.0.smithi122.stdout:1481: finishing copy_from to smithi12223906-5 2024-09-15T03:52:23.361 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 638 (ObjNum 525 snap 158 seq_num 525) dirty exists 2024-09-15T03:52:23.740 INFO:tasks.rados.rados.0.smithi122.stdout:1479: done (3 left) 2024-09-15T03:52:23.740 INFO:tasks.rados.rados.0.smithi122.stdout:1480: done (2 left) 2024-09-15T03:52:23.741 INFO:tasks.rados.rados.0.smithi122.stdout:1481: done (1 left) 2024-09-15T03:52:23.741 INFO:tasks.rados.rados.0.smithi122.stdout:1482: done (0 left) 2024-09-15T03:52:23.741 INFO:tasks.rados.rados.0.smithi122.stdout:1483: write oid 31 current snap is 162 2024-09-15T03:52:23.741 INFO:tasks.rados.rados.0.smithi122.stdout:1483: seq_num 548 ranges {410236=413868,1234250=729187,2715800=423063} 2024-09-15T03:52:23.746 INFO:tasks.rados.rados.0.smithi122.stdout:1483: writing smithi12223906-31 from 410236 to 824104 tid 1 2024-09-15T03:52:23.752 INFO:tasks.rados.rados.0.smithi122.stdout:1483: writing smithi12223906-31 from 1234250 to 1963437 tid 2 2024-09-15T03:52:23.759 INFO:tasks.rados.rados.0.smithi122.stdout:1483: writing smithi12223906-31 from 2715800 to 3138863 tid 3 2024-09-15T03:52:23.759 INFO:tasks.rados.rados.0.smithi122.stdout:1484: read oid 48 snap -1 2024-09-15T03:52:23.759 INFO:tasks.rados.rados.0.smithi122.stdout:1484: expect (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:52:23.759 INFO:tasks.rados.rados.0.smithi122.stdout:1485: rollback oid 23 current snap is 162 2024-09-15T03:52:23.759 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 157 2024-09-15T03:52:23.759 INFO:tasks.rados.rados.0.smithi122.stdout:1483: finishing write tid 1 to smithi12223906-31 2024-09-15T03:52:23.760 INFO:tasks.rados.rados.0.smithi122.stdout:1486: snap_remove snap 153 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient(hunting): tick 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 monclient(hunting): continuing hunt 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f6ccc138d40 legacy=0x7f6ccc1362f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f6ccc132b40 legacy=0x7f6ccc135be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f6ccc09faa0 legacy=0x7f6ccc1354d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient: _add_conns ranks=[2,1,7,5,4,8] 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient(hunting): picked mon.noname-b con 0x7f6cb40037d0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient(hunting): picked mon.noname-f con 0x7f6cb40073b0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient(hunting): picked mon.noname-i con 0x7f6cb400b130 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc137bf0 con 0x7f6cb40073b0 2024-09-15T03:52:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc09cfa0 con 0x7f6cb40037d0 2024-09-15T03:52:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc11c6e0 con 0x7f6cb400b130 2024-09-15T03:52:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc17fa700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cd2213700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6cb40037d0 0x7f6cb4005c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48916/0 (socket says 172.21.15.26:48916) 2024-09-15T03:52:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cd2213700 1 -- 172.21.15.26:0/844427424 learned_addr learned my addr 172.21.15.26:0/844427424 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827022865 0 0) 0x7f6ccc11c6e0 con 0x7f6cb400b130 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): my global_id is 134701 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb0003570 con 0x7f6cb400b130 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680344996 0 0) 0x7f6ccc137bf0 con 0x7f6cb40073b0 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): my global_id is 184285 2024-09-15T03:52:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ccc11c6e0 con 0x7f6cb40073b0 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139523193 0 0) 0x7f6ccc09cfa0 con 0x7f6cb40037d0 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): my global_id is 134822 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ccc137bf0 con 0x7f6cb40037d0 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3658837039 0 0) 0x7f6cb0003570 con 0x7f6cb400b130 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ccc09cfa0 con 0x7f6cb400b130 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc4001b30 con 0x7f6cb400b130 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): dump: 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 115304930 0 0) 0x7f6ccc11c6e0 con 0x7f6cb40073b0 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6cb0003570 con 0x7f6cb40073b0 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc80044d0 con 0x7f6cb40073b0 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient(hunting): dump: 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1094161963 0 0) 0x7f6ccc137bf0 con 0x7f6cb40037d0 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ccc11c6e0 con 0x7f6cb40037d0 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3810426297 0 0) 0x7f6ccc09cfa0 con 0x7f6cb400b130 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.26:6790/0 conn(0x7f6cb40037d0 legacy=0x7f6cb4005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.122:6789/0 conn(0x7f6cb40073b0 legacy=0x7f6cb4009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 1 monclient: found mon.i 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.756+0000 7f6cc27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc09ccc0 con 0x7f6cb400b130 2024-09-15T03:52:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 5 monclient: authenticate success, global_id 134701 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 10 monclient: _renew_subs 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc137770 con 0x7f6cb400b130 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 10 monclient: _renew_subs 2024-09-15T03:52:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc1324f0 con 0x7f6cb400b130 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd2a14700 1 -- 172.21.15.26:0/844427424 reap_dead start 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6cc4004fa0 con 0x7f6cb400b130 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc4005450 con 0x7f6cb400b130 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient: dump: 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd0a10700 4 set_mon_vals no callback set 2024-09-15T03:52:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6cc40055b0 con 0x7f6cb400b130 2024-09-15T03:52:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(342..342 src has 224..342) v4 ==== 3927+0+0 (unknown 3477121948 0 0) 0x7f6cc40441f0 con 0x7f6cb400b130 2024-09-15T03:52:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ca8006590 con 0x7f6ca8002840 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cd2213700 10 monclient: get_auth_request con 0x7f6cb0037de0 auth_method 0 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33402426 0 0) 0x7f6ca8006590 con 0x7f6ca8002840 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 10 monclient(hunting): my global_id is 134765 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.760+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ccc09cfa0 con 0x7f6ca8002840 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1019006099 0 0) 0x7f6ccc09cfa0 con 0x7f6ca8002840 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ca8006590 con 0x7f6ca8002840 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc8004be0 con 0x7f6ca8002840 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 10 monclient: dump: 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.781 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1294243088 0 0) 0x7f6ca8006590 con 0x7f6ca8002840 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ca800a130 con 0x7f6ca8002840 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6cc8006ff0 con 0x7f6ca8002840 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.26:6793/0 conn(0x7f6ca8002840 legacy=0x7f6ca8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.764+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.26:6793/0 conn(0x7f6ca8002840 legacy=0x7f6ca8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.768+0000 7f6cd3215700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.768+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ca8007520 con 0x7f6ca80068d0 2024-09-15T03:52:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169898130 0 0) 0x7f6ca8007520 con 0x7f6ca80068d0 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient(hunting): my global_id is 134774 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca8006590 con 0x7f6ca80068d0 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2882066798 0 0) 0x7f6ca8006590 con 0x7f6ca80068d0 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ca8007520 con 0x7f6ca80068d0 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6cc80057c0 con 0x7f6ca80068d0 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient: dump: 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.784 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 260488566 0 0) 0x7f6ca8007520 con 0x7f6ca80068d0 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ca8008760 con 0x7f6ca80068d0 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 4082309615) 0x7f6cc8006dd0 con 0x7f6ca80068d0 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.26:6793/0 conn(0x7f6ca80068d0 legacy=0x7f6ca8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cc27fc700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.26:6793/0 conn(0x7f6ca80068d0 legacy=0x7f6ca8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:23.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 60, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:23.786 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:23.787 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:23.788 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:23.789 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.790 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:23.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.791 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:23.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.791 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:23.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.792 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:23.793 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:23.793 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.793 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.793 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:23.794 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:23.795 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.796 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:23.798 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:23.800 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:23.802 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.804 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.805 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:23.806 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:23.807 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:23.808 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:52:23.808 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.808 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.808 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.808 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:23.808 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.122:6812/20895 conn(0x7f6cb0037de0 legacy=0x7f6cb003a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 10 monclient: shutdown 2024-09-15T03:52:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 >> v1:172.21.15.122:6792/0 conn(0x7f6cb400b130 legacy=0x7f6cb400d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 shutdown_connections 2024-09-15T03:52:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 >> 172.21.15.26:0/844427424 conn(0x7f6ccc06e370 msgr2=0x7f6ccc09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 shutdown_connections 2024-09-15T03:52:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.772+0000 7f6cd3215700 1 -- 172.21.15.26:0/844427424 wait complete. 2024-09-15T03:52:23.818 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:52:23.818 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon f 2024-09-15T03:52:23.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:52:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.924+0000 7fa41f1a2700 10 monclient: get_monmap_and_config 2024-09-15T03:52:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.924+0000 7fa41f1a2700 10 monclient: build_initial_monmap 2024-09-15T03:52:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.924+0000 7fa41f1a2700 10 monclient: monmap: 2024-09-15T03:52:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:23.931735+0000 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:23.931735+0000 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:23.935 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 1 Processor -- start 2024-09-15T03:52:23.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 1 -- start start 2024-09-15T03:52:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient: init 2024-09-15T03:52:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient: _add_conns ranks=[4,5,7,6,3,1,8,2,0] 2024-09-15T03:52:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient(hunting): picked mon.noname-c con 0x7fa418132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient(hunting): picked mon.noname-h con 0x7fa41809faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient(hunting): picked mon.noname-i con 0x7fa418138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41804ca10 con 0x7fa418132b40 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41809d5d0 con 0x7fa41809faa0 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa418133620 con 0x7fa418138d40 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient(hunting): authenticate will time out at 1288.361748s 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41e1a0700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fa418132b40 0x7fa418132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49082/0 (socket says 172.21.15.26:49082) 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41e1a0700 1 -- 172.21.15.26:0/1769356658 learned_addr learned my addr 172.21.15.26:0/1769356658 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498573385 0 0) 0x7fa41804ca10 con 0x7fa418132b40 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): my global_id is 184333 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa404003570 con 0x7fa418132b40 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664871243 0 0) 0x7fa418133620 con 0x7fa418138d40 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): my global_id is 134710 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41804ca10 con 0x7fa418138d40 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893415055 0 0) 0x7fa41809d5d0 con 0x7fa41809faa0 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient(hunting): my global_id is 5351 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa418133620 con 0x7fa41809faa0 2024-09-15T03:52:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1734801555 0 0) 0x7fa404003570 con 0x7fa418132b40 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 >> v1:172.21.15.122:6792/0 conn(0x7fa418138d40 legacy=0x7fa41813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 >> v1:172.21.15.122:6791/0 conn(0x7fa41809faa0 legacy=0x7fa41809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: _finish_hunting 0 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 monclient: found mon.noname-c 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4180a0cc0 con 0x7fa418132b40 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: _finish_auth 0 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 5 monclient: authenticate success, global_id 184333 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa414001b30 con 0x7fa418132b40 2024-09-15T03:52:23.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: dump: 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.945 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa414001da0 con 0x7fa418132b40 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 1 -- 172.21.15.26:0/1769356658 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa414004fb0 con 0x7fa418132b40 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41d19e700 10 monclient: dump: 2024-09-15T03:52:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.928+0000 7fa41f1a2700 4 set_mon_vals no callback set 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 10 monclient: shutdown 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 -- 172.21.15.26:0/1769356658 >> v1:172.21.15.26:6791/0 conn(0x7fa418132b40 legacy=0x7fa418132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 -- 172.21.15.26:0/1769356658 shutdown_connections 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 -- 172.21.15.26:0/1769356658 >> 172.21.15.26:0/1769356658 conn(0x7fa41806e370 msgr2=0x7fa4180a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 -- 172.21.15.26:0/1769356658 shutdown_connections 2024-09-15T03:52:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 -- 172.21.15.26:0/1769356658 wait complete. 2024-09-15T03:52:23.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 10 monclient: build_initial_monmap 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 10 monclient: monmap: 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:23.954 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 Processor -- start 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 1 -- start start 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.932+0000 7fa41f1a2700 10 monclient: init 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient: _add_conns ranks=[1,8,2,0,4,6,3,7,5] 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient(hunting): picked mon.noname-f con 0x7fa41809faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient(hunting): picked mon.noname-e con 0x7fa418132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient(hunting): picked mon.noname-b con 0x7fa418138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41809cfa0 con 0x7fa41809faa0 2024-09-15T03:52:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41811c640 con 0x7fa418138d40 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41809ccc0 con 0x7fa418132b40 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient(hunting): authenticate will time out at 1288.368234s 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41e9a1700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fa418138d40 0x7fa41814a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48920/0 (socket says 172.21.15.26:48920) 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41e9a1700 1 -- 172.21.15.26:0/193196217 learned_addr learned my addr 172.21.15.26:0/193196217 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41d99f700 1 --1- 172.21.15.26:0/193196217 >> v1:172.21.15.26:6793/0 conn(0x7fa418132b40 0x7fa418135f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37412/0 (socket says 172.21.15.26:37412) 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829001282 0 0) 0x7fa41811c640 con 0x7fa418138d40 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): my global_id is 134831 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3fc0035a0 con 0x7fa418138d40 2024-09-15T03:52:23.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243391816 0 0) 0x7fa41809ccc0 con 0x7fa418132b40 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): my global_id is 134783 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41811c640 con 0x7fa418132b40 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323756087 0 0) 0x7fa41809cfa0 con 0x7fa41809faa0 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): my global_id is 184294 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41809ccc0 con 0x7fa41809faa0 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1771323434 0 0) 0x7fa3fc0035a0 con 0x7fa418138d40 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa41809cfa0 con 0x7fa418138d40 2024-09-15T03:52:23.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa4100029b0 con 0x7fa418138d40 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.958 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 307306000 0 0) 0x7fa41811c640 con 0x7fa418132b40 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa3fc0035a0 con 0x7fa418132b40 2024-09-15T03:52:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa40c001a60 con 0x7fa418132b40 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.960 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1316117830 0 0) 0x7fa41809cfa0 con 0x7fa418138d40 2024-09-15T03:52:23.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.26:6793/0 conn(0x7fa418132b40 legacy=0x7fa418135f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.122:6789/0 conn(0x7fa41809faa0 legacy=0x7fa418135820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 monclient: found mon.b 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4181376a0 con 0x7fa418138d40 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 5 monclient: authenticate success, global_id 134831 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient: _renew_subs 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:23.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418137150 con 0x7fa418138d40 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient: _renew_subs 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa418137ec0 con 0x7fa418138d40 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa410002cf0 con 0x7fa418138d40 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa410001840 con 0x7fa418138d40 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41c99d700 4 set_mon_vals no callback set 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 10 monclient: dump: 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.964 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa4100019a0 con 0x7fa418138d40 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa400006590 con 0x7fa400002840 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(342..342 src has 224..342) v4 ==== 3927+0+0 (unknown 3477121948 0 0) 0x7fa410043d40 con 0x7fa418138d40 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa41d99f700 10 monclient: get_auth_request con 0x7fa3fc037c80 auth_method 0 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.936+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916778475 0 0) 0x7fa400006590 con 0x7fa400002840 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient(hunting): my global_id is 184303 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41809cfa0 con 0x7fa400002840 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2484785208 0 0) 0x7fa41809cfa0 con 0x7fa400002840 2024-09-15T03:52:23.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa400006590 con 0x7fa400002840 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa414004fb0 con 0x7fa400002840 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient: dump: 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1500534947 0 0) 0x7fa400006590 con 0x7fa400002840 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa40000a130 con 0x7fa400002840 2024-09-15T03:52:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa414007430 con 0x7fa400002840 2024-09-15T03:52:23.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:23.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:23.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.122:6789/0 conn(0x7fa400002840 legacy=0x7fa400004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.940+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.122:6789/0 conn(0x7fa400002840 legacy=0x7fa400004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:23.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa41f1a2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa400007520 con 0x7fa4000068d0 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002199201 0 0) 0x7fa400007520 con 0x7fa4000068d0 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 10 monclient(hunting): my global_id is 184312 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa400006590 con 0x7fa4000068d0 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4152136222 0 0) 0x7fa400006590 con 0x7fa4000068d0 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa400007520 con 0x7fa4000068d0 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa414005b10 con 0x7fa4000068d0 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:52:23.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 10 monclient: dump: 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 949375528 0 0) 0x7fa400007520 con 0x7fa4000068d0 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.952+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa400008760 con 0x7fa4000068d0 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7365 (unknown 0 0 998787058) 0x7fa414007010 con 0x7fa4000068d0 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa40a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa40a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.122:6789/0 conn(0x7fa4000068d0 legacy=0x7fa400006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa40a7fc700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.122:6789/0 conn(0x7fa4000068d0 legacy=0x7fa400006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 60, 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:23.974 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:23.975 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:23.976 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:23.977 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:23.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.979 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.986 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.989 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:23.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.992 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:23.993 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:23.994 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:23.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:23.994 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:23.994 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.122:6812/20895 conn(0x7fa3fc037c80 legacy=0x7fa3fc03a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 10 monclient: shutdown 2024-09-15T03:52:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41e9a1700 1 -- 172.21.15.26:0/193196217 reap_dead start 2024-09-15T03:52:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 >> v1:172.21.15.26:6790/0 conn(0x7fa418138d40 legacy=0x7fa41814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 shutdown_connections 2024-09-15T03:52:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 >> 172.21.15.26:0/193196217 conn(0x7fa41806e370 msgr2=0x7fa4180a33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 shutdown_connections 2024-09-15T03:52:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:23.956+0000 7fa41f1a2700 1 -- 172.21.15.26:0/193196217 wait complete. 2024-09-15T03:52:23.996 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:52:23.996 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon h 2024-09-15T03:52:23.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:52:24.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: get_monmap_and_config 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: build_initial_monmap 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: monmap: 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:24.112303+0000 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:24.112303+0000 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:24.115 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 1 Processor -- start 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 1 -- start start 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: init 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: _add_conns ranks=[4,6,0,2,8,1,5,7,3] 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient(hunting): picked mon.noname-c con 0x7f5d940a2aa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient(hunting): picked mon.noname-d con 0x7f5d9409b410 addr v1:172.21.15.26:6792/0 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient(hunting): picked mon.noname-a con 0x7f5d9406e160 addr v1:172.21.15.26:6789/0 2024-09-15T03:52:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9404ca10 con 0x7f5d9406e160 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d94099950 con 0x7f5d940a2aa0 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9409b7e0 con 0x7f5d9409b410 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient(hunting): authenticate will time out at 1288.541747s 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d92d9d700 1 -- >> v1:172.21.15.26:6789/0 conn(0x7f5d9406e160 legacy=0x7f5d94137d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6789/0 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d91d9b700 1 -- >> v1:172.21.15.26:6792/0 conn(0x7f5d9409b410 legacy=0x7f5d94134620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6792/0 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9259c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f5d940a2aa0 0x7f5d94097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49086/0 (socket says 172.21.15.26:49086) 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9259c700 1 -- 172.21.15.26:0/716864332 learned_addr learned my addr 172.21.15.26:0/716864332 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342098422 0 0) 0x7f5d94099950 con 0x7f5d940a2aa0 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient(hunting): my global_id is 184342 2024-09-15T03:52:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c003570 con 0x7f5d940a2aa0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115028325 0 0) 0x7f5d7c003570 con 0x7f5d940a2aa0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 >> v1:172.21.15.26:6792/0 conn(0x7f5d9409b410 legacy=0x7f5d94134620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 >> v1:172.21.15.26:6789/0 conn(0x7f5d9406e160 legacy=0x7f5d94137d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: _finish_hunting 0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 monclient: found mon.noname-c 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940995e0 con 0x7f5d940a2aa0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: _finish_auth 0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d880017b0 con 0x7f5d940a2aa0 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:52:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: dump: 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 5 monclient: authenticate success, global_id 184342 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5d88001ad0 con 0x7f5d940a2aa0 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 1 -- 172.21.15.26:0/716864332 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d88001cc0 con 0x7f5d940a2aa0 2024-09-15T03:52:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9359e700 4 set_mon_vals no callback set 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.108+0000 7f5d9159a700 10 monclient: dump: 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient: shutdown 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- 172.21.15.26:0/716864332 >> v1:172.21.15.26:6791/0 conn(0x7f5d940a2aa0 legacy=0x7f5d94097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- 172.21.15.26:0/716864332 shutdown_connections 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- 172.21.15.26:0/716864332 >> 172.21.15.26:0/716864332 conn(0x7f5d94063cd0 msgr2=0x7f5d940a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- 172.21.15.26:0/716864332 shutdown_connections 2024-09-15T03:52:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- 172.21.15.26:0/716864332 wait complete. 2024-09-15T03:52:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient: build_initial_monmap 2024-09-15T03:52:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient: monmap: 2024-09-15T03:52:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:24.130 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 Processor -- start 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- start start 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient: init 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient: _add_conns ranks=[4,2,8,7,3,6,1,0,5] 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient(hunting): picked mon.noname-c con 0x7f5d9406e160 addr v1:172.21.15.26:6791/0 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient(hunting): picked mon.noname-b con 0x7f5d9409b410 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient(hunting): picked mon.noname-e con 0x7f5d9406c6f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9409be80 con 0x7f5d9409b410 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9409b9b0 con 0x7f5d9406e160 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9406caf0 con 0x7f5d9406c6f0 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9359e700 10 monclient(hunting): authenticate will time out at 1288.546731s 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9259c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f5d9406e160 0x7f5d9406b8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:49094/0 (socket says 172.21.15.26:49094) 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d9259c700 1 -- 172.21.15.26:0/3629063134 learned_addr learned my addr 172.21.15.26:0/3629063134 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857273655 0 0) 0x7f5d9409b9b0 con 0x7f5d9406e160 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): my global_id is 184351 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d74003570 con 0x7f5d9406e160 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808538243 0 0) 0x7f5d9409be80 con 0x7f5d9409b410 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): my global_id is 134840 2024-09-15T03:52:24.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d9409b9b0 con 0x7f5d9409b410 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512507488 0 0) 0x7f5d8c003d80 con 0x7f5d9406c6f0 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): my global_id is 134792 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d9409be80 con 0x7f5d9406c6f0 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4148380712 0 0) 0x7f5d9409b9b0 con 0x7f5d9409b410 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d8c003d80 con 0x7f5d9409b410 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d840042c0 con 0x7f5d9409b410 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): dump: 2024-09-15T03:52:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.112+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2639717200 0 0) 0x7f5d74003570 con 0x7f5d9406e160 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d9409b9b0 con 0x7f5d9406e160 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d88001770 con 0x7f5d9406e160 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient(hunting): dump: 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1441323910 0 0) 0x7f5d8c003d80 con 0x7f5d9409b410 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.26:6793/0 conn(0x7f5d9406c6f0 legacy=0x7f5d941459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.26:6791/0 conn(0x7f5d9406e160 legacy=0x7f5d9406b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 monclient: found mon.b 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9412b350 con 0x7f5d9409b410 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 5 monclient: authenticate success, global_id 134840 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 10 monclient: _renew_subs 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d94119280 con 0x7f5d9409b410 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 10 monclient: _renew_subs 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9406d8d0 con 0x7f5d9409b410 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5d84004850 con 0x7f5d9409b410 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d84004d80 con 0x7f5d9409b410 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d90d99700 4 set_mon_vals no callback set 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: dump: 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5d84004ee0 con 0x7f5d9409b410 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5d78006590 con 0x7f5d78002840 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(342..342 src has 224..342) v4 ==== 3927+0+0 (unknown 3477121948 0 0) 0x7f5d84043df0 con 0x7f5d9409b410 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d92d9d700 10 monclient: get_auth_request con 0x7f5d74037d00 auth_method 0 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193374134 0 0) 0x7f5d78006590 con 0x7f5d78002840 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient(hunting): my global_id is 5360 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d8c003d80 con 0x7f5d78002840 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1741470633 0 0) 0x7f5d8c003d80 con 0x7f5d78002840 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d78006590 con 0x7f5d78002840 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d88005060 con 0x7f5d78002840 2024-09-15T03:52:24.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 10 monclient: dump: 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2669451661 0 0) 0x7f5d78006590 con 0x7f5d78002840 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.116+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d7800a130 con 0x7f5d78002840 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.120+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5d88007280 con 0x7f5d78002840 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.120+0000 7f5d827fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.120+0000 7f5d827fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.120+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.122:6791/0 conn(0x7f5d78002840 legacy=0x7f5d78004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.120+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.122:6791/0 conn(0x7f5d78002840 legacy=0x7f5d78004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.128+0000 7f5d9359e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.128+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5d78007520 con 0x7f5d780068d0 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493606959 0 0) 0x7f5d78007520 con 0x7f5d780068d0 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient(hunting): my global_id is 5369 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d78006590 con 0x7f5d780068d0 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 516272795 0 0) 0x7f5d78006590 con 0x7f5d780068d0 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d78007520 con 0x7f5d780068d0 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5d88005a70 con 0x7f5d780068d0 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:52:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient: dump: 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3507449609 0 0) 0x7f5d78007520 con 0x7f5d780068d0 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5d78008760 con 0x7f5d780068d0 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7707 (unknown 0 0 3585969410) 0x7f5d88006f10 con 0x7f5d780068d0 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.122:6791/0 conn(0x7f5d780068d0 legacy=0x7f5d78006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d827fc700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.122:6791/0 conn(0x7f5d780068d0 legacy=0x7f5d78006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 60, 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:24.147 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:24.148 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:24.149 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:24.150 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:24.151 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:24.152 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:24.153 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:24.155 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.156 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.157 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.158 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:24.159 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.160 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:24.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.163 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.164 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.165 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:24.166 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:24.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.122:6812/20895 conn(0x7f5d74037d00 legacy=0x7f5d7403a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 10 monclient: shutdown 2024-09-15T03:52:24.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 >> v1:172.21.15.26:6790/0 conn(0x7f5d9409b410 legacy=0x7f5d9406bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d92d9d700 1 -- 172.21.15.26:0/3629063134 reap_dead start 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d92d9d700 1 -- 172.21.15.26:0/3629063134 reap_dead start 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 shutdown_connections 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 >> 172.21.15.26:0/3629063134 conn(0x7f5d94063cd0 msgr2=0x7f5d9412ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 shutdown_connections 2024-09-15T03:52:24.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.132+0000 7f5d9359e700 1 -- 172.21.15.26:0/3629063134 wait complete. 2024-09-15T03:52:24.168 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:52:24.168 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-09-15T03:52:24.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient: get_monmap_and_config 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient: build_initial_monmap 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient: monmap: 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:52:24.286586+0000 2024-09-15T03:52:24.288 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:52:24.286586+0000 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:24.289 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 1 Processor -- start 2024-09-15T03:52:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 1 -- start start 2024-09-15T03:52:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient: init 2024-09-15T03:52:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient: _add_conns ranks=[8,3,7,5,2,4,1,6,0] 2024-09-15T03:52:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.280+0000 7f4f78570700 10 monclient(hunting): picked mon.noname-e con 0x7f4f7009b8d0 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient(hunting): picked mon.noname-g con 0x7f4f7009c250 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient(hunting): picked mon.noname-i con 0x7f4f7009e490 addr v1:172.21.15.122:6792/0 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7004ca10 con 0x7f4f7009c250 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7009d8a0 con 0x7f4f7009e490 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f701317e0 con 0x7f4f7009b8d0 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient(hunting): authenticate will time out at 1288.716237s 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f75b0b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f4f7009c250 legacy=0x7f4f701310d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7630c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f4f7009b8d0 0x7f4f701309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37434/0 (socket says 172.21.15.26:37434) 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7630c700 1 -- 172.21.15.26:0/1347552427 learned_addr learned my addr 172.21.15.26:0/1347552427 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290625987 0 0) 0x7f4f701317e0 con 0x7f4f7009b8d0 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient(hunting): my global_id is 134801 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f58003570 con 0x7f4f7009b8d0 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222622758 0 0) 0x7f4f7009d8a0 con 0x7f4f7009e490 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient(hunting): my global_id is 134719 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f701317e0 con 0x7f4f7009e490 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2358694310 0 0) 0x7f4f58003570 con 0x7f4f7009b8d0 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 >> v1:172.21.15.122:6792/0 conn(0x7f4f7009e490 legacy=0x7f4f70138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 >> v1:172.21.15.122:6790/0 conn(0x7f4f7009c250 legacy=0x7f4f701310d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: _finish_hunting 0 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 monclient: found mon.noname-e 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7009d5c0 con 0x7f4f7009b8d0 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: _finish_auth 0 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f600017b0 con 0x7f4f7009b8d0 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: dump: 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 5 monclient: authenticate success, global_id 134801 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f60001a90 con 0x7f4f7009b8d0 2024-09-15T03:52:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient: get_monmap_and_config success 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 4 set_mon_vals no callback set 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 1 -- 172.21.15.26:0/1347552427 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f60001c80 con 0x7f4f7009b8d0 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f7530a700 10 monclient: dump: 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient: shutdown 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- 172.21.15.26:0/1347552427 >> v1:172.21.15.26:6793/0 conn(0x7f4f7009b8d0 legacy=0x7f4f701309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- 172.21.15.26:0/1347552427 shutdown_connections 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- 172.21.15.26:0/1347552427 >> 172.21.15.26:0/1347552427 conn(0x7f4f70063f30 msgr2=0x7f4f700a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- 172.21.15.26:0/1347552427 shutdown_connections 2024-09-15T03:52:24.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 1 -- 172.21.15.26:0/1347552427 wait complete. 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient: build_initial_monmap 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.284+0000 7f4f78570700 10 monclient: monmap: 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:52:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 Processor -- start 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 -- start start 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: init 2024-09-15T03:52:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: _reopen_session rank -1 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: _add_conns ranks=[8,3,2,5,7,6,0,1,4] 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient(hunting): picked mon.noname-e con 0x7f4f7009c250 addr v1:172.21.15.26:6793/0 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient(hunting): picked mon.noname-g con 0x7f4f7009e490 addr v1:172.21.15.122:6790/0 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient(hunting): picked mon.noname-b con 0x7f4f70130dc0 addr v1:172.21.15.26:6790/0 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f70135d30 con 0x7f4f70130dc0 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7009cd20 con 0x7f4f7009e490 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7006e140 con 0x7f4f7009c250 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient(hunting): _renew_subs 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient(hunting): authenticate will time out at 1288.720873s 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f75b0b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f4f7009e490 legacy=0x7f4f701467f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f7630c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f4f7009c250 0x7f4f70138b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37436/0 (socket says 172.21.15.26:37436) 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f7630c700 1 -- 172.21.15.26:0/2576034382 learned_addr learned my addr 172.21.15.26:0/2576034382 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:52:24.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626405670 0 0) 0x7f4f70135d30 con 0x7f4f70130dc0 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): my global_id is 134849 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f5c003570 con 0x7f4f70130dc0 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347681247 0 0) 0x7f4f7006e140 con 0x7f4f7009c250 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): my global_id is 134810 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f70135d30 con 0x7f4f7009c250 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1722254196 0 0) 0x7f4f5c003570 con 0x7f4f70130dc0 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f7006e140 con 0x7f4f70130dc0 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2588545231 0 0) 0x7f4f70135d30 con 0x7f4f7009c250 2024-09-15T03:52:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f5c003570 con 0x7f4f7009c250 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f68001d00 con 0x7f4f70130dc0 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f60001770 con 0x7f4f7009c250 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient(hunting): dump: 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3014449500 0 0) 0x7f4f7006e140 con 0x7f4f70130dc0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.26:6793/0 conn(0x7f4f7009c250 legacy=0x7f4f70138b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.122:6790/0 conn(0x7f4f7009e490 legacy=0x7f4f701467f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 monclient: found mon.b 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f70137420 con 0x7f4f70130dc0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 5 monclient: authenticate success, global_id 134849 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: _renew_subs 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f70136b80 con 0x7f4f70130dc0 2024-09-15T03:52:24.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: _renew_subs 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f70135810 con 0x7f4f70130dc0 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f68003080 con 0x7f4f70130dc0 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f68002560 con 0x7f4f70130dc0 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f74b09700 4 set_mon_vals no callback set 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.288+0000 7f4f6e7fc700 10 monclient: dump: 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.311 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f78570700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:24.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4f50006590 con 0x7f4f50002840 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4f68002740 con 0x7f4f70130dc0 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(342..342 src has 224..342) v4 ==== 3927+0+0 (unknown 3477121948 0 0) 0x7f4f68041850 con 0x7f4f70130dc0 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f75b0b700 10 monclient: get_auth_request con 0x7f4f5c037b70 auth_method 0 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544627312 0 0) 0x7f4f50006590 con 0x7f4f50002840 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient(hunting): my global_id is 134728 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7006e140 con 0x7f4f50002840 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3021469586 0 0) 0x7f4f7006e140 con 0x7f4f50002840 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f50006590 con 0x7f4f50002840 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f600052c0 con 0x7f4f50002840 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient: dump: 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.314 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3863201562 0 0) 0x7f4f50006590 con 0x7f4f50002840 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f5000a130 con 0x7f4f50002840 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4f60007660 con 0x7f4f50002840 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.122:6792/0 conn(0x7f4f50002840 legacy=0x7f4f50004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.292+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.122:6792/0 conn(0x7f4f50002840 legacy=0x7f4f50004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f78570700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4f50007520 con 0x7f4f500068d0 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361895655 0 0) 0x7f4f50007520 con 0x7f4f500068d0 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 10 monclient(hunting): my global_id is 134737 2024-09-15T03:52:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f50006590 con 0x7f4f500068d0 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1984762901 0 0) 0x7f4f50006590 con 0x7f4f500068d0 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f50007520 con 0x7f4f500068d0 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f60005e90 con 0x7f4f500068d0 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.304+0000 7f4f6e7fc700 10 monclient: dump: 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:52:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1583918555 0 0) 0x7f4f50007520 con 0x7f4f500068d0 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4f50008760 con 0x7f4f500068d0 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7535 (unknown 0 0 3553174112) 0x7f4f600077d0 con 0x7f4f500068d0 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:52:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.122:6792/0 conn(0x7f4f500068d0 legacy=0x7f4f50006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f6e7fc700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.122:6792/0 conn(0x7f4f500068d0 legacy=0x7f4f50006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 60, 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:52:24.320 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:24.321 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:24.322 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:52:24.323 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:52:24.324 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.325 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:52:24.326 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.327 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.329 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:52:24.330 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.331 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.333 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.335 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:52:24.336 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:52:24.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.338 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:52:24.339 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.122:6812/20895 conn(0x7f4f5c037b70 legacy=0x7f4f5c03a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 10 monclient: shutdown 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 >> v1:172.21.15.26:6790/0 conn(0x7f4f70130dc0 legacy=0x7f4f70149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f76b0d700 1 -- 172.21.15.26:0/2576034382 reap_dead start 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f76b0d700 1 -- 172.21.15.26:0/2576034382 reap_dead start 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 shutdown_connections 2024-09-15T03:52:24.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 >> 172.21.15.26:0/2576034382 conn(0x7f4f70063f30 msgr2=0x7f4f7009ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:52:24.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 shutdown_connections 2024-09-15T03:52:24.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:52:24.308+0000 7f4f78570700 1 -- 172.21.15.26:0/2576034382 wait complete. 2024-09-15T03:52:24.342 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T03:52:24.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:52:24.758 INFO:tasks.rados.rados.0.smithi122.stdout:1483: finishing write tid 2 to smithi12223906-31 2024-09-15T03:52:24.758 INFO:tasks.rados.rados.0.smithi122.stdout:1484: expect (ObjNum 427 snap 127 seq_num 427) 2024-09-15T03:52:24.941 INFO:tasks.rados.rados.0.smithi122.stdout:1485: finishing rollback tid 0 to smithi12223906-23 2024-09-15T03:52:24.941 INFO:tasks.rados.rados.0.smithi122.stdout:1485: finishing rollback tid 1 to smithi12223906-23 2024-09-15T03:52:24.941 INFO:tasks.rados.rados.0.smithi122.stdout:1483: finishing write tid 3 to smithi12223906-31 2024-09-15T03:52:24.941 INFO:tasks.rados.rados.0.smithi122.stdout:1483: finishing write tid 4 to smithi12223906-31 2024-09-15T03:52:24.941 INFO:tasks.rados.rados.0.smithi122.stdout:1483: finishing write tid 5 to smithi12223906-31 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 234 (ObjNum 548 snap 162 seq_num 548) dirty exists 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1483: left oid 31 (ObjNum 548 snap 162 seq_num 548) 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1485: finishing rollback tid 2 to smithi12223906-23 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 716 (ObjNum 378 snap 108 seq_num 378) dirty exists 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1483: done (3 left) 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1484: done (2 left) 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1485: done (1 left) 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1486: done (0 left) 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1487: copy_from oid 27 from oid 46 current snap is 162 2024-09-15T03:52:24.942 INFO:tasks.rados.rados.0.smithi122.stdout:1488: snap_create 2024-09-15T03:52:24.943 INFO:tasks.rados.rados.0.smithi122.stdout:1487: finishing copy_from to smithi12223906-27 2024-09-15T03:52:24.943 INFO:tasks.rados.rados.0.smithi122.stdout:1487: got expected ENOENT (src dne) 2024-09-15T03:52:24.943 INFO:tasks.rados.rados.0.smithi122.stdout:1487: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:52:25.759 INFO:tasks.rados.rados.0.smithi122.stdout:1487: done (1 left) 2024-09-15T03:52:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:1488: done (0 left) 2024-09-15T03:52:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:1489: read oid 31 snap -1 2024-09-15T03:52:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:1489: expect (ObjNum 548 snap 162 seq_num 548) 2024-09-15T03:52:25.760 INFO:tasks.rados.rados.0.smithi122.stdout:1490: snap_remove snap 157 2024-09-15T03:52:26.778 INFO:tasks.rados.rados.0.smithi122.stdout:1490: done (1 left) 2024-09-15T03:52:26.778 INFO:tasks.rados.rados.0.smithi122.stdout:1491: copy_from oid 34 from oid 21 current snap is 163 2024-09-15T03:52:26.778 INFO:tasks.rados.rados.0.smithi122.stdout:1492: delete oid 36 current snap is 163 2024-09-15T03:52:26.778 INFO:tasks.rados.rados.0.smithi122.stdout:1489: expect (ObjNum 548 snap 162 seq_num 548) 2024-09-15T03:52:26.982 INFO:tasks.rados.rados.0.smithi122.stdout:1491: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:52:26.982 INFO:tasks.rados.rados.0.smithi122.stdout:1491: finishing copy_from to smithi12223906-34 2024-09-15T03:52:26.982 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 405 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:26.983 INFO:tasks.rados.rados.0.smithi122.stdout:1489: done (2 left) 2024-09-15T03:52:26.983 INFO:tasks.rados.rados.0.smithi122.stdout:1491: done (1 left) 2024-09-15T03:52:26.983 INFO:tasks.rados.rados.0.smithi122.stdout:1492: done (0 left) 2024-09-15T03:52:26.983 INFO:tasks.rados.rados.0.smithi122.stdout:1493: write oid 37 current snap is 163 2024-09-15T03:52:26.983 INFO:tasks.rados.rados.0.smithi122.stdout:1493: seq_num 550 ranges {443850=564366,1634861=732856,2770646=473204} 2024-09-15T03:52:26.987 INFO:tasks.rados.rados.0.smithi122.stdout:1493: writing smithi12223906-37 from 443850 to 1008216 tid 1 2024-09-15T03:52:26.997 INFO:tasks.rados.rados.0.smithi122.stdout:1493: writing smithi12223906-37 from 1634861 to 2367717 tid 2 2024-09-15T03:52:27.003 INFO:tasks.rados.rados.0.smithi122.stdout:1493: writing smithi12223906-37 from 2770646 to 3243850 tid 3 2024-09-15T03:52:27.003 INFO:tasks.rados.rados.0.smithi122.stdout:1494: write (excl) oid 24 current snap is 163 2024-09-15T03:52:27.004 INFO:tasks.rados.rados.0.smithi122.stdout:1493: finishing write tid 1 to smithi12223906-37 2024-09-15T03:52:27.004 INFO:tasks.rados.rados.0.smithi122.stdout:1494: seq_num 551 ranges {460657=639615,1693343=476514,2681716=578941} 2024-09-15T03:52:27.010 INFO:tasks.rados.rados.0.smithi122.stdout:1494: writing smithi12223906-24 from 460657 to 1100272 tid 1 2024-09-15T03:52:27.015 INFO:tasks.rados.rados.0.smithi122.stdout:1494: writing smithi12223906-24 from 1693343 to 2169857 tid 2 2024-09-15T03:52:27.021 INFO:tasks.rados.rados.0.smithi122.stdout:1494: writing smithi12223906-24 from 2681716 to 3260657 tid 3 2024-09-15T03:52:27.021 INFO:tasks.rados.rados.0.smithi122.stdout:1495: snap_create 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1493: finishing write tid 2 to smithi12223906-37 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1493: finishing write tid 3 to smithi12223906-37 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1493: finishing write tid 4 to smithi12223906-37 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1493: finishing write tid 5 to smithi12223906-37 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 409 (ObjNum 550 snap 163 seq_num 550) dirty exists 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1493: left oid 37 (ObjNum 550 snap 163 seq_num 550) 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1494: finishing write tid 1 to smithi12223906-24 2024-09-15T03:52:27.022 INFO:tasks.rados.rados.0.smithi122.stdout:1494: finishing write tid 2 to smithi12223906-24 2024-09-15T03:52:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:1494: finishing write tid 3 to smithi12223906-24 2024-09-15T03:52:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:1494: finishing write tid 4 to smithi12223906-24 2024-09-15T03:52:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:1494: finishing write tid 5 to smithi12223906-24 2024-09-15T03:52:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 746 (ObjNum 551 snap 163 seq_num 551) dirty exists 2024-09-15T03:52:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:1494: left oid 24 (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:52:27.785 INFO:tasks.rados.rados.0.smithi122.stdout:1493: done (2 left) 2024-09-15T03:52:27.785 INFO:tasks.rados.rados.0.smithi122.stdout:1494: done (1 left) 2024-09-15T03:52:27.785 INFO:tasks.rados.rados.0.smithi122.stdout:1495: done (0 left) 2024-09-15T03:52:27.785 INFO:tasks.rados.rados.0.smithi122.stdout:1496: copy_from oid 27 from oid 42 current snap is 164 2024-09-15T03:52:27.785 INFO:tasks.rados.rados.0.smithi122.stdout:1497: snap_remove snap 163 2024-09-15T03:52:28.788 INFO:tasks.rados.rados.0.smithi122.stdout:1497: done (1 left) 2024-09-15T03:52:28.788 INFO:tasks.rados.rados.0.smithi122.stdout:1498: snap_remove snap 161 2024-09-15T03:52:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:1498: done (1 left) 2024-09-15T03:52:29.794 INFO:tasks.rados.rados.0.smithi122.stdout:1499: snap_remove snap 159 2024-09-15T03:52:30.797 INFO:tasks.rados.rados.0.smithi122.stdout:1499: done (1 left) 2024-09-15T03:52:30.797 INFO:tasks.rados.rados.0.smithi122.stdout:1500: rollback oid 48 current snap is 164 2024-09-15T03:52:30.797 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 48 to 162 2024-09-15T03:52:30.798 INFO:tasks.rados.rados.0.smithi122.stdout:1501: snap_create 2024-09-15T03:52:30.798 INFO:tasks.rados.rados.0.smithi122.stdout:1496: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:52:30.798 INFO:tasks.rados.rados.0.smithi122.stdout:1496: finishing copy_from to smithi12223906-27 2024-09-15T03:52:30.798 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 394 (ObjNum 493 snap 149 seq_num 493) dirty exists 2024-09-15T03:52:30.805 INFO:tasks.rados.rados.0.smithi122.stdout:1500: finishing rollback tid 0 to smithi12223906-48 2024-09-15T03:52:30.807 INFO:tasks.rados.rados.0.smithi122.stdout:1500: finishing rollback tid 1 to smithi12223906-48 2024-09-15T03:52:30.808 INFO:tasks.rados.rados.0.smithi122.stdout:1500: finishing rollback tid 2 to smithi12223906-48 2024-09-15T03:52:30.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 412 (ObjNum 427 snap 127 seq_num 427) dirty exists 2024-09-15T03:52:31.722 INFO:tasks.rados.rados.0.smithi122.stdout:1496: done (2 left) 2024-09-15T03:52:31.722 INFO:tasks.rados.rados.0.smithi122.stdout:1500: done (1 left) 2024-09-15T03:52:31.722 INFO:tasks.rados.rados.0.smithi122.stdout:1501: done (0 left) 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1502: read oid 22 snap -1 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1502: expect deleted 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1503: read oid 40 snap -1 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1503: expect (ObjNum 527 snap 159 seq_num 527) 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1504: read oid 42 snap -1 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1504: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1505: write oid 30 current snap is 165 2024-09-15T03:52:31.723 INFO:tasks.rados.rados.0.smithi122.stdout:1505: seq_num 552 ranges {477464=714864,1751825=736525,3225492=51972} 2024-09-15T03:52:31.730 INFO:tasks.rados.rados.0.smithi122.stdout:1505: writing smithi12223906-30 from 477464 to 1192328 tid 1 2024-09-15T03:52:31.739 INFO:tasks.rados.rados.0.smithi122.stdout:1505: writing smithi12223906-30 from 1751825 to 2488350 tid 2 2024-09-15T03:52:31.744 INFO:tasks.rados.rados.0.smithi122.stdout:1505: writing smithi12223906-30 from 3225492 to 3277464 tid 3 2024-09-15T03:52:31.744 INFO:tasks.rados.rados.0.smithi122.stdout:1506: read oid 29 snap 164 2024-09-15T03:52:31.744 INFO:tasks.rados.rados.0.smithi122.stdout:1506: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:31.744 INFO:tasks.rados.rados.0.smithi122.stdout:1507: write (excl) oid 1 current snap is 165 2024-09-15T03:52:31.744 INFO:tasks.rados.rados.0.smithi122.stdout:1507: seq_num 553 ranges {494271=790113,1693954=480183,2620209=512129} 2024-09-15T03:52:31.751 INFO:tasks.rados.rados.0.smithi122.stdout:1507: writing smithi12223906-1 from 494271 to 1284384 tid 1 2024-09-15T03:52:31.756 INFO:tasks.rados.rados.0.smithi122.stdout:1507: writing smithi12223906-1 from 1693954 to 2174137 tid 2 2024-09-15T03:52:31.762 INFO:tasks.rados.rados.0.smithi122.stdout:1507: writing smithi12223906-1 from 2620209 to 3132338 tid 3 2024-09-15T03:52:31.762 INFO:tasks.rados.rados.0.smithi122.stdout:1508: snap_remove snap 162 2024-09-15T03:52:32.746 INFO:tasks.rados.rados.0.smithi122.stdout:1504: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:32.925 INFO:tasks.rados.rados.0.smithi122.stdout:1503: expect (ObjNum 527 snap 159 seq_num 527) 2024-09-15T03:52:33.017 INFO:tasks.rados.rados.0.smithi122.stdout:1505: finishing write tid 1 to smithi12223906-30 2024-09-15T03:52:33.017 INFO:tasks.rados.rados.0.smithi122.stdout:1505: finishing write tid 2 to smithi12223906-30 2024-09-15T03:52:33.017 INFO:tasks.rados.rados.0.smithi122.stdout:1506: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1505: finishing write tid 3 to smithi12223906-30 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1505: finishing write tid 4 to smithi12223906-30 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1505: finishing write tid 5 to smithi12223906-30 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 599 (ObjNum 552 snap 165 seq_num 552) dirty exists 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1505: left oid 30 (ObjNum 552 snap 165 seq_num 552) 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1507: finishing write tid 1 to smithi12223906-1 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1507: finishing write tid 2 to smithi12223906-1 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1507: finishing write tid 3 to smithi12223906-1 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1507: finishing write tid 4 to smithi12223906-1 2024-09-15T03:52:33.136 INFO:tasks.rados.rados.0.smithi122.stdout:1507: finishing write tid 5 to smithi12223906-1 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 583 (ObjNum 553 snap 165 seq_num 553) dirty exists 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1507: left oid 1 (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1502: done (6 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1503: done (5 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1504: done (4 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1505: done (3 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1506: done (2 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1507: done (1 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1508: done (0 left) 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1509: write oid 17 current snap is 165 2024-09-15T03:52:33.137 INFO:tasks.rados.rados.0.smithi122.stdout:1509: seq_num 554 ranges {511078=465362,1752436=740194,3163985=147093} 2024-09-15T03:52:33.148 INFO:tasks.rados.rados.0.smithi122.stdout:1509: writing smithi12223906-17 from 511078 to 976440 tid 1 2024-09-15T03:52:33.156 INFO:tasks.rados.rados.0.smithi122.stdout:1509: writing smithi12223906-17 from 1752436 to 2492630 tid 2 2024-09-15T03:52:33.161 INFO:tasks.rados.rados.0.smithi122.stdout:1509: writing smithi12223906-17 from 3163985 to 3311078 tid 3 2024-09-15T03:52:33.161 INFO:tasks.rados.rados.0.smithi122.stdout:1510: write (excl) oid 46 current snap is 165 2024-09-15T03:52:33.162 INFO:tasks.rados.rados.0.smithi122.stdout:1509: finishing write tid 1 to smithi12223906-17 2024-09-15T03:52:33.162 INFO:tasks.rados.rados.0.smithi122.stdout:1510: seq_num 555 ranges {527885=656964,1927271=483852,3191408=136477} 2024-09-15T03:52:33.168 INFO:tasks.rados.rados.0.smithi122.stdout:1510: writing smithi12223906-46 from 527885 to 1184849 tid 1 2024-09-15T03:52:33.175 INFO:tasks.rados.rados.0.smithi122.stdout:1510: writing smithi12223906-46 from 1927271 to 2411123 tid 2 2024-09-15T03:52:33.180 INFO:tasks.rados.rados.0.smithi122.stdout:1510: writing smithi12223906-46 from 3191408 to 3327885 tid 3 2024-09-15T03:52:33.180 INFO:tasks.rados.rados.0.smithi122.stdout:1511: write (excl) oid 6 current snap is 165 2024-09-15T03:52:33.180 INFO:tasks.rados.rados.0.smithi122.stdout:1511: seq_num 556 ranges {544692=732213,1985753=743863,3335184=9508} 2024-09-15T03:52:33.187 INFO:tasks.rados.rados.0.smithi122.stdout:1511: writing smithi12223906-6 from 544692 to 1276905 tid 1 2024-09-15T03:52:33.195 INFO:tasks.rados.rados.0.smithi122.stdout:1511: writing smithi12223906-6 from 1985753 to 2729616 tid 2 2024-09-15T03:52:33.199 INFO:tasks.rados.rados.0.smithi122.stdout:1511: writing smithi12223906-6 from 3335184 to 3344692 tid 3 2024-09-15T03:52:33.199 INFO:tasks.rados.rados.0.smithi122.stdout:1512: write (excl) oid 38 current snap is 165 2024-09-15T03:52:33.199 INFO:tasks.rados.rados.0.smithi122.stdout:1509: finishing write tid 2 to smithi12223906-17 2024-09-15T03:52:33.199 INFO:tasks.rados.rados.0.smithi122.stdout:1512: seq_num 557 ranges {561499=407462,1527882=487521,2729901=557217} 2024-09-15T03:52:33.204 INFO:tasks.rados.rados.0.smithi122.stdout:1512: writing smithi12223906-38 from 561499 to 968961 tid 1 2024-09-15T03:52:33.213 INFO:tasks.rados.rados.0.smithi122.stdout:1512: writing smithi12223906-38 from 1527882 to 2015403 tid 2 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:1512: writing smithi12223906-38 from 2729901 to 3287118 tid 3 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:1513: rollback oid 3 current snap is 165 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 164 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:1509: finishing write tid 3 to smithi12223906-17 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:1509: finishing write tid 4 to smithi12223906-17 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:1509: finishing write tid 5 to smithi12223906-17 2024-09-15T03:52:33.219 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 403 (ObjNum 554 snap 165 seq_num 554) dirty exists 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1509: left oid 17 (ObjNum 554 snap 165 seq_num 554) 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1510: finishing write tid 1 to smithi12223906-46 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1510: finishing write tid 2 to smithi12223906-46 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1510: finishing write tid 3 to smithi12223906-46 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1510: finishing write tid 4 to smithi12223906-46 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1510: finishing write tid 5 to smithi12223906-46 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 397 (ObjNum 555 snap 165 seq_num 555) dirty exists 2024-09-15T03:52:33.220 INFO:tasks.rados.rados.0.smithi122.stdout:1510: left oid 46 (ObjNum 555 snap 165 seq_num 555) 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1511: finishing write tid 1 to smithi12223906-6 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1511: finishing write tid 2 to smithi12223906-6 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1511: finishing write tid 3 to smithi12223906-6 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1511: finishing write tid 4 to smithi12223906-6 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1511: finishing write tid 5 to smithi12223906-6 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 407 (ObjNum 556 snap 165 seq_num 556) dirty exists 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1511: left oid 6 (ObjNum 556 snap 165 seq_num 556) 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1512: finishing write tid 1 to smithi12223906-38 2024-09-15T03:52:33.221 INFO:tasks.rados.rados.0.smithi122.stdout:1509: done (4 left) 2024-09-15T03:52:33.222 INFO:tasks.rados.rados.0.smithi122.stdout:1510: done (3 left) 2024-09-15T03:52:33.222 INFO:tasks.rados.rados.0.smithi122.stdout:1511: done (2 left) 2024-09-15T03:52:33.222 INFO:tasks.rados.rados.0.smithi122.stdout:1514: write (excl) oid 20 current snap is 165 2024-09-15T03:52:33.222 INFO:tasks.rados.rados.0.smithi122.stdout:1514: seq_num 558 ranges {578306=482711,1586364=631179,2757324=484842} 2024-09-15T03:52:33.224 INFO:tasks.rados.rados.0.smithi122.stdout:1514: writing smithi12223906-20 from 578306 to 1061017 tid 1 2024-09-15T03:52:33.230 INFO:tasks.rados.rados.0.smithi122.stdout:1514: writing smithi12223906-20 from 1586364 to 2217543 tid 2 2024-09-15T03:52:33.237 INFO:tasks.rados.rados.0.smithi122.stdout:1514: writing smithi12223906-20 from 2757324 to 3242166 tid 3 2024-09-15T03:52:33.237 INFO:tasks.rados.rados.0.smithi122.stdout:1515: write (excl) oid 8 current snap is 165 2024-09-15T03:52:33.237 INFO:tasks.rados.rados.0.smithi122.stdout:1512: finishing write tid 2 to smithi12223906-38 2024-09-15T03:52:33.238 INFO:tasks.rados.rados.0.smithi122.stdout:1515: seq_num 559 ranges {595113=557960,1644846=491190,2784747=610366} 2024-09-15T03:52:33.243 INFO:tasks.rados.rados.0.smithi122.stdout:1515: writing smithi12223906-8 from 595113 to 1153073 tid 1 2024-09-15T03:52:33.248 INFO:tasks.rados.rados.0.smithi122.stdout:1515: writing smithi12223906-8 from 1644846 to 2136036 tid 2 2024-09-15T03:52:33.255 INFO:tasks.rados.rados.0.smithi122.stdout:1515: writing smithi12223906-8 from 2784747 to 3395113 tid 3 2024-09-15T03:52:33.255 INFO:tasks.rados.rados.0.smithi122.stdout:1516: snap_remove snap 160 2024-09-15T03:52:33.754 INFO:tasks.rados.rados.0.smithi122.stdout:1512: finishing write tid 3 to smithi12223906-38 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1512: finishing write tid 4 to smithi12223906-38 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1512: finishing write tid 5 to smithi12223906-38 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 407 (ObjNum 557 snap 165 seq_num 557) dirty exists 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1512: left oid 38 (ObjNum 557 snap 165 seq_num 557) 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1513: finishing rollback tid 0 to smithi12223906-3 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1513: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1513: finishing rollback tid 2 to smithi12223906-3 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 764 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1514: finishing write tid 1 to smithi12223906-20 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1514: finishing write tid 2 to smithi12223906-20 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1514: finishing write tid 3 to smithi12223906-20 2024-09-15T03:52:33.755 INFO:tasks.rados.rados.0.smithi122.stdout:1514: finishing write tid 4 to smithi12223906-20 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1514: finishing write tid 5 to smithi12223906-20 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 650 (ObjNum 558 snap 165 seq_num 558) dirty exists 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1514: left oid 20 (ObjNum 558 snap 165 seq_num 558) 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: finishing write tid 1 to smithi12223906-8 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: finishing write tid 2 to smithi12223906-8 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: finishing write tid 3 to smithi12223906-8 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: finishing write tid 4 to smithi12223906-8 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: finishing write tid 5 to smithi12223906-8 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 411 (ObjNum 559 snap 165 seq_num 559) dirty exists 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: left oid 8 (ObjNum 559 snap 165 seq_num 559) 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1512: done (4 left) 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1513: done (3 left) 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1514: done (2 left) 2024-09-15T03:52:33.756 INFO:tasks.rados.rados.0.smithi122.stdout:1515: done (1 left) 2024-09-15T03:52:33.757 INFO:tasks.rados.rados.0.smithi122.stdout:1516: done (0 left) 2024-09-15T03:52:33.757 INFO:tasks.rados.rados.0.smithi122.stdout:1517: snap_create 2024-09-15T03:52:34.761 INFO:tasks.rados.rados.0.smithi122.stdout:1517: done (0 left) 2024-09-15T03:52:34.761 INFO:tasks.rados.rados.0.smithi122.stdout:1518: read oid 19 snap -1 2024-09-15T03:52:34.761 INFO:tasks.rados.rados.0.smithi122.stdout:1518: expect deleted 2024-09-15T03:52:34.761 INFO:tasks.rados.rados.0.smithi122.stdout:1519: copy_from oid 32 from oid 39 current snap is 166 2024-09-15T03:52:34.761 INFO:tasks.rados.rados.0.smithi122.stdout:1520: snap_create 2024-09-15T03:52:34.764 INFO:tasks.rados.rados.0.smithi122.stdout:1519: finishing copy_from to smithi12223906-32 2024-09-15T03:52:34.764 INFO:tasks.rados.rados.0.smithi122.stdout:1519: got expected ENOENT (src dne) 2024-09-15T03:52:34.765 INFO:tasks.rados.rados.0.smithi122.stdout:1519: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:52:35.768 INFO:tasks.rados.rados.0.smithi122.stdout:1518: done (2 left) 2024-09-15T03:52:35.769 INFO:tasks.rados.rados.0.smithi122.stdout:1519: done (1 left) 2024-09-15T03:52:35.769 INFO:tasks.rados.rados.0.smithi122.stdout:1520: done (0 left) 2024-09-15T03:52:35.769 INFO:tasks.rados.rados.0.smithi122.stdout:1521: copy_from oid 39 from oid 37 current snap is 167 2024-09-15T03:52:35.769 INFO:tasks.rados.rados.0.smithi122.stdout:1522: copy_from oid 45 from oid 3 current snap is 167 2024-09-15T03:52:35.769 INFO:tasks.rados.rados.0.smithi122.stdout:1523: write (excl) oid 15 current snap is 167 2024-09-15T03:52:35.769 INFO:tasks.rados.rados.0.smithi122.stdout:1523: seq_num 560 ranges {611920=633209,1703328=634848,2812170=507386} 2024-09-15T03:52:35.777 INFO:tasks.rados.rados.0.smithi122.stdout:1523: writing smithi12223906-15 from 611920 to 1245129 tid 1 2024-09-15T03:52:35.784 INFO:tasks.rados.rados.0.smithi122.stdout:1523: writing smithi12223906-15 from 1703328 to 2338176 tid 2 2024-09-15T03:52:35.790 INFO:tasks.rados.rados.0.smithi122.stdout:1523: writing smithi12223906-15 from 2812170 to 3319556 tid 3 2024-09-15T03:52:35.790 INFO:tasks.rados.rados.0.smithi122.stdout:1524: read oid 1 snap -1 2024-09-15T03:52:35.790 INFO:tasks.rados.rados.0.smithi122.stdout:1524: expect (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:35.790 INFO:tasks.rados.rados.0.smithi122.stdout:1525: write oid 42 current snap is 167 2024-09-15T03:52:35.790 INFO:tasks.rados.rados.0.smithi122.stdout:1522: finishing copy_from racing read to smithi12223906-45 2024-09-15T03:52:35.790 INFO:tasks.rados.rados.0.smithi122.stdout:1525: seq_num 561 ranges {628727=708458,2045457=494859,3123240=305487} 2024-09-15T03:52:35.797 INFO:tasks.rados.rados.0.smithi122.stdout:1525: writing smithi12223906-42 from 628727 to 1337185 tid 1 2024-09-15T03:52:35.804 INFO:tasks.rados.rados.0.smithi122.stdout:1525: writing smithi12223906-42 from 2045457 to 2540316 tid 2 2024-09-15T03:52:35.810 INFO:tasks.rados.rados.0.smithi122.stdout:1525: writing smithi12223906-42 from 3123240 to 3428727 tid 3 2024-09-15T03:52:35.810 INFO:tasks.rados.rados.0.smithi122.stdout:1526: rollback oid 25 current snap is 167 2024-09-15T03:52:35.810 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 164 2024-09-15T03:52:35.811 INFO:tasks.rados.rados.0.smithi122.stdout:1522: finishing copy_from to smithi12223906-45 2024-09-15T03:52:35.811 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 764 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:35.811 INFO:tasks.rados.rados.0.smithi122.stdout:1522: done (5 left) 2024-09-15T03:52:35.811 INFO:tasks.rados.rados.0.smithi122.stdout:1527: rollback oid 21 current snap is 167 2024-09-15T03:52:35.811 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 165 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1521: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1523: finishing write tid 1 to smithi12223906-15 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1521: finishing copy_from to smithi12223906-39 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 409 (ObjNum 550 snap 163 seq_num 550) dirty exists 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1523: finishing write tid 2 to smithi12223906-15 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1525: finishing write tid 1 to smithi12223906-42 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1523: finishing write tid 3 to smithi12223906-15 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1523: finishing write tid 4 to smithi12223906-15 2024-09-15T03:52:35.812 INFO:tasks.rados.rados.0.smithi122.stdout:1523: finishing write tid 5 to smithi12223906-15 2024-09-15T03:52:35.813 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 587 (ObjNum 560 snap 167 seq_num 560) dirty exists 2024-09-15T03:52:35.813 INFO:tasks.rados.rados.0.smithi122.stdout:1523: left oid 15 (ObjNum 560 snap 167 seq_num 560) 2024-09-15T03:52:35.813 INFO:tasks.rados.rados.0.smithi122.stdout:1525: finishing write tid 2 to smithi12223906-42 2024-09-15T03:52:35.813 INFO:tasks.rados.rados.0.smithi122.stdout:1524: expect (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:35.916 INFO:tasks.rados.rados.0.smithi122.stdout:1525: finishing write tid 3 to smithi12223906-42 2024-09-15T03:52:35.916 INFO:tasks.rados.rados.0.smithi122.stdout:1525: finishing write tid 4 to smithi12223906-42 2024-09-15T03:52:35.917 INFO:tasks.rados.rados.0.smithi122.stdout:1527: finishing rollback tid 0 to smithi12223906-21 2024-09-15T03:52:35.917 INFO:tasks.rados.rados.0.smithi122.stdout:1525: finishing write tid 5 to smithi12223906-42 2024-09-15T03:52:35.917 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 776 (ObjNum 561 snap 167 seq_num 561) dirty exists 2024-09-15T03:52:35.917 INFO:tasks.rados.rados.0.smithi122.stdout:1525: left oid 42 (ObjNum 561 snap 167 seq_num 561) 2024-09-15T03:52:35.917 INFO:tasks.rados.rados.0.smithi122.stdout:1527: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:52:35.918 INFO:tasks.rados.rados.0.smithi122.stdout:1526: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:52:35.918 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 0 (ObjNum 129 snap 0 seq_num 1842636960) dirty dne 2024-09-15T03:52:35.918 INFO:tasks.rados.rados.0.smithi122.stdout:1527: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:52:35.918 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 415 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:35.918 INFO:tasks.rados.rados.0.smithi122.stdout:1521: done (5 left) 2024-09-15T03:52:35.918 INFO:tasks.rados.rados.0.smithi122.stdout:1523: done (4 left) 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1524: done (3 left) 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1525: done (2 left) 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1526: done (1 left) 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1527: done (0 left) 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1528: read oid 36 snap -1 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1528: expect deleted 2024-09-15T03:52:35.919 INFO:tasks.rados.rados.0.smithi122.stdout:1529: copy_from oid 29 from oid 41 current snap is 167 2024-09-15T03:52:35.920 INFO:tasks.rados.rados.0.smithi122.stdout:1530: snap_remove snap 166 2024-09-15T03:52:36.793 INFO:tasks.rados.rados.0.smithi122.stdout:1530: done (2 left) 2024-09-15T03:52:36.793 INFO:tasks.rados.rados.0.smithi122.stdout:1531: delete oid 3 current snap is 167 2024-09-15T03:52:36.794 INFO:tasks.rados.rados.0.smithi122.stdout:1529: finishing copy_from to smithi12223906-29 2024-09-15T03:52:36.794 INFO:tasks.rados.rados.0.smithi122.stdout:1529: got expected ENOENT (src dne) 2024-09-15T03:52:36.794 INFO:tasks.rados.rados.0.smithi122.stdout:1529: finishing copy_from racing read to smithi12223906-29 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:1528: done (2 left) 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:1529: done (1 left) 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:1531: done (0 left) 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:1532: rollback oid 21 current snap is 167 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 164 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:1533: write (excl) oid 5 current snap is 167 2024-09-15T03:52:36.801 INFO:tasks.rados.rados.0.smithi122.stdout:1533: seq_num 563 ranges {662341=575309,1878774=498528,2894439=457555} 2024-09-15T03:52:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1533: writing smithi12223906-5 from 662341 to 1237650 tid 1 2024-09-15T03:52:36.814 INFO:tasks.rados.rados.0.smithi122.stdout:1533: writing smithi12223906-5 from 1878774 to 2377302 tid 2 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:1533: writing smithi12223906-5 from 2894439 to 3351994 tid 3 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:1534: copy_from oid 12 from oid 44 current snap is 167 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:1532: finishing rollback tid 0 to smithi12223906-21 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:1532: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:1532: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 418 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:36.820 INFO:tasks.rados.rados.0.smithi122.stdout:1533: finishing write tid 1 to smithi12223906-5 2024-09-15T03:52:36.821 INFO:tasks.rados.rados.0.smithi122.stdout:1533: finishing write tid 2 to smithi12223906-5 2024-09-15T03:52:36.821 INFO:tasks.rados.rados.0.smithi122.stdout:1532: done (2 left) 2024-09-15T03:52:36.821 INFO:tasks.rados.rados.0.smithi122.stdout:1535: snap_remove snap 164 2024-09-15T03:52:37.798 INFO:tasks.rados.rados.0.smithi122.stdout:1533: finishing write tid 3 to smithi12223906-5 2024-09-15T03:52:37.798 INFO:tasks.rados.rados.0.smithi122.stdout:1533: finishing write tid 4 to smithi12223906-5 2024-09-15T03:52:37.798 INFO:tasks.rados.rados.0.smithi122.stdout:1533: finishing write tid 5 to smithi12223906-5 2024-09-15T03:52:37.798 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 642 (ObjNum 563 snap 167 seq_num 563) dirty exists 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1533: left oid 5 (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1534: finishing copy_from racing read to smithi12223906-12 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1534: finishing copy_from to smithi12223906-12 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 712 (ObjNum 547 snap 161 seq_num 547) dirty exists 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1533: done (2 left) 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1534: done (1 left) 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1535: done (0 left) 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1536: write (excl) oid 49 current snap is 167 2024-09-15T03:52:37.799 INFO:tasks.rados.rados.0.smithi122.stdout:1536: seq_num 564 ranges {679148=650558,1937256=642186,3321862=157286} 2024-09-15T03:52:37.810 INFO:tasks.rados.rados.0.smithi122.stdout:1536: writing smithi12223906-49 from 679148 to 1329706 tid 1 2024-09-15T03:52:37.818 INFO:tasks.rados.rados.0.smithi122.stdout:1536: writing smithi12223906-49 from 1937256 to 2579442 tid 2 2024-09-15T03:52:37.823 INFO:tasks.rados.rados.0.smithi122.stdout:1536: writing smithi12223906-49 from 3321862 to 3479148 tid 3 2024-09-15T03:52:37.823 INFO:tasks.rados.rados.0.smithi122.stdout:1537: read oid 42 snap -1 2024-09-15T03:52:37.823 INFO:tasks.rados.rados.0.smithi122.stdout:1537: expect (ObjNum 561 snap 167 seq_num 561) 2024-09-15T03:52:37.824 INFO:tasks.rados.rados.0.smithi122.stdout:1538: write oid 25 current snap is 167 2024-09-15T03:52:37.824 INFO:tasks.rados.rados.0.smithi122.stdout:1536: finishing write tid 1 to smithi12223906-49 2024-09-15T03:52:37.824 INFO:tasks.rados.rados.0.smithi122.stdout:1536: finishing write tid 2 to smithi12223906-49 2024-09-15T03:52:37.824 INFO:tasks.rados.rados.0.smithi122.stdout:1538: seq_num 565 ranges {695955=725807,1879385=502197,2949285=480099} 2024-09-15T03:52:37.832 INFO:tasks.rados.rados.0.smithi122.stdout:1538: writing smithi12223906-25 from 695955 to 1421762 tid 1 2024-09-15T03:52:37.837 INFO:tasks.rados.rados.0.smithi122.stdout:1538: writing smithi12223906-25 from 1879385 to 2381582 tid 2 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1538: writing smithi12223906-25 from 2949285 to 3429384 tid 3 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1539: read oid 2 snap -1 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1539: expect (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1540: read oid 30 snap 165 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1540: expect (ObjNum 552 snap 165 seq_num 552) 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1541: read oid 29 snap 165 2024-09-15T03:52:37.844 INFO:tasks.rados.rados.0.smithi122.stdout:1541: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1542: copy_from oid 27 from oid 16 current snap is 167 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1543: copy_from oid 37 from oid 35 current snap is 167 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1536: finishing write tid 3 to smithi12223906-49 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1536: finishing write tid 4 to smithi12223906-49 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1536: finishing write tid 5 to smithi12223906-49 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 408 (ObjNum 564 snap 167 seq_num 564) dirty exists 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1536: left oid 49 (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:52:37.845 INFO:tasks.rados.rados.0.smithi122.stdout:1537: expect (ObjNum 561 snap 167 seq_num 561) 2024-09-15T03:52:38.107 INFO:tasks.rados.rados.0.smithi122.stdout:1538: finishing write tid 1 to smithi12223906-25 2024-09-15T03:52:38.107 INFO:tasks.rados.rados.0.smithi122.stdout:1538: finishing write tid 2 to smithi12223906-25 2024-09-15T03:52:38.107 INFO:tasks.rados.rados.0.smithi122.stdout:1536: done (7 left) 2024-09-15T03:52:38.107 INFO:tasks.rados.rados.0.smithi122.stdout:1537: done (6 left) 2024-09-15T03:52:38.107 INFO:tasks.rados.rados.0.smithi122.stdout:1544: read oid 6 snap -1 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1544: expect (ObjNum 556 snap 165 seq_num 556) 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1545: delete oid 11 current snap is 167 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1538: finishing write tid 3 to smithi12223906-25 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1538: finishing write tid 4 to smithi12223906-25 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1538: finishing write tid 5 to smithi12223906-25 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 789 (ObjNum 565 snap 167 seq_num 565) dirty exists 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1538: left oid 25 (ObjNum 565 snap 167 seq_num 565) 2024-09-15T03:52:38.108 INFO:tasks.rados.rados.0.smithi122.stdout:1541: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:38.231 INFO:tasks.rados.rados.0.smithi122.stdout:1539: expect (ObjNum 507 snap 151 seq_num 507) 2024-09-15T03:52:38.357 INFO:tasks.rados.rados.0.smithi122.stdout:1540: expect (ObjNum 552 snap 165 seq_num 552) 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1543: finishing copy_from racing read to smithi12223906-37 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1543: finishing copy_from to smithi12223906-37 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 570 (ObjNum 538 snap 160 seq_num 538) dirty exists 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1542: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1542: finishing copy_from to smithi12223906-27 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 567 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1538: done (7 left) 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1539: done (6 left) 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1540: done (5 left) 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1541: done (4 left) 2024-09-15T03:52:38.455 INFO:tasks.rados.rados.0.smithi122.stdout:1542: done (3 left) 2024-09-15T03:52:38.456 INFO:tasks.rados.rados.0.smithi122.stdout:1543: done (2 left) 2024-09-15T03:52:38.456 INFO:tasks.rados.rados.0.smithi122.stdout:1545: done (1 left) 2024-09-15T03:52:38.456 INFO:tasks.rados.rados.0.smithi122.stdout:1546: write oid 50 current snap is 167 2024-09-15T03:52:38.456 INFO:tasks.rados.rados.0.smithi122.stdout:1544: expect (ObjNum 556 snap 165 seq_num 556) 2024-09-15T03:52:38.690 INFO:tasks.rados.rados.0.smithi122.stdout:1546: seq_num 567 ranges {729569=476305,1996349=505866,3004131=525438} 2024-09-15T03:52:38.695 INFO:tasks.rados.rados.0.smithi122.stdout:1546: writing smithi12223906-50 from 729569 to 1205874 tid 1 2024-09-15T03:52:38.703 INFO:tasks.rados.rados.0.smithi122.stdout:1546: writing smithi12223906-50 from 1996349 to 2502215 tid 2 2024-09-15T03:52:38.708 INFO:tasks.rados.rados.0.smithi122.stdout:1546: writing smithi12223906-50 from 3004131 to 3529569 tid 3 2024-09-15T03:52:38.708 INFO:tasks.rados.rados.0.smithi122.stdout:1544: done (1 left) 2024-09-15T03:52:38.708 INFO:tasks.rados.rados.0.smithi122.stdout:1547: read oid 7 snap -1 2024-09-15T03:52:38.708 INFO:tasks.rados.rados.0.smithi122.stdout:1547: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:52:38.708 INFO:tasks.rados.rados.0.smithi122.stdout:1548: snap_create 2024-09-15T03:52:38.708 INFO:tasks.rados.rados.0.smithi122.stdout:1546: finishing write tid 1 to smithi12223906-50 2024-09-15T03:52:38.709 INFO:tasks.rados.rados.0.smithi122.stdout:1546: finishing write tid 2 to smithi12223906-50 2024-09-15T03:52:38.715 INFO:tasks.rados.rados.0.smithi122.stdout:1546: finishing write tid 3 to smithi12223906-50 2024-09-15T03:52:38.715 INFO:tasks.rados.rados.0.smithi122.stdout:1546: finishing write tid 4 to smithi12223906-50 2024-09-15T03:52:38.715 INFO:tasks.rados.rados.0.smithi122.stdout:1547: expect (ObjNum 318 snap 96 seq_num 318) 2024-09-15T03:52:38.916 INFO:tasks.rados.rados.0.smithi122.stdout:1546: finishing write tid 5 to smithi12223906-50 2024-09-15T03:52:38.916 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 751 (ObjNum 567 snap 167 seq_num 567) dirty exists 2024-09-15T03:52:38.917 INFO:tasks.rados.rados.0.smithi122.stdout:1546: left oid 50 (ObjNum 567 snap 167 seq_num 567) 2024-09-15T03:52:38.917 INFO:tasks.rados.rados.0.smithi122.stdout:1546: done (2 left) 2024-09-15T03:52:38.917 INFO:tasks.rados.rados.0.smithi122.stdout:1547: done (1 left) 2024-09-15T03:52:38.917 INFO:tasks.rados.rados.0.smithi122.stdout:1548: done (0 left) 2024-09-15T03:52:38.917 INFO:tasks.rados.rados.0.smithi122.stdout:1549: delete oid 23 current snap is 168 2024-09-15T03:52:38.924 INFO:tasks.rados.rados.0.smithi122.stdout:1549: done (0 left) 2024-09-15T03:52:38.924 INFO:tasks.rados.rados.0.smithi122.stdout:1550: read oid 37 snap 165 2024-09-15T03:52:38.924 INFO:tasks.rados.rados.0.smithi122.stdout:1550: expect (ObjNum 550 snap 163 seq_num 550) 2024-09-15T03:52:38.924 INFO:tasks.rados.rados.0.smithi122.stdout:1551: read oid 22 snap -1 2024-09-15T03:52:38.925 INFO:tasks.rados.rados.0.smithi122.stdout:1551: expect deleted 2024-09-15T03:52:38.925 INFO:tasks.rados.rados.0.smithi122.stdout:1552: snap_remove snap 167 2024-09-15T03:52:39.826 INFO:tasks.rados.rados.0.smithi122.stdout:1552: done (2 left) 2024-09-15T03:52:39.826 INFO:tasks.rados.rados.0.smithi122.stdout:1553: write (excl) oid 30 current snap is 168 2024-09-15T03:52:39.826 INFO:tasks.rados.rados.0.smithi122.stdout:1553: seq_num 569 ranges {763183=626803,1996960=793182,3226271=336912} 2024-09-15T03:52:39.833 INFO:tasks.rados.rados.0.smithi122.stdout:1553: writing smithi12223906-30 from 763183 to 1389986 tid 1 2024-09-15T03:52:39.842 INFO:tasks.rados.rados.0.smithi122.stdout:1553: writing smithi12223906-30 from 1996960 to 2790142 tid 2 2024-09-15T03:52:39.846 INFO:tasks.rados.rados.0.smithi122.stdout:1553: writing smithi12223906-30 from 3226271 to 3563183 tid 3 2024-09-15T03:52:39.846 INFO:tasks.rados.rados.0.smithi122.stdout:1554: copy_from oid 7 from oid 8 current snap is 168 2024-09-15T03:52:39.846 INFO:tasks.rados.rados.0.smithi122.stdout:1550: expect (ObjNum 550 snap 163 seq_num 550) 2024-09-15T03:52:39.953 INFO:tasks.rados.rados.0.smithi122.stdout:1553: finishing write tid 1 to smithi12223906-30 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1553: finishing write tid 2 to smithi12223906-30 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1553: finishing write tid 3 to smithi12223906-30 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1553: finishing write tid 4 to smithi12223906-30 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1553: finishing write tid 5 to smithi12223906-30 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 603 (ObjNum 569 snap 168 seq_num 569) dirty exists 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1553: left oid 30 (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1554: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1554: finishing copy_from to smithi12223906-7 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 411 (ObjNum 559 snap 165 seq_num 559) dirty exists 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1550: done (3 left) 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1551: done (2 left) 2024-09-15T03:52:39.954 INFO:tasks.rados.rados.0.smithi122.stdout:1553: done (1 left) 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1554: done (0 left) 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1555: read oid 23 snap -1 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1555: expect deleted 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1556: rollback oid 41 current snap is 168 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 165 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1557: rollback oid 38 current snap is 168 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 165 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1558: snap_create 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:1556: finishing rollback tid 1 to smithi12223906-41 2024-09-15T03:52:39.955 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 0 (ObjNum 305 snap 0 seq_num 1) dirty dne 2024-09-15T03:52:39.962 INFO:tasks.rados.rados.0.smithi122.stdout:1557: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:52:39.962 INFO:tasks.rados.rados.0.smithi122.stdout:1557: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:52:39.963 INFO:tasks.rados.rados.0.smithi122.stdout:1557: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:52:39.963 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 414 (ObjNum 557 snap 165 seq_num 557) dirty exists 2024-09-15T03:52:40.833 INFO:tasks.rados.rados.0.smithi122.stdout:1555: done (3 left) 2024-09-15T03:52:40.833 INFO:tasks.rados.rados.0.smithi122.stdout:1556: done (2 left) 2024-09-15T03:52:40.833 INFO:tasks.rados.rados.0.smithi122.stdout:1557: done (1 left) 2024-09-15T03:52:40.834 INFO:tasks.rados.rados.0.smithi122.stdout:1558: done (0 left) 2024-09-15T03:52:40.834 INFO:tasks.rados.rados.0.smithi122.stdout:1559: delete oid 8 current snap is 169 2024-09-15T03:52:40.842 INFO:tasks.rados.rados.0.smithi122.stdout:1559: done (0 left) 2024-09-15T03:52:40.842 INFO:tasks.rados.rados.0.smithi122.stdout:1560: read oid 9 snap -1 2024-09-15T03:52:40.842 INFO:tasks.rados.rados.0.smithi122.stdout:1560: expect (ObjNum 534 snap 160 seq_num 534) 2024-09-15T03:52:40.842 INFO:tasks.rados.rados.0.smithi122.stdout:1561: copy_from oid 19 from oid 32 current snap is 169 2024-09-15T03:52:40.842 INFO:tasks.rados.rados.0.smithi122.stdout:1562: read oid 31 snap 165 2024-09-15T03:52:40.843 INFO:tasks.rados.rados.0.smithi122.stdout:1562: expect (ObjNum 548 snap 162 seq_num 548) 2024-09-15T03:52:40.843 INFO:tasks.rados.rados.0.smithi122.stdout:1563: snap_remove snap 168 2024-09-15T03:52:41.724 INFO:tasks.rados.rados.0.smithi122.stdout:1560: expect (ObjNum 534 snap 160 seq_num 534) 2024-09-15T03:52:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:1561: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:52:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:1561: finishing copy_from to smithi12223906-19 2024-09-15T03:52:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 674 (ObjNum 505 snap 151 seq_num 505) dirty exists 2024-09-15T03:52:41.905 INFO:tasks.rados.rados.0.smithi122.stdout:1562: expect (ObjNum 548 snap 162 seq_num 548) 2024-09-15T03:52:42.101 INFO:tasks.rados.rados.0.smithi122.stdout:1560: done (3 left) 2024-09-15T03:52:42.101 INFO:tasks.rados.rados.0.smithi122.stdout:1561: done (2 left) 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1562: done (1 left) 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1563: done (0 left) 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1564: read oid 49 snap -1 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1564: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1565: read oid 27 snap -1 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1565: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1566: read oid 19 snap 165 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1566: expect deleted 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1567: read oid 24 snap -1 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1567: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1568: write oid 46 current snap is 169 2024-09-15T03:52:42.102 INFO:tasks.rados.rados.0.smithi122.stdout:1568: seq_num 571 ranges {796797=493654,1830277=796851,3397470=199327} 2024-09-15T03:52:42.117 INFO:tasks.rados.rados.0.smithi122.stdout:1568: writing smithi12223906-46 from 796797 to 1290451 tid 1 2024-09-15T03:52:42.124 INFO:tasks.rados.rados.0.smithi122.stdout:1568: writing smithi12223906-46 from 1830277 to 2627128 tid 2 2024-09-15T03:52:42.130 INFO:tasks.rados.rados.0.smithi122.stdout:1568: writing smithi12223906-46 from 3397470 to 3596797 tid 3 2024-09-15T03:52:42.130 INFO:tasks.rados.rados.0.smithi122.stdout:1569: read oid 30 snap -1 2024-09-15T03:52:42.130 INFO:tasks.rados.rados.0.smithi122.stdout:1569: expect (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:42.130 INFO:tasks.rados.rados.0.smithi122.stdout:1570: write oid 31 current snap is 169 2024-09-15T03:52:42.130 INFO:tasks.rados.rados.0.smithi122.stdout:1570: seq_num 572 ranges {413604=568903,1488759=656862,2624893=591709} 2024-09-15T03:52:42.135 INFO:tasks.rados.rados.0.smithi122.stdout:1570: writing smithi12223906-31 from 413604 to 982507 tid 1 2024-09-15T03:52:42.142 INFO:tasks.rados.rados.0.smithi122.stdout:1570: writing smithi12223906-31 from 1488759 to 2145621 tid 2 2024-09-15T03:52:42.148 INFO:tasks.rados.rados.0.smithi122.stdout:1570: writing smithi12223906-31 from 2624893 to 3216602 tid 3 2024-09-15T03:52:42.148 INFO:tasks.rados.rados.0.smithi122.stdout:1571: snap_create 2024-09-15T03:52:42.148 INFO:tasks.rados.rados.0.smithi122.stdout:1564: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:52:42.356 INFO:tasks.rados.rados.0.smithi122.stdout:1565: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:52:42.529 INFO:tasks.rados.rados.0.smithi122.stdout:1568: finishing write tid 1 to smithi12223906-46 2024-09-15T03:52:42.529 INFO:tasks.rados.rados.0.smithi122.stdout:1567: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:52:42.750 INFO:tasks.rados.rados.0.smithi122.stdout:1568: finishing write tid 2 to smithi12223906-46 2024-09-15T03:52:42.750 INFO:tasks.rados.rados.0.smithi122.stdout:1568: finishing write tid 3 to smithi12223906-46 2024-09-15T03:52:42.750 INFO:tasks.rados.rados.0.smithi122.stdout:1568: finishing write tid 4 to smithi12223906-46 2024-09-15T03:52:42.750 INFO:tasks.rados.rados.0.smithi122.stdout:1568: finishing write tid 5 to smithi12223906-46 2024-09-15T03:52:42.751 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 768 (ObjNum 571 snap 169 seq_num 571) dirty exists 2024-09-15T03:52:42.751 INFO:tasks.rados.rados.0.smithi122.stdout:1568: left oid 46 (ObjNum 571 snap 169 seq_num 571) 2024-09-15T03:52:42.751 INFO:tasks.rados.rados.0.smithi122.stdout:1569: expect (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1570: finishing write tid 1 to smithi12223906-31 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1570: finishing write tid 2 to smithi12223906-31 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1570: finishing write tid 3 to smithi12223906-31 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1570: finishing write tid 4 to smithi12223906-31 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1570: finishing write tid 5 to smithi12223906-31 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 238 (ObjNum 572 snap 169 seq_num 572) dirty exists 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1570: left oid 31 (ObjNum 572 snap 169 seq_num 572) 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1564: done (7 left) 2024-09-15T03:52:42.903 INFO:tasks.rados.rados.0.smithi122.stdout:1565: done (6 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1566: done (5 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1567: done (4 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1568: done (3 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1569: done (2 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1570: done (1 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1571: done (0 left) 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1572: write oid 19 current snap is 170 2024-09-15T03:52:42.904 INFO:tasks.rados.rados.0.smithi122.stdout:1572: seq_num 573 ranges {430411=644152,1830888=400520,2935963=402981} 2024-09-15T03:52:42.914 INFO:tasks.rados.rados.0.smithi122.stdout:1572: writing smithi12223906-19 from 430411 to 1074563 tid 1 2024-09-15T03:52:42.921 INFO:tasks.rados.rados.0.smithi122.stdout:1572: writing smithi12223906-19 from 1830888 to 2231408 tid 2 2024-09-15T03:52:42.927 INFO:tasks.rados.rados.0.smithi122.stdout:1572: writing smithi12223906-19 from 2935963 to 3338944 tid 3 2024-09-15T03:52:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:1573: write oid 17 current snap is 170 2024-09-15T03:52:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:1572: finishing write tid 1 to smithi12223906-19 2024-09-15T03:52:42.928 INFO:tasks.rados.rados.0.smithi122.stdout:1573: seq_num 574 ranges {447218=719401,1889370=660531,2963386=614253} 2024-09-15T03:52:42.934 INFO:tasks.rados.rados.0.smithi122.stdout:1573: writing smithi12223906-17 from 447218 to 1166619 tid 1 2024-09-15T03:52:42.943 INFO:tasks.rados.rados.0.smithi122.stdout:1573: writing smithi12223906-17 from 1889370 to 2549901 tid 2 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1573: writing smithi12223906-17 from 2963386 to 3577639 tid 3 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1574: snap_create 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1572: finishing write tid 2 to smithi12223906-19 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1572: finishing write tid 3 to smithi12223906-19 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1572: finishing write tid 4 to smithi12223906-19 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1572: finishing write tid 5 to smithi12223906-19 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 772 (ObjNum 573 snap 170 seq_num 573) dirty exists 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1572: left oid 19 (ObjNum 573 snap 170 seq_num 573) 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1573: finishing write tid 1 to smithi12223906-17 2024-09-15T03:52:42.949 INFO:tasks.rados.rados.0.smithi122.stdout:1573: finishing write tid 2 to smithi12223906-17 2024-09-15T03:52:42.953 INFO:tasks.rados.rados.0.smithi122.stdout:1573: finishing write tid 3 to smithi12223906-17 2024-09-15T03:52:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:1573: finishing write tid 4 to smithi12223906-17 2024-09-15T03:52:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:1573: finishing write tid 5 to smithi12223906-17 2024-09-15T03:52:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 413 (ObjNum 574 snap 170 seq_num 574) dirty exists 2024-09-15T03:52:42.954 INFO:tasks.rados.rados.0.smithi122.stdout:1573: left oid 17 (ObjNum 574 snap 170 seq_num 574) 2024-09-15T03:52:43.867 INFO:tasks.rados.rados.0.smithi122.stdout:1572: done (2 left) 2024-09-15T03:52:43.868 INFO:tasks.rados.rados.0.smithi122.stdout:1573: done (1 left) 2024-09-15T03:52:43.868 INFO:tasks.rados.rados.0.smithi122.stdout:1574: done (0 left) 2024-09-15T03:52:43.868 INFO:tasks.rados.rados.0.smithi122.stdout:1575: snap_create 2024-09-15T03:52:44.872 INFO:tasks.rados.rados.0.smithi122.stdout:1575: done (0 left) 2024-09-15T03:52:44.872 INFO:tasks.rados.rados.0.smithi122.stdout:1576: rollback oid 37 current snap is 172 2024-09-15T03:52:44.872 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 37 to 165 2024-09-15T03:52:44.872 INFO:tasks.rados.rados.0.smithi122.stdout:1577: delete oid 33 current snap is 172 2024-09-15T03:52:44.877 INFO:tasks.rados.rados.0.smithi122.stdout:1577: done (1 left) 2024-09-15T03:52:44.877 INFO:tasks.rados.rados.0.smithi122.stdout:1578: snap_remove snap 169 2024-09-15T03:52:45.890 INFO:tasks.rados.rados.0.smithi122.stdout:1578: done (1 left) 2024-09-15T03:52:45.891 INFO:tasks.rados.rados.0.smithi122.stdout:1579: read oid 29 snap -1 2024-09-15T03:52:45.891 INFO:tasks.rados.rados.0.smithi122.stdout:1579: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:45.891 INFO:tasks.rados.rados.0.smithi122.stdout:1580: write (excl) oid 12 current snap is 172 2024-09-15T03:52:45.891 INFO:tasks.rados.rados.0.smithi122.stdout:1576: finishing rollback tid 0 to smithi12223906-37 2024-09-15T03:52:45.891 INFO:tasks.rados.rados.0.smithi122.stdout:1580: seq_num 576 ranges {480832=469899,1606334=664200,3018232=662600} 2024-09-15T03:52:45.901 INFO:tasks.rados.rados.0.smithi122.stdout:1580: writing smithi12223906-12 from 480832 to 950731 tid 1 2024-09-15T03:52:45.909 INFO:tasks.rados.rados.0.smithi122.stdout:1580: writing smithi12223906-12 from 1606334 to 2270534 tid 2 2024-09-15T03:52:45.917 INFO:tasks.rados.rados.0.smithi122.stdout:1580: writing smithi12223906-12 from 3018232 to 3680832 tid 3 2024-09-15T03:52:45.917 INFO:tasks.rados.rados.0.smithi122.stdout:1581: delete oid 41 current snap is 172 2024-09-15T03:52:45.917 INFO:tasks.rados.rados.0.smithi122.stdout:1576: finishing rollback tid 1 to smithi12223906-37 2024-09-15T03:52:45.917 INFO:tasks.rados.rados.0.smithi122.stdout:1576: finishing rollback tid 2 to smithi12223906-37 2024-09-15T03:52:45.917 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 573 (ObjNum 550 snap 163 seq_num 550) dirty exists 2024-09-15T03:52:45.918 INFO:tasks.rados.rados.0.smithi122.stdout:1579: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:46.048 INFO:tasks.rados.rados.0.smithi122.stdout:1580: finishing write tid 1 to smithi12223906-12 2024-09-15T03:52:46.048 INFO:tasks.rados.rados.0.smithi122.stdout:1580: finishing write tid 2 to smithi12223906-12 2024-09-15T03:52:46.048 INFO:tasks.rados.rados.0.smithi122.stdout:1580: finishing write tid 3 to smithi12223906-12 2024-09-15T03:52:46.048 INFO:tasks.rados.rados.0.smithi122.stdout:1580: finishing write tid 4 to smithi12223906-12 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1580: finishing write tid 5 to smithi12223906-12 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 716 (ObjNum 576 snap 172 seq_num 576) dirty exists 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1580: left oid 12 (ObjNum 576 snap 172 seq_num 576) 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1576: done (3 left) 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1579: done (2 left) 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1580: done (1 left) 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1581: done (0 left) 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1582: write (excl) oid 23 current snap is 172 2024-09-15T03:52:46.049 INFO:tasks.rados.rados.0.smithi122.stdout:1582: seq_num 578 ranges {514446=736750,1723298=667869,3073078=641368} 2024-09-15T03:52:46.063 INFO:tasks.rados.rados.0.smithi122.stdout:1582: writing smithi12223906-23 from 514446 to 1251196 tid 1 2024-09-15T03:52:46.070 INFO:tasks.rados.rados.0.smithi122.stdout:1582: writing smithi12223906-23 from 1723298 to 2391167 tid 2 2024-09-15T03:52:46.077 INFO:tasks.rados.rados.0.smithi122.stdout:1582: writing smithi12223906-23 from 3073078 to 3714446 tid 3 2024-09-15T03:52:46.078 INFO:tasks.rados.rados.0.smithi122.stdout:1583: write oid 22 current snap is 172 2024-09-15T03:52:46.078 INFO:tasks.rados.rados.0.smithi122.stdout:1583: seq_num 579 ranges {531253=411999,1381780=411527,2300501=586966,3376974=354279} 2024-09-15T03:52:46.082 INFO:tasks.rados.rados.0.smithi122.stdout:1583: writing smithi12223906-22 from 531253 to 943252 tid 1 2024-09-15T03:52:46.087 INFO:tasks.rados.rados.0.smithi122.stdout:1583: writing smithi12223906-22 from 1381780 to 1793307 tid 2 2024-09-15T03:52:46.094 INFO:tasks.rados.rados.0.smithi122.stdout:1583: writing smithi12223906-22 from 2300501 to 2887467 tid 3 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1583: writing smithi12223906-22 from 3376974 to 3731253 tid 4 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1584: rollback oid 2 current snap is 172 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 170 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1582: finishing write tid 1 to smithi12223906-23 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1582: finishing write tid 2 to smithi12223906-23 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1585: copy_from oid 37 from oid 42 current snap is 172 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1582: finishing write tid 3 to smithi12223906-23 2024-09-15T03:52:46.099 INFO:tasks.rados.rados.0.smithi122.stdout:1582: finishing write tid 4 to smithi12223906-23 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1582: finishing write tid 5 to smithi12223906-23 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 763 (ObjNum 578 snap 172 seq_num 578) dirty exists 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1582: left oid 23 (ObjNum 578 snap 172 seq_num 578) 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1583: finishing write tid 1 to smithi12223906-22 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1583: finishing write tid 2 to smithi12223906-22 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1583: finishing write tid 3 to smithi12223906-22 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1582: done (3 left) 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1586: write (excl) oid 35 current snap is 172 2024-09-15T03:52:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:1586: seq_num 580 ranges {548060=487248,1440262=671538,2844277=798238} 2024-09-15T03:52:46.104 INFO:tasks.rados.rados.0.smithi122.stdout:1586: writing smithi12223906-35 from 548060 to 1035308 tid 1 2024-09-15T03:52:46.111 INFO:tasks.rados.rados.0.smithi122.stdout:1586: writing smithi12223906-35 from 1440262 to 2111800 tid 2 2024-09-15T03:52:46.121 INFO:tasks.rados.rados.0.smithi122.stdout:1586: writing smithi12223906-35 from 2844277 to 3642515 tid 3 2024-09-15T03:52:46.121 INFO:tasks.rados.rados.0.smithi122.stdout:1587: rollback oid 3 current snap is 172 2024-09-15T03:52:46.121 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 165 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:1583: finishing write tid 4 to smithi12223906-22 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:1583: finishing write tid 5 to smithi12223906-22 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:1583: finishing write tid 6 to smithi12223906-22 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 768 (ObjNum 579 snap 172 seq_num 579) dirty exists 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:1583: left oid 22 (ObjNum 579 snap 172 seq_num 579) 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:1583: done (4 left) 2024-09-15T03:52:46.122 INFO:tasks.rados.rados.0.smithi122.stdout:1588: snap_remove snap 171 2024-09-15T03:52:46.902 INFO:tasks.rados.rados.0.smithi122.stdout:1588: done (4 left) 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1589: snap_create 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1584: finishing rollback tid 0 to smithi12223906-2 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1584: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1584: finishing rollback tid 2 to smithi12223906-2 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 419 (ObjNum 507 snap 151 seq_num 507) dirty exists 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1586: finishing write tid 1 to smithi12223906-35 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1585: finishing copy_from racing read to smithi12223906-37 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1586: finishing write tid 2 to smithi12223906-35 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1585: finishing copy_from to smithi12223906-37 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 776 (ObjNum 561 snap 167 seq_num 561) dirty exists 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1587: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:52:46.903 INFO:tasks.rados.rados.0.smithi122.stdout:1587: finishing rollback tid 2 to smithi12223906-3 2024-09-15T03:52:46.904 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 791 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:46.904 INFO:tasks.rados.rados.0.smithi122.stdout:1586: finishing write tid 3 to smithi12223906-35 2024-09-15T03:52:46.904 INFO:tasks.rados.rados.0.smithi122.stdout:1586: finishing write tid 4 to smithi12223906-35 2024-09-15T03:52:46.904 INFO:tasks.rados.rados.0.smithi122.stdout:1586: finishing write tid 5 to smithi12223906-35 2024-09-15T03:52:46.904 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 574 (ObjNum 580 snap 172 seq_num 580) dirty exists 2024-09-15T03:52:46.904 INFO:tasks.rados.rados.0.smithi122.stdout:1586: left oid 35 (ObjNum 580 snap 172 seq_num 580) 2024-09-15T03:52:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:1584: done (4 left) 2024-09-15T03:52:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:1585: done (3 left) 2024-09-15T03:52:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:1586: done (2 left) 2024-09-15T03:52:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:1587: done (1 left) 2024-09-15T03:52:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:1589: done (0 left) 2024-09-15T03:52:47.909 INFO:tasks.rados.rados.0.smithi122.stdout:1590: snap_remove snap 165 2024-09-15T03:52:48.922 INFO:tasks.rados.rados.0.smithi122.stdout:1590: done (0 left) 2024-09-15T03:52:48.922 INFO:tasks.rados.rados.0.smithi122.stdout:1591: read oid 29 snap 172 2024-09-15T03:52:48.922 INFO:tasks.rados.rados.0.smithi122.stdout:1591: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:48.922 INFO:tasks.rados.rados.0.smithi122.stdout:1592: copy_from oid 22 from oid 11 current snap is 173 2024-09-15T03:52:48.922 INFO:tasks.rados.rados.0.smithi122.stdout:1593: snap_create 2024-09-15T03:52:48.929 INFO:tasks.rados.rados.0.smithi122.stdout:1591: expect (ObjNum 525 snap 158 seq_num 525) 2024-09-15T03:52:49.056 INFO:tasks.rados.rados.0.smithi122.stdout:1592: finishing copy_from to smithi12223906-22 2024-09-15T03:52:49.056 INFO:tasks.rados.rados.0.smithi122.stdout:1592: got expected ENOENT (src dne) 2024-09-15T03:52:49.056 INFO:tasks.rados.rados.0.smithi122.stdout:1592: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:52:49.921 INFO:tasks.rados.rados.0.smithi122.stdout:1591: done (2 left) 2024-09-15T03:52:49.921 INFO:tasks.rados.rados.0.smithi122.stdout:1592: done (1 left) 2024-09-15T03:52:49.921 INFO:tasks.rados.rados.0.smithi122.stdout:1593: done (0 left) 2024-09-15T03:52:49.921 INFO:tasks.rados.rados.0.smithi122.stdout:1594: rollback oid 24 current snap is 174 2024-09-15T03:52:49.921 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 172 2024-09-15T03:52:49.921 INFO:tasks.rados.rados.0.smithi122.stdout:1595: write oid 29 current snap is 174 2024-09-15T03:52:49.922 INFO:tasks.rados.rados.0.smithi122.stdout:1595: seq_num 581 ranges {564867=562497,1898744=415196,2755347=725863} 2024-09-15T03:52:49.929 INFO:tasks.rados.rados.0.smithi122.stdout:1595: writing smithi12223906-29 from 564867 to 1127364 tid 1 2024-09-15T03:52:49.936 INFO:tasks.rados.rados.0.smithi122.stdout:1595: writing smithi12223906-29 from 1898744 to 2313940 tid 2 2024-09-15T03:52:49.944 INFO:tasks.rados.rados.0.smithi122.stdout:1595: writing smithi12223906-29 from 2755347 to 3481210 tid 3 2024-09-15T03:52:49.944 INFO:tasks.rados.rados.0.smithi122.stdout:1596: copy_from oid 15 from oid 13 current snap is 174 2024-09-15T03:52:49.944 INFO:tasks.rados.rados.0.smithi122.stdout:1594: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:52:49.944 INFO:tasks.rados.rados.0.smithi122.stdout:1594: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:52:49.944 INFO:tasks.rados.rados.0.smithi122.stdout:1594: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:52:49.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 799 (ObjNum 551 snap 163 seq_num 551) dirty exists 2024-09-15T03:52:49.945 INFO:tasks.rados.rados.0.smithi122.stdout:1595: finishing write tid 1 to smithi12223906-29 2024-09-15T03:52:49.945 INFO:tasks.rados.rados.0.smithi122.stdout:1595: finishing write tid 2 to smithi12223906-29 2024-09-15T03:52:49.945 INFO:tasks.rados.rados.0.smithi122.stdout:1594: done (2 left) 2024-09-15T03:52:49.945 INFO:tasks.rados.rados.0.smithi122.stdout:1597: snap_create 2024-09-15T03:52:49.947 INFO:tasks.rados.rados.0.smithi122.stdout:1595: finishing write tid 3 to smithi12223906-29 2024-09-15T03:52:49.947 INFO:tasks.rados.rados.0.smithi122.stdout:1595: finishing write tid 4 to smithi12223906-29 2024-09-15T03:52:49.948 INFO:tasks.rados.rados.0.smithi122.stdout:1595: finishing write tid 5 to smithi12223906-29 2024-09-15T03:52:49.948 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 642 (ObjNum 581 snap 174 seq_num 581) dirty exists 2024-09-15T03:52:49.948 INFO:tasks.rados.rados.0.smithi122.stdout:1595: left oid 29 (ObjNum 581 snap 174 seq_num 581) 2024-09-15T03:52:49.958 INFO:tasks.rados.rados.0.smithi122.stdout:1596: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:52:49.962 INFO:tasks.rados.rados.0.smithi122.stdout:1596: finishing copy_from to smithi12223906-15 2024-09-15T03:52:49.963 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 385 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:52:50.957 INFO:tasks.rados.rados.0.smithi122.stdout:1595: done (2 left) 2024-09-15T03:52:50.957 INFO:tasks.rados.rados.0.smithi122.stdout:1596: done (1 left) 2024-09-15T03:52:50.957 INFO:tasks.rados.rados.0.smithi122.stdout:1597: done (0 left) 2024-09-15T03:52:50.957 INFO:tasks.rados.rados.0.smithi122.stdout:1598: write oid 2 current snap is 175 2024-09-15T03:52:50.957 INFO:tasks.rados.rados.0.smithi122.stdout:1598: seq_num 582 ranges {581674=637746,1840873=558854,3066417=537135} 2024-09-15T03:52:50.967 INFO:tasks.rados.rados.0.smithi122.stdout:1598: writing smithi12223906-2 from 581674 to 1219420 tid 1 2024-09-15T03:52:50.974 INFO:tasks.rados.rados.0.smithi122.stdout:1598: writing smithi12223906-2 from 1840873 to 2399727 tid 2 2024-09-15T03:52:50.982 INFO:tasks.rados.rados.0.smithi122.stdout:1598: writing smithi12223906-2 from 3066417 to 3603552 tid 3 2024-09-15T03:52:50.982 INFO:tasks.rados.rados.0.smithi122.stdout:1599: snap_remove snap 173 2024-09-15T03:52:51.730 INFO:tasks.rados.rados.0.smithi122.stdout:1599: done (1 left) 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1600: write oid 25 current snap is 175 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1598: finishing write tid 1 to smithi12223906-2 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1598: finishing write tid 2 to smithi12223906-2 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1598: finishing write tid 3 to smithi12223906-2 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1598: finishing write tid 4 to smithi12223906-2 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1598: finishing write tid 5 to smithi12223906-2 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 432 (ObjNum 582 snap 175 seq_num 582) dirty exists 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1598: left oid 2 (ObjNum 582 snap 175 seq_num 582) 2024-09-15T03:52:51.731 INFO:tasks.rados.rados.0.smithi122.stdout:1600: seq_num 583 ranges {598481=712995,1899355=418865,3093840=704641} 2024-09-15T03:52:51.739 INFO:tasks.rados.rados.0.smithi122.stdout:1600: writing smithi12223906-25 from 598481 to 1311476 tid 1 2024-09-15T03:52:51.746 INFO:tasks.rados.rados.0.smithi122.stdout:1600: writing smithi12223906-25 from 1899355 to 2318220 tid 2 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1600: writing smithi12223906-25 from 3093840 to 3798481 tid 3 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1598: done (1 left) 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1601: rollback oid 21 current snap is 175 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 174 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1600: finishing write tid 1 to smithi12223906-25 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1600: finishing write tid 2 to smithi12223906-25 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1602: read oid 24 snap -1 2024-09-15T03:52:51.755 INFO:tasks.rados.rados.0.smithi122.stdout:1602: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:52:51.756 INFO:tasks.rados.rados.0.smithi122.stdout:1603: read oid 5 snap -1 2024-09-15T03:52:51.756 INFO:tasks.rados.rados.0.smithi122.stdout:1603: expect (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:52:51.756 INFO:tasks.rados.rados.0.smithi122.stdout:1604: read oid 38 snap 172 2024-09-15T03:52:51.756 INFO:tasks.rados.rados.0.smithi122.stdout:1604: expect (ObjNum 557 snap 165 seq_num 557) 2024-09-15T03:52:51.756 INFO:tasks.rados.rados.0.smithi122.stdout:1605: copy_from oid 11 from oid 4 current snap is 175 2024-09-15T03:52:51.756 INFO:tasks.rados.rados.0.smithi122.stdout:1606: snap_create 2024-09-15T03:52:51.762 INFO:tasks.rados.rados.0.smithi122.stdout:1601: finishing rollback tid 0 to smithi12223906-21 2024-09-15T03:52:51.762 INFO:tasks.rados.rados.0.smithi122.stdout:1601: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:52:51.764 INFO:tasks.rados.rados.0.smithi122.stdout:1601: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:52:51.764 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 779 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:51.765 INFO:tasks.rados.rados.0.smithi122.stdout:1605: finishing copy_from racing read to smithi12223906-11 2024-09-15T03:52:51.768 INFO:tasks.rados.rados.0.smithi122.stdout:1605: finishing copy_from to smithi12223906-11 2024-09-15T03:52:51.768 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 227 (ObjNum 496 snap 150 seq_num 496) dirty exists 2024-09-15T03:52:51.769 INFO:tasks.rados.rados.0.smithi122.stdout:1603: expect (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:52:51.972 INFO:tasks.rados.rados.0.smithi122.stdout:1604: expect (ObjNum 557 snap 165 seq_num 557) 2024-09-15T03:52:52.187 INFO:tasks.rados.rados.0.smithi122.stdout:1602: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:52:52.405 INFO:tasks.rados.rados.0.smithi122.stdout:1600: finishing write tid 3 to smithi12223906-25 2024-09-15T03:52:52.405 INFO:tasks.rados.rados.0.smithi122.stdout:1600: finishing write tid 4 to smithi12223906-25 2024-09-15T03:52:52.405 INFO:tasks.rados.rados.0.smithi122.stdout:1600: finishing write tid 5 to smithi12223906-25 2024-09-15T03:52:52.405 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 806 (ObjNum 583 snap 175 seq_num 583) dirty exists 2024-09-15T03:52:52.405 INFO:tasks.rados.rados.0.smithi122.stdout:1600: left oid 25 (ObjNum 583 snap 175 seq_num 583) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1600: done (6 left) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1601: done (5 left) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1602: done (4 left) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1603: done (3 left) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1604: done (2 left) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1605: done (1 left) 2024-09-15T03:52:52.746 INFO:tasks.rados.rados.0.smithi122.stdout:1606: done (0 left) 2024-09-15T03:52:52.747 INFO:tasks.rados.rados.0.smithi122.stdout:1607: delete oid 38 current snap is 176 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1607: done (0 left) 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1608: rollback oid 20 current snap is 176 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 172 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1609: read oid 1 snap -1 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1609: expect (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1610: read oid 47 snap -1 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1610: expect (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1611: write (excl) oid 36 current snap is 176 2024-09-15T03:52:52.753 INFO:tasks.rados.rados.0.smithi122.stdout:1611: seq_num 585 ranges {632095=463493,1616319=422534,2748686=487304} 2024-09-15T03:52:52.761 INFO:tasks.rados.rados.0.smithi122.stdout:1611: writing smithi12223906-36 from 632095 to 1095588 tid 1 2024-09-15T03:52:52.765 INFO:tasks.rados.rados.0.smithi122.stdout:1611: writing smithi12223906-36 from 1616319 to 2038853 tid 2 2024-09-15T03:52:52.773 INFO:tasks.rados.rados.0.smithi122.stdout:1611: writing smithi12223906-36 from 2748686 to 3235990 tid 3 2024-09-15T03:52:52.773 INFO:tasks.rados.rados.0.smithi122.stdout:1612: read oid 33 snap -1 2024-09-15T03:52:52.773 INFO:tasks.rados.rados.0.smithi122.stdout:1612: expect deleted 2024-09-15T03:52:52.773 INFO:tasks.rados.rados.0.smithi122.stdout:1613: delete oid 22 current snap is 176 2024-09-15T03:52:52.773 INFO:tasks.rados.rados.0.smithi122.stdout:1609: expect (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1608: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:52:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1608: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:52:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1608: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:52:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 653 (ObjNum 558 snap 165 seq_num 558) dirty exists 2024-09-15T03:52:52.879 INFO:tasks.rados.rados.0.smithi122.stdout:1610: expect (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: finishing write tid 1 to smithi12223906-36 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: finishing write tid 2 to smithi12223906-36 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: finishing write tid 3 to smithi12223906-36 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: finishing write tid 4 to smithi12223906-36 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: finishing write tid 5 to smithi12223906-36 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 810 (ObjNum 585 snap 176 seq_num 585) dirty exists 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: left oid 36 (ObjNum 585 snap 176 seq_num 585) 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1608: done (5 left) 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1609: done (4 left) 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1610: done (3 left) 2024-09-15T03:52:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:1611: done (2 left) 2024-09-15T03:52:53.104 INFO:tasks.rados.rados.0.smithi122.stdout:1612: done (1 left) 2024-09-15T03:52:53.104 INFO:tasks.rados.rados.0.smithi122.stdout:1613: done (0 left) 2024-09-15T03:52:53.104 INFO:tasks.rados.rados.0.smithi122.stdout:1614: copy_from oid 7 from oid 43 current snap is 176 2024-09-15T03:52:53.104 INFO:tasks.rados.rados.0.smithi122.stdout:1615: delete oid 41 current snap is 176 2024-09-15T03:52:53.105 INFO:tasks.rados.rados.0.smithi122.stdout:1615: done (1 left) 2024-09-15T03:52:53.105 INFO:tasks.rados.rados.0.smithi122.stdout:1616: write (excl) oid 38 current snap is 176 2024-09-15T03:52:53.105 INFO:tasks.rados.rados.0.smithi122.stdout:1616: seq_num 588 ranges {682516=405593,1791765=569861,2830955=721120} 2024-09-15T03:52:53.116 INFO:tasks.rados.rados.0.smithi122.stdout:1616: writing smithi12223906-38 from 682516 to 1088109 tid 1 2024-09-15T03:52:53.125 INFO:tasks.rados.rados.0.smithi122.stdout:1616: writing smithi12223906-38 from 1791765 to 2361626 tid 2 2024-09-15T03:52:53.133 INFO:tasks.rados.rados.0.smithi122.stdout:1616: writing smithi12223906-38 from 2830955 to 3552075 tid 3 2024-09-15T03:52:53.133 INFO:tasks.rados.rados.0.smithi122.stdout:1617: rollback oid 13 current snap is 176 2024-09-15T03:52:53.133 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 174 2024-09-15T03:52:53.133 INFO:tasks.rados.rados.0.smithi122.stdout:1614: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:52:53.133 INFO:tasks.rados.rados.0.smithi122.stdout:1616: finishing write tid 1 to smithi12223906-38 2024-09-15T03:52:53.133 INFO:tasks.rados.rados.0.smithi122.stdout:1614: finishing copy_from to smithi12223906-7 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 694 (ObjNum 532 snap 160 seq_num 532) dirty exists 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:1616: finishing write tid 2 to smithi12223906-38 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:1614: done (2 left) 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:1618: copy_from oid 6 from oid 3 current snap is 176 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:1619: copy_from oid 48 from oid 40 current snap is 176 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:1620: write (excl) oid 20 current snap is 176 2024-09-15T03:52:53.134 INFO:tasks.rados.rados.0.smithi122.stdout:1620: seq_num 589 ranges {699323=480842,1850247=429872,2858378=532392} 2024-09-15T03:52:53.140 INFO:tasks.rados.rados.0.smithi122.stdout:1620: writing smithi12223906-20 from 699323 to 1180165 tid 1 2024-09-15T03:52:53.146 INFO:tasks.rados.rados.0.smithi122.stdout:1620: writing smithi12223906-20 from 1850247 to 2280119 tid 2 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:1620: writing smithi12223906-20 from 2858378 to 3390770 tid 3 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:1621: write (excl) oid 17 current snap is 176 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:1616: finishing write tid 3 to smithi12223906-38 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:1617: finishing rollback tid 0 to smithi12223906-13 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:1617: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:1617: finishing rollback tid 2 to smithi12223906-13 2024-09-15T03:52:53.153 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 775 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:52:53.154 INFO:tasks.rados.rados.0.smithi122.stdout:1616: finishing write tid 4 to smithi12223906-38 2024-09-15T03:52:53.154 INFO:tasks.rados.rados.0.smithi122.stdout:1620: finishing write tid 1 to smithi12223906-20 2024-09-15T03:52:53.154 INFO:tasks.rados.rados.0.smithi122.stdout:1616: finishing write tid 5 to smithi12223906-38 2024-09-15T03:52:53.154 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 420 (ObjNum 588 snap 176 seq_num 588) dirty exists 2024-09-15T03:52:53.154 INFO:tasks.rados.rados.0.smithi122.stdout:1616: left oid 38 (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:52:53.154 INFO:tasks.rados.rados.0.smithi122.stdout:1621: seq_num 590 ranges {716130=556091,1792376=573530,2769448=460017,3821956=94174} 2024-09-15T03:52:53.159 INFO:tasks.rados.rados.0.smithi122.stdout:1621: writing smithi12223906-17 from 716130 to 1272221 tid 1 2024-09-15T03:52:53.166 INFO:tasks.rados.rados.0.smithi122.stdout:1621: writing smithi12223906-17 from 1792376 to 2365906 tid 2 2024-09-15T03:52:53.171 INFO:tasks.rados.rados.0.smithi122.stdout:1621: writing smithi12223906-17 from 2769448 to 3229465 tid 3 2024-09-15T03:52:53.175 INFO:tasks.rados.rados.0.smithi122.stdout:1621: writing smithi12223906-17 from 3821956 to 3916130 tid 4 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1616: done (5 left) 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1617: done (4 left) 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1622: copy_from oid 31 from oid 19 current snap is 176 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1618: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1619: finishing copy_from racing read to smithi12223906-48 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1618: finishing copy_from to smithi12223906-6 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 791 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1619: finishing copy_from to smithi12223906-48 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 594 (ObjNum 527 snap 159 seq_num 527) dirty exists 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1620: finishing write tid 2 to smithi12223906-20 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1618: done (4 left) 2024-09-15T03:52:53.176 INFO:tasks.rados.rados.0.smithi122.stdout:1619: done (3 left) 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1623: read oid 30 snap -1 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1623: expect (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1624: snap_create 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1620: finishing write tid 3 to smithi12223906-20 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1620: finishing write tid 4 to smithi12223906-20 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1620: finishing write tid 5 to smithi12223906-20 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 657 (ObjNum 589 snap 176 seq_num 589) dirty exists 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1620: left oid 20 (ObjNum 589 snap 176 seq_num 589) 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1621: finishing write tid 1 to smithi12223906-17 2024-09-15T03:52:53.177 INFO:tasks.rados.rados.0.smithi122.stdout:1621: finishing write tid 2 to smithi12223906-17 2024-09-15T03:52:53.182 INFO:tasks.rados.rados.0.smithi122.stdout:1623: expect (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:53.333 INFO:tasks.rados.rados.0.smithi122.stdout:1621: finishing write tid 3 to smithi12223906-17 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:1621: finishing write tid 4 to smithi12223906-17 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:1621: finishing write tid 5 to smithi12223906-17 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:1621: finishing write tid 6 to smithi12223906-17 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 440 (ObjNum 590 snap 176 seq_num 590) dirty exists 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:1621: left oid 17 (ObjNum 590 snap 176 seq_num 590) 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:1622: finishing copy_from racing read to smithi12223906-31 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:1622: finishing copy_from to smithi12223906-31 2024-09-15T03:52:53.334 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 772 (ObjNum 573 snap 170 seq_num 573) dirty exists 2024-09-15T03:52:53.974 INFO:tasks.rados.rados.0.smithi122.stdout:1620: done (4 left) 2024-09-15T03:52:53.975 INFO:tasks.rados.rados.0.smithi122.stdout:1621: done (3 left) 2024-09-15T03:52:53.975 INFO:tasks.rados.rados.0.smithi122.stdout:1622: done (2 left) 2024-09-15T03:52:53.975 INFO:tasks.rados.rados.0.smithi122.stdout:1623: done (1 left) 2024-09-15T03:52:53.975 INFO:tasks.rados.rados.0.smithi122.stdout:1624: done (0 left) 2024-09-15T03:52:53.975 INFO:tasks.rados.rados.0.smithi122.stdout:1625: snap_remove snap 175 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1625: done (0 left) 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1626: rollback oid 42 current snap is 177 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 170 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1627: read oid 12 snap 174 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1627: expect (ObjNum 576 snap 172 seq_num 576) 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1628: rollback oid 21 current snap is 177 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 176 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1629: write oid 18 current snap is 177 2024-09-15T03:52:54.980 INFO:tasks.rados.rados.0.smithi122.stdout:1629: seq_num 591 ranges {732937=631340,1850858=433541,2796871=671289,3888195=44742} 2024-09-15T03:52:54.991 INFO:tasks.rados.rados.0.smithi122.stdout:1629: writing smithi12223906-18 from 732937 to 1364277 tid 1 2024-09-15T03:52:54.997 INFO:tasks.rados.rados.0.smithi122.stdout:1629: writing smithi12223906-18 from 1850858 to 2284399 tid 2 2024-09-15T03:52:55.004 INFO:tasks.rados.rados.0.smithi122.stdout:1629: writing smithi12223906-18 from 2796871 to 3468160 tid 3 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1629: writing smithi12223906-18 from 3888195 to 3932937 tid 4 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1630: read oid 30 snap -1 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1630: expect (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1628: finishing rollback tid 0 to smithi12223906-21 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1631: read oid 25 snap -1 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1631: expect (ObjNum 583 snap 175 seq_num 583) 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1632: write (excl) oid 40 current snap is 177 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1626: finishing rollback tid 0 to smithi12223906-42 2024-09-15T03:52:55.008 INFO:tasks.rados.rados.0.smithi122.stdout:1632: seq_num 592 ranges {749744=706589,1909340=577199,3224294=482561} 2024-09-15T03:52:55.016 INFO:tasks.rados.rados.0.smithi122.stdout:1632: writing smithi12223906-40 from 749744 to 1456333 tid 1 2024-09-15T03:52:55.021 INFO:tasks.rados.rados.0.smithi122.stdout:1632: writing smithi12223906-40 from 1909340 to 2486539 tid 2 2024-09-15T03:52:55.029 INFO:tasks.rados.rados.0.smithi122.stdout:1632: writing smithi12223906-40 from 3224294 to 3706855 tid 3 2024-09-15T03:52:55.029 INFO:tasks.rados.rados.0.smithi122.stdout:1633: read oid 10 snap -1 2024-09-15T03:52:55.030 INFO:tasks.rados.rados.0.smithi122.stdout:1633: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:55.030 INFO:tasks.rados.rados.0.smithi122.stdout:1634: snap_create 2024-09-15T03:52:55.030 INFO:tasks.rados.rados.0.smithi122.stdout:1626: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:52:55.030 INFO:tasks.rados.rados.0.smithi122.stdout:1626: finishing rollback tid 2 to smithi12223906-42 2024-09-15T03:52:55.030 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 814 (ObjNum 561 snap 167 seq_num 561) dirty exists 2024-09-15T03:52:55.030 INFO:tasks.rados.rados.0.smithi122.stdout:1627: expect (ObjNum 576 snap 172 seq_num 576) 2024-09-15T03:52:55.274 INFO:tasks.rados.rados.0.smithi122.stdout:1628: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:52:55.274 INFO:tasks.rados.rados.0.smithi122.stdout:1628: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:52:55.274 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 782 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:52:55.274 INFO:tasks.rados.rados.0.smithi122.stdout:1629: finishing write tid 1 to smithi12223906-18 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1629: finishing write tid 2 to smithi12223906-18 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1629: finishing write tid 3 to smithi12223906-18 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1629: finishing write tid 4 to smithi12223906-18 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1629: finishing write tid 5 to smithi12223906-18 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1629: finishing write tid 6 to smithi12223906-18 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 786 (ObjNum 591 snap 177 seq_num 591) dirty exists 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1629: left oid 18 (ObjNum 591 snap 177 seq_num 591) 2024-09-15T03:52:55.275 INFO:tasks.rados.rados.0.smithi122.stdout:1630: expect (ObjNum 569 snap 168 seq_num 569) 2024-09-15T03:52:55.429 INFO:tasks.rados.rados.0.smithi122.stdout:1631: expect (ObjNum 583 snap 175 seq_num 583) 2024-09-15T03:52:55.585 INFO:tasks.rados.rados.0.smithi122.stdout:1632: finishing write tid 1 to smithi12223906-40 2024-09-15T03:52:55.585 INFO:tasks.rados.rados.0.smithi122.stdout:1632: finishing write tid 2 to smithi12223906-40 2024-09-15T03:52:55.585 INFO:tasks.rados.rados.0.smithi122.stdout:1633: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:52:55.747 INFO:tasks.rados.rados.0.smithi122.stdout:1632: finishing write tid 3 to smithi12223906-40 2024-09-15T03:52:55.747 INFO:tasks.rados.rados.0.smithi122.stdout:1632: finishing write tid 4 to smithi12223906-40 2024-09-15T03:52:55.747 INFO:tasks.rados.rados.0.smithi122.stdout:1632: finishing write tid 5 to smithi12223906-40 2024-09-15T03:52:55.747 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 607 (ObjNum 592 snap 177 seq_num 592) dirty exists 2024-09-15T03:52:55.747 INFO:tasks.rados.rados.0.smithi122.stdout:1632: left oid 40 (ObjNum 592 snap 177 seq_num 592) 2024-09-15T03:52:56.004 INFO:tasks.rados.rados.0.smithi122.stdout:1626: done (8 left) 2024-09-15T03:52:56.004 INFO:tasks.rados.rados.0.smithi122.stdout:1627: done (7 left) 2024-09-15T03:52:56.004 INFO:tasks.rados.rados.0.smithi122.stdout:1628: done (6 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1629: done (5 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1630: done (4 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1631: done (3 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1632: done (2 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1633: done (1 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1634: done (0 left) 2024-09-15T03:52:56.005 INFO:tasks.rados.rados.0.smithi122.stdout:1635: snap_create 2024-09-15T03:52:57.010 INFO:tasks.rados.rados.0.smithi122.stdout:1635: done (0 left) 2024-09-15T03:52:57.010 INFO:tasks.rados.rados.0.smithi122.stdout:1636: write (excl) oid 32 current snap is 179 2024-09-15T03:52:57.011 INFO:tasks.rados.rados.0.smithi122.stdout:1636: seq_num 593 ranges {766551=498191,1684175=720857,2851717=693833} 2024-09-15T03:52:57.017 INFO:tasks.rados.rados.0.smithi122.stdout:1636: writing smithi12223906-32 from 766551 to 1264742 tid 1 2024-09-15T03:52:57.024 INFO:tasks.rados.rados.0.smithi122.stdout:1636: writing smithi12223906-32 from 1684175 to 2405032 tid 2 2024-09-15T03:52:57.030 INFO:tasks.rados.rados.0.smithi122.stdout:1636: writing smithi12223906-32 from 2851717 to 3545550 tid 3 2024-09-15T03:52:57.030 INFO:tasks.rados.rados.0.smithi122.stdout:1637: snap_create 2024-09-15T03:52:57.031 INFO:tasks.rados.rados.0.smithi122.stdout:1636: finishing write tid 1 to smithi12223906-32 2024-09-15T03:52:57.031 INFO:tasks.rados.rados.0.smithi122.stdout:1636: finishing write tid 2 to smithi12223906-32 2024-09-15T03:52:57.036 INFO:tasks.rados.rados.0.smithi122.stdout:1636: finishing write tid 3 to smithi12223906-32 2024-09-15T03:52:57.036 INFO:tasks.rados.rados.0.smithi122.stdout:1636: finishing write tid 4 to smithi12223906-32 2024-09-15T03:52:57.036 INFO:tasks.rados.rados.0.smithi122.stdout:1636: finishing write tid 5 to smithi12223906-32 2024-09-15T03:52:57.036 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 791 (ObjNum 593 snap 179 seq_num 593) dirty exists 2024-09-15T03:52:57.036 INFO:tasks.rados.rados.0.smithi122.stdout:1636: left oid 32 (ObjNum 593 snap 179 seq_num 593) 2024-09-15T03:52:58.027 INFO:tasks.rados.rados.0.smithi122.stdout:1636: done (1 left) 2024-09-15T03:52:58.027 INFO:tasks.rados.rados.0.smithi122.stdout:1637: done (0 left) 2024-09-15T03:52:58.027 INFO:tasks.rados.rados.0.smithi122.stdout:1638: copy_from oid 22 from oid 15 current snap is 180 2024-09-15T03:52:58.027 INFO:tasks.rados.rados.0.smithi122.stdout:1639: write oid 39 current snap is 180 2024-09-15T03:52:58.027 INFO:tasks.rados.rados.0.smithi122.stdout:1639: seq_num 594 ranges {783358=573440,2026304=580868,3279140=621458} 2024-09-15T03:52:58.035 INFO:tasks.rados.rados.0.smithi122.stdout:1639: writing smithi12223906-39 from 783358 to 1356798 tid 1 2024-09-15T03:52:58.042 INFO:tasks.rados.rados.0.smithi122.stdout:1639: writing smithi12223906-39 from 2026304 to 2607172 tid 2 2024-09-15T03:52:58.051 INFO:tasks.rados.rados.0.smithi122.stdout:1639: writing smithi12223906-39 from 3279140 to 3900598 tid 3 2024-09-15T03:52:58.051 INFO:tasks.rados.rados.0.smithi122.stdout:1640: write oid 35 current snap is 180 2024-09-15T03:52:58.051 INFO:tasks.rados.rados.0.smithi122.stdout:1640: seq_num 595 ranges {400165=648689,1684786=315379} 2024-09-15T03:52:58.055 INFO:tasks.rados.rados.0.smithi122.stdout:1640: writing smithi12223906-35 from 400165 to 1048854 tid 1 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1640: writing smithi12223906-35 from 1684786 to 2000165 tid 2 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1641: rollback oid 8 current snap is 180 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 179 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1638: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1639: finishing write tid 1 to smithi12223906-39 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1642: read oid 1 snap -1 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1642: expect (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1643: write (excl) oid 45 current snap is 180 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1638: finishing copy_from to smithi12223906-22 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 385 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1639: finishing write tid 2 to smithi12223906-39 2024-09-15T03:52:58.062 INFO:tasks.rados.rados.0.smithi122.stdout:1639: finishing write tid 3 to smithi12223906-39 2024-09-15T03:52:58.063 INFO:tasks.rados.rados.0.smithi122.stdout:1639: finishing write tid 4 to smithi12223906-39 2024-09-15T03:52:58.063 INFO:tasks.rados.rados.0.smithi122.stdout:1643: seq_num 596 ranges {416972=723938,1743268=273704} 2024-09-15T03:52:58.068 INFO:tasks.rados.rados.0.smithi122.stdout:1643: writing smithi12223906-45 from 416972 to 1140910 tid 1 2024-09-15T03:52:58.074 INFO:tasks.rados.rados.0.smithi122.stdout:1643: writing smithi12223906-45 from 1743268 to 2016972 tid 2 2024-09-15T03:52:58.074 INFO:tasks.rados.rados.0.smithi122.stdout:1638: done (5 left) 2024-09-15T03:52:58.075 INFO:tasks.rados.rados.0.smithi122.stdout:1644: read oid 36 snap -1 2024-09-15T03:52:58.075 INFO:tasks.rados.rados.0.smithi122.stdout:1644: expect (ObjNum 585 snap 176 seq_num 585) 2024-09-15T03:52:58.075 INFO:tasks.rados.rados.0.smithi122.stdout:1645: read oid 48 snap -1 2024-09-15T03:52:58.076 INFO:tasks.rados.rados.0.smithi122.stdout:1645: expect (ObjNum 527 snap 159 seq_num 527) 2024-09-15T03:52:58.076 INFO:tasks.rados.rados.0.smithi122.stdout:1646: snap_remove snap 178 2024-09-15T03:52:59.035 INFO:tasks.rados.rados.0.smithi122.stdout:1646: done (7 left) 2024-09-15T03:52:59.035 INFO:tasks.rados.rados.0.smithi122.stdout:1647: delete oid 9 current snap is 180 2024-09-15T03:52:59.035 INFO:tasks.rados.rados.0.smithi122.stdout:1639: finishing write tid 5 to smithi12223906-39 2024-09-15T03:52:59.035 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 819 (ObjNum 594 snap 180 seq_num 594) dirty exists 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:1639: left oid 39 (ObjNum 594 snap 180 seq_num 594) 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:1641: finishing rollback tid 1 to smithi12223906-8 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 0 (ObjNum 49 snap 0 seq_num 1842299904) dirty dne 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:1640: finishing write tid 1 to smithi12223906-35 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:1640: finishing write tid 2 to smithi12223906-35 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:1640: finishing write tid 3 to smithi12223906-35 2024-09-15T03:52:59.036 INFO:tasks.rados.rados.0.smithi122.stdout:1642: expect (ObjNum 553 snap 165 seq_num 553) 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:1640: finishing write tid 4 to smithi12223906-35 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 577 (ObjNum 595 snap 180 seq_num 595) dirty exists 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:1640: left oid 35 (ObjNum 595 snap 180 seq_num 595) 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:1643: finishing write tid 1 to smithi12223906-45 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:1643: finishing write tid 2 to smithi12223906-45 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:1643: finishing write tid 3 to smithi12223906-45 2024-09-15T03:52:59.141 INFO:tasks.rados.rados.0.smithi122.stdout:1643: finishing write tid 4 to smithi12223906-45 2024-09-15T03:52:59.142 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 778 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:52:59.142 INFO:tasks.rados.rados.0.smithi122.stdout:1643: left oid 45 (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:52:59.142 INFO:tasks.rados.rados.0.smithi122.stdout:1645: expect (ObjNum 527 snap 159 seq_num 527) 2024-09-15T03:52:59.234 INFO:tasks.rados.rados.0.smithi122.stdout:1644: expect (ObjNum 585 snap 176 seq_num 585) 2024-09-15T03:52:59.361 INFO:tasks.rados.rados.0.smithi122.stdout:1639: done (7 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1640: done (6 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1641: done (5 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1642: done (4 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1643: done (3 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1644: done (2 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1645: done (1 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1647: done (0 left) 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1648: write (excl) oid 37 current snap is 180 2024-09-15T03:52:59.362 INFO:tasks.rados.rados.0.smithi122.stdout:1648: seq_num 598 ranges {450586=474436,1343879=588206} 2024-09-15T03:52:59.373 INFO:tasks.rados.rados.0.smithi122.stdout:1648: writing smithi12223906-37 from 450586 to 925022 tid 1 2024-09-15T03:52:59.380 INFO:tasks.rados.rados.0.smithi122.stdout:1648: writing smithi12223906-37 from 1343879 to 1932085 tid 2 2024-09-15T03:52:59.380 INFO:tasks.rados.rados.0.smithi122.stdout:1649: read oid 23 snap -1 2024-09-15T03:52:59.381 INFO:tasks.rados.rados.0.smithi122.stdout:1649: expect (ObjNum 578 snap 172 seq_num 578) 2024-09-15T03:52:59.381 INFO:tasks.rados.rados.0.smithi122.stdout:1650: write (excl) oid 15 current snap is 180 2024-09-15T03:52:59.381 INFO:tasks.rados.rados.0.smithi122.stdout:1650: seq_num 599 ranges {467393=549685,1802361=265032} 2024-09-15T03:52:59.386 INFO:tasks.rados.rados.0.smithi122.stdout:1650: writing smithi12223906-15 from 467393 to 1017078 tid 1 2024-09-15T03:52:59.393 INFO:tasks.rados.rados.0.smithi122.stdout:1650: writing smithi12223906-15 from 1802361 to 2067393 tid 2 2024-09-15T03:52:59.393 INFO:tasks.rados.rados.0.smithi122.stdout:1651: write oid 32 current snap is 180 2024-09-15T03:52:59.393 INFO:tasks.rados.rados.0.smithi122.stdout:1648: finishing write tid 1 to smithi12223906-37 2024-09-15T03:52:59.393 INFO:tasks.rados.rados.0.smithi122.stdout:1651: seq_num 600 ranges {484200=624934,1860843=223357} 2024-09-15T03:52:59.399 INFO:tasks.rados.rados.0.smithi122.stdout:1651: writing smithi12223906-32 from 484200 to 1109134 tid 1 2024-09-15T03:52:59.405 INFO:tasks.rados.rados.0.smithi122.stdout:1651: writing smithi12223906-32 from 1860843 to 2084200 tid 2 2024-09-15T03:52:59.405 INFO:tasks.rados.rados.0.smithi122.stdout:1652: write (excl) oid 2 current snap is 180 2024-09-15T03:52:59.405 INFO:tasks.rados.rados.0.smithi122.stdout:1652: seq_num 601 ranges {501007=416536,1635678=465329} 2024-09-15T03:52:59.410 INFO:tasks.rados.rados.0.smithi122.stdout:1652: writing smithi12223906-2 from 501007 to 917543 tid 1 2024-09-15T03:52:59.417 INFO:tasks.rados.rados.0.smithi122.stdout:1652: writing smithi12223906-2 from 1635678 to 2101007 tid 2 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1653: delete oid 22 current snap is 180 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1648: finishing write tid 2 to smithi12223906-37 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1648: finishing write tid 3 to smithi12223906-37 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1648: finishing write tid 4 to smithi12223906-37 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 785 (ObjNum 598 snap 180 seq_num 598) dirty exists 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1648: left oid 37 (ObjNum 598 snap 180 seq_num 598) 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1650: finishing write tid 1 to smithi12223906-15 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1650: finishing write tid 2 to smithi12223906-15 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1650: finishing write tid 3 to smithi12223906-15 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1650: finishing write tid 4 to smithi12223906-15 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 591 (ObjNum 599 snap 180 seq_num 599) dirty exists 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1650: left oid 15 (ObjNum 599 snap 180 seq_num 599) 2024-09-15T03:52:59.418 INFO:tasks.rados.rados.0.smithi122.stdout:1649: expect (ObjNum 578 snap 172 seq_num 578) 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1651: finishing write tid 1 to smithi12223906-32 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1651: finishing write tid 2 to smithi12223906-32 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1651: finishing write tid 3 to smithi12223906-32 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1651: finishing write tid 4 to smithi12223906-32 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 798 (ObjNum 600 snap 180 seq_num 600) dirty exists 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1651: left oid 32 (ObjNum 600 snap 180 seq_num 600) 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1652: finishing write tid 1 to smithi12223906-2 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1648: done (5 left) 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1649: done (4 left) 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1650: done (3 left) 2024-09-15T03:52:59.539 INFO:tasks.rados.rados.0.smithi122.stdout:1651: done (2 left) 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1653: done (1 left) 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1654: read oid 33 snap -1 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1654: expect deleted 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1655: rollback oid 15 current snap is 180 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 179 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1652: finishing write tid 2 to smithi12223906-2 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1652: finishing write tid 3 to smithi12223906-2 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1652: finishing write tid 4 to smithi12223906-2 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 446 (ObjNum 601 snap 180 seq_num 601) dirty exists 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1652: left oid 2 (ObjNum 601 snap 180 seq_num 601) 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1652: done (2 left) 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1656: write (excl) oid 37 current snap is 180 2024-09-15T03:52:59.540 INFO:tasks.rados.rados.0.smithi122.stdout:1656: seq_num 603 ranges {534621=567034,1636289=498332} 2024-09-15T03:52:59.546 INFO:tasks.rados.rados.0.smithi122.stdout:1656: writing smithi12223906-37 from 534621 to 1101655 tid 1 2024-09-15T03:52:59.555 INFO:tasks.rados.rados.0.smithi122.stdout:1656: writing smithi12223906-37 from 1636289 to 2134621 tid 2 2024-09-15T03:52:59.555 INFO:tasks.rados.rados.0.smithi122.stdout:1657: read oid 26 snap -1 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1657: expect (ObjNum 520 snap 156 seq_num 520) 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1658: rollback oid 9 current snap is 180 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 179 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1655: finishing rollback tid 0 to smithi12223906-15 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1655: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1654: done (4 left) 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1659: read oid 7 snap 174 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1659: expect (ObjNum 559 snap 165 seq_num 559) 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1660: read oid 40 snap -1 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1660: expect (ObjNum 592 snap 177 seq_num 592) 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1661: read oid 33 snap -1 2024-09-15T03:52:59.556 INFO:tasks.rados.rados.0.smithi122.stdout:1661: expect deleted 2024-09-15T03:52:59.557 INFO:tasks.rados.rados.0.smithi122.stdout:1662: write oid 27 current snap is 180 2024-09-15T03:52:59.557 INFO:tasks.rados.rados.0.smithi122.stdout:1662: seq_num 604 ranges {551428=642283,1694771=456657} 2024-09-15T03:52:59.563 INFO:tasks.rados.rados.0.smithi122.stdout:1662: writing smithi12223906-27 from 551428 to 1193711 tid 1 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1662: writing smithi12223906-27 from 1694771 to 2151428 tid 2 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1663: read oid 5 snap -1 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1663: expect (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1664: copy_from oid 32 from oid 25 current snap is 180 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1665: rollback oid 50 current snap is 180 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 172 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1655: finishing rollback tid 2 to smithi12223906-15 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 594 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1656: finishing write tid 1 to smithi12223906-37 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1658: finishing rollback tid 1 to smithi12223906-9 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1656: finishing write tid 2 to smithi12223906-37 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1656: finishing write tid 3 to smithi12223906-37 2024-09-15T03:52:59.569 INFO:tasks.rados.rados.0.smithi122.stdout:1656: finishing write tid 4 to smithi12223906-37 2024-09-15T03:52:59.570 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 788 (ObjNum 603 snap 180 seq_num 603) dirty exists 2024-09-15T03:52:59.570 INFO:tasks.rados.rados.0.smithi122.stdout:1656: left oid 37 (ObjNum 603 snap 180 seq_num 603) 2024-09-15T03:52:59.570 INFO:tasks.rados.rados.0.smithi122.stdout:1657: expect (ObjNum 520 snap 156 seq_num 520) 2024-09-15T03:52:59.654 INFO:tasks.rados.rados.0.smithi122.stdout:1658: finishing rollback tid 2 to smithi12223906-9 2024-09-15T03:52:59.654 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 794 (ObjNum 534 snap 160 seq_num 534) dirty exists 2024-09-15T03:52:59.654 INFO:tasks.rados.rados.0.smithi122.stdout:1655: done (10 left) 2024-09-15T03:52:59.654 INFO:tasks.rados.rados.0.smithi122.stdout:1656: done (9 left) 2024-09-15T03:52:59.655 INFO:tasks.rados.rados.0.smithi122.stdout:1657: done (8 left) 2024-09-15T03:52:59.655 INFO:tasks.rados.rados.0.smithi122.stdout:1658: done (7 left) 2024-09-15T03:52:59.655 INFO:tasks.rados.rados.0.smithi122.stdout:1661: done (6 left) 2024-09-15T03:52:59.655 INFO:tasks.rados.rados.0.smithi122.stdout:1666: delete oid 14 current snap is 180 2024-09-15T03:52:59.655 INFO:tasks.rados.rados.0.smithi122.stdout:1660: expect (ObjNum 592 snap 177 seq_num 592) 2024-09-15T03:52:59.830 INFO:tasks.rados.rados.0.smithi122.stdout:1659: expect (ObjNum 559 snap 165 seq_num 559) 2024-09-15T03:53:00.043 INFO:tasks.rados.rados.0.smithi122.stdout:1659: done (6 left) 2024-09-15T03:53:00.043 INFO:tasks.rados.rados.0.smithi122.stdout:1660: done (5 left) 2024-09-15T03:53:00.043 INFO:tasks.rados.rados.0.smithi122.stdout:1666: done (4 left) 2024-09-15T03:53:00.043 INFO:tasks.rados.rados.0.smithi122.stdout:1667: write (excl) oid 1 current snap is 180 2024-09-15T03:53:00.043 INFO:tasks.rados.rados.0.smithi122.stdout:1663: expect (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:53:00.232 INFO:tasks.rados.rados.0.smithi122.stdout:1662: finishing write tid 1 to smithi12223906-27 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1662: finishing write tid 2 to smithi12223906-27 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1664: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1662: finishing write tid 3 to smithi12223906-27 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1662: finishing write tid 4 to smithi12223906-27 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 803 (ObjNum 604 snap 180 seq_num 604) dirty exists 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1662: left oid 27 (ObjNum 604 snap 180 seq_num 604) 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1665: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1665: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:53:00.233 INFO:tasks.rados.rados.0.smithi122.stdout:1667: seq_num 606 ranges {585042=792781,2095382=89660} 2024-09-15T03:53:00.240 INFO:tasks.rados.rados.0.smithi122.stdout:1667: writing smithi12223906-1 from 585042 to 1377823 tid 1 2024-09-15T03:53:00.245 INFO:tasks.rados.rados.0.smithi122.stdout:1667: writing smithi12223906-1 from 2095382 to 2185042 tid 2 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1662: done (4 left) 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1663: done (3 left) 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1668: read oid 28 snap -1 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1668: expect deleted 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1664: finishing copy_from to smithi12223906-32 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 806 (ObjNum 583 snap 175 seq_num 583) dirty exists 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1665: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 808 (ObjNum 567 snap 167 seq_num 567) dirty exists 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1669: read oid 21 snap -1 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1669: expect (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1664: done (4 left) 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1665: done (3 left) 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1670: read oid 34 snap 170 2024-09-15T03:53:00.246 INFO:tasks.rados.rados.0.smithi122.stdout:1670: expect (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:53:00.247 INFO:tasks.rados.rados.0.smithi122.stdout:1671: snap_create 2024-09-15T03:53:00.252 INFO:tasks.rados.rados.0.smithi122.stdout:1667: finishing write tid 1 to smithi12223906-1 2024-09-15T03:53:00.252 INFO:tasks.rados.rados.0.smithi122.stdout:1667: finishing write tid 2 to smithi12223906-1 2024-09-15T03:53:00.253 INFO:tasks.rados.rados.0.smithi122.stdout:1667: finishing write tid 3 to smithi12223906-1 2024-09-15T03:53:00.253 INFO:tasks.rados.rados.0.smithi122.stdout:1667: finishing write tid 4 to smithi12223906-1 2024-09-15T03:53:00.253 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 597 (ObjNum 606 snap 180 seq_num 606) dirty exists 2024-09-15T03:53:00.253 INFO:tasks.rados.rados.0.smithi122.stdout:1667: left oid 1 (ObjNum 606 snap 180 seq_num 606) 2024-09-15T03:53:00.254 INFO:tasks.rados.rados.0.smithi122.stdout:1670: expect (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:53:00.394 INFO:tasks.rados.rados.0.smithi122.stdout:1669: expect (ObjNum 526 snap 158 seq_num 526) 2024-09-15T03:53:01.052 INFO:tasks.rados.rados.0.smithi122.stdout:1667: done (4 left) 2024-09-15T03:53:01.052 INFO:tasks.rados.rados.0.smithi122.stdout:1668: done (3 left) 2024-09-15T03:53:01.052 INFO:tasks.rados.rados.0.smithi122.stdout:1669: done (2 left) 2024-09-15T03:53:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:1670: done (1 left) 2024-09-15T03:53:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:1671: done (0 left) 2024-09-15T03:53:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:1672: copy_from oid 32 from oid 16 current snap is 181 2024-09-15T03:53:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:1673: delete oid 46 current snap is 181 2024-09-15T03:53:01.058 INFO:tasks.rados.rados.0.smithi122.stdout:1673: done (1 left) 2024-09-15T03:53:01.058 INFO:tasks.rados.rados.0.smithi122.stdout:1674: write (excl) oid 43 current snap is 181 2024-09-15T03:53:01.058 INFO:tasks.rados.rados.0.smithi122.stdout:1674: seq_num 608 ranges {618656=543279,1812346=406310} 2024-09-15T03:53:01.064 INFO:tasks.rados.rados.0.smithi122.stdout:1674: writing smithi12223906-43 from 618656 to 1161935 tid 1 2024-09-15T03:53:01.070 INFO:tasks.rados.rados.0.smithi122.stdout:1674: writing smithi12223906-43 from 1812346 to 2218656 tid 2 2024-09-15T03:53:01.070 INFO:tasks.rados.rados.0.smithi122.stdout:1675: snap_create 2024-09-15T03:53:01.070 INFO:tasks.rados.rados.0.smithi122.stdout:1672: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:53:01.070 INFO:tasks.rados.rados.0.smithi122.stdout:1672: finishing copy_from to smithi12223906-32 2024-09-15T03:53:01.071 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 567 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:53:01.076 INFO:tasks.rados.rados.0.smithi122.stdout:1674: finishing write tid 1 to smithi12223906-43 2024-09-15T03:53:01.076 INFO:tasks.rados.rados.0.smithi122.stdout:1674: finishing write tid 2 to smithi12223906-43 2024-09-15T03:53:01.076 INFO:tasks.rados.rados.0.smithi122.stdout:1674: finishing write tid 3 to smithi12223906-43 2024-09-15T03:53:01.076 INFO:tasks.rados.rados.0.smithi122.stdout:1674: finishing write tid 4 to smithi12223906-43 2024-09-15T03:53:01.076 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 827 (ObjNum 608 snap 181 seq_num 608) dirty exists 2024-09-15T03:53:01.077 INFO:tasks.rados.rados.0.smithi122.stdout:1674: left oid 43 (ObjNum 608 snap 181 seq_num 608) 2024-09-15T03:53:01.732 INFO:tasks.rados.rados.0.smithi122.stdout:1672: done (2 left) 2024-09-15T03:53:01.732 INFO:tasks.rados.rados.0.smithi122.stdout:1674: done (1 left) 2024-09-15T03:53:01.732 INFO:tasks.rados.rados.0.smithi122.stdout:1675: done (0 left) 2024-09-15T03:53:01.732 INFO:tasks.rados.rados.0.smithi122.stdout:1676: read oid 35 snap -1 2024-09-15T03:53:01.732 INFO:tasks.rados.rados.0.smithi122.stdout:1676: expect (ObjNum 595 snap 180 seq_num 595) 2024-09-15T03:53:01.733 INFO:tasks.rados.rados.0.smithi122.stdout:1677: snap_remove snap 179 2024-09-15T03:53:02.733 INFO:tasks.rados.rados.0.smithi122.stdout:1677: done (1 left) 2024-09-15T03:53:02.733 INFO:tasks.rados.rados.0.smithi122.stdout:1678: snap_remove snap 170 2024-09-15T03:53:03.741 INFO:tasks.rados.rados.0.smithi122.stdout:1678: done (1 left) 2024-09-15T03:53:03.742 INFO:tasks.rados.rados.0.smithi122.stdout:1679: snap_create 2024-09-15T03:53:03.742 INFO:tasks.rados.rados.0.smithi122.stdout:1676: expect (ObjNum 595 snap 180 seq_num 595) 2024-09-15T03:53:04.746 INFO:tasks.rados.rados.0.smithi122.stdout:1676: done (1 left) 2024-09-15T03:53:04.746 INFO:tasks.rados.rados.0.smithi122.stdout:1679: done (0 left) 2024-09-15T03:53:04.746 INFO:tasks.rados.rados.0.smithi122.stdout:1680: read oid 38 snap 181 2024-09-15T03:53:04.746 INFO:tasks.rados.rados.0.smithi122.stdout:1680: expect (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:53:04.746 INFO:tasks.rados.rados.0.smithi122.stdout:1681: write oid 29 current snap is 183 2024-09-15T03:53:04.746 INFO:tasks.rados.rados.0.smithi122.stdout:1681: seq_num 609 ranges {635463=734881,1987181=248282} 2024-09-15T03:53:04.757 INFO:tasks.rados.rados.0.smithi122.stdout:1681: writing smithi12223906-29 from 635463 to 1370344 tid 1 2024-09-15T03:53:04.763 INFO:tasks.rados.rados.0.smithi122.stdout:1681: writing smithi12223906-29 from 1987181 to 2235463 tid 2 2024-09-15T03:53:04.763 INFO:tasks.rados.rados.0.smithi122.stdout:1682: read oid 24 snap -1 2024-09-15T03:53:04.763 INFO:tasks.rados.rados.0.smithi122.stdout:1682: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:53:04.763 INFO:tasks.rados.rados.0.smithi122.stdout:1683: write (excl) oid 34 current snap is 183 2024-09-15T03:53:04.763 INFO:tasks.rados.rados.0.smithi122.stdout:1683: seq_num 610 ranges {652270=410130,1529310=493867} 2024-09-15T03:53:04.769 INFO:tasks.rados.rados.0.smithi122.stdout:1683: writing smithi12223906-34 from 652270 to 1062400 tid 1 2024-09-15T03:53:04.775 INFO:tasks.rados.rados.0.smithi122.stdout:1683: writing smithi12223906-34 from 1529310 to 2023177 tid 2 2024-09-15T03:53:04.775 INFO:tasks.rados.rados.0.smithi122.stdout:1684: write oid 31 current snap is 183 2024-09-15T03:53:04.775 INFO:tasks.rados.rados.0.smithi122.stdout:1684: seq_num 611 ranges {669077=485379,1587792=681285} 2024-09-15T03:53:04.781 INFO:tasks.rados.rados.0.smithi122.stdout:1684: writing smithi12223906-31 from 669077 to 1154456 tid 1 2024-09-15T03:53:04.788 INFO:tasks.rados.rados.0.smithi122.stdout:1684: writing smithi12223906-31 from 1587792 to 2269077 tid 2 2024-09-15T03:53:04.788 INFO:tasks.rados.rados.0.smithi122.stdout:1685: write oid 37 current snap is 183 2024-09-15T03:53:04.788 INFO:tasks.rados.rados.0.smithi122.stdout:1680: expect (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:53:04.917 INFO:tasks.rados.rados.0.smithi122.stdout:1681: finishing write tid 1 to smithi12223906-29 2024-09-15T03:53:04.917 INFO:tasks.rados.rados.0.smithi122.stdout:1681: finishing write tid 2 to smithi12223906-29 2024-09-15T03:53:04.917 INFO:tasks.rados.rados.0.smithi122.stdout:1681: finishing write tid 3 to smithi12223906-29 2024-09-15T03:53:04.917 INFO:tasks.rados.rados.0.smithi122.stdout:1681: finishing write tid 4 to smithi12223906-29 2024-09-15T03:53:04.917 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 697 (ObjNum 609 snap 183 seq_num 609) dirty exists 2024-09-15T03:53:04.917 INFO:tasks.rados.rados.0.smithi122.stdout:1681: left oid 29 (ObjNum 609 snap 183 seq_num 609) 2024-09-15T03:53:04.918 INFO:tasks.rados.rados.0.smithi122.stdout:1682: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:53:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:1683: finishing write tid 1 to smithi12223906-34 2024-09-15T03:53:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:1683: finishing write tid 2 to smithi12223906-34 2024-09-15T03:53:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:1683: finishing write tid 3 to smithi12223906-34 2024-09-15T03:53:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:1683: finishing write tid 4 to smithi12223906-34 2024-09-15T03:53:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 719 (ObjNum 610 snap 183 seq_num 610) dirty exists 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1683: left oid 34 (ObjNum 610 snap 183 seq_num 610) 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1684: finishing write tid 1 to smithi12223906-31 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1684: finishing write tid 2 to smithi12223906-31 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1684: finishing write tid 3 to smithi12223906-31 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1684: finishing write tid 4 to smithi12223906-31 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 775 (ObjNum 611 snap 183 seq_num 611) dirty exists 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1684: left oid 31 (ObjNum 611 snap 183 seq_num 611) 2024-09-15T03:53:05.153 INFO:tasks.rados.rados.0.smithi122.stdout:1685: seq_num 612 ranges {685884=560628,2046274=239610} 2024-09-15T03:53:05.169 INFO:tasks.rados.rados.0.smithi122.stdout:1685: writing smithi12223906-37 from 685884 to 1246512 tid 1 2024-09-15T03:53:05.174 INFO:tasks.rados.rados.0.smithi122.stdout:1685: writing smithi12223906-37 from 2046274 to 2285884 tid 2 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1680: done (5 left) 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1681: done (4 left) 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1682: done (3 left) 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1683: done (2 left) 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1684: done (1 left) 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1686: write oid 3 current snap is 183 2024-09-15T03:53:05.175 INFO:tasks.rados.rados.0.smithi122.stdout:1686: seq_num 613 ranges {702691=635877,2104756=197935} 2024-09-15T03:53:05.182 INFO:tasks.rados.rados.0.smithi122.stdout:1686: writing smithi12223906-3 from 702691 to 1338568 tid 1 2024-09-15T03:53:05.190 INFO:tasks.rados.rados.0.smithi122.stdout:1686: writing smithi12223906-3 from 2104756 to 2302691 tid 2 2024-09-15T03:53:05.190 INFO:tasks.rados.rados.0.smithi122.stdout:1687: copy_from oid 16 from oid 5 current snap is 183 2024-09-15T03:53:05.190 INFO:tasks.rados.rados.0.smithi122.stdout:1685: finishing write tid 1 to smithi12223906-37 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1685: finishing write tid 2 to smithi12223906-37 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1685: finishing write tid 3 to smithi12223906-37 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1685: finishing write tid 4 to smithi12223906-37 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 791 (ObjNum 612 snap 183 seq_num 612) dirty exists 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1685: left oid 37 (ObjNum 612 snap 183 seq_num 612) 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1685: done (2 left) 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1688: read oid 33 snap -1 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1688: expect deleted 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1689: copy_from oid 39 from oid 21 current snap is 183 2024-09-15T03:53:05.191 INFO:tasks.rados.rados.0.smithi122.stdout:1690: delete oid 27 current snap is 183 2024-09-15T03:53:05.195 INFO:tasks.rados.rados.0.smithi122.stdout:1686: finishing write tid 1 to smithi12223906-3 2024-09-15T03:53:05.196 INFO:tasks.rados.rados.0.smithi122.stdout:1686: finishing write tid 2 to smithi12223906-3 2024-09-15T03:53:05.196 INFO:tasks.rados.rados.0.smithi122.stdout:1688: done (4 left) 2024-09-15T03:53:05.196 INFO:tasks.rados.rados.0.smithi122.stdout:1690: done (3 left) 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:1691: copy_from oid 12 from oid 40 current snap is 183 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:1686: finishing write tid 3 to smithi12223906-3 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:1692: write (excl) oid 4 current snap is 183 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:1686: finishing write tid 4 to smithi12223906-3 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 849 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:1686: left oid 3 (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:53:05.197 INFO:tasks.rados.rados.0.smithi122.stdout:1692: seq_num 615 ranges {736305=786375,2105367=230938} 2024-09-15T03:53:05.211 INFO:tasks.rados.rados.0.smithi122.stdout:1692: writing smithi12223906-4 from 736305 to 1522680 tid 1 2024-09-15T03:53:05.216 INFO:tasks.rados.rados.0.smithi122.stdout:1692: writing smithi12223906-4 from 2105367 to 2336305 tid 2 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1686: done (4 left) 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1693: read oid 49 snap -1 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1693: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1694: read oid 26 snap 176 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1694: expect (ObjNum 520 snap 156 seq_num 520) 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1695: read oid 17 snap -1 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1695: expect (ObjNum 590 snap 176 seq_num 590) 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1696: write oid 35 current snap is 183 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1687: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:53:05.217 INFO:tasks.rados.rados.0.smithi122.stdout:1696: seq_num 616 ranges {753112=577977,1880202=472910} 2024-09-15T03:53:05.224 INFO:tasks.rados.rados.0.smithi122.stdout:1696: writing smithi12223906-35 from 753112 to 1331089 tid 1 2024-09-15T03:53:05.230 INFO:tasks.rados.rados.0.smithi122.stdout:1696: writing smithi12223906-35 from 1880202 to 2353112 tid 2 2024-09-15T03:53:05.230 INFO:tasks.rados.rados.0.smithi122.stdout:1697: snap_create 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:1689: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:1689: finishing copy_from to smithi12223906-39 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 782 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:1687: finishing copy_from to smithi12223906-16 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 642 (ObjNum 563 snap 167 seq_num 563) dirty exists 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:1691: finishing copy_from racing read to smithi12223906-12 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:1691: finishing copy_from to smithi12223906-12 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 607 (ObjNum 592 snap 177 seq_num 592) dirty exists 2024-09-15T03:53:05.231 INFO:tasks.rados.rados.0.smithi122.stdout:1693: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:1692: finishing write tid 1 to smithi12223906-4 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:1692: finishing write tid 2 to smithi12223906-4 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:1692: finishing write tid 3 to smithi12223906-4 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:1692: finishing write tid 4 to smithi12223906-4 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 778 (ObjNum 615 snap 183 seq_num 615) dirty exists 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:1692: left oid 4 (ObjNum 615 snap 183 seq_num 615) 2024-09-15T03:53:05.435 INFO:tasks.rados.rados.0.smithi122.stdout:1694: expect (ObjNum 520 snap 156 seq_num 520) 2024-09-15T03:53:05.522 INFO:tasks.rados.rados.0.smithi122.stdout:1695: expect (ObjNum 590 snap 176 seq_num 590) 2024-09-15T03:53:05.749 INFO:tasks.rados.rados.0.smithi122.stdout:1696: finishing write tid 1 to smithi12223906-35 2024-09-15T03:53:05.749 INFO:tasks.rados.rados.0.smithi122.stdout:1696: finishing write tid 2 to smithi12223906-35 2024-09-15T03:53:05.749 INFO:tasks.rados.rados.0.smithi122.stdout:1696: finishing write tid 3 to smithi12223906-35 2024-09-15T03:53:05.749 INFO:tasks.rados.rados.0.smithi122.stdout:1696: finishing write tid 4 to smithi12223906-35 2024-09-15T03:53:05.750 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 580 (ObjNum 616 snap 183 seq_num 616) dirty exists 2024-09-15T03:53:05.750 INFO:tasks.rados.rados.0.smithi122.stdout:1696: left oid 35 (ObjNum 616 snap 183 seq_num 616) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1687: done (8 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1689: done (7 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1691: done (6 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1692: done (5 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1693: done (4 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1694: done (3 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1695: done (2 left) 2024-09-15T03:53:05.775 INFO:tasks.rados.rados.0.smithi122.stdout:1696: done (1 left) 2024-09-15T03:53:05.776 INFO:tasks.rados.rados.0.smithi122.stdout:1697: done (0 left) 2024-09-15T03:53:05.776 INFO:tasks.rados.rados.0.smithi122.stdout:1698: write oid 7 current snap is 184 2024-09-15T03:53:05.776 INFO:tasks.rados.rados.0.smithi122.stdout:1698: seq_num 617 ranges {769919=653226,1938684=431235} 2024-09-15T03:53:05.780 INFO:tasks.rados.rados.0.smithi122.stdout:1698: writing smithi12223906-7 from 769919 to 1423145 tid 1 2024-09-15T03:53:05.785 INFO:tasks.rados.rados.0.smithi122.stdout:1698: writing smithi12223906-7 from 1938684 to 2369919 tid 2 2024-09-15T03:53:05.785 INFO:tasks.rados.rados.0.smithi122.stdout:1699: write oid 39 current snap is 184 2024-09-15T03:53:05.786 INFO:tasks.rados.rados.0.smithi122.stdout:1699: seq_num 618 ranges {786726=728475,2280813=105913} 2024-09-15T03:53:05.794 INFO:tasks.rados.rados.0.smithi122.stdout:1699: writing smithi12223906-39 from 786726 to 1515201 tid 1 2024-09-15T03:53:05.798 INFO:tasks.rados.rados.0.smithi122.stdout:1699: writing smithi12223906-39 from 2280813 to 2386726 tid 2 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:1700: snap_create 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:1698: finishing write tid 1 to smithi12223906-7 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:1698: finishing write tid 2 to smithi12223906-7 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:1698: finishing write tid 3 to smithi12223906-7 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:1698: finishing write tid 4 to smithi12223906-7 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 700 (ObjNum 617 snap 184 seq_num 617) dirty exists 2024-09-15T03:53:05.799 INFO:tasks.rados.rados.0.smithi122.stdout:1698: left oid 7 (ObjNum 617 snap 184 seq_num 617) 2024-09-15T03:53:05.809 INFO:tasks.rados.rados.0.smithi122.stdout:1699: finishing write tid 1 to smithi12223906-39 2024-09-15T03:53:05.809 INFO:tasks.rados.rados.0.smithi122.stdout:1699: finishing write tid 2 to smithi12223906-39 2024-09-15T03:53:05.809 INFO:tasks.rados.rados.0.smithi122.stdout:1699: finishing write tid 3 to smithi12223906-39 2024-09-15T03:53:05.810 INFO:tasks.rados.rados.0.smithi122.stdout:1699: finishing write tid 4 to smithi12223906-39 2024-09-15T03:53:05.810 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 855 (ObjNum 618 snap 184 seq_num 618) dirty exists 2024-09-15T03:53:05.810 INFO:tasks.rados.rados.0.smithi122.stdout:1699: left oid 39 (ObjNum 618 snap 184 seq_num 618) 2024-09-15T03:53:06.735 INFO:tasks.rados.rados.0.smithi122.stdout:1698: done (2 left) 2024-09-15T03:53:06.735 INFO:tasks.rados.rados.0.smithi122.stdout:1699: done (1 left) 2024-09-15T03:53:06.736 INFO:tasks.rados.rados.0.smithi122.stdout:1700: done (0 left) 2024-09-15T03:53:06.736 INFO:tasks.rados.rados.0.smithi122.stdout:1701: rollback oid 48 current snap is 185 2024-09-15T03:53:06.736 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 48 to 181 2024-09-15T03:53:06.736 INFO:tasks.rados.rados.0.smithi122.stdout:1702: read oid 37 snap -1 2024-09-15T03:53:06.736 INFO:tasks.rados.rados.0.smithi122.stdout:1702: expect (ObjNum 612 snap 183 seq_num 612) 2024-09-15T03:53:06.736 INFO:tasks.rados.rados.0.smithi122.stdout:1703: delete oid 20 current snap is 185 2024-09-15T03:53:06.742 INFO:tasks.rados.rados.0.smithi122.stdout:1703: done (2 left) 2024-09-15T03:53:06.742 INFO:tasks.rados.rados.0.smithi122.stdout:1704: read oid 8 snap -1 2024-09-15T03:53:06.743 INFO:tasks.rados.rados.0.smithi122.stdout:1704: expect deleted 2024-09-15T03:53:06.743 INFO:tasks.rados.rados.0.smithi122.stdout:1705: rollback oid 2 current snap is 185 2024-09-15T03:53:06.743 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 184 2024-09-15T03:53:06.743 INFO:tasks.rados.rados.0.smithi122.stdout:1706: delete oid 40 current snap is 185 2024-09-15T03:53:06.750 INFO:tasks.rados.rados.0.smithi122.stdout:1702: expect (ObjNum 612 snap 183 seq_num 612) 2024-09-15T03:53:06.920 INFO:tasks.rados.rados.0.smithi122.stdout:1701: finishing rollback tid 0 to smithi12223906-48 2024-09-15T03:53:06.920 INFO:tasks.rados.rados.0.smithi122.stdout:1701: finishing rollback tid 1 to smithi12223906-48 2024-09-15T03:53:06.920 INFO:tasks.rados.rados.0.smithi122.stdout:1701: finishing rollback tid 2 to smithi12223906-48 2024-09-15T03:53:06.920 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 794 (ObjNum 527 snap 159 seq_num 527) dirty exists 2024-09-15T03:53:06.920 INFO:tasks.rados.rados.0.smithi122.stdout:1705: finishing rollback tid 0 to smithi12223906-2 2024-09-15T03:53:06.920 INFO:tasks.rados.rados.0.smithi122.stdout:1705: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1705: finishing rollback tid 2 to smithi12223906-2 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 456 (ObjNum 601 snap 180 seq_num 601) dirty exists 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1701: done (4 left) 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1702: done (3 left) 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1704: done (2 left) 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1705: done (1 left) 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1706: done (0 left) 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1707: write oid 11 current snap is 185 2024-09-15T03:53:06.921 INFO:tasks.rados.rados.0.smithi122.stdout:1707: seq_num 621 ranges {437147=554222,1656259=655870} 2024-09-15T03:53:06.924 INFO:tasks.rados.rados.0.smithi122.stdout:1707: writing smithi12223906-11 from 437147 to 991369 tid 1 2024-09-15T03:53:06.934 INFO:tasks.rados.rados.0.smithi122.stdout:1707: writing smithi12223906-11 from 1656259 to 2312129 tid 2 2024-09-15T03:53:06.934 INFO:tasks.rados.rados.0.smithi122.stdout:1708: snap_remove snap 172 2024-09-15T03:53:07.781 INFO:tasks.rados.rados.0.smithi122.stdout:1708: done (1 left) 2024-09-15T03:53:07.781 INFO:tasks.rados.rados.0.smithi122.stdout:1709: write oid 22 current snap is 185 2024-09-15T03:53:07.781 INFO:tasks.rados.rados.0.smithi122.stdout:1707: finishing write tid 1 to smithi12223906-11 2024-09-15T03:53:07.782 INFO:tasks.rados.rados.0.smithi122.stdout:1709: seq_num 622 ranges {453954=629471,1598388=515881} 2024-09-15T03:53:07.790 INFO:tasks.rados.rados.0.smithi122.stdout:1709: writing smithi12223906-22 from 453954 to 1083425 tid 1 2024-09-15T03:53:07.797 INFO:tasks.rados.rados.0.smithi122.stdout:1709: writing smithi12223906-22 from 1598388 to 2114269 tid 2 2024-09-15T03:53:07.797 INFO:tasks.rados.rados.0.smithi122.stdout:1710: write oid 23 current snap is 185 2024-09-15T03:53:07.797 INFO:tasks.rados.rados.0.smithi122.stdout:1707: finishing write tid 2 to smithi12223906-11 2024-09-15T03:53:07.797 INFO:tasks.rados.rados.0.smithi122.stdout:1710: seq_num 623 ranges {470761=704720,1656870=659539} 2024-09-15T03:53:07.804 INFO:tasks.rados.rados.0.smithi122.stdout:1710: writing smithi12223906-23 from 470761 to 1175481 tid 1 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1710: writing smithi12223906-23 from 1656870 to 2316409 tid 2 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1711: snap_create 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1707: finishing write tid 3 to smithi12223906-11 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1707: finishing write tid 4 to smithi12223906-11 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 645 (ObjNum 621 snap 185 seq_num 621) dirty exists 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1707: left oid 11 (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1709: finishing write tid 1 to smithi12223906-22 2024-09-15T03:53:07.812 INFO:tasks.rados.rados.0.smithi122.stdout:1709: finishing write tid 2 to smithi12223906-22 2024-09-15T03:53:07.813 INFO:tasks.rados.rados.0.smithi122.stdout:1709: finishing write tid 3 to smithi12223906-22 2024-09-15T03:53:07.813 INFO:tasks.rados.rados.0.smithi122.stdout:1709: finishing write tid 4 to smithi12223906-22 2024-09-15T03:53:07.813 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 841 (ObjNum 622 snap 185 seq_num 622) dirty exists 2024-09-15T03:53:07.813 INFO:tasks.rados.rados.0.smithi122.stdout:1709: left oid 22 (ObjNum 622 snap 185 seq_num 622) 2024-09-15T03:53:07.813 INFO:tasks.rados.rados.0.smithi122.stdout:1710: finishing write tid 1 to smithi12223906-23 2024-09-15T03:53:07.814 INFO:tasks.rados.rados.0.smithi122.stdout:1710: finishing write tid 2 to smithi12223906-23 2024-09-15T03:53:07.815 INFO:tasks.rados.rados.0.smithi122.stdout:1710: finishing write tid 3 to smithi12223906-23 2024-09-15T03:53:07.815 INFO:tasks.rados.rados.0.smithi122.stdout:1710: finishing write tid 4 to smithi12223906-23 2024-09-15T03:53:07.815 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 845 (ObjNum 623 snap 185 seq_num 623) dirty exists 2024-09-15T03:53:07.815 INFO:tasks.rados.rados.0.smithi122.stdout:1710: left oid 23 (ObjNum 623 snap 185 seq_num 623) 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1707: done (3 left) 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1709: done (2 left) 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1710: done (1 left) 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1711: done (0 left) 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1712: read oid 32 snap -1 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1712: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:53:08.792 INFO:tasks.rados.rados.0.smithi122.stdout:1713: delete oid 40 current snap is 186 2024-09-15T03:53:08.794 INFO:tasks.rados.rados.0.smithi122.stdout:1713: done (1 left) 2024-09-15T03:53:08.794 INFO:tasks.rados.rados.0.smithi122.stdout:1714: snap_create 2024-09-15T03:53:08.804 INFO:tasks.rados.rados.0.smithi122.stdout:1712: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:53:09.802 INFO:tasks.rados.rados.0.smithi122.stdout:1712: done (1 left) 2024-09-15T03:53:09.802 INFO:tasks.rados.rados.0.smithi122.stdout:1714: done (0 left) 2024-09-15T03:53:09.802 INFO:tasks.rados.rados.0.smithi122.stdout:1715: write oid 16 current snap is 187 2024-09-15T03:53:09.802 INFO:tasks.rados.rados.0.smithi122.stdout:1715: seq_num 625 ranges {504375=571571,1490187=663208} 2024-09-15T03:53:09.810 INFO:tasks.rados.rados.0.smithi122.stdout:1715: writing smithi12223906-16 from 504375 to 1075946 tid 1 2024-09-15T03:53:09.817 INFO:tasks.rados.rados.0.smithi122.stdout:1715: writing smithi12223906-16 from 1490187 to 2153395 tid 2 2024-09-15T03:53:09.817 INFO:tasks.rados.rados.0.smithi122.stdout:1716: write (excl) oid 46 current snap is 187 2024-09-15T03:53:09.817 INFO:tasks.rados.rados.0.smithi122.stdout:1716: seq_num 626 ranges {521182=646820,1948669=523219} 2024-09-15T03:53:09.824 INFO:tasks.rados.rados.0.smithi122.stdout:1716: writing smithi12223906-46 from 521182 to 1168002 tid 1 2024-09-15T03:53:09.831 INFO:tasks.rados.rados.0.smithi122.stdout:1716: writing smithi12223906-46 from 1948669 to 2471888 tid 2 2024-09-15T03:53:09.831 INFO:tasks.rados.rados.0.smithi122.stdout:1717: write (excl) oid 2 current snap is 187 2024-09-15T03:53:09.831 INFO:tasks.rados.rados.0.smithi122.stdout:1717: seq_num 627 ranges {537989=722069,1890798=647191} 2024-09-15T03:53:09.838 INFO:tasks.rados.rados.0.smithi122.stdout:1717: writing smithi12223906-2 from 537989 to 1260058 tid 1 2024-09-15T03:53:09.845 INFO:tasks.rados.rados.0.smithi122.stdout:1717: writing smithi12223906-2 from 1890798 to 2537989 tid 2 2024-09-15T03:53:09.846 INFO:tasks.rados.rados.0.smithi122.stdout:1718: write oid 25 current snap is 187 2024-09-15T03:53:09.846 INFO:tasks.rados.rados.0.smithi122.stdout:1718: seq_num 628 ranges {554796=797318,1949280=526888} 2024-09-15T03:53:09.853 INFO:tasks.rados.rados.0.smithi122.stdout:1718: writing smithi12223906-25 from 554796 to 1352114 tid 1 2024-09-15T03:53:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:1718: writing smithi12223906-25 from 1949280 to 2476168 tid 2 2024-09-15T03:53:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:1719: snap_create 2024-09-15T03:53:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:1715: finishing write tid 1 to smithi12223906-16 2024-09-15T03:53:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:1715: finishing write tid 2 to smithi12223906-16 2024-09-15T03:53:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:1715: finishing write tid 3 to smithi12223906-16 2024-09-15T03:53:09.860 INFO:tasks.rados.rados.0.smithi122.stdout:1715: finishing write tid 4 to smithi12223906-16 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 648 (ObjNum 625 snap 187 seq_num 625) dirty exists 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1715: left oid 16 (ObjNum 625 snap 187 seq_num 625) 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1716: finishing write tid 1 to smithi12223906-46 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1716: finishing write tid 2 to smithi12223906-46 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1716: finishing write tid 3 to smithi12223906-46 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1716: finishing write tid 4 to smithi12223906-46 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 782 (ObjNum 626 snap 187 seq_num 626) dirty exists 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1716: left oid 46 (ObjNum 626 snap 187 seq_num 626) 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1717: finishing write tid 1 to smithi12223906-2 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1717: finishing write tid 2 to smithi12223906-2 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1717: finishing write tid 3 to smithi12223906-2 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:1717: finishing write tid 4 to smithi12223906-2 2024-09-15T03:53:09.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 464 (ObjNum 627 snap 187 seq_num 627) dirty exists 2024-09-15T03:53:09.862 INFO:tasks.rados.rados.0.smithi122.stdout:1717: left oid 2 (ObjNum 627 snap 187 seq_num 627) 2024-09-15T03:53:09.868 INFO:tasks.rados.rados.0.smithi122.stdout:1718: finishing write tid 1 to smithi12223906-25 2024-09-15T03:53:09.868 INFO:tasks.rados.rados.0.smithi122.stdout:1718: finishing write tid 2 to smithi12223906-25 2024-09-15T03:53:09.868 INFO:tasks.rados.rados.0.smithi122.stdout:1718: finishing write tid 3 to smithi12223906-25 2024-09-15T03:53:09.869 INFO:tasks.rados.rados.0.smithi122.stdout:1718: finishing write tid 4 to smithi12223906-25 2024-09-15T03:53:09.869 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 873 (ObjNum 628 snap 187 seq_num 628) dirty exists 2024-09-15T03:53:09.869 INFO:tasks.rados.rados.0.smithi122.stdout:1718: left oid 25 (ObjNum 628 snap 187 seq_num 628) 2024-09-15T03:53:10.810 INFO:tasks.rados.rados.0.smithi122.stdout:1715: done (4 left) 2024-09-15T03:53:10.811 INFO:tasks.rados.rados.0.smithi122.stdout:1716: done (3 left) 2024-09-15T03:53:10.811 INFO:tasks.rados.rados.0.smithi122.stdout:1717: done (2 left) 2024-09-15T03:53:10.811 INFO:tasks.rados.rados.0.smithi122.stdout:1718: done (1 left) 2024-09-15T03:53:10.811 INFO:tasks.rados.rados.0.smithi122.stdout:1719: done (0 left) 2024-09-15T03:53:10.811 INFO:tasks.rados.rados.0.smithi122.stdout:1720: snap_create 2024-09-15T03:53:11.738 INFO:tasks.rados.rados.0.smithi122.stdout:1720: done (0 left) 2024-09-15T03:53:11.738 INFO:tasks.rados.rados.0.smithi122.stdout:1721: snap_create 2024-09-15T03:53:12.738 INFO:tasks.rados.rados.0.smithi122.stdout:1721: done (0 left) 2024-09-15T03:53:12.738 INFO:tasks.rados.rados.0.smithi122.stdout:1722: snap_remove snap 187 2024-09-15T03:53:13.742 INFO:tasks.rados.rados.0.smithi122.stdout:1722: done (0 left) 2024-09-15T03:53:13.742 INFO:tasks.rados.rados.0.smithi122.stdout:1723: snap_create 2024-09-15T03:53:14.747 INFO:tasks.rados.rados.0.smithi122.stdout:1723: done (0 left) 2024-09-15T03:53:14.748 INFO:tasks.rados.rados.0.smithi122.stdout:1724: delete oid 47 current snap is 191 2024-09-15T03:53:14.755 INFO:tasks.rados.rados.0.smithi122.stdout:1724: done (0 left) 2024-09-15T03:53:14.755 INFO:tasks.rados.rados.0.smithi122.stdout:1725: snap_remove snap 181 2024-09-15T03:53:15.771 INFO:tasks.rados.rados.0.smithi122.stdout:1725: done (0 left) 2024-09-15T03:53:15.771 INFO:tasks.rados.rados.0.smithi122.stdout:1726: rollback oid 24 current snap is 191 2024-09-15T03:53:15.771 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 183 2024-09-15T03:53:15.771 INFO:tasks.rados.rados.0.smithi122.stdout:1727: snap_create 2024-09-15T03:53:15.780 INFO:tasks.rados.rados.0.smithi122.stdout:1726: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:53:15.780 INFO:tasks.rados.rados.0.smithi122.stdout:1726: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:53:15.780 INFO:tasks.rados.rados.0.smithi122.stdout:1726: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:53:15.780 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 881 (ObjNum 551 snap 163 seq_num 551) dirty exists 2024-09-15T03:53:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:1726: done (1 left) 2024-09-15T03:53:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:1727: done (0 left) 2024-09-15T03:53:16.738 INFO:tasks.rados.rados.0.smithi122.stdout:1728: write (excl) oid 6 current snap is 192 2024-09-15T03:53:16.739 INFO:tasks.rados.rados.0.smithi122.stdout:1728: seq_num 630 ranges {588410=547816,1666244=414204,2484603=103807} 2024-09-15T03:53:16.745 INFO:tasks.rados.rados.0.smithi122.stdout:1728: writing smithi12223906-6 from 588410 to 1136226 tid 1 2024-09-15T03:53:16.751 INFO:tasks.rados.rados.0.smithi122.stdout:1728: writing smithi12223906-6 from 1666244 to 2080448 tid 2 2024-09-15T03:53:16.754 INFO:tasks.rados.rados.0.smithi122.stdout:1728: writing smithi12223906-6 from 2484603 to 2588410 tid 3 2024-09-15T03:53:16.754 INFO:tasks.rados.rados.0.smithi122.stdout:1729: snap_remove snap 180 2024-09-15T03:53:17.742 INFO:tasks.rados.rados.0.smithi122.stdout:1729: done (1 left) 2024-09-15T03:53:17.742 INFO:tasks.rados.rados.0.smithi122.stdout:1730: write (excl) oid 34 current snap is 192 2024-09-15T03:53:17.743 INFO:tasks.rados.rados.0.smithi122.stdout:1730: seq_num 631 ranges {605217=739418,2124726=480491} 2024-09-15T03:53:17.751 INFO:tasks.rados.rados.0.smithi122.stdout:1730: writing smithi12223906-34 from 605217 to 1344635 tid 1 2024-09-15T03:53:17.759 INFO:tasks.rados.rados.0.smithi122.stdout:1730: writing smithi12223906-34 from 2124726 to 2605217 tid 2 2024-09-15T03:53:17.759 INFO:tasks.rados.rados.0.smithi122.stdout:1731: delete oid 4 current snap is 192 2024-09-15T03:53:17.759 INFO:tasks.rados.rados.0.smithi122.stdout:1728: finishing write tid 1 to smithi12223906-6 2024-09-15T03:53:17.759 INFO:tasks.rados.rados.0.smithi122.stdout:1728: finishing write tid 2 to smithi12223906-6 2024-09-15T03:53:17.760 INFO:tasks.rados.rados.0.smithi122.stdout:1728: finishing write tid 3 to smithi12223906-6 2024-09-15T03:53:17.760 INFO:tasks.rados.rados.0.smithi122.stdout:1728: finishing write tid 4 to smithi12223906-6 2024-09-15T03:53:17.760 INFO:tasks.rados.rados.0.smithi122.stdout:1728: finishing write tid 5 to smithi12223906-6 2024-09-15T03:53:17.760 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 798 (ObjNum 630 snap 192 seq_num 630) dirty exists 2024-09-15T03:53:17.760 INFO:tasks.rados.rados.0.smithi122.stdout:1728: left oid 6 (ObjNum 630 snap 192 seq_num 630) 2024-09-15T03:53:17.760 INFO:tasks.rados.rados.0.smithi122.stdout:1730: finishing write tid 1 to smithi12223906-34 2024-09-15T03:53:17.763 INFO:tasks.rados.rados.0.smithi122.stdout:1730: finishing write tid 2 to smithi12223906-34 2024-09-15T03:53:17.764 INFO:tasks.rados.rados.0.smithi122.stdout:1728: done (2 left) 2024-09-15T03:53:17.764 INFO:tasks.rados.rados.0.smithi122.stdout:1731: done (1 left) 2024-09-15T03:53:17.765 INFO:tasks.rados.rados.0.smithi122.stdout:1732: write oid 36 current snap is 192 2024-09-15T03:53:17.765 INFO:tasks.rados.rados.0.smithi122.stdout:1732: seq_num 633 ranges {638831=489916,1841690=677884} 2024-09-15T03:53:17.774 INFO:tasks.rados.rados.0.smithi122.stdout:1732: writing smithi12223906-36 from 638831 to 1128747 tid 1 2024-09-15T03:53:17.785 INFO:tasks.rados.rados.0.smithi122.stdout:1732: writing smithi12223906-36 from 1841690 to 2519574 tid 2 2024-09-15T03:53:17.786 INFO:tasks.rados.rados.0.smithi122.stdout:1733: snap_remove snap 177 2024-09-15T03:53:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:1730: finishing write tid 3 to smithi12223906-34 2024-09-15T03:53:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:1730: finishing write tid 4 to smithi12223906-34 2024-09-15T03:53:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 722 (ObjNum 631 snap 192 seq_num 631) dirty exists 2024-09-15T03:53:18.780 INFO:tasks.rados.rados.0.smithi122.stdout:1730: left oid 34 (ObjNum 631 snap 192 seq_num 631) 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1732: finishing write tid 1 to smithi12223906-36 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1732: finishing write tid 2 to smithi12223906-36 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1732: finishing write tid 3 to smithi12223906-36 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1732: finishing write tid 4 to smithi12223906-36 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 897 (ObjNum 633 snap 192 seq_num 633) dirty exists 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1732: left oid 36 (ObjNum 633 snap 192 seq_num 633) 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1730: done (2 left) 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1732: done (1 left) 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1733: done (0 left) 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1734: read oid 49 snap -1 2024-09-15T03:53:18.781 INFO:tasks.rados.rados.0.smithi122.stdout:1734: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:53:18.782 INFO:tasks.rados.rados.0.smithi122.stdout:1735: write oid 32 current snap is 192 2024-09-15T03:53:18.782 INFO:tasks.rados.rados.0.smithi122.stdout:1735: seq_num 634 ranges {655638=565165,1900172=421542} 2024-09-15T03:53:18.792 INFO:tasks.rados.rados.0.smithi122.stdout:1735: writing smithi12223906-32 from 655638 to 1220803 tid 1 2024-09-15T03:53:18.800 INFO:tasks.rados.rados.0.smithi122.stdout:1735: writing smithi12223906-32 from 1900172 to 2321714 tid 2 2024-09-15T03:53:18.800 INFO:tasks.rados.rados.0.smithi122.stdout:1736: snap_create 2024-09-15T03:53:18.800 INFO:tasks.rados.rados.0.smithi122.stdout:1734: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:53:19.007 INFO:tasks.rados.rados.0.smithi122.stdout:1735: finishing write tid 1 to smithi12223906-32 2024-09-15T03:53:19.008 INFO:tasks.rados.rados.0.smithi122.stdout:1735: finishing write tid 2 to smithi12223906-32 2024-09-15T03:53:19.008 INFO:tasks.rados.rados.0.smithi122.stdout:1735: finishing write tid 3 to smithi12223906-32 2024-09-15T03:53:19.008 INFO:tasks.rados.rados.0.smithi122.stdout:1735: finishing write tid 4 to smithi12223906-32 2024-09-15T03:53:19.008 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 867 (ObjNum 634 snap 192 seq_num 634) dirty exists 2024-09-15T03:53:19.008 INFO:tasks.rados.rados.0.smithi122.stdout:1735: left oid 32 (ObjNum 634 snap 192 seq_num 634) 2024-09-15T03:53:19.798 INFO:tasks.rados.rados.0.smithi122.stdout:1734: done (2 left) 2024-09-15T03:53:19.798 INFO:tasks.rados.rados.0.smithi122.stdout:1735: done (1 left) 2024-09-15T03:53:19.798 INFO:tasks.rados.rados.0.smithi122.stdout:1736: done (0 left) 2024-09-15T03:53:19.799 INFO:tasks.rados.rados.0.smithi122.stdout:1737: snap_create 2024-09-15T03:53:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:1737: done (0 left) 2024-09-15T03:53:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:1738: read oid 4 snap -1 2024-09-15T03:53:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:1738: expect deleted 2024-09-15T03:53:20.804 INFO:tasks.rados.rados.0.smithi122.stdout:1739: read oid 38 snap -1 2024-09-15T03:53:20.805 INFO:tasks.rados.rados.0.smithi122.stdout:1739: expect (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:53:20.805 INFO:tasks.rados.rados.0.smithi122.stdout:1740: read oid 46 snap -1 2024-09-15T03:53:20.805 INFO:tasks.rados.rados.0.smithi122.stdout:1740: expect (ObjNum 626 snap 187 seq_num 626) 2024-09-15T03:53:20.805 INFO:tasks.rados.rados.0.smithi122.stdout:1741: snap_create 2024-09-15T03:53:20.814 INFO:tasks.rados.rados.0.smithi122.stdout:1740: expect (ObjNum 626 snap 187 seq_num 626) 2024-09-15T03:53:20.901 INFO:tasks.rados.rados.0.smithi122.stdout:1739: expect (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:53:21.741 INFO:tasks.rados.rados.0.smithi122.stdout:1738: done (3 left) 2024-09-15T03:53:21.742 INFO:tasks.rados.rados.0.smithi122.stdout:1739: done (2 left) 2024-09-15T03:53:21.742 INFO:tasks.rados.rados.0.smithi122.stdout:1740: done (1 left) 2024-09-15T03:53:21.742 INFO:tasks.rados.rados.0.smithi122.stdout:1741: done (0 left) 2024-09-15T03:53:21.742 INFO:tasks.rados.rados.0.smithi122.stdout:1742: copy_from oid 9 from oid 41 current snap is 195 2024-09-15T03:53:21.742 INFO:tasks.rados.rados.0.smithi122.stdout:1743: write oid 10 current snap is 195 2024-09-15T03:53:21.742 INFO:tasks.rados.rados.0.smithi122.stdout:1743: seq_num 635 ranges {672445=640414,1842301=681553} 2024-09-15T03:53:21.749 INFO:tasks.rados.rados.0.smithi122.stdout:1743: writing smithi12223906-10 from 672445 to 1312859 tid 1 2024-09-15T03:53:21.756 INFO:tasks.rados.rados.0.smithi122.stdout:1743: writing smithi12223906-10 from 1842301 to 2523854 tid 2 2024-09-15T03:53:21.756 INFO:tasks.rados.rados.0.smithi122.stdout:1744: read oid 8 snap -1 2024-09-15T03:53:21.756 INFO:tasks.rados.rados.0.smithi122.stdout:1744: expect deleted 2024-09-15T03:53:21.756 INFO:tasks.rados.rados.0.smithi122.stdout:1745: snap_remove snap 183 2024-09-15T03:53:22.755 INFO:tasks.rados.rados.0.smithi122.stdout:1745: done (3 left) 2024-09-15T03:53:22.755 INFO:tasks.rados.rados.0.smithi122.stdout:1746: write (excl) oid 6 current snap is 195 2024-09-15T03:53:22.755 INFO:tasks.rados.rados.0.smithi122.stdout:1746: seq_num 636 ranges {689252=715663,1900783=425211} 2024-09-15T03:53:22.764 INFO:tasks.rados.rados.0.smithi122.stdout:1746: writing smithi12223906-6 from 689252 to 1404915 tid 1 2024-09-15T03:53:22.770 INFO:tasks.rados.rados.0.smithi122.stdout:1746: writing smithi12223906-6 from 1900783 to 2325994 tid 2 2024-09-15T03:53:22.770 INFO:tasks.rados.rados.0.smithi122.stdout:1747: snap_create 2024-09-15T03:53:22.770 INFO:tasks.rados.rados.0.smithi122.stdout:1742: finishing copy_from to smithi12223906-9 2024-09-15T03:53:22.770 INFO:tasks.rados.rados.0.smithi122.stdout:1742: got expected ENOENT (src dne) 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:1742: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:1743: finishing write tid 1 to smithi12223906-10 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:1743: finishing write tid 2 to smithi12223906-10 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:1743: finishing write tid 3 to smithi12223906-10 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:1743: finishing write tid 4 to smithi12223906-10 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 797 (ObjNum 635 snap 195 seq_num 635) dirty exists 2024-09-15T03:53:22.771 INFO:tasks.rados.rados.0.smithi122.stdout:1743: left oid 10 (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:53:22.778 INFO:tasks.rados.rados.0.smithi122.stdout:1746: finishing write tid 1 to smithi12223906-6 2024-09-15T03:53:22.778 INFO:tasks.rados.rados.0.smithi122.stdout:1746: finishing write tid 2 to smithi12223906-6 2024-09-15T03:53:22.778 INFO:tasks.rados.rados.0.smithi122.stdout:1746: finishing write tid 3 to smithi12223906-6 2024-09-15T03:53:22.779 INFO:tasks.rados.rados.0.smithi122.stdout:1746: finishing write tid 4 to smithi12223906-6 2024-09-15T03:53:22.779 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 801 (ObjNum 636 snap 195 seq_num 636) dirty exists 2024-09-15T03:53:22.779 INFO:tasks.rados.rados.0.smithi122.stdout:1746: left oid 6 (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1742: done (4 left) 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1743: done (3 left) 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1744: done (2 left) 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1746: done (1 left) 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1747: done (0 left) 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1748: read oid 8 snap -1 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1748: expect deleted 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1749: read oid 20 snap 188 2024-09-15T03:53:23.769 INFO:tasks.rados.rados.0.smithi122.stdout:1749: expect deleted 2024-09-15T03:53:23.770 INFO:tasks.rados.rados.0.smithi122.stdout:1750: snap_create 2024-09-15T03:53:24.772 INFO:tasks.rados.rados.0.smithi122.stdout:1748: done (2 left) 2024-09-15T03:53:24.773 INFO:tasks.rados.rados.0.smithi122.stdout:1749: done (1 left) 2024-09-15T03:53:24.773 INFO:tasks.rados.rados.0.smithi122.stdout:1750: done (0 left) 2024-09-15T03:53:24.773 INFO:tasks.rados.rados.0.smithi122.stdout:1751: write (excl) oid 50 current snap is 197 2024-09-15T03:53:24.773 INFO:tasks.rados.rados.0.smithi122.stdout:1751: seq_num 637 ranges {706059=790912,1959265=685222} 2024-09-15T03:53:24.783 INFO:tasks.rados.rados.0.smithi122.stdout:1751: writing smithi12223906-50 from 706059 to 1496971 tid 1 2024-09-15T03:53:24.790 INFO:tasks.rados.rados.0.smithi122.stdout:1751: writing smithi12223906-50 from 1959265 to 2644487 tid 2 2024-09-15T03:53:24.790 INFO:tasks.rados.rados.0.smithi122.stdout:1752: read oid 10 snap -1 2024-09-15T03:53:24.790 INFO:tasks.rados.rados.0.smithi122.stdout:1752: expect (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:53:24.790 INFO:tasks.rados.rados.0.smithi122.stdout:1753: write (excl) oid 47 current snap is 197 2024-09-15T03:53:24.790 INFO:tasks.rados.rados.0.smithi122.stdout:1753: seq_num 638 ranges {722866=466161,1617747=428880,2587634=135232} 2024-09-15T03:53:24.797 INFO:tasks.rados.rados.0.smithi122.stdout:1753: writing smithi12223906-47 from 722866 to 1189027 tid 1 2024-09-15T03:53:24.802 INFO:tasks.rados.rados.0.smithi122.stdout:1753: writing smithi12223906-47 from 1617747 to 2046627 tid 2 2024-09-15T03:53:24.806 INFO:tasks.rados.rados.0.smithi122.stdout:1753: writing smithi12223906-47 from 2587634 to 2722866 tid 3 2024-09-15T03:53:24.806 INFO:tasks.rados.rados.0.smithi122.stdout:1754: snap_create 2024-09-15T03:53:24.806 INFO:tasks.rados.rados.0.smithi122.stdout:1752: expect (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:53:25.042 INFO:tasks.rados.rados.0.smithi122.stdout:1751: finishing write tid 1 to smithi12223906-50 2024-09-15T03:53:25.042 INFO:tasks.rados.rados.0.smithi122.stdout:1751: finishing write tid 2 to smithi12223906-50 2024-09-15T03:53:25.042 INFO:tasks.rados.rados.0.smithi122.stdout:1751: finishing write tid 3 to smithi12223906-50 2024-09-15T03:53:25.042 INFO:tasks.rados.rados.0.smithi122.stdout:1751: finishing write tid 4 to smithi12223906-50 2024-09-15T03:53:25.042 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 875 (ObjNum 637 snap 197 seq_num 637) dirty exists 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1751: left oid 50 (ObjNum 637 snap 197 seq_num 637) 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1753: finishing write tid 1 to smithi12223906-47 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1753: finishing write tid 2 to smithi12223906-47 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1753: finishing write tid 3 to smithi12223906-47 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1753: finishing write tid 4 to smithi12223906-47 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1753: finishing write tid 5 to smithi12223906-47 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 647 (ObjNum 638 snap 197 seq_num 638) dirty exists 2024-09-15T03:53:25.043 INFO:tasks.rados.rados.0.smithi122.stdout:1753: left oid 47 (ObjNum 638 snap 197 seq_num 638) 2024-09-15T03:53:25.786 INFO:tasks.rados.rados.0.smithi122.stdout:1751: done (3 left) 2024-09-15T03:53:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:1752: done (2 left) 2024-09-15T03:53:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:1753: done (1 left) 2024-09-15T03:53:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:1754: done (0 left) 2024-09-15T03:53:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:1755: read oid 22 snap 176 2024-09-15T03:53:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:1755: expect deleted 2024-09-15T03:53:25.787 INFO:tasks.rados.rados.0.smithi122.stdout:1756: snap_create 2024-09-15T03:53:26.744 INFO:tasks.rados.rados.0.smithi122.stdout:1755: done (1 left) 2024-09-15T03:53:26.744 INFO:tasks.rados.rados.0.smithi122.stdout:1756: done (0 left) 2024-09-15T03:53:26.744 INFO:tasks.rados.rados.0.smithi122.stdout:1757: snap_remove snap 185 2024-09-15T03:53:27.750 INFO:tasks.rados.rados.0.smithi122.stdout:1757: done (0 left) 2024-09-15T03:53:27.751 INFO:tasks.rados.rados.0.smithi122.stdout:1758: delete oid 35 current snap is 199 2024-09-15T03:53:27.761 INFO:tasks.rados.rados.0.smithi122.stdout:1758: done (0 left) 2024-09-15T03:53:27.761 INFO:tasks.rados.rados.0.smithi122.stdout:1759: write oid 1 current snap is 199 2024-09-15T03:53:27.761 INFO:tasks.rados.rados.0.smithi122.stdout:1759: seq_num 640 ranges {756480=733012,2134711=432549} 2024-09-15T03:53:27.770 INFO:tasks.rados.rados.0.smithi122.stdout:1759: writing smithi12223906-1 from 756480 to 1489492 tid 1 2024-09-15T03:53:27.777 INFO:tasks.rados.rados.0.smithi122.stdout:1759: writing smithi12223906-1 from 2134711 to 2567260 tid 2 2024-09-15T03:53:27.777 INFO:tasks.rados.rados.0.smithi122.stdout:1760: read oid 29 snap -1 2024-09-15T03:53:27.777 INFO:tasks.rados.rados.0.smithi122.stdout:1760: expect (ObjNum 609 snap 183 seq_num 609) 2024-09-15T03:53:27.777 INFO:tasks.rados.rados.0.smithi122.stdout:1761: delete oid 32 current snap is 199 2024-09-15T03:53:27.777 INFO:tasks.rados.rados.0.smithi122.stdout:1759: finishing write tid 1 to smithi12223906-1 2024-09-15T03:53:27.781 INFO:tasks.rados.rados.0.smithi122.stdout:1760: expect (ObjNum 609 snap 183 seq_num 609) 2024-09-15T03:53:27.941 INFO:tasks.rados.rados.0.smithi122.stdout:1759: finishing write tid 2 to smithi12223906-1 2024-09-15T03:53:27.941 INFO:tasks.rados.rados.0.smithi122.stdout:1759: finishing write tid 3 to smithi12223906-1 2024-09-15T03:53:27.941 INFO:tasks.rados.rados.0.smithi122.stdout:1759: finishing write tid 4 to smithi12223906-1 2024-09-15T03:53:27.941 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 651 (ObjNum 640 snap 199 seq_num 640) dirty exists 2024-09-15T03:53:27.942 INFO:tasks.rados.rados.0.smithi122.stdout:1759: left oid 1 (ObjNum 640 snap 199 seq_num 640) 2024-09-15T03:53:27.942 INFO:tasks.rados.rados.0.smithi122.stdout:1759: done (2 left) 2024-09-15T03:53:27.942 INFO:tasks.rados.rados.0.smithi122.stdout:1760: done (1 left) 2024-09-15T03:53:27.942 INFO:tasks.rados.rados.0.smithi122.stdout:1761: done (0 left) 2024-09-15T03:53:27.942 INFO:tasks.rados.rados.0.smithi122.stdout:1762: write oid 2 current snap is 199 2024-09-15T03:53:27.942 INFO:tasks.rados.rados.0.smithi122.stdout:1762: seq_num 642 ranges {790094=483510,1851675=436218} 2024-09-15T03:53:27.954 INFO:tasks.rados.rados.0.smithi122.stdout:1762: writing smithi12223906-2 from 790094 to 1273604 tid 1 2024-09-15T03:53:27.960 INFO:tasks.rados.rados.0.smithi122.stdout:1762: writing smithi12223906-2 from 1851675 to 2287893 tid 2 2024-09-15T03:53:27.960 INFO:tasks.rados.rados.0.smithi122.stdout:1763: read oid 47 snap 190 2024-09-15T03:53:27.960 INFO:tasks.rados.rados.0.smithi122.stdout:1763: expect (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:53:27.960 INFO:tasks.rados.rados.0.smithi122.stdout:1764: rollback oid 42 current snap is 199 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 192 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1765: read oid 41 snap -1 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1765: expect deleted 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1766: copy_from oid 28 from oid 46 current snap is 199 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1767: read oid 16 snap -1 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1767: expect (ObjNum 625 snap 187 seq_num 625) 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1768: write (excl) oid 21 current snap is 199 2024-09-15T03:53:27.961 INFO:tasks.rados.rados.0.smithi122.stdout:1768: seq_num 643 ranges {406901=558759,1393804=579876,2608396=198505} 2024-09-15T03:53:27.969 INFO:tasks.rados.rados.0.smithi122.stdout:1768: writing smithi12223906-21 from 406901 to 965660 tid 1 2024-09-15T03:53:27.972 INFO:tasks.rados.rados.0.smithi122.stdout:1768: writing smithi12223906-21 from 1393804 to 1973680 tid 2 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1768: writing smithi12223906-21 from 2608396 to 2806901 tid 3 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1769: read oid 4 snap -1 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1769: expect deleted 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1770: read oid 45 snap -1 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1770: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1771: rollback oid 8 current snap is 199 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 182 2024-09-15T03:53:27.978 INFO:tasks.rados.rados.0.smithi122.stdout:1767: expect (ObjNum 625 snap 187 seq_num 625) 2024-09-15T03:53:28.139 INFO:tasks.rados.rados.0.smithi122.stdout:1762: finishing write tid 1 to smithi12223906-2 2024-09-15T03:53:28.139 INFO:tasks.rados.rados.0.smithi122.stdout:1762: finishing write tid 2 to smithi12223906-2 2024-09-15T03:53:28.139 INFO:tasks.rados.rados.0.smithi122.stdout:1762: finishing write tid 3 to smithi12223906-2 2024-09-15T03:53:28.139 INFO:tasks.rados.rados.0.smithi122.stdout:1762: finishing write tid 4 to smithi12223906-2 2024-09-15T03:53:28.139 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 478 (ObjNum 642 snap 199 seq_num 642) dirty exists 2024-09-15T03:53:28.139 INFO:tasks.rados.rados.0.smithi122.stdout:1762: left oid 2 (ObjNum 642 snap 199 seq_num 642) 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1766: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1762: done (9 left) 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1765: done (8 left) 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1767: done (7 left) 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1772: delete oid 32 current snap is 199 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1766: finishing copy_from to smithi12223906-28 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 782 (ObjNum 626 snap 187 seq_num 626) dirty exists 2024-09-15T03:53:28.140 INFO:tasks.rados.rados.0.smithi122.stdout:1763: expect (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:53:28.359 INFO:tasks.rados.rados.0.smithi122.stdout:1764: finishing rollback tid 0 to smithi12223906-42 2024-09-15T03:53:28.359 INFO:tasks.rados.rados.0.smithi122.stdout:1764: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1764: finishing rollback tid 2 to smithi12223906-42 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 929 (ObjNum 561 snap 167 seq_num 561) dirty exists 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1771: finishing rollback tid 1 to smithi12223906-8 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 0 (ObjNum 65 snap 0 seq_num 1842594912) dirty dne 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1768: finishing write tid 1 to smithi12223906-21 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1768: finishing write tid 2 to smithi12223906-21 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1768: finishing write tid 3 to smithi12223906-21 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1768: finishing write tid 4 to smithi12223906-21 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1768: finishing write tid 5 to smithi12223906-21 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 801 (ObjNum 643 snap 199 seq_num 643) dirty exists 2024-09-15T03:53:28.360 INFO:tasks.rados.rados.0.smithi122.stdout:1768: left oid 21 (ObjNum 643 snap 199 seq_num 643) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1763: done (7 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1764: done (6 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1766: done (5 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1768: done (4 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1769: done (3 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1771: done (2 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1772: done (1 left) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1773: read oid 25 snap -1 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1773: expect (ObjNum 628 snap 187 seq_num 628) 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1774: write oid 23 current snap is 199 2024-09-15T03:53:28.361 INFO:tasks.rados.rados.0.smithi122.stdout:1770: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:53:28.460 INFO:tasks.rados.rados.0.smithi122.stdout:1773: expect (ObjNum 628 snap 187 seq_num 628) 2024-09-15T03:53:28.592 INFO:tasks.rados.rados.0.smithi122.stdout:1774: seq_num 645 ranges {440515=709257,1910768=583545} 2024-09-15T03:53:28.597 INFO:tasks.rados.rados.0.smithi122.stdout:1774: writing smithi12223906-23 from 440515 to 1149772 tid 1 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1774: writing smithi12223906-23 from 1910768 to 2494313 tid 2 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1770: done (2 left) 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1773: done (1 left) 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1775: read oid 35 snap 192 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1775: expect (ObjNum 616 snap 183 seq_num 616) 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1776: read oid 13 snap -1 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1776: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1777: read oid 2 snap 184 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1777: expect (ObjNum 601 snap 180 seq_num 601) 2024-09-15T03:53:28.605 INFO:tasks.rados.rados.0.smithi122.stdout:1778: delete oid 30 current snap is 199 2024-09-15T03:53:28.611 INFO:tasks.rados.rados.0.smithi122.stdout:1777: expect (ObjNum 601 snap 180 seq_num 601) 2024-09-15T03:53:28.719 INFO:tasks.rados.rados.0.smithi122.stdout:1777: done (4 left) 2024-09-15T03:53:28.720 INFO:tasks.rados.rados.0.smithi122.stdout:1778: done (3 left) 2024-09-15T03:53:28.720 INFO:tasks.rados.rados.0.smithi122.stdout:1779: write oid 44 current snap is 199 2024-09-15T03:53:28.720 INFO:tasks.rados.rados.0.smithi122.stdout:1774: finishing write tid 1 to smithi12223906-23 2024-09-15T03:53:28.720 INFO:tasks.rados.rados.0.smithi122.stdout:1776: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:53:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1774: finishing write tid 2 to smithi12223906-23 2024-09-15T03:53:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1774: finishing write tid 3 to smithi12223906-23 2024-09-15T03:53:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1774: finishing write tid 4 to smithi12223906-23 2024-09-15T03:53:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 886 (ObjNum 645 snap 199 seq_num 645) dirty exists 2024-09-15T03:53:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1774: left oid 23 (ObjNum 645 snap 199 seq_num 645) 2024-09-15T03:53:28.927 INFO:tasks.rados.rados.0.smithi122.stdout:1775: expect (ObjNum 616 snap 183 seq_num 616) 2024-09-15T03:53:29.043 INFO:tasks.rados.rados.0.smithi122.stdout:1779: seq_num 647 ranges {474129=576108,1627732=587214,2718088=156041} 2024-09-15T03:53:29.048 INFO:tasks.rados.rados.0.smithi122.stdout:1779: writing smithi12223906-44 from 474129 to 1050237 tid 1 2024-09-15T03:53:29.054 INFO:tasks.rados.rados.0.smithi122.stdout:1779: writing smithi12223906-44 from 1627732 to 2214946 tid 2 2024-09-15T03:53:29.058 INFO:tasks.rados.rados.0.smithi122.stdout:1779: writing smithi12223906-44 from 2718088 to 2874129 tid 3 2024-09-15T03:53:29.058 INFO:tasks.rados.rados.0.smithi122.stdout:1774: done (3 left) 2024-09-15T03:53:29.058 INFO:tasks.rados.rados.0.smithi122.stdout:1775: done (2 left) 2024-09-15T03:53:29.058 INFO:tasks.rados.rados.0.smithi122.stdout:1776: done (1 left) 2024-09-15T03:53:29.058 INFO:tasks.rados.rados.0.smithi122.stdout:1780: delete oid 8 current snap is 199 2024-09-15T03:53:29.061 INFO:tasks.rados.rados.0.smithi122.stdout:1780: done (1 left) 2024-09-15T03:53:29.061 INFO:tasks.rados.rados.0.smithi122.stdout:1781: rollback oid 26 current snap is 199 2024-09-15T03:53:29.061 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 191 2024-09-15T03:53:29.061 INFO:tasks.rados.rados.0.smithi122.stdout:1782: snap_create 2024-09-15T03:53:29.066 INFO:tasks.rados.rados.0.smithi122.stdout:1779: finishing write tid 1 to smithi12223906-44 2024-09-15T03:53:29.066 INFO:tasks.rados.rados.0.smithi122.stdout:1781: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:53:29.066 INFO:tasks.rados.rados.0.smithi122.stdout:1779: finishing write tid 2 to smithi12223906-44 2024-09-15T03:53:29.066 INFO:tasks.rados.rados.0.smithi122.stdout:1779: finishing write tid 3 to smithi12223906-44 2024-09-15T03:53:29.066 INFO:tasks.rados.rados.0.smithi122.stdout:1781: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:53:29.066 INFO:tasks.rados.rados.0.smithi122.stdout:1779: finishing write tid 4 to smithi12223906-44 2024-09-15T03:53:29.067 INFO:tasks.rados.rados.0.smithi122.stdout:1779: finishing write tid 5 to smithi12223906-44 2024-09-15T03:53:29.067 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 891 (ObjNum 647 snap 199 seq_num 647) dirty exists 2024-09-15T03:53:29.067 INFO:tasks.rados.rados.0.smithi122.stdout:1779: left oid 44 (ObjNum 647 snap 199 seq_num 647) 2024-09-15T03:53:29.067 INFO:tasks.rados.rados.0.smithi122.stdout:1781: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:53:29.067 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 423 (ObjNum 520 snap 156 seq_num 520) dirty exists 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1779: done (2 left) 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1781: done (1 left) 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1782: done (0 left) 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1783: read oid 14 snap -1 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1783: expect deleted 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1784: write oid 22 current snap is 200 2024-09-15T03:53:29.753 INFO:tasks.rados.rados.0.smithi122.stdout:1784: seq_num 649 ranges {507743=726606,1744696=590883,2772934=134809} 2024-09-15T03:53:29.761 INFO:tasks.rados.rados.0.smithi122.stdout:1784: writing smithi12223906-22 from 507743 to 1234349 tid 1 2024-09-15T03:53:29.767 INFO:tasks.rados.rados.0.smithi122.stdout:1784: writing smithi12223906-22 from 1744696 to 2335579 tid 2 2024-09-15T03:53:29.771 INFO:tasks.rados.rados.0.smithi122.stdout:1784: writing smithi12223906-22 from 2772934 to 2907743 tid 3 2024-09-15T03:53:29.771 INFO:tasks.rados.rados.0.smithi122.stdout:1785: write oid 4 current snap is 200 2024-09-15T03:53:29.771 INFO:tasks.rados.rados.0.smithi122.stdout:1785: seq_num 650 ranges {524550=401855,1403178=450894,2516710=407840} 2024-09-15T03:53:29.776 INFO:tasks.rados.rados.0.smithi122.stdout:1785: writing smithi12223906-4 from 524550 to 926405 tid 1 2024-09-15T03:53:29.781 INFO:tasks.rados.rados.0.smithi122.stdout:1785: writing smithi12223906-4 from 1403178 to 1854072 tid 2 2024-09-15T03:53:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:1785: writing smithi12223906-4 from 2516710 to 2924550 tid 3 2024-09-15T03:53:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:1786: rollback oid 27 current snap is 200 2024-09-15T03:53:29.789 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 176 2024-09-15T03:53:29.789 INFO:tasks.rados.rados.0.smithi122.stdout:1784: finishing write tid 1 to smithi12223906-22 2024-09-15T03:53:29.789 INFO:tasks.rados.rados.0.smithi122.stdout:1784: finishing write tid 2 to smithi12223906-22 2024-09-15T03:53:29.789 INFO:tasks.rados.rados.0.smithi122.stdout:1784: finishing write tid 3 to smithi12223906-22 2024-09-15T03:53:29.790 INFO:tasks.rados.rados.0.smithi122.stdout:1784: finishing write tid 4 to smithi12223906-22 2024-09-15T03:53:29.790 INFO:tasks.rados.rados.0.smithi122.stdout:1784: finishing write tid 5 to smithi12223906-22 2024-09-15T03:53:29.790 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 896 (ObjNum 649 snap 200 seq_num 649) dirty exists 2024-09-15T03:53:29.790 INFO:tasks.rados.rados.0.smithi122.stdout:1784: left oid 22 (ObjNum 649 snap 200 seq_num 649) 2024-09-15T03:53:29.790 INFO:tasks.rados.rados.0.smithi122.stdout:1783: done (3 left) 2024-09-15T03:53:29.791 INFO:tasks.rados.rados.0.smithi122.stdout:1784: done (2 left) 2024-09-15T03:53:29.791 INFO:tasks.rados.rados.0.smithi122.stdout:1787: copy_from oid 25 from oid 3 current snap is 200 2024-09-15T03:53:29.791 INFO:tasks.rados.rados.0.smithi122.stdout:1785: finishing write tid 1 to smithi12223906-4 2024-09-15T03:53:29.791 INFO:tasks.rados.rados.0.smithi122.stdout:1788: snap_create 2024-09-15T03:53:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:1785: finishing write tid 2 to smithi12223906-4 2024-09-15T03:53:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:1785: finishing write tid 3 to smithi12223906-4 2024-09-15T03:53:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:1785: finishing write tid 4 to smithi12223906-4 2024-09-15T03:53:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:1785: finishing write tid 5 to smithi12223906-4 2024-09-15T03:53:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 783 (ObjNum 650 snap 200 seq_num 650) dirty exists 2024-09-15T03:53:29.793 INFO:tasks.rados.rados.0.smithi122.stdout:1785: left oid 4 (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:53:29.794 INFO:tasks.rados.rados.0.smithi122.stdout:1786: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:53:29.795 INFO:tasks.rados.rados.0.smithi122.stdout:1786: finishing rollback tid 2 to smithi12223906-27 2024-09-15T03:53:29.796 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 898 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:53:29.803 INFO:tasks.rados.rados.0.smithi122.stdout:1787: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:53:29.804 INFO:tasks.rados.rados.0.smithi122.stdout:1787: finishing copy_from to smithi12223906-25 2024-09-15T03:53:29.804 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 849 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:53:30.759 INFO:tasks.rados.rados.0.smithi122.stdout:1785: done (3 left) 2024-09-15T03:53:30.759 INFO:tasks.rados.rados.0.smithi122.stdout:1786: done (2 left) 2024-09-15T03:53:30.759 INFO:tasks.rados.rados.0.smithi122.stdout:1787: done (1 left) 2024-09-15T03:53:30.759 INFO:tasks.rados.rados.0.smithi122.stdout:1788: done (0 left) 2024-09-15T03:53:30.759 INFO:tasks.rados.rados.0.smithi122.stdout:1789: write oid 32 current snap is 201 2024-09-15T03:53:30.760 INFO:tasks.rados.rados.0.smithi122.stdout:1789: seq_num 651 ranges {541357=477104,1745307=594552} 2024-09-15T03:53:30.764 INFO:tasks.rados.rados.0.smithi122.stdout:1789: writing smithi12223906-32 from 541357 to 1018461 tid 1 2024-09-15T03:53:30.772 INFO:tasks.rados.rados.0.smithi122.stdout:1789: writing smithi12223906-32 from 1745307 to 2339859 tid 2 2024-09-15T03:53:30.772 INFO:tasks.rados.rados.0.smithi122.stdout:1790: delete oid 16 current snap is 201 2024-09-15T03:53:30.772 INFO:tasks.rados.rados.0.smithi122.stdout:1789: finishing write tid 1 to smithi12223906-32 2024-09-15T03:53:30.777 INFO:tasks.rados.rados.0.smithi122.stdout:1789: finishing write tid 2 to smithi12223906-32 2024-09-15T03:53:30.778 INFO:tasks.rados.rados.0.smithi122.stdout:1790: done (1 left) 2024-09-15T03:53:30.778 INFO:tasks.rados.rados.0.smithi122.stdout:1791: write (excl) oid 13 current snap is 201 2024-09-15T03:53:30.778 INFO:tasks.rados.rados.0.smithi122.stdout:1789: finishing write tid 3 to smithi12223906-32 2024-09-15T03:53:30.778 INFO:tasks.rados.rados.0.smithi122.stdout:1791: seq_num 653 ranges {574971=627602,1862271=598221} 2024-09-15T03:53:30.784 INFO:tasks.rados.rados.0.smithi122.stdout:1791: writing smithi12223906-13 from 574971 to 1202573 tid 1 2024-09-15T03:53:30.793 INFO:tasks.rados.rados.0.smithi122.stdout:1791: writing smithi12223906-13 from 1862271 to 2460492 tid 2 2024-09-15T03:53:30.793 INFO:tasks.rados.rados.0.smithi122.stdout:1792: write (excl) oid 49 current snap is 201 2024-09-15T03:53:30.793 INFO:tasks.rados.rados.0.smithi122.stdout:1792: seq_num 654 ranges {591778=419204,1637106=741879,2910049=81729} 2024-09-15T03:53:30.798 INFO:tasks.rados.rados.0.smithi122.stdout:1792: writing smithi12223906-49 from 591778 to 1010982 tid 1 2024-09-15T03:53:30.805 INFO:tasks.rados.rados.0.smithi122.stdout:1792: writing smithi12223906-49 from 1637106 to 2378985 tid 2 2024-09-15T03:53:30.809 INFO:tasks.rados.rados.0.smithi122.stdout:1792: writing smithi12223906-49 from 2910049 to 2991778 tid 3 2024-09-15T03:53:30.809 INFO:tasks.rados.rados.0.smithi122.stdout:1793: read oid 5 snap -1 2024-09-15T03:53:30.809 INFO:tasks.rados.rados.0.smithi122.stdout:1793: expect (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:53:30.809 INFO:tasks.rados.rados.0.smithi122.stdout:1794: write (excl) oid 30 current snap is 201 2024-09-15T03:53:30.809 INFO:tasks.rados.rados.0.smithi122.stdout:1794: seq_num 655 ranges {608585=494453,1579235=601890,2821119=187466} 2024-09-15T03:53:30.815 INFO:tasks.rados.rados.0.smithi122.stdout:1794: writing smithi12223906-30 from 608585 to 1103038 tid 1 2024-09-15T03:53:30.821 INFO:tasks.rados.rados.0.smithi122.stdout:1794: writing smithi12223906-30 from 1579235 to 2181125 tid 2 2024-09-15T03:53:30.826 INFO:tasks.rados.rados.0.smithi122.stdout:1794: writing smithi12223906-30 from 2821119 to 3008585 tid 3 2024-09-15T03:53:30.826 INFO:tasks.rados.rados.0.smithi122.stdout:1795: write oid 17 current snap is 201 2024-09-15T03:53:30.826 INFO:tasks.rados.rados.0.smithi122.stdout:1789: finishing write tid 4 to smithi12223906-32 2024-09-15T03:53:30.826 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 901 (ObjNum 651 snap 201 seq_num 651) dirty exists 2024-09-15T03:53:30.826 INFO:tasks.rados.rados.0.smithi122.stdout:1789: left oid 32 (ObjNum 651 snap 201 seq_num 651) 2024-09-15T03:53:30.827 INFO:tasks.rados.rados.0.smithi122.stdout:1795: seq_num 656 ranges {625392=569702,1637717=745548,2848542=176850} 2024-09-15T03:53:30.832 INFO:tasks.rados.rados.0.smithi122.stdout:1795: writing smithi12223906-17 from 625392 to 1195094 tid 1 2024-09-15T03:53:30.840 INFO:tasks.rados.rados.0.smithi122.stdout:1795: writing smithi12223906-17 from 1637717 to 2383265 tid 2 2024-09-15T03:53:30.845 INFO:tasks.rados.rados.0.smithi122.stdout:1795: writing smithi12223906-17 from 2848542 to 3025392 tid 3 2024-09-15T03:53:30.845 INFO:tasks.rados.rados.0.smithi122.stdout:1789: done (5 left) 2024-09-15T03:53:30.845 INFO:tasks.rados.rados.0.smithi122.stdout:1796: rollback oid 8 current snap is 201 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 193 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1791: finishing write tid 1 to smithi12223906-13 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1791: finishing write tid 2 to smithi12223906-13 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1791: finishing write tid 3 to smithi12223906-13 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1791: finishing write tid 4 to smithi12223906-13 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 785 (ObjNum 653 snap 201 seq_num 653) dirty exists 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1791: left oid 13 (ObjNum 653 snap 201 seq_num 653) 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1792: finishing write tid 1 to smithi12223906-49 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1792: finishing write tid 2 to smithi12223906-49 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1792: finishing write tid 3 to smithi12223906-49 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1792: finishing write tid 4 to smithi12223906-49 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1791: done (5 left) 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1797: read oid 1 snap -1 2024-09-15T03:53:30.846 INFO:tasks.rados.rados.0.smithi122.stdout:1797: expect (ObjNum 640 snap 199 seq_num 640) 2024-09-15T03:53:30.847 INFO:tasks.rados.rados.0.smithi122.stdout:1798: delete oid 46 current snap is 201 2024-09-15T03:53:30.847 INFO:tasks.rados.rados.0.smithi122.stdout:1792: finishing write tid 5 to smithi12223906-49 2024-09-15T03:53:30.847 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 483 (ObjNum 654 snap 201 seq_num 654) dirty exists 2024-09-15T03:53:30.847 INFO:tasks.rados.rados.0.smithi122.stdout:1792: left oid 49 (ObjNum 654 snap 201 seq_num 654) 2024-09-15T03:53:30.847 INFO:tasks.rados.rados.0.smithi122.stdout:1794: finishing write tid 1 to smithi12223906-30 2024-09-15T03:53:30.847 INFO:tasks.rados.rados.0.smithi122.stdout:1793: expect (ObjNum 563 snap 167 seq_num 563) 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1794: finishing write tid 2 to smithi12223906-30 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1794: finishing write tid 3 to smithi12223906-30 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1794: finishing write tid 4 to smithi12223906-30 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1794: finishing write tid 5 to smithi12223906-30 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 613 (ObjNum 655 snap 201 seq_num 655) dirty exists 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1794: left oid 30 (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1795: finishing write tid 1 to smithi12223906-17 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1796: finishing rollback tid 1 to smithi12223906-8 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1795: finishing write tid 2 to smithi12223906-17 2024-09-15T03:53:31.052 INFO:tasks.rados.rados.0.smithi122.stdout:1792: done (6 left) 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1793: done (5 left) 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1794: done (4 left) 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1796: done (3 left) 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1798: done (2 left) 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1795: finishing write tid 3 to smithi12223906-17 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1795: finishing write tid 4 to smithi12223906-17 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:1795: finishing write tid 5 to smithi12223906-17 2024-09-15T03:53:31.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 488 (ObjNum 656 snap 201 seq_num 656) dirty exists 2024-09-15T03:53:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:1795: left oid 17 (ObjNum 656 snap 201 seq_num 656) 2024-09-15T03:53:31.054 INFO:tasks.rados.rados.0.smithi122.stdout:1797: expect (ObjNum 640 snap 199 seq_num 640) 2024-09-15T03:53:31.211 INFO:tasks.rados.rados.0.smithi122.stdout:1799: read oid 37 snap -1 2024-09-15T03:53:31.211 INFO:tasks.rados.rados.0.smithi122.stdout:1799: expect (ObjNum 612 snap 183 seq_num 612) 2024-09-15T03:53:31.211 INFO:tasks.rados.rados.0.smithi122.stdout:1795: done (2 left) 2024-09-15T03:53:31.211 INFO:tasks.rados.rados.0.smithi122.stdout:1797: done (1 left) 2024-09-15T03:53:31.211 INFO:tasks.rados.rados.0.smithi122.stdout:1800: rollback oid 26 current snap is 201 2024-09-15T03:53:31.212 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 188 2024-09-15T03:53:31.212 INFO:tasks.rados.rados.0.smithi122.stdout:1801: write (excl) oid 24 current snap is 201 2024-09-15T03:53:31.212 INFO:tasks.rados.rados.0.smithi122.stdout:1801: seq_num 658 ranges {659006=720200,2154681=749217} 2024-09-15T03:53:31.226 INFO:tasks.rados.rados.0.smithi122.stdout:1801: writing smithi12223906-24 from 659006 to 1379206 tid 1 2024-09-15T03:53:31.234 INFO:tasks.rados.rados.0.smithi122.stdout:1801: writing smithi12223906-24 from 2154681 to 2903898 tid 2 2024-09-15T03:53:31.234 INFO:tasks.rados.rados.0.smithi122.stdout:1802: rollback oid 16 current snap is 201 2024-09-15T03:53:31.234 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 182 2024-09-15T03:53:31.234 INFO:tasks.rados.rados.0.smithi122.stdout:1799: expect (ObjNum 612 snap 183 seq_num 612) 2024-09-15T03:53:31.411 INFO:tasks.rados.rados.0.smithi122.stdout:1800: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1800: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1800: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 426 (ObjNum 520 snap 156 seq_num 520) dirty exists 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1802: finishing rollback tid 1 to smithi12223906-16 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1802: finishing rollback tid 2 to smithi12223906-16 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 654 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1801: finishing write tid 1 to smithi12223906-24 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1801: finishing write tid 2 to smithi12223906-24 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1801: finishing write tid 3 to smithi12223906-24 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:1801: finishing write tid 4 to smithi12223906-24 2024-09-15T03:53:31.412 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 935 (ObjNum 658 snap 201 seq_num 658) dirty exists 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1801: left oid 24 (ObjNum 658 snap 201 seq_num 658) 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1799: done (3 left) 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1800: done (2 left) 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1801: done (1 left) 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1802: done (0 left) 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1803: copy_from oid 30 from oid 40 current snap is 201 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1804: write oid 3 current snap is 201 2024-09-15T03:53:31.413 INFO:tasks.rados.rados.0.smithi122.stdout:1804: seq_num 659 ranges {675813=795449,2096810=609228} 2024-09-15T03:53:31.429 INFO:tasks.rados.rados.0.smithi122.stdout:1804: writing smithi12223906-3 from 675813 to 1471262 tid 1 2024-09-15T03:53:31.435 INFO:tasks.rados.rados.0.smithi122.stdout:1804: writing smithi12223906-3 from 2096810 to 2706038 tid 2 2024-09-15T03:53:31.435 INFO:tasks.rados.rados.0.smithi122.stdout:1805: copy_from oid 39 from oid 23 current snap is 201 2024-09-15T03:53:31.435 INFO:tasks.rados.rados.0.smithi122.stdout:1803: finishing copy_from to smithi12223906-30 2024-09-15T03:53:31.436 INFO:tasks.rados.rados.0.smithi122.stdout:1803: got expected ENOENT (src dne) 2024-09-15T03:53:31.436 INFO:tasks.rados.rados.0.smithi122.stdout:1806: write oid 31 current snap is 201 2024-09-15T03:53:31.436 INFO:tasks.rados.rados.0.smithi122.stdout:1806: seq_num 660 ranges {692620=470698,1755292=752886} 2024-09-15T03:53:31.441 INFO:tasks.rados.rados.0.smithi122.stdout:1806: writing smithi12223906-31 from 692620 to 1163318 tid 1 2024-09-15T03:53:31.450 INFO:tasks.rados.rados.0.smithi122.stdout:1806: writing smithi12223906-31 from 1755292 to 2508178 tid 2 2024-09-15T03:53:31.450 INFO:tasks.rados.rados.0.smithi122.stdout:1807: write (excl) oid 13 current snap is 201 2024-09-15T03:53:31.450 INFO:tasks.rados.rados.0.smithi122.stdout:1803: finishing copy_from racing read to smithi12223906-30 2024-09-15T03:53:31.450 INFO:tasks.rados.rados.0.smithi122.stdout:1807: seq_num 661 ranges {709427=545947,1813774=612897,2985657=123770} 2024-09-15T03:53:31.456 INFO:tasks.rados.rados.0.smithi122.stdout:1807: writing smithi12223906-13 from 709427 to 1255374 tid 1 2024-09-15T03:53:31.463 INFO:tasks.rados.rados.0.smithi122.stdout:1807: writing smithi12223906-13 from 1813774 to 2426671 tid 2 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1807: writing smithi12223906-13 from 2985657 to 3109427 tid 3 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1803: done (4 left) 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1808: read oid 9 snap 200 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1808: expect (ObjNum 534 snap 160 seq_num 534) 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1809: delete oid 33 current snap is 201 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1804: finishing write tid 1 to smithi12223906-3 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1804: finishing write tid 2 to smithi12223906-3 2024-09-15T03:53:31.468 INFO:tasks.rados.rados.0.smithi122.stdout:1804: finishing write tid 3 to smithi12223906-3 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1804: finishing write tid 4 to smithi12223906-3 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 939 (ObjNum 659 snap 201 seq_num 659) dirty exists 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1804: left oid 3 (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1806: finishing write tid 1 to smithi12223906-31 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1806: finishing write tid 2 to smithi12223906-31 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1806: finishing write tid 3 to smithi12223906-31 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1806: finishing write tid 4 to smithi12223906-31 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 786 (ObjNum 660 snap 201 seq_num 660) dirty exists 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1806: left oid 31 (ObjNum 660 snap 201 seq_num 660) 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1805: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1805: finishing copy_from to smithi12223906-39 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 886 (ObjNum 645 snap 199 seq_num 645) dirty exists 2024-09-15T03:53:31.469 INFO:tasks.rados.rados.0.smithi122.stdout:1807: finishing write tid 1 to smithi12223906-13 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:1807: finishing write tid 2 to smithi12223906-13 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:1804: done (5 left) 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:1805: done (4 left) 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:1806: done (3 left) 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:1809: done (2 left) 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:1810: rollback oid 27 current snap is 201 2024-09-15T03:53:31.471 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 192 2024-09-15T03:53:31.472 INFO:tasks.rados.rados.0.smithi122.stdout:1811: snap_create 2024-09-15T03:53:31.472 INFO:tasks.rados.rados.0.smithi122.stdout:1807: finishing write tid 3 to smithi12223906-13 2024-09-15T03:53:31.472 INFO:tasks.rados.rados.0.smithi122.stdout:1807: finishing write tid 4 to smithi12223906-13 2024-09-15T03:53:31.472 INFO:tasks.rados.rados.0.smithi122.stdout:1807: finishing write tid 5 to smithi12223906-13 2024-09-15T03:53:31.472 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 790 (ObjNum 661 snap 201 seq_num 661) dirty exists 2024-09-15T03:53:31.472 INFO:tasks.rados.rados.0.smithi122.stdout:1807: left oid 13 (ObjNum 661 snap 201 seq_num 661) 2024-09-15T03:53:31.473 INFO:tasks.rados.rados.0.smithi122.stdout:1808: expect (ObjNum 534 snap 160 seq_num 534) 2024-09-15T03:53:31.646 INFO:tasks.rados.rados.0.smithi122.stdout:1810: finishing rollback tid 0 to smithi12223906-27 2024-09-15T03:53:31.646 INFO:tasks.rados.rados.0.smithi122.stdout:1810: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:53:31.646 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 905 (ObjNum 193 snap 0 seq_num 1843102480) dirty dne 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1807: done (3 left) 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1808: done (2 left) 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1810: done (1 left) 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1811: done (0 left) 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1812: copy_from oid 47 from oid 25 current snap is 202 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1813: rollback oid 40 current snap is 202 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 200 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1814: read oid 30 snap -1 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1814: expect (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1815: write (excl) oid 49 current snap is 202 2024-09-15T03:53:31.748 INFO:tasks.rados.rados.0.smithi122.stdout:1815: seq_num 663 ranges {743041=412798,1930738=616566,3040503=102538} 2024-09-15T03:53:31.754 INFO:tasks.rados.rados.0.smithi122.stdout:1815: writing smithi12223906-49 from 743041 to 1155839 tid 1 2024-09-15T03:53:31.765 INFO:tasks.rados.rados.0.smithi122.stdout:1815: writing smithi12223906-49 from 1930738 to 2547304 tid 2 2024-09-15T03:53:31.768 INFO:tasks.rados.rados.0.smithi122.stdout:1815: writing smithi12223906-49 from 3040503 to 3143041 tid 3 2024-09-15T03:53:31.769 INFO:tasks.rados.rados.0.smithi122.stdout:1813: finishing rollback tid 1 to smithi12223906-40 2024-09-15T03:53:31.769 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 608 (ObjNum 65 snap 0 seq_num 1843099728) dirty dne 2024-09-15T03:53:31.769 INFO:tasks.rados.rados.0.smithi122.stdout:1812: finishing copy_from racing read to smithi12223906-47 2024-09-15T03:53:31.769 INFO:tasks.rados.rados.0.smithi122.stdout:1814: expect (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:31.865 INFO:tasks.rados.rados.0.smithi122.stdout:1812: finishing copy_from to smithi12223906-47 2024-09-15T03:53:31.865 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 849 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:53:31.865 INFO:tasks.rados.rados.0.smithi122.stdout:1815: finishing write tid 1 to smithi12223906-49 2024-09-15T03:53:31.865 INFO:tasks.rados.rados.0.smithi122.stdout:1812: done (3 left) 2024-09-15T03:53:31.866 INFO:tasks.rados.rados.0.smithi122.stdout:1813: done (2 left) 2024-09-15T03:53:31.866 INFO:tasks.rados.rados.0.smithi122.stdout:1814: done (1 left) 2024-09-15T03:53:31.866 INFO:tasks.rados.rados.0.smithi122.stdout:1816: snap_remove snap 184 2024-09-15T03:53:32.798 INFO:tasks.rados.rados.0.smithi122.stdout:1816: done (1 left) 2024-09-15T03:53:32.798 INFO:tasks.rados.rados.0.smithi122.stdout:1817: snap_create 2024-09-15T03:53:32.798 INFO:tasks.rados.rados.0.smithi122.stdout:1815: finishing write tid 2 to smithi12223906-49 2024-09-15T03:53:32.799 INFO:tasks.rados.rados.0.smithi122.stdout:1815: finishing write tid 3 to smithi12223906-49 2024-09-15T03:53:32.799 INFO:tasks.rados.rados.0.smithi122.stdout:1815: finishing write tid 4 to smithi12223906-49 2024-09-15T03:53:32.799 INFO:tasks.rados.rados.0.smithi122.stdout:1815: finishing write tid 5 to smithi12223906-49 2024-09-15T03:53:32.799 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 493 (ObjNum 663 snap 202 seq_num 663) dirty exists 2024-09-15T03:53:32.799 INFO:tasks.rados.rados.0.smithi122.stdout:1815: left oid 49 (ObjNum 663 snap 202 seq_num 663) 2024-09-15T03:53:33.795 INFO:tasks.rados.rados.0.smithi122.stdout:1815: done (1 left) 2024-09-15T03:53:33.796 INFO:tasks.rados.rados.0.smithi122.stdout:1817: done (0 left) 2024-09-15T03:53:33.796 INFO:tasks.rados.rados.0.smithi122.stdout:1818: write (excl) oid 43 current snap is 203 2024-09-15T03:53:33.796 INFO:tasks.rados.rados.0.smithi122.stdout:1818: seq_num 664 ranges {759848=488047,1989220=760224} 2024-09-15T03:53:33.807 INFO:tasks.rados.rados.0.smithi122.stdout:1818: writing smithi12223906-43 from 759848 to 1247895 tid 1 2024-09-15T03:53:33.815 INFO:tasks.rados.rados.0.smithi122.stdout:1818: writing smithi12223906-43 from 1989220 to 2749444 tid 2 2024-09-15T03:53:33.815 INFO:tasks.rados.rados.0.smithi122.stdout:1819: snap_create 2024-09-15T03:53:33.821 INFO:tasks.rados.rados.0.smithi122.stdout:1818: finishing write tid 1 to smithi12223906-43 2024-09-15T03:53:33.822 INFO:tasks.rados.rados.0.smithi122.stdout:1818: finishing write tid 2 to smithi12223906-43 2024-09-15T03:53:33.822 INFO:tasks.rados.rados.0.smithi122.stdout:1818: finishing write tid 3 to smithi12223906-43 2024-09-15T03:53:33.822 INFO:tasks.rados.rados.0.smithi122.stdout:1818: finishing write tid 4 to smithi12223906-43 2024-09-15T03:53:33.822 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 957 (ObjNum 664 snap 203 seq_num 664) dirty exists 2024-09-15T03:53:33.822 INFO:tasks.rados.rados.0.smithi122.stdout:1818: left oid 43 (ObjNum 664 snap 203 seq_num 664) 2024-09-15T03:53:34.819 INFO:tasks.rados.rados.0.smithi122.stdout:1818: done (1 left) 2024-09-15T03:53:34.819 INFO:tasks.rados.rados.0.smithi122.stdout:1819: done (0 left) 2024-09-15T03:53:34.819 INFO:tasks.rados.rados.0.smithi122.stdout:1820: write (excl) oid 9 current snap is 204 2024-09-15T03:53:34.819 INFO:tasks.rados.rados.0.smithi122.stdout:1820: seq_num 665 ranges {776655=563296,2047702=620235,3095349=81306} 2024-09-15T03:53:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:1820: writing smithi12223906-9 from 776655 to 1339951 tid 1 2024-09-15T03:53:34.834 INFO:tasks.rados.rados.0.smithi122.stdout:1820: writing smithi12223906-9 from 2047702 to 2667937 tid 2 2024-09-15T03:53:34.837 INFO:tasks.rados.rados.0.smithi122.stdout:1820: writing smithi12223906-9 from 3095349 to 3176655 tid 3 2024-09-15T03:53:34.837 INFO:tasks.rados.rados.0.smithi122.stdout:1821: read oid 10 snap 194 2024-09-15T03:53:34.837 INFO:tasks.rados.rados.0.smithi122.stdout:1821: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:53:34.837 INFO:tasks.rados.rados.0.smithi122.stdout:1822: delete oid 28 current snap is 204 2024-09-15T03:53:34.837 INFO:tasks.rados.rados.0.smithi122.stdout:1820: finishing write tid 1 to smithi12223906-9 2024-09-15T03:53:34.840 INFO:tasks.rados.rados.0.smithi122.stdout:1820: finishing write tid 2 to smithi12223906-9 2024-09-15T03:53:34.840 INFO:tasks.rados.rados.0.smithi122.stdout:1820: finishing write tid 3 to smithi12223906-9 2024-09-15T03:53:34.840 INFO:tasks.rados.rados.0.smithi122.stdout:1820: finishing write tid 4 to smithi12223906-9 2024-09-15T03:53:34.841 INFO:tasks.rados.rados.0.smithi122.stdout:1820: finishing write tid 5 to smithi12223906-9 2024-09-15T03:53:34.841 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 805 (ObjNum 665 snap 204 seq_num 665) dirty exists 2024-09-15T03:53:34.841 INFO:tasks.rados.rados.0.smithi122.stdout:1820: left oid 9 (ObjNum 665 snap 204 seq_num 665) 2024-09-15T03:53:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:1820: done (2 left) 2024-09-15T03:53:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:1822: done (1 left) 2024-09-15T03:53:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:1823: read oid 41 snap -1 2024-09-15T03:53:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:1823: expect deleted 2024-09-15T03:53:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:1824: read oid 13 snap -1 2024-09-15T03:53:34.843 INFO:tasks.rados.rados.0.smithi122.stdout:1824: expect (ObjNum 661 snap 201 seq_num 661) 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1825: read oid 6 snap -1 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1825: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1826: read oid 19 snap -1 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1826: expect (ObjNum 573 snap 170 seq_num 573) 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1827: read oid 30 snap -1 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1827: expect (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:34.844 INFO:tasks.rados.rados.0.smithi122.stdout:1828: read oid 11 snap -1 2024-09-15T03:53:34.845 INFO:tasks.rados.rados.0.smithi122.stdout:1828: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:53:34.845 INFO:tasks.rados.rados.0.smithi122.stdout:1829: write oid 28 current snap is 204 2024-09-15T03:53:34.845 INFO:tasks.rados.rados.0.smithi122.stdout:1829: seq_num 667 ranges {410269=713794,1648313=507551,2917489=292780} 2024-09-15T03:53:34.852 INFO:tasks.rados.rados.0.smithi122.stdout:1829: writing smithi12223906-28 from 410269 to 1124063 tid 1 2024-09-15T03:53:34.859 INFO:tasks.rados.rados.0.smithi122.stdout:1829: writing smithi12223906-28 from 1648313 to 2155864 tid 2 2024-09-15T03:53:34.867 INFO:tasks.rados.rados.0.smithi122.stdout:1829: writing smithi12223906-28 from 2917489 to 3210269 tid 3 2024-09-15T03:53:34.867 INFO:tasks.rados.rados.0.smithi122.stdout:1830: delete oid 26 current snap is 204 2024-09-15T03:53:34.867 INFO:tasks.rados.rados.0.smithi122.stdout:1821: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:53:35.045 INFO:tasks.rados.rados.0.smithi122.stdout:1827: expect (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:1828: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:53:35.266 INFO:tasks.rados.rados.0.smithi122.stdout:1825: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:53:35.451 INFO:tasks.rados.rados.0.smithi122.stdout:1824: expect (ObjNum 661 snap 201 seq_num 661) 2024-09-15T03:53:35.646 INFO:tasks.rados.rados.0.smithi122.stdout:1829: finishing write tid 1 to smithi12223906-28 2024-09-15T03:53:35.646 INFO:tasks.rados.rados.0.smithi122.stdout:1829: finishing write tid 2 to smithi12223906-28 2024-09-15T03:53:35.646 INFO:tasks.rados.rados.0.smithi122.stdout:1826: expect (ObjNum 573 snap 170 seq_num 573) 2024-09-15T03:53:35.861 INFO:tasks.rados.rados.0.smithi122.stdout:1829: finishing write tid 3 to smithi12223906-28 2024-09-15T03:53:35.861 INFO:tasks.rados.rados.0.smithi122.stdout:1829: finishing write tid 4 to smithi12223906-28 2024-09-15T03:53:35.861 INFO:tasks.rados.rados.0.smithi122.stdout:1829: finishing write tid 5 to smithi12223906-28 2024-09-15T03:53:35.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 787 (ObjNum 667 snap 204 seq_num 667) dirty exists 2024-09-15T03:53:35.861 INFO:tasks.rados.rados.0.smithi122.stdout:1829: left oid 28 (ObjNum 667 snap 204 seq_num 667) 2024-09-15T03:53:35.861 INFO:tasks.rados.rados.0.smithi122.stdout:1821: done (8 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1823: done (7 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1824: done (6 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1825: done (5 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1826: done (4 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1827: done (3 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1828: done (2 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1829: done (1 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1830: done (0 left) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1831: copy_from oid 41 from oid 6 current snap is 204 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1832: read oid 2 snap -1 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1832: expect (ObjNum 642 snap 199 seq_num 642) 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1833: write oid 24 current snap is 204 2024-09-15T03:53:35.862 INFO:tasks.rados.rados.0.smithi122.stdout:1833: seq_num 669 ranges {443883=464292,1365277=511220,2572335=671548} 2024-09-15T03:53:35.872 INFO:tasks.rados.rados.0.smithi122.stdout:1833: writing smithi12223906-24 from 443883 to 908175 tid 1 2024-09-15T03:53:35.877 INFO:tasks.rados.rados.0.smithi122.stdout:1833: writing smithi12223906-24 from 1365277 to 1876497 tid 2 2024-09-15T03:53:35.885 INFO:tasks.rados.rados.0.smithi122.stdout:1833: writing smithi12223906-24 from 2572335 to 3243883 tid 3 2024-09-15T03:53:35.886 INFO:tasks.rados.rados.0.smithi122.stdout:1834: read oid 14 snap 199 2024-09-15T03:53:35.886 INFO:tasks.rados.rados.0.smithi122.stdout:1834: expect deleted 2024-09-15T03:53:35.886 INFO:tasks.rados.rados.0.smithi122.stdout:1835: write oid 18 current snap is 204 2024-09-15T03:53:35.886 INFO:tasks.rados.rados.0.smithi122.stdout:1835: seq_num 670 ranges {460690=655894,1540112=771231,2716111=539778} 2024-09-15T03:53:35.892 INFO:tasks.rados.rados.0.smithi122.stdout:1835: writing smithi12223906-18 from 460690 to 1116584 tid 1 2024-09-15T03:53:35.898 INFO:tasks.rados.rados.0.smithi122.stdout:1835: writing smithi12223906-18 from 1540112 to 2311343 tid 2 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1835: writing smithi12223906-18 from 2716111 to 3255889 tid 3 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1836: read oid 43 snap 193 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1836: expect (ObjNum 608 snap 181 seq_num 608) 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1837: read oid 9 snap -1 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1837: expect (ObjNum 665 snap 204 seq_num 665) 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1838: delete oid 30 current snap is 204 2024-09-15T03:53:35.904 INFO:tasks.rados.rados.0.smithi122.stdout:1832: expect (ObjNum 642 snap 199 seq_num 642) 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:1831: finishing copy_from racing read to smithi12223906-41 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:1831: finishing copy_from to smithi12223906-41 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 801 (ObjNum 636 snap 195 seq_num 636) dirty exists 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:1833: finishing write tid 1 to smithi12223906-24 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:1833: finishing write tid 2 to smithi12223906-24 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:1831: done (7 left) 2024-09-15T03:53:36.072 INFO:tasks.rados.rados.0.smithi122.stdout:1832: done (6 left) 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1838: done (5 left) 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1839: write oid 21 current snap is 204 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1833: finishing write tid 3 to smithi12223906-24 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1833: finishing write tid 4 to smithi12223906-24 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1833: finishing write tid 5 to smithi12223906-24 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 962 (ObjNum 669 snap 204 seq_num 669) dirty exists 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1833: left oid 24 (ObjNum 669 snap 204 seq_num 669) 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1835: finishing write tid 1 to smithi12223906-18 2024-09-15T03:53:36.073 INFO:tasks.rados.rados.0.smithi122.stdout:1839: seq_num 672 ranges {494304=406392,1540723=774900,2770957=523347} 2024-09-15T03:53:36.086 INFO:tasks.rados.rados.0.smithi122.stdout:1839: writing smithi12223906-21 from 494304 to 900696 tid 1 2024-09-15T03:53:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:1839: writing smithi12223906-21 from 1540723 to 2315623 tid 2 2024-09-15T03:53:36.100 INFO:tasks.rados.rados.0.smithi122.stdout:1839: writing smithi12223906-21 from 2770957 to 3294304 tid 3 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1833: done (5 left) 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1834: done (4 left) 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1840: delete oid 42 current snap is 204 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1835: finishing write tid 2 to smithi12223906-18 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1835: finishing write tid 3 to smithi12223906-18 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1835: finishing write tid 4 to smithi12223906-18 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1835: finishing write tid 5 to smithi12223906-18 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 918 (ObjNum 670 snap 204 seq_num 670) dirty exists 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1835: left oid 18 (ObjNum 670 snap 204 seq_num 670) 2024-09-15T03:53:36.101 INFO:tasks.rados.rados.0.smithi122.stdout:1837: expect (ObjNum 665 snap 204 seq_num 665) 2024-09-15T03:53:36.292 INFO:tasks.rados.rados.0.smithi122.stdout:1836: expect (ObjNum 608 snap 181 seq_num 608) 2024-09-15T03:53:36.408 INFO:tasks.rados.rados.0.smithi122.stdout:1839: finishing write tid 1 to smithi12223906-21 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1839: finishing write tid 2 to smithi12223906-21 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1839: finishing write tid 3 to smithi12223906-21 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1839: finishing write tid 4 to smithi12223906-21 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1839: finishing write tid 5 to smithi12223906-21 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 805 (ObjNum 672 snap 204 seq_num 672) dirty exists 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1839: left oid 21 (ObjNum 672 snap 204 seq_num 672) 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1835: done (4 left) 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1836: done (3 left) 2024-09-15T03:53:36.409 INFO:tasks.rados.rados.0.smithi122.stdout:1837: done (2 left) 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1839: done (1 left) 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1840: done (0 left) 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1841: copy_from oid 16 from oid 19 current snap is 204 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1842: read oid 15 snap -1 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1842: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1843: read oid 8 snap 194 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1843: expect deleted 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1844: copy_from oid 44 from oid 41 current snap is 204 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1845: write (excl) oid 28 current snap is 204 2024-09-15T03:53:36.413 INFO:tasks.rados.rados.0.smithi122.stdout:1845: seq_num 674 ranges {527918=556890,1657687=778569,3225803=102115} 2024-09-15T03:53:36.423 INFO:tasks.rados.rados.0.smithi122.stdout:1845: writing smithi12223906-28 from 527918 to 1084808 tid 1 2024-09-15T03:53:36.431 INFO:tasks.rados.rados.0.smithi122.stdout:1845: writing smithi12223906-28 from 1657687 to 2436256 tid 2 2024-09-15T03:53:36.436 INFO:tasks.rados.rados.0.smithi122.stdout:1845: writing smithi12223906-28 from 3225803 to 3327918 tid 3 2024-09-15T03:53:36.436 INFO:tasks.rados.rados.0.smithi122.stdout:1846: delete oid 30 current snap is 204 2024-09-15T03:53:36.436 INFO:tasks.rados.rados.0.smithi122.stdout:1842: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:53:36.663 INFO:tasks.rados.rados.0.smithi122.stdout:1841: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1845: finishing write tid 1 to smithi12223906-28 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1844: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1844: finishing copy_from to smithi12223906-44 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 801 (ObjNum 636 snap 195 seq_num 636) dirty exists 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1841: finishing copy_from to smithi12223906-16 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 772 (ObjNum 573 snap 170 seq_num 573) dirty exists 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1845: finishing write tid 2 to smithi12223906-28 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1845: finishing write tid 3 to smithi12223906-28 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1845: finishing write tid 4 to smithi12223906-28 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1845: finishing write tid 5 to smithi12223906-28 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 791 (ObjNum 674 snap 204 seq_num 674) dirty exists 2024-09-15T03:53:36.664 INFO:tasks.rados.rados.0.smithi122.stdout:1845: left oid 28 (ObjNum 674 snap 204 seq_num 674) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1841: done (5 left) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1842: done (4 left) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1843: done (3 left) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1844: done (2 left) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1845: done (1 left) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1846: done (0 left) 2024-09-15T03:53:36.665 INFO:tasks.rados.rados.0.smithi122.stdout:1847: delete oid 15 current snap is 204 2024-09-15T03:53:36.671 INFO:tasks.rados.rados.0.smithi122.stdout:1847: done (0 left) 2024-09-15T03:53:36.671 INFO:tasks.rados.rados.0.smithi122.stdout:1848: write (excl) oid 48 current snap is 204 2024-09-15T03:53:36.671 INFO:tasks.rados.rados.0.smithi122.stdout:1848: seq_num 677 ranges {578339=498990,1833133=525896,2791719=586620} 2024-09-15T03:53:36.682 INFO:tasks.rados.rados.0.smithi122.stdout:1848: writing smithi12223906-48 from 578339 to 1077329 tid 1 2024-09-15T03:53:36.691 INFO:tasks.rados.rados.0.smithi122.stdout:1848: writing smithi12223906-48 from 1833133 to 2359029 tid 2 2024-09-15T03:53:36.697 INFO:tasks.rados.rados.0.smithi122.stdout:1848: writing smithi12223906-48 from 2791719 to 3378339 tid 3 2024-09-15T03:53:36.697 INFO:tasks.rados.rados.0.smithi122.stdout:1849: read oid 4 snap -1 2024-09-15T03:53:36.697 INFO:tasks.rados.rados.0.smithi122.stdout:1849: expect (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:53:36.697 INFO:tasks.rados.rados.0.smithi122.stdout:1850: delete oid 33 current snap is 204 2024-09-15T03:53:36.697 INFO:tasks.rados.rados.0.smithi122.stdout:1848: finishing write tid 1 to smithi12223906-48 2024-09-15T03:53:36.698 INFO:tasks.rados.rados.0.smithi122.stdout:1848: finishing write tid 2 to smithi12223906-48 2024-09-15T03:53:36.701 INFO:tasks.rados.rados.0.smithi122.stdout:1850: done (2 left) 2024-09-15T03:53:36.701 INFO:tasks.rados.rados.0.smithi122.stdout:1851: rollback oid 49 current snap is 204 2024-09-15T03:53:36.701 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 195 2024-09-15T03:53:36.701 INFO:tasks.rados.rados.0.smithi122.stdout:1852: read oid 26 snap -1 2024-09-15T03:53:36.702 INFO:tasks.rados.rados.0.smithi122.stdout:1852: expect deleted 2024-09-15T03:53:36.702 INFO:tasks.rados.rados.0.smithi122.stdout:1853: rollback oid 35 current snap is 204 2024-09-15T03:53:36.702 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 195 2024-09-15T03:53:36.702 INFO:tasks.rados.rados.0.smithi122.stdout:1854: copy_from oid 34 from oid 2 current snap is 204 2024-09-15T03:53:36.702 INFO:tasks.rados.rados.0.smithi122.stdout:1855: delete oid 5 current snap is 204 2024-09-15T03:53:36.705 INFO:tasks.rados.rados.0.smithi122.stdout:1849: expect (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:1853: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:1853: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 583 (ObjNum 616 snap 183 seq_num 616) dirty exists 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:1848: finishing write tid 3 to smithi12223906-48 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:1848: finishing write tid 4 to smithi12223906-48 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:1848: finishing write tid 5 to smithi12223906-48 2024-09-15T03:53:36.807 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 809 (ObjNum 677 snap 204 seq_num 677) dirty exists 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1848: left oid 48 (ObjNum 677 snap 204 seq_num 677) 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1851: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1851: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1851: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 503 (ObjNum 564 snap 167 seq_num 564) dirty exists 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1854: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1854: finishing copy_from to smithi12223906-34 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 478 (ObjNum 642 snap 199 seq_num 642) dirty exists 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1848: done (6 left) 2024-09-15T03:53:36.808 INFO:tasks.rados.rados.0.smithi122.stdout:1849: done (5 left) 2024-09-15T03:53:36.809 INFO:tasks.rados.rados.0.smithi122.stdout:1851: done (4 left) 2024-09-15T03:53:36.809 INFO:tasks.rados.rados.0.smithi122.stdout:1852: done (3 left) 2024-09-15T03:53:36.809 INFO:tasks.rados.rados.0.smithi122.stdout:1853: done (2 left) 2024-09-15T03:53:36.809 INFO:tasks.rados.rados.0.smithi122.stdout:1854: done (1 left) 2024-09-15T03:53:36.809 INFO:tasks.rados.rados.0.smithi122.stdout:1855: done (0 left) 2024-09-15T03:53:36.809 INFO:tasks.rados.rados.0.smithi122.stdout:1856: snap_remove snap 189 2024-09-15T03:53:37.759 INFO:tasks.rados.rados.0.smithi122.stdout:1856: done (0 left) 2024-09-15T03:53:37.759 INFO:tasks.rados.rados.0.smithi122.stdout:1857: write oid 31 current snap is 204 2024-09-15T03:53:37.759 INFO:tasks.rados.rados.0.smithi122.stdout:1857: seq_num 680 ranges {628760=724737,1892226=673223,3157635=271125} 2024-09-15T03:53:37.768 INFO:tasks.rados.rados.0.smithi122.stdout:1857: writing smithi12223906-31 from 628760 to 1353497 tid 1 2024-09-15T03:53:37.775 INFO:tasks.rados.rados.0.smithi122.stdout:1857: writing smithi12223906-31 from 1892226 to 2565449 tid 2 2024-09-15T03:53:37.780 INFO:tasks.rados.rados.0.smithi122.stdout:1857: writing smithi12223906-31 from 3157635 to 3428760 tid 3 2024-09-15T03:53:37.780 INFO:tasks.rados.rados.0.smithi122.stdout:1858: snap_create 2024-09-15T03:53:37.780 INFO:tasks.rados.rados.0.smithi122.stdout:1857: finishing write tid 1 to smithi12223906-31 2024-09-15T03:53:37.782 INFO:tasks.rados.rados.0.smithi122.stdout:1857: finishing write tid 2 to smithi12223906-31 2024-09-15T03:53:37.785 INFO:tasks.rados.rados.0.smithi122.stdout:1857: finishing write tid 3 to smithi12223906-31 2024-09-15T03:53:37.785 INFO:tasks.rados.rados.0.smithi122.stdout:1857: finishing write tid 4 to smithi12223906-31 2024-09-15T03:53:37.785 INFO:tasks.rados.rados.0.smithi122.stdout:1857: finishing write tid 5 to smithi12223906-31 2024-09-15T03:53:37.785 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 790 (ObjNum 680 snap 204 seq_num 680) dirty exists 2024-09-15T03:53:37.785 INFO:tasks.rados.rados.0.smithi122.stdout:1857: left oid 31 (ObjNum 680 snap 204 seq_num 680) 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:1857: done (1 left) 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:1858: done (0 left) 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:1859: rollback oid 34 current snap is 205 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 193 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:1860: copy_from oid 19 from oid 13 current snap is 205 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:1861: read oid 25 snap -1 2024-09-15T03:53:38.769 INFO:tasks.rados.rados.0.smithi122.stdout:1861: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:53:38.770 INFO:tasks.rados.rados.0.smithi122.stdout:1862: snap_create 2024-09-15T03:53:38.776 INFO:tasks.rados.rados.0.smithi122.stdout:1859: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:53:38.777 INFO:tasks.rados.rados.0.smithi122.stdout:1859: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:53:38.777 INFO:tasks.rados.rados.0.smithi122.stdout:1859: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:53:38.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 725 (ObjNum 631 snap 192 seq_num 631) dirty exists 2024-09-15T03:53:38.779 INFO:tasks.rados.rados.0.smithi122.stdout:1861: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:53:38.942 INFO:tasks.rados.rados.0.smithi122.stdout:1860: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:53:38.943 INFO:tasks.rados.rados.0.smithi122.stdout:1860: finishing copy_from to smithi12223906-19 2024-09-15T03:53:38.943 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 790 (ObjNum 661 snap 201 seq_num 661) dirty exists 2024-09-15T03:53:39.783 INFO:tasks.rados.rados.0.smithi122.stdout:1859: done (3 left) 2024-09-15T03:53:39.783 INFO:tasks.rados.rados.0.smithi122.stdout:1860: done (2 left) 2024-09-15T03:53:39.783 INFO:tasks.rados.rados.0.smithi122.stdout:1861: done (1 left) 2024-09-15T03:53:39.783 INFO:tasks.rados.rados.0.smithi122.stdout:1862: done (0 left) 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1863: read oid 40 snap 200 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1863: expect deleted 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1864: read oid 8 snap -1 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1864: expect deleted 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1865: read oid 21 snap -1 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1865: expect (ObjNum 672 snap 204 seq_num 672) 2024-09-15T03:53:39.784 INFO:tasks.rados.rados.0.smithi122.stdout:1866: snap_create 2024-09-15T03:53:39.793 INFO:tasks.rados.rados.0.smithi122.stdout:1865: expect (ObjNum 672 snap 204 seq_num 672) 2024-09-15T03:53:40.793 INFO:tasks.rados.rados.0.smithi122.stdout:1863: done (3 left) 2024-09-15T03:53:40.793 INFO:tasks.rados.rados.0.smithi122.stdout:1864: done (2 left) 2024-09-15T03:53:40.793 INFO:tasks.rados.rados.0.smithi122.stdout:1865: done (1 left) 2024-09-15T03:53:40.794 INFO:tasks.rados.rados.0.smithi122.stdout:1866: done (0 left) 2024-09-15T03:53:40.794 INFO:tasks.rados.rados.0.smithi122.stdout:1867: snap_create 2024-09-15T03:53:41.750 INFO:tasks.rados.rados.0.smithi122.stdout:1867: done (0 left) 2024-09-15T03:53:41.750 INFO:tasks.rados.rados.0.smithi122.stdout:1868: snap_remove snap 205 2024-09-15T03:53:42.756 INFO:tasks.rados.rados.0.smithi122.stdout:1868: done (0 left) 2024-09-15T03:53:42.757 INFO:tasks.rados.rados.0.smithi122.stdout:1869: snap_create 2024-09-15T03:53:43.753 INFO:tasks.rados.rados.0.smithi122.stdout:1869: done (0 left) 2024-09-15T03:53:43.753 INFO:tasks.rados.rados.0.smithi122.stdout:1870: snap_create 2024-09-15T03:53:44.757 INFO:tasks.rados.rados.0.smithi122.stdout:1870: done (0 left) 2024-09-15T03:53:44.757 INFO:tasks.rados.rados.0.smithi122.stdout:1871: read oid 25 snap 198 2024-09-15T03:53:44.757 INFO:tasks.rados.rados.0.smithi122.stdout:1871: expect (ObjNum 628 snap 187 seq_num 628) 2024-09-15T03:53:44.757 INFO:tasks.rados.rados.0.smithi122.stdout:1872: delete oid 23 current snap is 210 2024-09-15T03:53:44.763 INFO:tasks.rados.rados.0.smithi122.stdout:1872: done (1 left) 2024-09-15T03:53:44.763 INFO:tasks.rados.rados.0.smithi122.stdout:1873: write oid 9 current snap is 210 2024-09-15T03:53:44.763 INFO:tasks.rados.rados.0.smithi122.stdout:1873: seq_num 682 ranges {662374=475235,1609190=676892,2812481=624101} 2024-09-15T03:53:44.770 INFO:tasks.rados.rados.0.smithi122.stdout:1873: writing smithi12223906-9 from 662374 to 1137609 tid 1 2024-09-15T03:53:44.778 INFO:tasks.rados.rados.0.smithi122.stdout:1873: writing smithi12223906-9 from 1609190 to 2286082 tid 2 2024-09-15T03:53:44.785 INFO:tasks.rados.rados.0.smithi122.stdout:1873: writing smithi12223906-9 from 2812481 to 3436582 tid 3 2024-09-15T03:53:44.785 INFO:tasks.rados.rados.0.smithi122.stdout:1874: write oid 37 current snap is 210 2024-09-15T03:53:44.785 INFO:tasks.rados.rados.0.smithi122.stdout:1874: seq_num 683 ranges {679181=550484,1667672=536903,2839904=435373} 2024-09-15T03:53:44.791 INFO:tasks.rados.rados.0.smithi122.stdout:1874: writing smithi12223906-37 from 679181 to 1229665 tid 1 2024-09-15T03:53:44.797 INFO:tasks.rados.rados.0.smithi122.stdout:1874: writing smithi12223906-37 from 1667672 to 2204575 tid 2 2024-09-15T03:53:44.803 INFO:tasks.rados.rados.0.smithi122.stdout:1874: writing smithi12223906-37 from 2839904 to 3275277 tid 3 2024-09-15T03:53:44.804 INFO:tasks.rados.rados.0.smithi122.stdout:1875: read oid 34 snap -1 2024-09-15T03:53:44.804 INFO:tasks.rados.rados.0.smithi122.stdout:1875: expect (ObjNum 631 snap 192 seq_num 631) 2024-09-15T03:53:44.804 INFO:tasks.rados.rados.0.smithi122.stdout:1876: snap_create 2024-09-15T03:53:44.804 INFO:tasks.rados.rados.0.smithi122.stdout:1871: expect (ObjNum 628 snap 187 seq_num 628) 2024-09-15T03:53:44.952 INFO:tasks.rados.rados.0.smithi122.stdout:1873: finishing write tid 1 to smithi12223906-9 2024-09-15T03:53:44.952 INFO:tasks.rados.rados.0.smithi122.stdout:1873: finishing write tid 2 to smithi12223906-9 2024-09-15T03:53:44.952 INFO:tasks.rados.rados.0.smithi122.stdout:1873: finishing write tid 3 to smithi12223906-9 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:1873: finishing write tid 4 to smithi12223906-9 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:1873: finishing write tid 5 to smithi12223906-9 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 809 (ObjNum 682 snap 210 seq_num 682) dirty exists 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:1873: left oid 9 (ObjNum 682 snap 210 seq_num 682) 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:1874: finishing write tid 1 to smithi12223906-37 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:1874: finishing write tid 2 to smithi12223906-37 2024-09-15T03:53:44.953 INFO:tasks.rados.rados.0.smithi122.stdout:1875: expect (ObjNum 631 snap 192 seq_num 631) 2024-09-15T03:53:45.119 INFO:tasks.rados.rados.0.smithi122.stdout:1874: finishing write tid 3 to smithi12223906-37 2024-09-15T03:53:45.120 INFO:tasks.rados.rados.0.smithi122.stdout:1874: finishing write tid 4 to smithi12223906-37 2024-09-15T03:53:45.120 INFO:tasks.rados.rados.0.smithi122.stdout:1874: finishing write tid 5 to smithi12223906-37 2024-09-15T03:53:45.120 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 813 (ObjNum 683 snap 210 seq_num 683) dirty exists 2024-09-15T03:53:45.120 INFO:tasks.rados.rados.0.smithi122.stdout:1874: left oid 37 (ObjNum 683 snap 210 seq_num 683) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1871: done (4 left) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1873: done (3 left) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1874: done (2 left) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1875: done (1 left) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1876: done (0 left) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1877: read oid 10 snap -1 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1877: expect (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1878: read oid 47 snap 176 2024-09-15T03:53:45.770 INFO:tasks.rados.rados.0.smithi122.stdout:1878: expect (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:53:45.771 INFO:tasks.rados.rados.0.smithi122.stdout:1879: rollback oid 28 current snap is 211 2024-09-15T03:53:45.771 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 28 to 186 2024-09-15T03:53:45.771 INFO:tasks.rados.rados.0.smithi122.stdout:1880: copy_from oid 5 from oid 45 current snap is 211 2024-09-15T03:53:45.771 INFO:tasks.rados.rados.0.smithi122.stdout:1881: snap_remove snap 194 2024-09-15T03:53:46.750 INFO:tasks.rados.rados.0.smithi122.stdout:1881: done (4 left) 2024-09-15T03:53:46.751 INFO:tasks.rados.rados.0.smithi122.stdout:1882: write (excl) oid 29 current snap is 211 2024-09-15T03:53:46.751 INFO:tasks.rados.rados.0.smithi122.stdout:1882: seq_num 684 ranges {695988=625733,2009801=680561,3150974=345014} 2024-09-15T03:53:46.759 INFO:tasks.rados.rados.0.smithi122.stdout:1882: writing smithi12223906-29 from 695988 to 1321721 tid 1 2024-09-15T03:53:46.767 INFO:tasks.rados.rados.0.smithi122.stdout:1882: writing smithi12223906-29 from 2009801 to 2690362 tid 2 2024-09-15T03:53:46.772 INFO:tasks.rados.rados.0.smithi122.stdout:1882: writing smithi12223906-29 from 3150974 to 3495988 tid 3 2024-09-15T03:53:46.773 INFO:tasks.rados.rados.0.smithi122.stdout:1883: read oid 30 snap 203 2024-09-15T03:53:46.773 INFO:tasks.rados.rados.0.smithi122.stdout:1883: expect (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:46.773 INFO:tasks.rados.rados.0.smithi122.stdout:1884: read oid 27 snap -1 2024-09-15T03:53:46.773 INFO:tasks.rados.rados.0.smithi122.stdout:1884: expect deleted 2024-09-15T03:53:46.773 INFO:tasks.rados.rados.0.smithi122.stdout:1885: write oid 23 current snap is 211 2024-09-15T03:53:46.773 INFO:tasks.rados.rados.0.smithi122.stdout:1885: seq_num 685 ranges {712795=417335,1784636=540572,2894750=457917} 2024-09-15T03:53:46.779 INFO:tasks.rados.rados.0.smithi122.stdout:1885: writing smithi12223906-23 from 712795 to 1130130 tid 1 2024-09-15T03:53:46.786 INFO:tasks.rados.rados.0.smithi122.stdout:1885: writing smithi12223906-23 from 1784636 to 2325208 tid 2 2024-09-15T03:53:46.792 INFO:tasks.rados.rados.0.smithi122.stdout:1885: writing smithi12223906-23 from 2894750 to 3352667 tid 3 2024-09-15T03:53:46.792 INFO:tasks.rados.rados.0.smithi122.stdout:1886: read oid 4 snap -1 2024-09-15T03:53:46.792 INFO:tasks.rados.rados.0.smithi122.stdout:1886: expect (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:53:46.792 INFO:tasks.rados.rados.0.smithi122.stdout:1887: write (excl) oid 43 current snap is 211 2024-09-15T03:53:46.792 INFO:tasks.rados.rados.0.smithi122.stdout:1887: seq_num 686 ranges {729602=492584,1843118=684230,3322173=207429} 2024-09-15T03:53:46.799 INFO:tasks.rados.rados.0.smithi122.stdout:1887: writing smithi12223906-43 from 729602 to 1222186 tid 1 2024-09-15T03:53:46.806 INFO:tasks.rados.rados.0.smithi122.stdout:1887: writing smithi12223906-43 from 1843118 to 2527348 tid 2 2024-09-15T03:53:46.812 INFO:tasks.rados.rados.0.smithi122.stdout:1887: writing smithi12223906-43 from 3322173 to 3529602 tid 3 2024-09-15T03:53:46.812 INFO:tasks.rados.rados.0.smithi122.stdout:1888: write (excl) oid 7 current snap is 211 2024-09-15T03:53:46.812 INFO:tasks.rados.rados.0.smithi122.stdout:1888: seq_num 687 ranges {746409=567833,1901600=544241,3065949=480460} 2024-09-15T03:53:46.819 INFO:tasks.rados.rados.0.smithi122.stdout:1888: writing smithi12223906-7 from 746409 to 1314242 tid 1 2024-09-15T03:53:46.826 INFO:tasks.rados.rados.0.smithi122.stdout:1888: writing smithi12223906-7 from 1901600 to 2445841 tid 2 2024-09-15T03:53:46.832 INFO:tasks.rados.rados.0.smithi122.stdout:1888: writing smithi12223906-7 from 3065949 to 3546409 tid 3 2024-09-15T03:53:46.832 INFO:tasks.rados.rados.0.smithi122.stdout:1889: delete oid 1 current snap is 211 2024-09-15T03:53:46.832 INFO:tasks.rados.rados.0.smithi122.stdout:1877: expect (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:53:47.066 INFO:tasks.rados.rados.0.smithi122.stdout:1879: finishing rollback tid 0 to smithi12223906-28 2024-09-15T03:53:47.066 INFO:tasks.rados.rados.0.smithi122.stdout:1879: finishing rollback tid 1 to smithi12223906-28 2024-09-15T03:53:47.067 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 793 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:53:47.067 INFO:tasks.rados.rados.0.smithi122.stdout:1878: expect (ObjNum 545 snap 161 seq_num 545) 2024-09-15T03:53:47.281 INFO:tasks.rados.rados.0.smithi122.stdout:1880: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:53:47.281 INFO:tasks.rados.rados.0.smithi122.stdout:1880: finishing copy_from to smithi12223906-5 2024-09-15T03:53:47.281 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 778 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:53:47.282 INFO:tasks.rados.rados.0.smithi122.stdout:1882: finishing write tid 1 to smithi12223906-29 2024-09-15T03:53:47.282 INFO:tasks.rados.rados.0.smithi122.stdout:1882: finishing write tid 2 to smithi12223906-29 2024-09-15T03:53:47.282 INFO:tasks.rados.rados.0.smithi122.stdout:1883: expect (ObjNum 655 snap 201 seq_num 655) 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1882: finishing write tid 3 to smithi12223906-29 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1882: finishing write tid 4 to smithi12223906-29 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1882: finishing write tid 5 to smithi12223906-29 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 797 (ObjNum 684 snap 211 seq_num 684) dirty exists 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1882: left oid 29 (ObjNum 684 snap 211 seq_num 684) 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1885: finishing write tid 1 to smithi12223906-23 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1885: finishing write tid 2 to smithi12223906-23 2024-09-15T03:53:47.372 INFO:tasks.rados.rados.0.smithi122.stdout:1885: finishing write tid 3 to smithi12223906-23 2024-09-15T03:53:47.373 INFO:tasks.rados.rados.0.smithi122.stdout:1885: finishing write tid 4 to smithi12223906-23 2024-09-15T03:53:47.373 INFO:tasks.rados.rados.0.smithi122.stdout:1885: finishing write tid 5 to smithi12223906-23 2024-09-15T03:53:47.373 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 951 (ObjNum 685 snap 211 seq_num 685) dirty exists 2024-09-15T03:53:47.373 INFO:tasks.rados.rados.0.smithi122.stdout:1885: left oid 23 (ObjNum 685 snap 211 seq_num 685) 2024-09-15T03:53:47.373 INFO:tasks.rados.rados.0.smithi122.stdout:1886: expect (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:53:47.461 INFO:tasks.rados.rados.0.smithi122.stdout:1887: finishing write tid 1 to smithi12223906-43 2024-09-15T03:53:47.462 INFO:tasks.rados.rados.0.smithi122.stdout:1887: finishing write tid 2 to smithi12223906-43 2024-09-15T03:53:47.462 INFO:tasks.rados.rados.0.smithi122.stdout:1887: finishing write tid 3 to smithi12223906-43 2024-09-15T03:53:47.462 INFO:tasks.rados.rados.0.smithi122.stdout:1887: finishing write tid 4 to smithi12223906-43 2024-09-15T03:53:47.462 INFO:tasks.rados.rados.0.smithi122.stdout:1887: finishing write tid 5 to smithi12223906-43 2024-09-15T03:53:47.462 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 995 (ObjNum 686 snap 211 seq_num 686) dirty exists 2024-09-15T03:53:47.462 INFO:tasks.rados.rados.0.smithi122.stdout:1887: left oid 43 (ObjNum 686 snap 211 seq_num 686) 2024-09-15T03:53:47.463 INFO:tasks.rados.rados.0.smithi122.stdout:1888: finishing write tid 1 to smithi12223906-7 2024-09-15T03:53:47.463 INFO:tasks.rados.rados.0.smithi122.stdout:1888: finishing write tid 2 to smithi12223906-7 2024-09-15T03:53:47.463 INFO:tasks.rados.rados.0.smithi122.stdout:1888: finishing write tid 3 to smithi12223906-7 2024-09-15T03:53:47.463 INFO:tasks.rados.rados.0.smithi122.stdout:1888: finishing write tid 4 to smithi12223906-7 2024-09-15T03:53:47.463 INFO:tasks.rados.rados.0.smithi122.stdout:1888: finishing write tid 5 to smithi12223906-7 2024-09-15T03:53:47.463 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 801 (ObjNum 687 snap 211 seq_num 687) dirty exists 2024-09-15T03:53:47.464 INFO:tasks.rados.rados.0.smithi122.stdout:1888: left oid 7 (ObjNum 687 snap 211 seq_num 687) 2024-09-15T03:53:47.464 INFO:tasks.rados.rados.0.smithi122.stdout:1877: done (11 left) 2024-09-15T03:53:47.464 INFO:tasks.rados.rados.0.smithi122.stdout:1878: done (10 left) 2024-09-15T03:53:47.464 INFO:tasks.rados.rados.0.smithi122.stdout:1879: done (9 left) 2024-09-15T03:53:47.464 INFO:tasks.rados.rados.0.smithi122.stdout:1880: done (8 left) 2024-09-15T03:53:47.464 INFO:tasks.rados.rados.0.smithi122.stdout:1882: done (7 left) 2024-09-15T03:53:47.465 INFO:tasks.rados.rados.0.smithi122.stdout:1883: done (6 left) 2024-09-15T03:53:47.465 INFO:tasks.rados.rados.0.smithi122.stdout:1884: done (5 left) 2024-09-15T03:53:47.465 INFO:tasks.rados.rados.0.smithi122.stdout:1885: done (4 left) 2024-09-15T03:53:47.466 INFO:tasks.rados.rados.0.smithi122.stdout:1886: done (3 left) 2024-09-15T03:53:47.466 INFO:tasks.rados.rados.0.smithi122.stdout:1887: done (2 left) 2024-09-15T03:53:47.466 INFO:tasks.rados.rados.0.smithi122.stdout:1888: done (1 left) 2024-09-15T03:53:47.466 INFO:tasks.rados.rados.0.smithi122.stdout:1889: done (0 left) 2024-09-15T03:53:47.466 INFO:tasks.rados.rados.0.smithi122.stdout:1890: delete oid 19 current snap is 211 2024-09-15T03:53:47.468 INFO:tasks.rados.rados.0.smithi122.stdout:1890: done (0 left) 2024-09-15T03:53:47.468 INFO:tasks.rados.rados.0.smithi122.stdout:1891: snap_remove snap 203 2024-09-15T03:53:47.789 INFO:tasks.rados.rados.0.smithi122.stdout:1891: done (0 left) 2024-09-15T03:53:47.789 INFO:tasks.rados.rados.0.smithi122.stdout:1892: snap_remove snap 199 2024-09-15T03:53:48.801 INFO:tasks.rados.rados.0.smithi122.stdout:1892: done (0 left) 2024-09-15T03:53:48.801 INFO:tasks.rados.rados.0.smithi122.stdout:1893: write oid 22 current snap is 211 2024-09-15T03:53:48.801 INFO:tasks.rados.rados.0.smithi122.stdout:1893: seq_num 690 ranges {796830=793580,2360693=691568} 2024-09-15T03:53:48.811 INFO:tasks.rados.rados.0.smithi122.stdout:1893: writing smithi12223906-22 from 796830 to 1590410 tid 1 2024-09-15T03:53:48.820 INFO:tasks.rados.rados.0.smithi122.stdout:1893: writing smithi12223906-22 from 2360693 to 3052261 tid 2 2024-09-15T03:53:48.820 INFO:tasks.rados.rados.0.smithi122.stdout:1894: write oid 31 current snap is 211 2024-09-15T03:53:48.820 INFO:tasks.rados.rados.0.smithi122.stdout:1893: finishing write tid 1 to smithi12223906-22 2024-09-15T03:53:48.820 INFO:tasks.rados.rados.0.smithi122.stdout:1894: seq_num 691 ranges {413637=468829,1619175=435226,2542935=758255} 2024-09-15T03:53:48.824 INFO:tasks.rados.rados.0.smithi122.stdout:1894: writing smithi12223906-31 from 413637 to 882466 tid 1 2024-09-15T03:53:48.831 INFO:tasks.rados.rados.0.smithi122.stdout:1894: writing smithi12223906-31 from 1619175 to 2054401 tid 2 2024-09-15T03:53:48.839 INFO:tasks.rados.rados.0.smithi122.stdout:1894: writing smithi12223906-31 from 2542935 to 3301190 tid 3 2024-09-15T03:53:48.839 INFO:tasks.rados.rados.0.smithi122.stdout:1895: snap_remove snap 206 2024-09-15T03:53:49.807 INFO:tasks.rados.rados.0.smithi122.stdout:1895: done (2 left) 2024-09-15T03:53:49.807 INFO:tasks.rados.rados.0.smithi122.stdout:1896: delete oid 2 current snap is 211 2024-09-15T03:53:49.807 INFO:tasks.rados.rados.0.smithi122.stdout:1893: finishing write tid 2 to smithi12223906-22 2024-09-15T03:53:49.807 INFO:tasks.rados.rados.0.smithi122.stdout:1893: finishing write tid 3 to smithi12223906-22 2024-09-15T03:53:49.807 INFO:tasks.rados.rados.0.smithi122.stdout:1893: finishing write tid 4 to smithi12223906-22 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 969 (ObjNum 690 snap 211 seq_num 690) dirty exists 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1893: left oid 22 (ObjNum 690 snap 211 seq_num 690) 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1894: finishing write tid 1 to smithi12223906-31 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1894: finishing write tid 2 to smithi12223906-31 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1894: finishing write tid 3 to smithi12223906-31 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1894: finishing write tid 4 to smithi12223906-31 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1894: finishing write tid 5 to smithi12223906-31 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 794 (ObjNum 691 snap 211 seq_num 691) dirty exists 2024-09-15T03:53:49.808 INFO:tasks.rados.rados.0.smithi122.stdout:1894: left oid 31 (ObjNum 691 snap 211 seq_num 691) 2024-09-15T03:53:49.811 INFO:tasks.rados.rados.0.smithi122.stdout:1893: done (2 left) 2024-09-15T03:53:49.812 INFO:tasks.rados.rados.0.smithi122.stdout:1894: done (1 left) 2024-09-15T03:53:49.812 INFO:tasks.rados.rados.0.smithi122.stdout:1896: done (0 left) 2024-09-15T03:53:49.812 INFO:tasks.rados.rados.0.smithi122.stdout:1897: copy_from oid 26 from oid 18 current snap is 211 2024-09-15T03:53:49.812 INFO:tasks.rados.rados.0.smithi122.stdout:1898: write oid 28 current snap is 211 2024-09-15T03:53:49.812 INFO:tasks.rados.rados.0.smithi122.stdout:1898: seq_num 693 ranges {447251=735680,1736139=438895,2597781=780799} 2024-09-15T03:53:49.818 INFO:tasks.rados.rados.0.smithi122.stdout:1898: writing smithi12223906-28 from 447251 to 1182931 tid 1 2024-09-15T03:53:49.824 INFO:tasks.rados.rados.0.smithi122.stdout:1898: writing smithi12223906-28 from 1736139 to 2175034 tid 2 2024-09-15T03:53:49.831 INFO:tasks.rados.rados.0.smithi122.stdout:1898: writing smithi12223906-28 from 2597781 to 3378580 tid 3 2024-09-15T03:53:49.832 INFO:tasks.rados.rados.0.smithi122.stdout:1899: snap_create 2024-09-15T03:53:49.832 INFO:tasks.rados.rados.0.smithi122.stdout:1898: finishing write tid 1 to smithi12223906-28 2024-09-15T03:53:49.832 INFO:tasks.rados.rados.0.smithi122.stdout:1897: finishing copy_from racing read to smithi12223906-26 2024-09-15T03:53:49.832 INFO:tasks.rados.rados.0.smithi122.stdout:1898: finishing write tid 2 to smithi12223906-28 2024-09-15T03:53:49.832 INFO:tasks.rados.rados.0.smithi122.stdout:1897: finishing copy_from to smithi12223906-26 2024-09-15T03:53:49.833 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 918 (ObjNum 670 snap 204 seq_num 670) dirty exists 2024-09-15T03:53:49.837 INFO:tasks.rados.rados.0.smithi122.stdout:1898: finishing write tid 3 to smithi12223906-28 2024-09-15T03:53:49.838 INFO:tasks.rados.rados.0.smithi122.stdout:1898: finishing write tid 4 to smithi12223906-28 2024-09-15T03:53:49.838 INFO:tasks.rados.rados.0.smithi122.stdout:1898: finishing write tid 5 to smithi12223906-28 2024-09-15T03:53:49.838 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 805 (ObjNum 693 snap 211 seq_num 693) dirty exists 2024-09-15T03:53:49.838 INFO:tasks.rados.rados.0.smithi122.stdout:1898: left oid 28 (ObjNum 693 snap 211 seq_num 693) 2024-09-15T03:53:50.814 INFO:tasks.rados.rados.0.smithi122.stdout:1897: done (2 left) 2024-09-15T03:53:50.814 INFO:tasks.rados.rados.0.smithi122.stdout:1898: done (1 left) 2024-09-15T03:53:50.814 INFO:tasks.rados.rados.0.smithi122.stdout:1899: done (0 left) 2024-09-15T03:53:50.814 INFO:tasks.rados.rados.0.smithi122.stdout:1900: delete oid 31 current snap is 212 2024-09-15T03:53:50.830 INFO:tasks.rados.rados.0.smithi122.stdout:1900: done (0 left) 2024-09-15T03:53:50.830 INFO:tasks.rados.rados.0.smithi122.stdout:1901: read oid 44 snap -1 2024-09-15T03:53:50.830 INFO:tasks.rados.rados.0.smithi122.stdout:1901: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:53:50.830 INFO:tasks.rados.rados.0.smithi122.stdout:1902: snap_remove snap 174 2024-09-15T03:53:51.760 INFO:tasks.rados.rados.0.smithi122.stdout:1901: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1901: done (1 left) 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1902: done (0 left) 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1903: copy_from oid 16 from oid 36 current snap is 212 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1904: copy_from oid 37 from oid 43 current snap is 212 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1905: copy_from oid 15 from oid 39 current snap is 212 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1906: read oid 46 snap -1 2024-09-15T03:53:51.965 INFO:tasks.rados.rados.0.smithi122.stdout:1906: expect deleted 2024-09-15T03:53:51.966 INFO:tasks.rados.rados.0.smithi122.stdout:1907: snap_create 2024-09-15T03:53:51.993 INFO:tasks.rados.rados.0.smithi122.stdout:1905: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:53:51.993 INFO:tasks.rados.rados.0.smithi122.stdout:1903: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:53:51.993 INFO:tasks.rados.rados.0.smithi122.stdout:1903: finishing copy_from to smithi12223906-16 2024-09-15T03:53:51.994 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 897 (ObjNum 633 snap 192 seq_num 633) dirty exists 2024-09-15T03:53:51.995 INFO:tasks.rados.rados.0.smithi122.stdout:1904: finishing copy_from racing read to smithi12223906-37 2024-09-15T03:53:51.999 INFO:tasks.rados.rados.0.smithi122.stdout:1905: finishing copy_from to smithi12223906-15 2024-09-15T03:53:52.000 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 886 (ObjNum 645 snap 199 seq_num 645) dirty exists 2024-09-15T03:53:52.000 INFO:tasks.rados.rados.0.smithi122.stdout:1904: finishing copy_from to smithi12223906-37 2024-09-15T03:53:52.000 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 995 (ObjNum 686 snap 211 seq_num 686) dirty exists 2024-09-15T03:53:52.840 INFO:tasks.rados.rados.0.smithi122.stdout:1903: done (4 left) 2024-09-15T03:53:52.840 INFO:tasks.rados.rados.0.smithi122.stdout:1904: done (3 left) 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1905: done (2 left) 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1906: done (1 left) 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1907: done (0 left) 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1908: rollback oid 28 current snap is 213 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 28 to 204 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1909: rollback oid 21 current snap is 213 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 182 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1910: rollback oid 41 current snap is 213 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 197 2024-09-15T03:53:52.841 INFO:tasks.rados.rados.0.smithi122.stdout:1911: delete oid 43 current snap is 213 2024-09-15T03:53:52.848 INFO:tasks.rados.rados.0.smithi122.stdout:1910: finishing rollback tid 0 to smithi12223906-41 2024-09-15T03:53:52.848 INFO:tasks.rados.rados.0.smithi122.stdout:1908: finishing rollback tid 0 to smithi12223906-28 2024-09-15T03:53:52.848 INFO:tasks.rados.rados.0.smithi122.stdout:1909: finishing rollback tid 0 to smithi12223906-21 2024-09-15T03:53:52.849 INFO:tasks.rados.rados.0.smithi122.stdout:1911: done (3 left) 2024-09-15T03:53:52.849 INFO:tasks.rados.rados.0.smithi122.stdout:1912: snap_remove snap 196 2024-09-15T03:53:53.854 INFO:tasks.rados.rados.0.smithi122.stdout:1912: done (3 left) 2024-09-15T03:53:53.854 INFO:tasks.rados.rados.0.smithi122.stdout:1913: copy_from oid 39 from oid 45 current snap is 213 2024-09-15T03:53:53.854 INFO:tasks.rados.rados.0.smithi122.stdout:1914: rollback oid 34 current snap is 213 2024-09-15T03:53:53.854 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 197 2024-09-15T03:53:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1915: copy_from oid 44 from oid 43 current snap is 213 2024-09-15T03:53:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1916: write oid 6 current snap is 213 2024-09-15T03:53:53.855 INFO:tasks.rados.rados.0.smithi122.stdout:1916: seq_num 696 ranges {497672=561427,1795232=702575,2963697=730968} 2024-09-15T03:53:53.859 INFO:tasks.rados.rados.0.smithi122.stdout:1916: writing smithi12223906-6 from 497672 to 1059099 tid 1 2024-09-15T03:53:53.868 INFO:tasks.rados.rados.0.smithi122.stdout:1916: writing smithi12223906-6 from 1795232 to 2497807 tid 2 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1916: writing smithi12223906-6 from 2963697 to 3694665 tid 3 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1917: copy_from oid 18 from oid 42 current snap is 213 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1908: finishing rollback tid 1 to smithi12223906-28 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1908: finishing rollback tid 2 to smithi12223906-28 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 808 (ObjNum 674 snap 204 seq_num 674) dirty exists 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1909: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1909: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 998 (ObjNum 526 snap 158 seq_num 526) dirty exists 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1910: finishing rollback tid 1 to smithi12223906-41 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 804 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:53:53.876 INFO:tasks.rados.rados.0.smithi122.stdout:1908: done (7 left) 2024-09-15T03:53:53.877 INFO:tasks.rados.rados.0.smithi122.stdout:1909: done (6 left) 2024-09-15T03:53:53.877 INFO:tasks.rados.rados.0.smithi122.stdout:1910: done (5 left) 2024-09-15T03:53:53.877 INFO:tasks.rados.rados.0.smithi122.stdout:1918: snap_remove snap 209 2024-09-15T03:53:54.342 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-09-15T03:53:54.342 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-09-15T03:53:54.343 INFO:tasks.ceph.mon.d:Restarting daemon 2024-09-15T03:53:54.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-09-15T03:53:54.346 INFO:tasks.ceph.mon.d:Started 2024-09-15T03:53:54.346 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:53:54.346 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T03:53:54.346 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T03:53:54.346 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T03:53:54.348 INFO:tasks.ceph.mon.g:Started 2024-09-15T03:53:54.349 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-15T03:53:54.349 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-15T03:53:54.349 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T03:53:54.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T03:53:54.351 INFO:tasks.ceph.mon.a:Started 2024-09-15T03:53:54.351 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:53:54.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:53:54.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient: get_monmap_and_config 2024-09-15T03:53:54.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient: build_initial_monmap 2024-09-15T03:53:54.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient: monmap: 2024-09-15T03:53:54.459 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:53:54.459 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:53:54.456984+0000 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:53:54.456984+0000 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:53:54.460 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 1 Processor -- start 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 1 -- start start 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient: init 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient: _reopen_session rank -1 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient: _add_conns ranks=[6,2,5,7,8,4,3,0,1] 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient(hunting): picked mon.noname-d con 0x7f486009ef50 addr v1:172.21.15.26:6792/0 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient(hunting): picked mon.noname-b con 0x7f4860130c50 addr v1:172.21.15.26:6790/0 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient(hunting): picked mon.noname-h con 0x7f4860136cc0 addr v1:172.21.15.122:6791/0 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486004ca10 con 0x7f4860130c50 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486009cd80 con 0x7f4860136cc0 2024-09-15T03:53:54.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486009f570 con 0x7f486009ef50 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient(hunting): _renew_subs 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 10 monclient(hunting): authenticate will time out at 1378.886997s 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f4864ec8700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f486009ef50 0x7f486009e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:52846/0 (socket says 172.21.15.26:52846) 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f4864ec8700 1 -- 172.21.15.26:0/2524645041 learned_addr learned my addr 172.21.15.26:0/2524645041 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504230143 0 0) 0x7f486004ca10 con 0x7f4860130c50 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient(hunting): my global_id is 134858 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4848003570 con 0x7f4860130c50 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145692860 0 0) 0x7f486009cd80 con 0x7f4860136cc0 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:53:54.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient(hunting): my global_id is 5378 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f486004ca10 con 0x7f4860136cc0 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 333384504 0 0) 0x7f4848003570 con 0x7f4860130c50 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 >> v1:172.21.15.26:6792/0 conn(0x7f486009ef50 legacy=0x7f486009e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 >> v1:172.21.15.122:6791/0 conn(0x7f4860136cc0 legacy=0x7f48601390a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 monclient: found mon.noname-b 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486009d060 con 0x7f4860130c50 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4858001b30 con 0x7f4860130c50 2024-09-15T03:53:54.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f485f7fe700 10 monclient: dump: 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:53:54.467 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 5 monclient: authenticate success, global_id 134858 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.454+0000 7f486712c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4858001e10 con 0x7f4860130c50 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: get_monmap_and_config success 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485f7fe700 1 -- 172.21.15.26:0/2524645041 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f48580025c0 con 0x7f4860130c50 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:53:54.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 4 set_mon_vals no callback set 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485f7fe700 10 monclient: dump: 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:53:54.469 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: shutdown 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- 172.21.15.26:0/2524645041 >> v1:172.21.15.26:6790/0 conn(0x7f4860130c50 legacy=0x7f4860131020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- 172.21.15.26:0/2524645041 shutdown_connections 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- 172.21.15.26:0/2524645041 >> 172.21.15.26:0/2524645041 conn(0x7f4860067800 msgr2=0x7f4860063ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- 172.21.15.26:0/2524645041 shutdown_connections 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- 172.21.15.26:0/2524645041 wait complete. 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: build_initial_monmap 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: monmap: 2024-09-15T03:53:54.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:53:54.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:53:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:53:54.472 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 Processor -- start 2024-09-15T03:53:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- start start 2024-09-15T03:53:54.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: init 2024-09-15T03:53:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: _reopen_session rank -1 2024-09-15T03:53:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient: _add_conns ranks=[1,7,3,2,4,8,6,5,0] 2024-09-15T03:53:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient(hunting): picked mon.noname-f con 0x7f4860130c50 addr v1:172.21.15.122:6789/0 2024-09-15T03:53:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient(hunting): picked mon.noname-i con 0x7f4860136cc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:53:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient(hunting): picked mon.noname-g con 0x7f4860119500 addr v1:172.21.15.122:6790/0 2024-09-15T03:53:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486009bd00 con 0x7f4860130c50 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486009e840 con 0x7f4860119500 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4860119c90 con 0x7f4860136cc0 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient(hunting): _renew_subs 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f486712c700 10 monclient(hunting): authenticate will time out at 1378.891147s 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f4864ec8700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f4860130c50 0x7f48601186e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:54038/0 (socket says 172.21.15.26:54038) 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f4864ec8700 1 -- 172.21.15.26:0/738005564 learned_addr learned my addr 172.21.15.26:0/738005564 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485ffff700 1 --1- 172.21.15.26:0/738005564 >> v1:172.21.15.122:6792/0 conn(0x7f4860136cc0 0x7f4860118df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:36214/0 (socket says 172.21.15.26:36214) 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270827796 0 0) 0x7f486009bd00 con 0x7f4860130c50 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 10 monclient(hunting): my global_id is 184321 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f484c003570 con 0x7f4860130c50 2024-09-15T03:53:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593926017 0 0) 0x7f4860119c90 con 0x7f4860136cc0 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 10 monclient(hunting): my global_id is 134746 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.458+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f486009bd00 con 0x7f4860136cc0 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 517226827 0 0) 0x7f484c003570 con 0x7f4860130c50 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4860119c90 con 0x7f4860130c50 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4854001950 con 0x7f4860130c50 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): dump: 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:53:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3555754891 0 0) 0x7f486009bd00 con 0x7f4860136cc0 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f484c003570 con 0x7f4860136cc0 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f48580022c0 con 0x7f4860136cc0 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient(hunting): dump: 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:53:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 789292821 0 0) 0x7f4860119c90 con 0x7f4860130c50 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 >> v1:172.21.15.122:6792/0 conn(0x7f4860136cc0 legacy=0x7f4860118df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 >> v1:172.21.15.122:6790/0 conn(0x7f4860119500 legacy=0x7f4860148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient: _finish_hunting 0 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 monclient: found mon.f 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486011aa00 con 0x7f4860130c50 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 10 monclient: _finish_auth 0 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f485cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 5 monclient: authenticate success, global_id 184321 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 monclient: _renew_subs 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486011af30 con 0x7f4860130c50 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 monclient: _renew_subs 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f486006e190 con 0x7f4860130c50 2024-09-15T03:53:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.462+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486004bbf0 con 0x7f4860130c50 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f48540048f0 con 0x7f4860130c50 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4854004da0 con 0x7f4860130c50 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 10 monclient: dump: 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:53:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4854004f40 con 0x7f4860130c50 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485effd700 4 set_mon_vals no callback set 2024-09-15T03:53:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(429..429 src has 224..429) v4 ==== 4003+0+0 (unknown 3652937004 0 0) 0x7f4854043ec0 con 0x7f4860130c50 2024-09-15T03:53:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485ffff700 10 monclient: get_auth_request con 0x7f484c037bc0 auth_method 0 2024-09-15T03:53:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f4854014d40 con 0x7f4860130c50 2024-09-15T03:53:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:53:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.466+0000 7f485cff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:53:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.694+0000 7f486712c700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:53:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.694+0000 7f486712c700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:53:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.694+0000 7f486712c700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:53:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:54.694+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 --> v1:172.21.15.122:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f486006f230 con 0x7f4860130c50 2024-09-15T03:53:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:57.458+0000 7f48477fe700 10 monclient: tick 2024-09-15T03:53:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:57.458+0000 7f48477fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:53:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:57.458+0000 7f48477fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:53:58.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:58.458+0000 7f48477fe700 10 monclient: tick 2024-09-15T03:53:58.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:58.458+0000 7f48477fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:53:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.458+0000 7f48477fe700 10 monclient: tick 2024-09-15T03:53:59.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.458+0000 7f48477fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:53:59.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.506+0000 7f485cff9700 1 -- 172.21.15.26:0/738005564 <== mon.1 v1:172.21.15.122:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 284387766) 0x7f4854014740 con 0x7f4860130c50 2024-09-15T03:53:59.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.506+0000 7f485cff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:53:59.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.506+0000 7f485cff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:53:59.511 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":62,"quorum":[1,2,4,5,6,7,8],"quorum_names":["f","b","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 >> v1:172.21.15.122:6812/20895 conn(0x7f484c037bc0 legacy=0x7f484c03a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 10 monclient: shutdown 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 >> v1:172.21.15.122:6789/0 conn(0x7f4860130c50 legacy=0x7f48601186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 shutdown_connections 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 >> 172.21.15.26:0/738005564 conn(0x7f4860067800 msgr2=0x7f486006f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 shutdown_connections 2024-09-15T03:53:59.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:53:59.510+0000 7f486712c700 1 -- 172.21.15.26:0/738005564 wait complete. 2024-09-15T03:53:59.585 INFO:tasks.rados.rados.0.smithi122.stdout:1918: done (5 left) 2024-09-15T03:53:59.586 INFO:tasks.rados.rados.0.smithi122.stdout:1919: write (excl) oid 22 current snap is 213 2024-09-15T03:53:59.586 INFO:tasks.rados.rados.0.smithi122.stdout:1919: seq_num 697 ranges {514479=636676,1853714=446233,2991120=542240} 2024-09-15T03:53:59.592 INFO:tasks.rados.rados.0.smithi122.stdout:1919: writing smithi12223906-22 from 514479 to 1151155 tid 1 2024-09-15T03:53:59.602 INFO:tasks.rados.rados.0.smithi122.stdout:1919: writing smithi12223906-22 from 1853714 to 2299947 tid 2 2024-09-15T03:53:59.609 INFO:tasks.rados.rados.0.smithi122.stdout:1919: writing smithi12223906-22 from 2991120 to 3533360 tid 3 2024-09-15T03:53:59.609 INFO:tasks.rados.rados.0.smithi122.stdout:1920: read oid 50 snap -1 2024-09-15T03:53:59.609 INFO:tasks.rados.rados.0.smithi122.stdout:1920: expect (ObjNum 637 snap 197 seq_num 637) 2024-09-15T03:53:59.610 INFO:tasks.rados.rados.0.smithi122.stdout:1921: snap_remove snap 182 2024-09-15T03:54:00.607 INFO:tasks.rados.rados.0.smithi122.stdout:1921: done (7 left) 2024-09-15T03:54:00.608 INFO:tasks.rados.rados.0.smithi122.stdout:1922: read oid 8 snap -1 2024-09-15T03:54:00.608 INFO:tasks.rados.rados.0.smithi122.stdout:1922: expect deleted 2024-09-15T03:54:00.608 INFO:tasks.rados.rados.0.smithi122.stdout:1923: snap_remove snap 212 2024-09-15T03:54:01.612 INFO:tasks.rados.rados.0.smithi122.stdout:1923: done (8 left) 2024-09-15T03:54:01.612 INFO:tasks.rados.rados.0.smithi122.stdout:1924: snap_remove snap 210 2024-09-15T03:54:01.761 INFO:tasks.rados.rados.0.smithi122.stdout:1924: done (8 left) 2024-09-15T03:54:01.761 INFO:tasks.rados.rados.0.smithi122.stdout:1925: rollback oid 27 current snap is 213 2024-09-15T03:54:01.761 INFO:tasks.rados.rados.0.smithi122.stdout:1915: finishing copy_from to smithi12223906-44 2024-09-15T03:54:01.761 INFO:tasks.rados.rados.0.smithi122.stdout:1915: got expected ENOENT (src dne) 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1915: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1914: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1914: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1914: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 728 (ObjNum 631 snap 192 seq_num 631) dirty exists 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1913: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1913: finishing copy_from to smithi12223906-39 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 778 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1916: finishing write tid 1 to smithi12223906-6 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1916: finishing write tid 2 to smithi12223906-6 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1917: finishing copy_from to smithi12223906-18 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1917: got expected ENOENT (src dne) 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1917: finishing copy_from racing read to smithi12223906-18 2024-09-15T03:54:01.762 INFO:tasks.rados.rados.0.smithi122.stdout:1916: finishing write tid 3 to smithi12223906-6 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1916: finishing write tid 4 to smithi12223906-6 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1916: finishing write tid 5 to smithi12223906-6 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 813 (ObjNum 696 snap 213 seq_num 696) dirty exists 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1916: left oid 6 (ObjNum 696 snap 213 seq_num 696) 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1919: finishing write tid 1 to smithi12223906-22 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1919: finishing write tid 2 to smithi12223906-22 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1919: finishing write tid 3 to smithi12223906-22 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1919: finishing write tid 4 to smithi12223906-22 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1919: finishing write tid 5 to smithi12223906-22 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1008 (ObjNum 697 snap 213 seq_num 697) dirty exists 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1919: left oid 22 (ObjNum 697 snap 213 seq_num 697) 2024-09-15T03:54:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:1920: expect (ObjNum 637 snap 197 seq_num 637) 2024-09-15T03:54:01.849 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 208 2024-09-15T03:54:01.849 INFO:tasks.rados.rados.0.smithi122.stdout:1913: done (8 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1914: done (7 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1915: done (6 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1916: done (5 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1917: done (4 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1919: done (3 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1920: done (2 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1922: done (1 left) 2024-09-15T03:54:01.850 INFO:tasks.rados.rados.0.smithi122.stdout:1926: snap_create 2024-09-15T03:54:01.853 INFO:tasks.rados.rados.0.smithi122.stdout:1925: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:54:01.853 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 905 (ObjNum 417 snap 0 seq_num 1457507696) dirty dne 2024-09-15T03:54:02.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:54:02.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient: get_monmap_and_config 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient: build_initial_monmap 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient: monmap: 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:02.644822+0000 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:02.644822+0000 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:02.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:02.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 1 Processor -- start 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 1 -- start start 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient: init 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient: _add_conns ranks=[0,7,6,3,4,8,2,1,5] 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient(hunting): picked mon.noname-a con 0x7fb90c132aa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:02.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient(hunting): picked mon.noname-i con 0x7fb90c09fab0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient(hunting): picked mon.noname-d con 0x7fb90c09fe80 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c04ca10 con 0x7fb90c132aa0 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c1338d0 con 0x7fb90c09fe80 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c132e70 con 0x7fb90c09fab0 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90bfff700 10 monclient(hunting): authenticate will time out at 1387.074626s 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fb90c09fe80 0x7fb90c13b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44472/0 (socket says 172.21.15.26:44472) 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb90c132aa0 0x7fb90c1331c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47232/0 (socket says 172.21.15.26:47232) 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb90b7fe700 1 -- 172.21.15.26:0/318571461 learned_addr learned my addr 172.21.15.26:0/318571461 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639235794 0 0) 0x7fb90c132e70 con 0x7fb90c09fab0 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:02.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 10 monclient(hunting): my global_id is 134755 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8f8003570 con 0x7fb90c09fab0 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3928816282 0 0) 0x7fb8f8003570 con 0x7fb90c09fab0 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.642+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 >> v1:172.21.15.26:6792/0 conn(0x7fb90c09fe80 legacy=0x7fb90c13b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 >> v1:172.21.15.26:6789/0 conn(0x7fb90c132aa0 legacy=0x7fb90c1331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 1 monclient: found mon.noname-i 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0a0c20 con 0x7fb90c09fab0 2024-09-15T03:54:02.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: _finish_auth 0 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb8f40017b0 con 0x7fb90c09fab0 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 5 monclient: authenticate success, global_id 134755 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: dump: 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb8f4001a60 con 0x7fb90c09fab0 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 1 -- 172.21.15.26:0/318571461 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb8f4002240 con 0x7fb90c09fab0 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 4 set_mon_vals no callback set 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb909ffb700 10 monclient: dump: 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:02.656 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 10 monclient: shutdown 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 -- 172.21.15.26:0/318571461 >> v1:172.21.15.122:6792/0 conn(0x7fb90c09fab0 legacy=0x7fb90c137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 -- 172.21.15.26:0/318571461 shutdown_connections 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 -- 172.21.15.26:0/318571461 >> 172.21.15.26:0/318571461 conn(0x7fb90c06e430 msgr2=0x7fb90c0a5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 -- 172.21.15.26:0/318571461 shutdown_connections 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 -- 172.21.15.26:0/318571461 wait complete. 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 10 monclient: build_initial_monmap 2024-09-15T03:54:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 10 monclient: monmap: 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:02.661 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 Processor -- start 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 1 -- start start 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.646+0000 7fb90bfff700 10 monclient: init 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient: _add_conns ranks=[2,0,6,1,5,7,3,8,4] 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient(hunting): picked mon.noname-b con 0x7fb90c09fab0 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient(hunting): picked mon.noname-a con 0x7fb90c09fe80 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient(hunting): picked mon.noname-d con 0x7fb90c132aa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c0a1b10 con 0x7fb90c09fe80 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c0a1590 con 0x7fb90c09fab0 2024-09-15T03:54:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90c06d1f0 con 0x7fb90c132aa0 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient(hunting): authenticate will time out at 1387.080618s 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90affd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb90c09fab0 0x7fb90c145bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:55626/0 (socket says 172.21.15.26:55626) 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90affd700 1 -- 172.21.15.26:0/1581809149 learned_addr learned my addr 172.21.15.26:0/1581809149 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056130452 0 0) 0x7fb90c0a1590 con 0x7fb90c09fab0 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): my global_id is 134867 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec003570 con 0x7fb90c09fab0 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3079050564 0 0) 0x7fb8ec003570 con 0x7fb90c09fab0 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb90c0a1590 con 0x7fb90c09fab0 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb8fc001960 con 0x7fb90c09fab0 2024-09-15T03:54:02.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): dump: 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 929815020 0 0) 0x7fb90c0a1590 con 0x7fb90c09fab0 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 >> v1:172.21.15.26:6792/0 conn(0x7fb90c132aa0 legacy=0x7fb90c14df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 >> v1:172.21.15.26:6789/0 conn(0x7fb90c09fe80 legacy=0x7fb90c131c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 monclient: found mon.b 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c06b450 con 0x7fb90c09fab0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: _finish_auth 0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 5 monclient: authenticate success, global_id 134867 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient: _renew_subs 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90c146ad0 con 0x7fb90c09fab0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient: _renew_subs 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90c132340 con 0x7fb90c09fab0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb8fc001d90 con 0x7fb90c09fab0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb8fc004ee0 con 0x7fb90c09fab0 2024-09-15T03:54:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb9097fa700 4 set_mon_vals no callback set 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.650+0000 7fb902ffd700 10 monclient: dump: 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:02.667 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb90bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb90bfff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:02.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb90bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb8fc005110 con 0x7fb90c09fab0 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f00050b0 con 0x7fb90c09fab0 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(433..433 src has 224..433) v4 ==== 4063+0+0 (unknown 3226152462 0 0) 0x7fb8fc043fc0 con 0x7fb90c09fab0 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.654+0000 7fb90a7fc700 10 monclient: get_auth_request con 0x7fb8ec0374f0 auth_method 0 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.658+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fb8fc014990 con 0x7fb90c09fab0 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.662+0000 7fb902ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:02.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.662+0000 7fb902ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:02.758 INFO:tasks.rados.rados.0.smithi122.stdout:1925: done (1 left) 2024-09-15T03:54:02.758 INFO:tasks.rados.rados.0.smithi122.stdout:1926: done (0 left) 2024-09-15T03:54:02.758 INFO:tasks.rados.rados.0.smithi122.stdout:1927: snap_remove snap 176 2024-09-15T03:54:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb90bfff700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:54:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb90bfff700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb90bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb8f00058e0 con 0x7fb90c09fab0 2024-09-15T03:54:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb902ffd700 1 -- 172.21.15.26:0/1581809149 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2688 (unknown 254746657 0 2646708009) 0x7fb8fc014990 con 0x7fb90c09fab0 2024-09-15T03:54:02.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb902ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:02.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.886+0000 7fb902ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:02.894 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":62,"quorum":[1,2,4,5,6,7,8],"quorum_names":["f","b","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:54:02.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 >> v1:172.21.15.122:6812/20895 conn(0x7fb8ec0374f0 legacy=0x7fb8ec0399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 10 monclient: shutdown 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 >> v1:172.21.15.26:6790/0 conn(0x7fb90c09fab0 legacy=0x7fb90c145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 shutdown_connections 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 >> 172.21.15.26:0/1581809149 conn(0x7fb90c06e430 msgr2=0x7fb90c0a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 shutdown_connections 2024-09-15T03:54:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:02.890+0000 7fb90bfff700 1 -- 172.21.15.26:0/1581809149 wait complete. 2024-09-15T03:54:03.773 INFO:tasks.rados.rados.0.smithi122.stdout:1927: done (0 left) 2024-09-15T03:54:03.773 INFO:tasks.rados.rados.0.smithi122.stdout:1928: write (excl) oid 18 current snap is 214 2024-09-15T03:54:03.773 INFO:tasks.rados.rados.0.smithi122.stdout:1928: seq_num 698 ranges {531286=711925,1912196=706244,3018543=712743} 2024-09-15T03:54:03.780 INFO:tasks.rados.rados.0.smithi122.stdout:1928: writing smithi12223906-18 from 531286 to 1243211 tid 1 2024-09-15T03:54:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:1928: writing smithi12223906-18 from 1912196 to 2618440 tid 2 2024-09-15T03:54:03.797 INFO:tasks.rados.rados.0.smithi122.stdout:1928: writing smithi12223906-18 from 3018543 to 3731286 tid 3 2024-09-15T03:54:03.797 INFO:tasks.rados.rados.0.smithi122.stdout:1929: copy_from oid 49 from oid 39 current snap is 214 2024-09-15T03:54:03.797 INFO:tasks.rados.rados.0.smithi122.stdout:1930: delete oid 42 current snap is 214 2024-09-15T03:54:03.797 INFO:tasks.rados.rados.0.smithi122.stdout:1928: finishing write tid 1 to smithi12223906-18 2024-09-15T03:54:03.805 INFO:tasks.rados.rados.0.smithi122.stdout:1928: finishing write tid 2 to smithi12223906-18 2024-09-15T03:54:03.806 INFO:tasks.rados.rados.0.smithi122.stdout:1928: finishing write tid 3 to smithi12223906-18 2024-09-15T03:54:03.806 INFO:tasks.rados.rados.0.smithi122.stdout:1930: done (2 left) 2024-09-15T03:54:03.806 INFO:tasks.rados.rados.0.smithi122.stdout:1931: rollback oid 20 current snap is 214 2024-09-15T03:54:03.806 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 201 2024-09-15T03:54:03.806 INFO:tasks.rados.rados.0.smithi122.stdout:1928: finishing write tid 4 to smithi12223906-18 2024-09-15T03:54:03.806 INFO:tasks.rados.rados.0.smithi122.stdout:1932: rollback oid 44 current snap is 214 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 208 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:1933: snap_create 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:1928: finishing write tid 5 to smithi12223906-18 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1038 (ObjNum 698 snap 214 seq_num 698) dirty exists 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:1928: left oid 18 (ObjNum 698 snap 214 seq_num 698) 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:1931: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:54:03.807 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 0 (ObjNum 289 snap 0 seq_num 1843085536) dirty dne 2024-09-15T03:54:03.809 INFO:tasks.rados.rados.0.smithi122.stdout:1929: finishing copy_from racing read to smithi12223906-49 2024-09-15T03:54:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1932: finishing rollback tid 0 to smithi12223906-44 2024-09-15T03:54:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1932: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:54:03.810 INFO:tasks.rados.rados.0.smithi122.stdout:1932: finishing rollback tid 2 to smithi12223906-44 2024-09-15T03:54:03.811 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1046 (ObjNum 636 snap 195 seq_num 636) dirty exists 2024-09-15T03:54:03.812 INFO:tasks.rados.rados.0.smithi122.stdout:1929: finishing copy_from to smithi12223906-49 2024-09-15T03:54:03.812 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 778 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:54:05.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:54:06.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient: get_monmap_and_config 2024-09-15T03:54:06.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient: build_initial_monmap 2024-09-15T03:54:06.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient: monmap: 2024-09-15T03:54:06.020 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:06.020 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:06.020 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:06.017660+0000 2024-09-15T03:54:06.020 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:06.017660+0000 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:06.021 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:06.022 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 1 Processor -- start 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 1 -- start start 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient: init 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient: _add_conns ranks=[1,6,8,3,4,0,5,2,7] 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient(hunting): picked mon.noname-f con 0x7f1a4009f590 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:06.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient(hunting): picked mon.noname-d con 0x7f1a401331b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient(hunting): picked mon.noname-e con 0x7f1a40139730 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4004ca10 con 0x7f1a4009f590 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a40096720 con 0x7f1a401331b0 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4009f960 con 0x7f1a40139730 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a4576a700 10 monclient(hunting): authenticate will time out at 1390.447566s 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a3e7fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1a401331b0 0x7f1a40138010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:44476/0 (socket says 172.21.15.26:44476) 2024-09-15T03:54:06.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:06.014+0000 7f1a3e7fc700 1 -- 172.21.15.26:0/4144302501 learned_addr learned my addr 172.21.15.26:0/4144302501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient(hunting): tick 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 monclient(hunting): continuing hunt 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 -- 172.21.15.26:0/4144302501 >> v1:172.21.15.26:6793/0 conn(0x7f1a40139730 legacy=0x7f1a4013bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 -- 172.21.15.26:0/4144302501 >> v1:172.21.15.26:6792/0 conn(0x7f1a401331b0 legacy=0x7f1a40138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 -- 172.21.15.26:0/4144302501 >> v1:172.21.15.122:6789/0 conn(0x7f1a4009f590 legacy=0x7f1a40098bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient: _add_conns ranks=[5,3,4,2,7,0] 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient(hunting): picked mon.noname-h con 0x7f1a20003830 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient(hunting): picked mon.noname-g con 0x7f1a200074b0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient(hunting): picked mon.noname-c con 0x7f1a2000b230 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 -- 172.21.15.26:0/4144302501 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4006e070 con 0x7f1a200074b0 2024-09-15T03:54:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 -- 172.21.15.26:0/4144302501 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a20010030 con 0x7f1a2000b230 2024-09-15T03:54:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 1 -- 172.21.15.26:0/4144302501 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a20011260 con 0x7f1a20003830 2024-09-15T03:54:09.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.014+0000 7f1a3cff9700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785180151 0 0) 0x7f1a20010030 con 0x7f1a2000b230 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient(hunting): my global_id is 184360 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a2c003570 con 0x7f1a2000b230 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3117939380 0 0) 0x7f1a2c003570 con 0x7f1a2000b230 2024-09-15T03:54:09.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 >> v1:172.21.15.122:6791/0 conn(0x7f1a20003830 legacy=0x7f1a20005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 >> v1:172.21.15.122:6790/0 conn(0x7f1a200074b0 legacy=0x7f1a20009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 monclient: found mon.noname-c 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3f7fe700 1 -- 172.21.15.26:0/4144302501 reap_dead start 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a20012550 con 0x7f1a2000b230 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: _finish_auth 0 2024-09-15T03:54:09.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a38001bb0 con 0x7f1a2000b230 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 5 monclient: authenticate success, global_id 184360 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: dump: 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.618 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:09.619 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:09.620 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a38002f70 con 0x7f1a2000b230 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 1 -- 172.21.15.26:0/4144302501 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a38004d50 con 0x7f1a2000b230 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a3dffb700 10 monclient: dump: 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:09.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 4 set_mon_vals no callback set 2024-09-15T03:54:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 10 monclient: shutdown 2024-09-15T03:54:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 1 -- 172.21.15.26:0/4144302501 >> v1:172.21.15.26:6791/0 conn(0x7f1a2000b230 legacy=0x7f1a2000d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 1 -- 172.21.15.26:0/4144302501 shutdown_connections 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 1 -- 172.21.15.26:0/4144302501 >> 172.21.15.26:0/4144302501 conn(0x7f1a400a2e20 msgr2=0x7f1a4009ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 1 -- 172.21.15.26:0/4144302501 shutdown_connections 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.610+0000 7f1a4576a700 1 -- 172.21.15.26:0/4144302501 wait complete. 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient: build_initial_monmap 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient: monmap: 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 1 Processor -- start 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 1 -- start start 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient: init 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient: _add_conns ranks=[8,6,7,0,1,3,5,2,4] 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient(hunting): picked mon.noname-e con 0x7f1a2000b230 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient(hunting): picked mon.noname-d con 0x7f1a401383e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient(hunting): picked mon.noname-i con 0x7f1a401387b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a40135600 con 0x7f1a401383e0 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4009c660 con 0x7f1a401387b0 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4006e2a0 con 0x7f1a2000b230 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a4576a700 10 monclient(hunting): authenticate will time out at 1394.045988s 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a3effd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f1a2000b230 0x7f1a40141fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51610/0 (socket says 172.21.15.26:51610) 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a3effd700 1 -- 172.21.15.26:0/255097921 learned_addr learned my addr 172.21.15.26:0/255097921 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.614+0000 7f1a3e7fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1a401383e0 0x7f1a40147320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60220/0 (socket says 172.21.15.26:60220) 2024-09-15T03:54:09.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690166313 0 0) 0x7f1a40135600 con 0x7f1a401383e0 2024-09-15T03:54:09.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): my global_id is 194109 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a2000cef0 con 0x7f1a401383e0 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310224237 0 0) 0x7f1a4009c660 con 0x7f1a401387b0 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): my global_id is 134764 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a40135600 con 0x7f1a401387b0 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218367848 0 0) 0x7f1a4006e2a0 con 0x7f1a2000b230 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): my global_id is 134819 2024-09-15T03:54:09.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a4009c660 con 0x7f1a2000b230 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3646828523 0 0) 0x7f1a2000cef0 con 0x7f1a401383e0 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a4006e2a0 con 0x7f1a401383e0 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a28001950 con 0x7f1a401383e0 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): dump: 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:09.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2050494535 0 0) 0x7f1a4006e2a0 con 0x7f1a401383e0 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 >> v1:172.21.15.26:6793/0 conn(0x7f1a2000b230 legacy=0x7f1a40141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 >> v1:172.21.15.122:6792/0 conn(0x7f1a401387b0 legacy=0x7f1a4014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 monclient: found mon.d 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a401358e0 con 0x7f1a401383e0 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: _finish_auth 0 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:09.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 5 monclient: authenticate success, global_id 194109 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 10 monclient: _renew_subs 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a40135bc0 con 0x7f1a401383e0 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 10 monclient: _renew_subs 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a40142d60 con 0x7f1a401383e0 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a280048a0 con 0x7f1a401383e0 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a28004dd0 con 0x7f1a401383e0 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a3d7fa700 4 set_mon_vals no callback set 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.638+0000 7f1a36ffd700 10 monclient: dump: 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:09.651 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:09.652 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:09.653 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:09.653 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:09.653 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:09.653 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a4576a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1a28004f30 con 0x7f1a401383e0 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a4576a700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a4576a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a240050b0 con 0x7f1a401383e0 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(435..435 src has 224..435) v4 ==== 3971+0+0 (unknown 478903487 0 0) 0x7f1a28043ce0 con 0x7f1a401383e0 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.642+0000 7f1a3effd700 10 monclient: get_auth_request con 0x7f1a20038c40 auth_method 0 2024-09-15T03:54:09.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.646+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f1a28014b30 con 0x7f1a401383e0 2024-09-15T03:54:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.646+0000 7f1a36ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.646+0000 7f1a36ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a4576a700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:54:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a4576a700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:09.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a4576a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:09.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1a240058e0 con 0x7f1a401383e0 2024-09-15T03:54:09.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a36ffd700 1 -- 172.21.15.26:0/255097921 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 3800395692) 0x7f1a28013420 con 0x7f1a401383e0 2024-09-15T03:54:09.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a36ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:09.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.874+0000 7f1a36ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:09.880 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:54:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 >> v1:172.21.15.122:6812/20895 conn(0x7f1a20038c40 legacy=0x7f1a2003b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 10 monclient: shutdown 2024-09-15T03:54:09.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 >> v1:172.21.15.26:6792/0 conn(0x7f1a401383e0 legacy=0x7f1a40147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 shutdown_connections 2024-09-15T03:54:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 >> 172.21.15.26:0/255097921 conn(0x7f1a400a2e20 msgr2=0x7f1a40132150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 shutdown_connections 2024-09-15T03:54:09.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.878+0000 7f1a4576a700 1 -- 172.21.15.26:0/255097921 wait complete. 2024-09-15T03:54:09.895 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:54:09.895 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:54:09.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:54:10.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.994+0000 7fdfd6bc2700 10 monclient: get_monmap_and_config 2024-09-15T03:54:10.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.994+0000 7fdfd6bc2700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.994+0000 7fdfd6bc2700 10 monclient: monmap: 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:09.998444+0000 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:09.998444+0000 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.002 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.002 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.994+0000 7fdfd6bc2700 1 Processor -- start 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.994+0000 7fdfd6bc2700 1 -- start start 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.994+0000 7fdfd6bc2700 10 monclient: init 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient: _add_conns ranks=[4,5,2,7,1,6,3,0,8] 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient(hunting): picked mon.noname-c con 0x7fdfd00a4b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient(hunting): picked mon.noname-h con 0x7fdfd006b5c0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient(hunting): picked mon.noname-b con 0x7fdfd0139410 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd004ca10 con 0x7fdfd0139410 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd0068dc0 con 0x7fdfd00a4b40 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd00a06e0 con 0x7fdfd006b5c0 2024-09-15T03:54:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient(hunting): authenticate will time out at 1394.428427s 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd495e700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fdfd00a4b40 0x7fdfd009aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55114/0 (socket says 172.21.15.26:55114) 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd495e700 1 -- 172.21.15.26:0/1906548798 learned_addr learned my addr 172.21.15.26:0/1906548798 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd515f700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fdfd0139410 0x7fdfd013b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33538/0 (socket says 172.21.15.26:33538) 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257076759 0 0) 0x7fdfd0068dc0 con 0x7fdfd00a4b40 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): my global_id is 184369 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb8003570 con 0x7fdfd00a4b40 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282530662 0 0) 0x7fdfd004ca10 con 0x7fdfd0139410 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): my global_id is 134876 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd0068dc0 con 0x7fdfd0139410 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360230851 0 0) 0x7fdfd00a06e0 con 0x7fdfd006b5c0 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient(hunting): my global_id is 5387 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd004ca10 con 0x7fdfd006b5c0 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2393904621 0 0) 0x7fdfb8003570 con 0x7fdfd00a4b40 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 >> v1:172.21.15.122:6791/0 conn(0x7fdfd006b5c0 legacy=0x7fdfd0137cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 >> v1:172.21.15.26:6790/0 conn(0x7fdfd0139410 legacy=0x7fdfd013b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 monclient: found mon.noname-c 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd00a4f10 con 0x7fdfd00a4b40 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 5 monclient: authenticate success, global_id 184369 2024-09-15T03:54:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdfcc0017b0 con 0x7fdfd00a4b40 2024-09-15T03:54:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: dump: 2024-09-15T03:54:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfcc0019a0 con 0x7fdfd00a4b40 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 1 -- 172.21.15.26:0/1906548798 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdfcc0048f0 con 0x7fdfd00a4b40 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 4 set_mon_vals no callback set 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfc77fe700 10 monclient: dump: 2024-09-15T03:54:10.012 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient: shutdown 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1906548798 >> v1:172.21.15.26:6791/0 conn(0x7fdfd00a4b40 legacy=0x7fdfd009aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1906548798 shutdown_connections 2024-09-15T03:54:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1906548798 >> 172.21.15.26:0/1906548798 conn(0x7fdfd0063a10 msgr2=0x7fdfd00a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1906548798 shutdown_connections 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1906548798 wait complete. 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:09.998+0000 7fdfd6bc2700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: monmap: 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.017 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 Processor -- start 2024-09-15T03:54:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 -- start start 2024-09-15T03:54:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: init 2024-09-15T03:54:10.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: _add_conns ranks=[6,8,7,3,4,1,0,5,2] 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient(hunting): picked mon.noname-d con 0x7fdfd00a4b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient(hunting): picked mon.noname-e con 0x7fdfd0139410 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient(hunting): picked mon.noname-i con 0x7fdfd0141fb0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd006e120 con 0x7fdfd00a4b40 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd01429c0 con 0x7fdfd0141fb0 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd0098d60 con 0x7fdfd0139410 2024-09-15T03:54:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient(hunting): authenticate will time out at 1394.432997s 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd495e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fdfd00a4b40 0x7fdfd009dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60230/0 (socket says 172.21.15.26:60230) 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd495e700 1 -- 172.21.15.26:0/1130708885 learned_addr learned my addr 172.21.15.26:0/1130708885 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc7fff700 1 --1- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6793/0 conn(0x7fdfd0139410 0x7fdfd0146e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51616/0 (socket says 172.21.15.26:51616) 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295125015 0 0) 0x7fdfd0098d60 con 0x7fdfd0139410 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): my global_id is 134828 2024-09-15T03:54:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfbc003570 con 0x7fdfd0139410 2024-09-15T03:54:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2701080472 0 0) 0x7fdfd006e120 con 0x7fdfd00a4b40 2024-09-15T03:54:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): my global_id is 194118 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd0098d60 con 0x7fdfd00a4b40 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 473575907 0 0) 0x7fdfbc003570 con 0x7fdfd0139410 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfd006e120 con 0x7fdfd0139410 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdfc0001a60 con 0x7fdfd0139410 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): dump: 2024-09-15T03:54:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3334735833 0 0) 0x7fdfd006e120 con 0x7fdfd0139410 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.122:6792/0 conn(0x7fdfd0141fb0 legacy=0x7fdfd014a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6792/0 conn(0x7fdfd00a4b40 legacy=0x7fdfd009dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 monclient: found mon.e 2024-09-15T03:54:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd0099420 con 0x7fdfd0139410 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 5 monclient: authenticate success, global_id 134828 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: _renew_subs 2024-09-15T03:54:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd006e5e0 con 0x7fdfd0139410 2024-09-15T03:54:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: _renew_subs 2024-09-15T03:54:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd009e380 con 0x7fdfd0139410 2024-09-15T03:54:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdfc0002040 con 0x7fdfd0139410 2024-09-15T03:54:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc6ffd700 4 set_mon_vals no callback set 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdfc00024f0 con 0x7fdfd0139410 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.002+0000 7fdfc4ff9700 10 monclient: dump: 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdfc00026d0 con 0x7fdfd0139410 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfd6bc2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdfb0006590 con 0x7fdfb0002840 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(435..435 src has 224..435) v4 ==== 3971+0+0 (unknown 478903487 0 0) 0x7fdfc0043f80 con 0x7fdfd0139410 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547175235 0 0) 0x7fdfb0006590 con 0x7fdfb0002840 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient(hunting): my global_id is 194103 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd006e120 con 0x7fdfb0002840 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfd515f700 10 monclient: get_auth_request con 0x7fdfbc037460 auth_method 0 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3990862951 0 0) 0x7fdfd006e120 con 0x7fdfb0002840 2024-09-15T03:54:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfb0006590 con 0x7fdfb0002840 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdfcc004f00 con 0x7fdfb0002840 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient: dump: 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2270245140 0 0) 0x7fdfb0006590 con 0x7fdfb0002840 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfb000a130 con 0x7fdfb0002840 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdfcc0072f0 con 0x7fdfb0002840 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6789/0 conn(0x7fdfb0002840 legacy=0x7fdfb0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.006+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6789/0 conn(0x7fdfb0002840 legacy=0x7fdfb0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfd6bc2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdfb0007520 con 0x7fdfb00068d0 2024-09-15T03:54:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301935258 0 0) 0x7fdfb0007520 con 0x7fdfb00068d0 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfc4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfc4ff9700 10 monclient(hunting): my global_id is 194112 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.014+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb0006590 con 0x7fdfb00068d0 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4164916592 0 0) 0x7fdfb0006590 con 0x7fdfb00068d0 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfb0007520 con 0x7fdfb00068d0 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdfcc005a90 con 0x7fdfb00068d0 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 10 monclient: dump: 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 890444943 0 0) 0x7fdfb0007520 con 0x7fdfb00068d0 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdfb0008760 con 0x7fdfb00068d0 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 228798226) 0x7fdfcc00a2e0 con 0x7fdfb00068d0 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6789/0 conn(0x7fdfb00068d0 legacy=0x7fdfb0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfc4ff9700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6789/0 conn(0x7fdfb00068d0 legacy=0x7fdfb0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:10.044 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.045 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.046 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:10.047 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.048 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.049 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.052 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:10.053 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.054 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.055 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:10.057 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.058 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.059 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.060 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.060 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:10.060 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.060 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:10.061 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.062 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.062 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.062 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:10.062 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.064 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.065 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:10.066 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.122:6812/20895 conn(0x7fdfbc037460 legacy=0x7fdfbc039910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 10 monclient: shutdown 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 >> v1:172.21.15.26:6793/0 conn(0x7fdfd0139410 legacy=0x7fdfd0146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd515f700 1 -- 172.21.15.26:0/1130708885 reap_dead start 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd515f700 1 -- 172.21.15.26:0/1130708885 reap_dead start 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 shutdown_connections 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 >> 172.21.15.26:0/1130708885 conn(0x7fdfd0063a10 msgr2=0x7fdfd006c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 shutdown_connections 2024-09-15T03:54:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.018+0000 7fdfd6bc2700 1 -- 172.21.15.26:0/1130708885 wait complete. 2024-09-15T03:54:10.070 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:10.070 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:54:10.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:54:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.178+0000 7f9620ad1700 10 monclient: get_monmap_and_config 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.178+0000 7f9620ad1700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.178+0000 7f9620ad1700 10 monclient: monmap: 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:10.183276+0000 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:10.183276+0000 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.190 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.191 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.192 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.192 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 Processor -- start 2024-09-15T03:54:10.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- start start 2024-09-15T03:54:10.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient: init 2024-09-15T03:54:10.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient: _add_conns ranks=[1,4,2,6,7,8,5,3,0] 2024-09-15T03:54:10.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient(hunting): picked mon.noname-f con 0x7f961c132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient(hunting): picked mon.noname-c con 0x7f961c09faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient(hunting): picked mon.noname-b con 0x7f961c138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c04ca10 con 0x7f961c132b40 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c09d5d0 con 0x7f961c138d40 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c133620 con 0x7f961c09faa0 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient(hunting): authenticate will time out at 1394.612631s 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961affd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f961c09faa0 0x7f961c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55116/0 (socket says 172.21.15.26:55116) 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961bfff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f961c138d40 0x7f961c13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33546/0 (socket says 172.21.15.26:33546) 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961affd700 1 -- 172.21.15.26:0/1659361605 learned_addr learned my addr 172.21.15.26:0/1659361605 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877285335 0 0) 0x7f961c133620 con 0x7f961c09faa0 2024-09-15T03:54:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): my global_id is 184378 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9604003570 con 0x7f961c09faa0 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356753266 0 0) 0x7f961c09d5d0 con 0x7f961c138d40 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): my global_id is 134885 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c133620 con 0x7f961c138d40 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566070221 0 0) 0x7f961c04ca10 con 0x7f961c132b40 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient(hunting): my global_id is 184330 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c09d5d0 con 0x7f961c132b40 2024-09-15T03:54:10.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326159721 0 0) 0x7f9604003570 con 0x7f961c09faa0 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 >> v1:172.21.15.26:6790/0 conn(0x7f961c138d40 legacy=0x7f961c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 >> v1:172.21.15.122:6789/0 conn(0x7f961c132b40 legacy=0x7f961c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 monclient: found mon.noname-c 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c0a0cc0 con 0x7f961c09faa0 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f960c0017b0 con 0x7f961c09faa0 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:10.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: dump: 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.197 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 5 monclient: authenticate success, global_id 184378 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f960c0019a0 con 0x7f961c09faa0 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 1 -- 172.21.15.26:0/1659361605 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f960c0048f0 con 0x7f961c09faa0 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f961a7fc700 10 monclient: dump: 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 4 set_mon_vals no callback set 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 10 monclient: shutdown 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- 172.21.15.26:0/1659361605 >> v1:172.21.15.26:6791/0 conn(0x7f961c09faa0 legacy=0x7f961c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- 172.21.15.26:0/1659361605 shutdown_connections 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- 172.21.15.26:0/1659361605 >> 172.21.15.26:0/1659361605 conn(0x7f961c06e370 msgr2=0x7f961c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- 172.21.15.26:0/1659361605 shutdown_connections 2024-09-15T03:54:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.182+0000 7f9620ad1700 1 -- 172.21.15.26:0/1659361605 wait complete. 2024-09-15T03:54:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient: monmap: 2024-09-15T03:54:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 1 Processor -- start 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 1 -- start start 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient: init 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient: _add_conns ranks=[1,8,0,5,6,2,4,3,7] 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient(hunting): picked mon.noname-f con 0x7f961c09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient(hunting): picked mon.noname-e con 0x7f961c132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient(hunting): picked mon.noname-a con 0x7f961c138d40 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c137bd0 con 0x7f961c138d40 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c1378f0 con 0x7f961c09faa0 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c135dc0 con 0x7f961c132b40 2024-09-15T03:54:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f9620ad1700 10 monclient(hunting): authenticate will time out at 1394.619148s 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f961bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f961c138d40 0x7f961c14a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39742/0 (socket says 172.21.15.26:39742) 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f961affd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f961c132b40 0x7f961c134940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51628/0 (socket says 172.21.15.26:51628) 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f961bfff700 1 -- 172.21.15.26:0/4117319921 learned_addr learned my addr 172.21.15.26:0/4117319921 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372527643 0 0) 0x7f961c137bd0 con 0x7f961c138d40 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): my global_id is 194121 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9608003570 con 0x7f961c138d40 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694353110 0 0) 0x7f961c135dc0 con 0x7f961c132b40 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): my global_id is 134837 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c137bd0 con 0x7f961c132b40 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172581317 0 0) 0x7f961c1378f0 con 0x7f961c09faa0 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 10 monclient(hunting): my global_id is 184339 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.186+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c135dc0 con 0x7f961c09faa0 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2841366478 0 0) 0x7f9608003570 con 0x7f961c138d40 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f961c1378f0 con 0x7f961c138d40 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9614001d90 con 0x7f961c138d40 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): dump: 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.210 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 590978170 0 0) 0x7f961c137bd0 con 0x7f961c132b40 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9608003570 con 0x7f961c132b40 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f960c001770 con 0x7f961c132b40 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): dump: 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1245895230 0 0) 0x7f961c1378f0 con 0x7f961c138d40 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.26:6793/0 conn(0x7f961c132b40 legacy=0x7f961c134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.122:6789/0 conn(0x7f961c09faa0 legacy=0x7f961c134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 monclient: found mon.a 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c136990 con 0x7f961c138d40 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 5 monclient: authenticate success, global_id 194121 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 10 monclient: _renew_subs 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c135050 con 0x7f961c138d40 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 10 monclient: _renew_subs 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:10.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1385b0 con 0x7f961c138d40 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9614004e60 con 0x7f961c138d40 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9614005310 con 0x7f961c138d40 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9619ffb700 4 set_mon_vals no callback set 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: dump: 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.214 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.215 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f95fc006590 con 0x7f95fc002840 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f961400fcd0 con 0x7f961c138d40 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(435..435 src has 224..435) v4 ==== 3971+0+0 (unknown 478903487 0 0) 0x7f96140441a0 con 0x7f961c138d40 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599270737 0 0) 0x7f95fc006590 con 0x7f95fc002840 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient(hunting): my global_id is 134894 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c1378f0 con 0x7f95fc002840 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f961affd700 10 monclient: get_auth_request con 0x7f9608037cf0 auth_method 0 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1872607280 0 0) 0x7f961c1378f0 con 0x7f95fc002840 2024-09-15T03:54:10.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95fc006590 con 0x7f95fc002840 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9610001d50 con 0x7f95fc002840 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: dump: 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.217 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3136185014 0 0) 0x7f95fc006590 con 0x7f95fc002840 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95fc00a130 con 0x7f95fc002840 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9610002360 con 0x7f95fc002840 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.26:6790/0 conn(0x7f95fc002840 legacy=0x7f95fc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.190+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.26:6790/0 conn(0x7f95fc002840 legacy=0x7f95fc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f9620ad1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f95fc009640 con 0x7f95fc0090b0 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958880249 0 0) 0x7f95fc009640 con 0x7f95fc0090b0 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient(hunting): my global_id is 134903 2024-09-15T03:54:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95fc006590 con 0x7f95fc0090b0 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1785888744 0 0) 0x7f95fc006590 con 0x7f95fc0090b0 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95fc009640 con 0x7f95fc0090b0 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9610002b80 con 0x7f95fc0090b0 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient: dump: 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3862039502 0 0) 0x7f95fc009640 con 0x7f95fc0090b0 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95fc0098f0 con 0x7f95fc0090b0 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2703928) 0x7f9610003070 con 0x7f95fc0090b0 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.26:6790/0 conn(0x7f95fc0090b0 legacy=0x7f95fc008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.202+0000 7f96037fe700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.26:6790/0 conn(0x7f95fc0090b0 legacy=0x7f95fc008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:10.223 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:10.224 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.225 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.226 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:10.227 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.228 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:10.229 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.230 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.232 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:10.233 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.234 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.235 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.236 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:10.238 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.239 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.240 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.241 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:10.243 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.122:6812/20895 conn(0x7f9608037cf0 legacy=0x7f960803a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 10 monclient: shutdown 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f961bfff700 1 -- 172.21.15.26:0/4117319921 reap_dead start 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 >> v1:172.21.15.26:6789/0 conn(0x7f961c138d40 legacy=0x7f961c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 shutdown_connections 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 >> 172.21.15.26:0/4117319921 conn(0x7f961c06e370 msgr2=0x7f961c12eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 shutdown_connections 2024-09-15T03:54:10.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.206+0000 7f9620ad1700 1 -- 172.21.15.26:0/4117319921 wait complete. 2024-09-15T03:54:10.246 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:10.246 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:54:10.246 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.358+0000 7f6c19146700 10 monclient: get_monmap_and_config 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.358+0000 7f6c19146700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.358+0000 7f6c19146700 10 monclient: monmap: 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:10.363012+0000 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:10.363012+0000 2024-09-15T03:54:10.365 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.366 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.358+0000 7f6c19146700 1 Processor -- start 2024-09-15T03:54:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.358+0000 7f6c19146700 1 -- start start 2024-09-15T03:54:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.358+0000 7f6c19146700 10 monclient: init 2024-09-15T03:54:10.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient: _add_conns ranks=[4,0,8,5,2,1,3,6,7] 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient(hunting): picked mon.noname-c con 0x7f6c14132c90 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient(hunting): picked mon.noname-a con 0x7f6c14133060 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient(hunting): picked mon.noname-e con 0x7f6c14133470 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1404ca10 con 0x7f6c14133060 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c14068f90 con 0x7f6c14132c90 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c140a1430 con 0x7f6c14133470 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient(hunting): authenticate will time out at 1394.792411s 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c1259c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c14133060 0x7f6c14137fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39750/0 (socket says 172.21.15.26:39750) 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c1259c700 1 -- 172.21.15.26:0/2233442596 learned_addr learned my addr 172.21.15.26:0/2233442596 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323692075 0 0) 0x7f6c1404ca10 con 0x7f6c14133060 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): my global_id is 194130 2024-09-15T03:54:10.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bfc003570 con 0x7f6c14133060 2024-09-15T03:54:10.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408894500 0 0) 0x7f6c14068f90 con 0x7f6c14132c90 2024-09-15T03:54:10.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): my global_id is 184387 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c1404ca10 con 0x7f6c14132c90 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293514050 0 0) 0x7f6c140a1430 con 0x7f6c14133470 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient(hunting): my global_id is 134846 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c14068f90 con 0x7f6c14133470 2024-09-15T03:54:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468269383 0 0) 0x7f6bfc003570 con 0x7f6c14133060 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 >> v1:172.21.15.26:6793/0 conn(0x7f6c14133470 legacy=0x7f6c1413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 >> v1:172.21.15.26:6791/0 conn(0x7f6c14132c90 legacy=0x7f6c1409ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 monclient: found mon.noname-a 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1409da90 con 0x7f6c14133060 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c040017b0 con 0x7f6c14133060 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: dump: 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 5 monclient: authenticate success, global_id 194130 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c04001a90 con 0x7f6c14133060 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 1 -- 172.21.15.26:0/2233442596 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c04001c80 con 0x7f6c14133060 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 4 set_mon_vals no callback set 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c11d9b700 10 monclient: dump: 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.383 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient: shutdown 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- 172.21.15.26:0/2233442596 >> v1:172.21.15.26:6789/0 conn(0x7f6c14133060 legacy=0x7f6c14137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- 172.21.15.26:0/2233442596 shutdown_connections 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- 172.21.15.26:0/2233442596 >> 172.21.15.26:0/2233442596 conn(0x7f6c1406e510 msgr2=0x7f6c140a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- 172.21.15.26:0/2233442596 shutdown_connections 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 1 -- 172.21.15.26:0/2233442596 wait complete. 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.362+0000 7f6c19146700 10 monclient: monmap: 2024-09-15T03:54:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 Processor -- start 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 -- start start 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: init 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: _add_conns ranks=[1,2,6,4,8,7,5,3,0] 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient(hunting): picked mon.noname-f con 0x7f6c14132c90 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient(hunting): picked mon.noname-b con 0x7f6c14133060 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient(hunting): picked mon.noname-d con 0x7f6c14133470 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c14134200 con 0x7f6c14132c90 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1409d120 con 0x7f6c14133060 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c140a1e90 con 0x7f6c14133470 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient(hunting): authenticate will time out at 1394.796884s 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c1359e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f6c14133470 0x7f6c1414a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60244/0 (socket says 172.21.15.26:60244) 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c1359e700 1 -- 172.21.15.26:0/140561769 learned_addr learned my addr 172.21.15.26:0/140561769 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619563876 0 0) 0x7f6c140a1e90 con 0x7f6c14133470 2024-09-15T03:54:10.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): my global_id is 194127 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c00003570 con 0x7f6c14133470 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250851190 0 0) 0x7f6c1409d120 con 0x7f6c14133060 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): my global_id is 134912 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c140a1e90 con 0x7f6c14133060 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 46363757 0 0) 0x7f6c00003570 con 0x7f6c14133470 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c1409d120 con 0x7f6c14133470 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c0c002dd0 con 0x7f6c14133470 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:10.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): dump: 2024-09-15T03:54:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 313903186 0 0) 0x7f6c140a1e90 con 0x7f6c14133060 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c00003570 con 0x7f6c14133060 2024-09-15T03:54:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c04001770 con 0x7f6c14133060 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient(hunting): dump: 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3129436958 0 0) 0x7f6c00003570 con 0x7f6c14133060 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.26:6792/0 conn(0x7f6c14133470 legacy=0x7f6c1414a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.122:6789/0 conn(0x7f6c14132c90 legacy=0x7f6c1412fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 monclient: found mon.b 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c14120390 con 0x7f6c14133060 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 5 monclient: authenticate success, global_id 134912 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: _renew_subs 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1411e330 con 0x7f6c14133060 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: _renew_subs 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c14142b70 con 0x7f6c14133060 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6c04001c70 con 0x7f6c14133060 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6c10d99700 4 set_mon_vals no callback set 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c04001100 con 0x7f6c14133060 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 10 monclient: dump: 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.366+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6c0400fb20 con 0x7f6c14133060 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6c19146700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6bf4006590 con 0x7f6bf4002840 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(435..435 src has 224..435) v4 ==== 3971+0+0 (unknown 478903487 0 0) 0x7f6c04043fa0 con 0x7f6c14133060 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920282029 0 0) 0x7f6bf4006590 con 0x7f6bf4002840 2024-09-15T03:54:10.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient(hunting): my global_id is 184396 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c00003570 con 0x7f6bf4002840 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6c1359e700 10 monclient: get_auth_request con 0x7f6c00037b40 auth_method 0 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4004836717 0 0) 0x7f6c00003570 con 0x7f6bf4002840 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6bf4006590 con 0x7f6bf4002840 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c080018e0 con 0x7f6bf4002840 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient: dump: 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.398 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1973158704 0 0) 0x7f6bf4006590 con 0x7f6bf4002840 2024-09-15T03:54:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bf400a130 con 0x7f6bf4002840 2024-09-15T03:54:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6c08001c00 con 0x7f6bf4002840 2024-09-15T03:54:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.26:6791/0 conn(0x7f6bf4002840 legacy=0x7f6bf4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.370+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.26:6791/0 conn(0x7f6bf4002840 legacy=0x7f6bf4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:10.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6bf4009640 con 0x7f6bf40090b0 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556293125 0 0) 0x7f6bf4009640 con 0x7f6bf40090b0 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient(hunting): my global_id is 184405 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf4006590 con 0x7f6bf40090b0 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3339727079 0 0) 0x7f6bf4006590 con 0x7f6bf40090b0 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6bf4009640 con 0x7f6bf40090b0 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6c08002460 con 0x7f6bf40090b0 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient: dump: 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.402 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.403 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 779558899 0 0) 0x7f6bf4009640 con 0x7f6bf40090b0 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6bf40098f0 con 0x7f6bf40090b0 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1381184508) 0x7f6c080027f0 con 0x7f6bf40090b0 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.26:6791/0 conn(0x7f6bf40090b0 legacy=0x7f6bf4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6bfaffd700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.26:6791/0 conn(0x7f6bf40090b0 legacy=0x7f6bf4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:10.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:10.405 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.406 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.407 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:10.408 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.409 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.412 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:10.413 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:10.417 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.418 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.419 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.420 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.422 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.423 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:10.424 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.122:6812/20895 conn(0x7f6c00037b40 legacy=0x7f6c00039ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 10 monclient: shutdown 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 >> v1:172.21.15.26:6790/0 conn(0x7f6c14133060 legacy=0x7f6c14147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c1359e700 1 -- 172.21.15.26:0/140561769 reap_dead start 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c1359e700 1 -- 172.21.15.26:0/140561769 reap_dead start 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 shutdown_connections 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 >> 172.21.15.26:0/140561769 conn(0x7f6c1406e510 msgr2=0x7f6c140a4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 shutdown_connections 2024-09-15T03:54:10.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.378+0000 7f6c19146700 1 -- 172.21.15.26:0/140561769 wait complete. 2024-09-15T03:54:10.427 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:10.427 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:54:10.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:54:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: get_monmap_and_config 2024-09-15T03:54:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: monmap: 2024-09-15T03:54:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:10.543967+0000 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:10.543967+0000 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 1 Processor -- start 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 1 -- start start 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: init 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: _add_conns ranks=[4,0,7,8,1,2,5,6,3] 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient(hunting): picked mon.noname-c con 0x7fd378132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient(hunting): picked mon.noname-a con 0x7fd37809faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient(hunting): picked mon.noname-i con 0x7fd378138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37804ca10 con 0x7fd37809faa0 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37809d5d0 con 0x7fd378132b40 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378133620 con 0x7fd378138d40 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient(hunting): authenticate will time out at 1394.973711s 2024-09-15T03:54:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37e4e5700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd378132b40 0x7fd378132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55148/0 (socket says 172.21.15.26:55148) 2024-09-15T03:54:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37dce4700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd37809faa0 0x7fd37809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39754/0 (socket says 172.21.15.26:39754) 2024-09-15T03:54:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37e4e5700 1 -- 172.21.15.26:0/3614528015 learned_addr learned my addr 172.21.15.26:0/3614528015 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539406814 0 0) 0x7fd37804ca10 con 0x7fd37809faa0 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): my global_id is 194139 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd364003570 con 0x7fd37809faa0 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568414948 0 0) 0x7fd378133620 con 0x7fd378138d40 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): my global_id is 134782 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37804ca10 con 0x7fd378138d40 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069793382 0 0) 0x7fd37809d5d0 con 0x7fd378132b40 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient(hunting): my global_id is 184414 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378133620 con 0x7fd378132b40 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365674667 0 0) 0x7fd37804ca10 con 0x7fd378138d40 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 >> v1:172.21.15.26:6791/0 conn(0x7fd378132b40 legacy=0x7fd378132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 >> v1:172.21.15.26:6789/0 conn(0x7fd37809faa0 legacy=0x7fd37809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 monclient: found mon.noname-i 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780a0cc0 con 0x7fd378138d40 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd374001790 con 0x7fd378138d40 2024-09-15T03:54:10.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: dump: 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.554 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 5 monclient: authenticate success, global_id 134782 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd374001a40 con 0x7fd378138d40 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 1 -- 172.21.15.26:0/3614528015 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd374001f70 con 0x7fd378138d40 2024-09-15T03:54:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37d4e3700 10 monclient: dump: 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.542+0000 7fd37f4e7700 4 set_mon_vals no callback set 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient: shutdown 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- 172.21.15.26:0/3614528015 >> v1:172.21.15.122:6792/0 conn(0x7fd378138d40 legacy=0x7fd37813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- 172.21.15.26:0/3614528015 shutdown_connections 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- 172.21.15.26:0/3614528015 >> 172.21.15.26:0/3614528015 conn(0x7fd37806e370 msgr2=0x7fd3780a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- 172.21.15.26:0/3614528015 shutdown_connections 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- 172.21.15.26:0/3614528015 wait complete. 2024-09-15T03:54:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient: monmap: 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.559 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.559 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 Processor -- start 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- start start 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient: init 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient: _add_conns ranks=[7,6,5,3,4,2,8,1,0] 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient(hunting): picked mon.noname-i con 0x7fd37809faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient(hunting): picked mon.noname-d con 0x7fd378132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient(hunting): picked mon.noname-h con 0x7fd378138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37809ccc0 con 0x7fd378138d40 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd378137700 con 0x7fd378132b40 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37811c6e0 con 0x7fd37809faa0 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.546+0000 7fd37f4e7700 10 monclient(hunting): authenticate will time out at 1394.979969s 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37dce4700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd378132b40 0x7fd378135b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60254/0 (socket says 172.21.15.26:60254) 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37dce4700 1 -- 172.21.15.26:0/145624398 learned_addr learned my addr 172.21.15.26:0/145624398 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739051748 0 0) 0x7fd378137700 con 0x7fd378132b40 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): my global_id is 194136 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003570 con 0x7fd378132b40 2024-09-15T03:54:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886168475 0 0) 0x7fd37811c6e0 con 0x7fd37809faa0 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): my global_id is 134791 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378137700 con 0x7fd37809faa0 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956512939 0 0) 0x7fd37809ccc0 con 0x7fd378138d40 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): my global_id is 5396 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37811c6e0 con 0x7fd378138d40 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2937363684 0 0) 0x7fd35c003570 con 0x7fd378132b40 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd37809ccc0 con 0x7fd378132b40 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd36c001290 con 0x7fd378132b40 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 683090793 0 0) 0x7fd37809ccc0 con 0x7fd378132b40 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.122:6792/0 conn(0x7fd37809faa0 legacy=0x7fd378135400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.122:6791/0 conn(0x7fd378138d40 legacy=0x7fd378136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 monclient: found mon.d 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378136fb0 con 0x7fd378132b40 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 5 monclient: authenticate success, global_id 194136 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 10 monclient: _renew_subs 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd378137b80 con 0x7fd378132b40 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 10 monclient: _renew_subs 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd378132690 con 0x7fd378132b40 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd36c004ae0 con 0x7fd378132b40 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd36c004f90 con 0x7fd378132b40 2024-09-15T03:54:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: dump: 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37cce2700 4 set_mon_vals no callback set 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd36c00f950 con 0x7fd378132b40 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd360006590 con 0x7fd360002840 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(435..435 src has 224..435) v4 ==== 3971+0+0 (unknown 478903487 0 0) 0x7fd36c043ef0 con 0x7fd378132b40 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097850641 0 0) 0x7fd360006590 con 0x7fd360002840 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient(hunting): my global_id is 194145 2024-09-15T03:54:10.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37809ccc0 con 0x7fd360002840 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd37ece6700 10 monclient: get_auth_request con 0x7fd35c037680 auth_method 0 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1766645774 0 0) 0x7fd37809ccc0 con 0x7fd360002840 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd360006590 con 0x7fd360002840 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd370004ef0 con 0x7fd360002840 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 10 monclient: dump: 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 376010959 0 0) 0x7fd360006590 con 0x7fd360002840 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.550+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd36000a130 con 0x7fd360002840 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.554+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd370007280 con 0x7fd360002840 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.554+0000 7fd36a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.554+0000 7fd36a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:10.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.554+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.26:6792/0 conn(0x7fd360002840 legacy=0x7fd360004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.554+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.26:6792/0 conn(0x7fd360002840 legacy=0x7fd360004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd360009640 con 0x7fd3600090b0 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027220316 0 0) 0x7fd360009640 con 0x7fd3600090b0 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient(hunting): my global_id is 194154 2024-09-15T03:54:10.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd360006590 con 0x7fd3600090b0 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 956380186 0 0) 0x7fd360006590 con 0x7fd3600090b0 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd360009640 con 0x7fd3600090b0 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd370005a90 con 0x7fd3600090b0 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient: dump: 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.575 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 673438943 0 0) 0x7fd360009640 con 0x7fd3600090b0 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd3600098f0 con 0x7fd3600090b0 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 119330132) 0x7fd370006fe0 con 0x7fd3600090b0 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:10.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.26:6792/0 conn(0x7fd3600090b0 legacy=0x7fd360008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd36a7fc700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.26:6792/0 conn(0x7fd3600090b0 legacy=0x7fd360008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:10.577 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.578 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.579 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:10.580 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.581 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.583 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.584 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.587 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.588 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:10.591 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.592 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.593 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.594 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:10.595 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.122:6812/20895 conn(0x7fd35c037680 legacy=0x7fd35c039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 10 monclient: shutdown 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 >> v1:172.21.15.26:6792/0 conn(0x7fd378132b40 legacy=0x7fd378135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37ece6700 1 -- 172.21.15.26:0/145624398 reap_dead start 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37ece6700 1 -- 172.21.15.26:0/145624398 reap_dead start 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 shutdown_connections 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 >> 172.21.15.26:0/145624398 conn(0x7fd37806e370 msgr2=0x7fd3780a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 shutdown_connections 2024-09-15T03:54:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.566+0000 7fd37f4e7700 1 -- 172.21.15.26:0/145624398 wait complete. 2024-09-15T03:54:10.598 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:10.598 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:54:10.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:54:10.656 INFO:tasks.rados.rados.0.smithi122.stdout:1928: done (4 left) 2024-09-15T03:54:10.656 INFO:tasks.rados.rados.0.smithi122.stdout:1929: done (3 left) 2024-09-15T03:54:10.656 INFO:tasks.rados.rados.0.smithi122.stdout:1931: done (2 left) 2024-09-15T03:54:10.656 INFO:tasks.rados.rados.0.smithi122.stdout:1932: done (1 left) 2024-09-15T03:54:10.656 INFO:tasks.rados.rados.0.smithi122.stdout:1933: done (0 left) 2024-09-15T03:54:10.656 INFO:tasks.rados.rados.0.smithi122.stdout:1934: delete oid 29 current snap is 215 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:1934: done (0 left) 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:1935: read oid 46 snap -1 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:1935: expect deleted 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:1936: read oid 3 snap -1 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:1936: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:1937: rollback oid 23 current snap is 215 2024-09-15T03:54:10.660 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 201 2024-09-15T03:54:10.661 INFO:tasks.rados.rados.0.smithi122.stdout:1938: write oid 28 current snap is 215 2024-09-15T03:54:10.661 INFO:tasks.rados.rados.0.smithi122.stdout:1938: seq_num 701 ranges {581707=654025,1687642=453571,2700812=703681} 2024-09-15T03:54:10.669 INFO:tasks.rados.rados.0.smithi122.stdout:1938: writing smithi12223906-28 from 581707 to 1235732 tid 1 2024-09-15T03:54:10.673 INFO:tasks.rados.rados.0.smithi122.stdout:1938: writing smithi12223906-28 from 1687642 to 2141213 tid 2 2024-09-15T03:54:10.681 INFO:tasks.rados.rados.0.smithi122.stdout:1938: writing smithi12223906-28 from 2700812 to 3404493 tid 3 2024-09-15T03:54:10.682 INFO:tasks.rados.rados.0.smithi122.stdout:1939: delete oid 31 current snap is 215 2024-09-15T03:54:10.682 INFO:tasks.rados.rados.0.smithi122.stdout:1937: finishing rollback tid 0 to smithi12223906-23 2024-09-15T03:54:10.682 INFO:tasks.rados.rados.0.smithi122.stdout:1937: finishing rollback tid 1 to smithi12223906-23 2024-09-15T03:54:10.682 INFO:tasks.rados.rados.0.smithi122.stdout:1937: finishing rollback tid 2 to smithi12223906-23 2024-09-15T03:54:10.682 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 1050 (ObjNum 645 snap 199 seq_num 645) dirty exists 2024-09-15T03:54:10.682 INFO:tasks.rados.rados.0.smithi122.stdout:1936: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient: get_monmap_and_config 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient: monmap: 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:10.713166+0000 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:10.713166+0000 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.716 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 1 Processor -- start 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 1 -- start start 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient: init 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient: _add_conns ranks=[1,6,3,4,5,7,0,8,2] 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient(hunting): picked mon.noname-f con 0x7f4404132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient(hunting): picked mon.noname-d con 0x7f440409faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient(hunting): picked mon.noname-g con 0x7f4404138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440404ca10 con 0x7f4404132b40 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440409d5d0 con 0x7f4404138d40 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4404133620 con 0x7f440409faa0 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44094f8700 10 monclient(hunting): authenticate will time out at 1395.142728s 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44037fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f440409faa0 0x7f440409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60292/0 (socket says 172.21.15.26:60292) 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f44037fe700 1 -- 172.21.15.26:0/3973464847 learned_addr learned my addr 172.21.15.26:0/3973464847 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4403fff700 1 --1- 172.21.15.26:0/3973464847 >> v1:172.21.15.122:6789/0 conn(0x7f4404132b40 0x7f4404132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:38670/0 (socket says 172.21.15.26:38670) 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904598340 0 0) 0x7f440404ca10 con 0x7f4404132b40 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): my global_id is 184357 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43ec003570 con 0x7f4404132b40 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196243375 0 0) 0x7f440409d5d0 con 0x7f4404138d40 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): my global_id is 194106 2024-09-15T03:54:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440404ca10 con 0x7f4404138d40 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445011346 0 0) 0x7f4404133620 con 0x7f440409faa0 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient(hunting): my global_id is 194163 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440409d5d0 con 0x7f440409faa0 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1613013769 0 0) 0x7f43ec003570 con 0x7f4404132b40 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 >> v1:172.21.15.26:6792/0 conn(0x7f440409faa0 legacy=0x7f440409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 >> v1:172.21.15.122:6790/0 conn(0x7f4404138d40 legacy=0x7f440413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 monclient: found mon.noname-f 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040a0cc0 con 0x7f4404132b40 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43fc001b30 con 0x7f4404132b40 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.710+0000 7f4402ffd700 10 monclient: dump: 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 5 monclient: authenticate success, global_id 184357 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43fc001da0 con 0x7f4404132b40 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f4402ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 4 set_mon_vals no callback set 2024-09-15T03:54:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f4402ffd700 1 -- 172.21.15.26:0/3973464847 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43fc004ef0 con 0x7f4404132b40 2024-09-15T03:54:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f4402ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f4402ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f4402ffd700 10 monclient: dump: 2024-09-15T03:54:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: shutdown 2024-09-15T03:54:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- 172.21.15.26:0/3973464847 >> v1:172.21.15.122:6789/0 conn(0x7f4404132b40 legacy=0x7f4404132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- 172.21.15.26:0/3973464847 shutdown_connections 2024-09-15T03:54:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- 172.21.15.26:0/3973464847 >> 172.21.15.26:0/3973464847 conn(0x7f440406e370 msgr2=0x7f44040a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- 172.21.15.26:0/3973464847 shutdown_connections 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- 172.21.15.26:0/3973464847 wait complete. 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: monmap: 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.743 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 Processor -- start 2024-09-15T03:54:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- start start 2024-09-15T03:54:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: init 2024-09-15T03:54:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient: _add_conns ranks=[7,2,1,6,3,4,0,8,5] 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient(hunting): picked mon.noname-i con 0x7f440409faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient(hunting): picked mon.noname-b con 0x7f4404132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient(hunting): picked mon.noname-f con 0x7f4404138d40 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440409cfa0 con 0x7f4404138d40 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440409ccc0 con 0x7f4404132b40 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440411cbc0 con 0x7f440409faa0 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44094f8700 10 monclient(hunting): authenticate will time out at 1395.147703s 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44037fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f4404132b40 0x7f4404135ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33588/0 (socket says 172.21.15.26:33588) 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.714+0000 7f44037fe700 1 -- 172.21.15.26:0/4039007737 learned_addr learned my addr 172.21.15.26:0/4039007737 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526887468 0 0) 0x7f440409cfa0 con 0x7f4404138d40 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): my global_id is 184366 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f0003570 con 0x7f4404138d40 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737999838 0 0) 0x7f440409ccc0 con 0x7f4404132b40 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): my global_id is 134921 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440409cfa0 con 0x7f4404132b40 2024-09-15T03:54:10.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894815435 0 0) 0x7f440411cbc0 con 0x7f440409faa0 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): my global_id is 134800 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440409ccc0 con 0x7f440409faa0 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1912680197 0 0) 0x7f43f0003570 con 0x7f4404138d40 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f440411cbc0 con 0x7f4404138d40 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43f4001980 con 0x7f4404138d40 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): dump: 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3115783691 0 0) 0x7f440409cfa0 con 0x7f4404132b40 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43f0003570 con 0x7f4404132b40 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43f8001a20 con 0x7f4404132b40 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): dump: 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3380611606 0 0) 0x7f440411cbc0 con 0x7f4404138d40 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.122:6792/0 conn(0x7f440409faa0 legacy=0x7f4404135790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.26:6790/0 conn(0x7f4404132b40 legacy=0x7f4404135ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 monclient: found mon.f 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44041375d0 con 0x7f4404138d40 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 5 monclient: authenticate success, global_id 184366 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 10 monclient: _renew_subs 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44041370c0 con 0x7f4404138d40 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 10 monclient: _renew_subs 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4404137c90 con 0x7f4404138d40 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f43f4001e50 con 0x7f4404138d40 2024-09-15T03:54:10.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43f4002380 con 0x7f4404138d40 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44027fc700 4 set_mon_vals no callback set 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient: dump: 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.752 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f43e4006590 con 0x7f43e4002840 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f43f400ce00 con 0x7f4404138d40 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(436..436 src has 224..436) v4 ==== 3943+0+0 (unknown 3288886531 0 0) 0x7f43f4041480 con 0x7f4404138d40 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148028691 0 0) 0x7f43e4006590 con 0x7f43e4002840 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 10 monclient(hunting): my global_id is 134855 2024-09-15T03:54:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440411cbc0 con 0x7f43e4002840 2024-09-15T03:54:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.718+0000 7f44037fe700 10 monclient: get_auth_request con 0x7f43f0037cd0 auth_method 0 2024-09-15T03:54:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2962905223 0 0) 0x7f440411cbc0 con 0x7f43e4002840 2024-09-15T03:54:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43e4006590 con 0x7f43e4002840 2024-09-15T03:54:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43fc001b80 con 0x7f43e4002840 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 10 monclient: dump: 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3888986330 0 0) 0x7f43e4006590 con 0x7f43e4002840 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43e400a130 con 0x7f43e4002840 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f43fc0075c0 con 0x7f43e4002840 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.26:6793/0 conn(0x7f43e4002840 legacy=0x7f43e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.722+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.26:6793/0 conn(0x7f43e4002840 legacy=0x7f43e4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:10.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f43e4009640 con 0x7f43e40090b0 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862762013 0 0) 0x7f43e4009640 con 0x7f43e40090b0 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient(hunting): my global_id is 134864 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43e4006590 con 0x7f43e40090b0 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2951849176 0 0) 0x7f43e4006590 con 0x7f43e40090b0 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43e4009640 con 0x7f43e40090b0 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43fc005d70 con 0x7f43e40090b0 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient: dump: 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2421150479 0 0) 0x7f43e4009640 con 0x7f43e40090b0 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43e40098f0 con 0x7f43e40090b0 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 3957776633) 0x7f43fc007730 con 0x7f43e40090b0 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.26:6793/0 conn(0x7f43e40090b0 legacy=0x7f43e4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f43ebfff700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.26:6793/0 conn(0x7f43e40090b0 legacy=0x7f43e4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:10.766 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.767 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.768 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.769 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:10.770 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:10.771 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.772 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.775 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.778 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.781 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:10.782 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:10.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.784 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.786 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.787 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:10.788 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.789 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.789 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:10.789 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:10.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.122:6812/20895 conn(0x7f43f0037cd0 legacy=0x7f43f003a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 10 monclient: shutdown 2024-09-15T03:54:10.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 >> v1:172.21.15.122:6789/0 conn(0x7f4404138d40 legacy=0x7f440414a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f4408cf7700 1 -- 172.21.15.26:0/4039007737 reap_dead start 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f4408cf7700 1 -- 172.21.15.26:0/4039007737 reap_dead start 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 shutdown_connections 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.730+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 >> 172.21.15.26:0/4039007737 conn(0x7f440406e370 msgr2=0x7f44040a4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.734+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 shutdown_connections 2024-09-15T03:54:10.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.734+0000 7f44094f8700 1 -- 172.21.15.26:0/4039007737 wait complete. 2024-09-15T03:54:10.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:10.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:54:10.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:54:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.902+0000 7f37b2eb9700 10 monclient: get_monmap_and_config 2024-09-15T03:54:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.902+0000 7f37b2eb9700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.902+0000 7f37b2eb9700 10 monclient: monmap: 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:10.906482+0000 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:10.906482+0000 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.902+0000 7f37b2eb9700 1 Processor -- start 2024-09-15T03:54:10.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.902+0000 7f37b2eb9700 1 -- start start 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.902+0000 7f37b2eb9700 10 monclient: init 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: _add_conns ranks=[8,7,0,6,1,5,4,2,3] 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient(hunting): picked mon.noname-e con 0x7f37ac1335e0 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient(hunting): picked mon.noname-i con 0x7f37ac09fc80 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient(hunting): picked mon.noname-a con 0x7f37ac0a0050 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac04ca10 con 0x7f37ac0a0050 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06eab0 con 0x7f37ac09fc80 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0a1330 con 0x7f37ac1335e0 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient(hunting): authenticate will time out at 1395.336465s 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b1eb7700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f37ac1335e0 0x7f37ac132da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51670/0 (socket says 172.21.15.26:51670) 2024-09-15T03:54:10.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b1eb7700 1 -- 172.21.15.26:0/647061485 learned_addr learned my addr 172.21.15.26:0/647061485 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b26b8700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37ac0a0050 0x7f37ac13b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39756/0 (socket says 172.21.15.26:39756) 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830050806 0 0) 0x7f37ac0a1330 con 0x7f37ac1335e0 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): my global_id is 134873 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3798003570 con 0x7f37ac1335e0 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683072842 0 0) 0x7f37ac06eab0 con 0x7f37ac09fc80 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): my global_id is 134809 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac0a1330 con 0x7f37ac09fc80 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470795823 0 0) 0x7f37ac04ca10 con 0x7f37ac0a0050 2024-09-15T03:54:10.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient(hunting): my global_id is 194148 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac06eab0 con 0x7f37ac0a0050 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047052471 0 0) 0x7f3798003570 con 0x7f37ac1335e0 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 >> v1:172.21.15.122:6792/0 conn(0x7f37ac09fc80 legacy=0x7f37ac137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 >> v1:172.21.15.26:6789/0 conn(0x7f37ac0a0050 legacy=0x7f37ac13b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 monclient: found mon.noname-e 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0a10d0 con 0x7f37ac1335e0 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a80017b0 con 0x7f37ac1335e0 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: dump: 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.916 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 5 monclient: authenticate success, global_id 134873 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37a8001a20 con 0x7f37ac1335e0 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 1 -- 172.21.15.26:0/647061485 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a8004c30 con 0x7f37ac1335e0 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 4 set_mon_vals no callback set 2024-09-15T03:54:10.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b0eb5700 10 monclient: dump: 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: shutdown 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- 172.21.15.26:0/647061485 >> v1:172.21.15.26:6793/0 conn(0x7f37ac1335e0 legacy=0x7f37ac132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- 172.21.15.26:0/647061485 shutdown_connections 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- 172.21.15.26:0/647061485 >> 172.21.15.26:0/647061485 conn(0x7f37ac06e6e0 msgr2=0x7f37ac0a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- 172.21.15.26:0/647061485 shutdown_connections 2024-09-15T03:54:10.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- 172.21.15.26:0/647061485 wait complete. 2024-09-15T03:54:10.920 INFO:tasks.rados.rados.0.smithi122.stdout:1938: finishing write tid 1 to smithi12223906-28 2024-09-15T03:54:10.920 INFO:tasks.rados.rados.0.smithi122.stdout:1938: finishing write tid 2 to smithi12223906-28 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1938: finishing write tid 3 to smithi12223906-28 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1938: finishing write tid 4 to smithi12223906-28 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1938: finishing write tid 5 to smithi12223906-28 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 813 (ObjNum 701 snap 215 seq_num 701) dirty exists 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1938: left oid 28 (ObjNum 701 snap 215 seq_num 701) 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1935: done (4 left) 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1936: done (3 left) 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1937: done (2 left) 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1938: done (1 left) 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1939: done (0 left) 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1940: write (excl) oid 26 current snap is 215 2024-09-15T03:54:10.921 INFO:tasks.rados.rados.0.smithi122.stdout:1940: seq_num 703 ranges {615321=404523,1804606=457240,2755658=726225} 2024-09-15T03:54:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: build_initial_monmap 2024-09-15T03:54:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: monmap: 2024-09-15T03:54:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:10.927 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:10.927 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:10.927 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:10.927 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 Processor -- start 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 1 -- start start 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.906+0000 7f37b2eb9700 10 monclient: init 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient: _add_conns ranks=[0,4,6,8,1,7,5,3,2] 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient(hunting): picked mon.noname-a con 0x7f37ac09fc80 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient(hunting): picked mon.noname-c con 0x7f37ac0a0050 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient(hunting): picked mon.noname-d con 0x7f37ac1335e0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0a1b70 con 0x7f37ac09fc80 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac120090 con 0x7f37ac0a0050 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac11e2f0 con 0x7f37ac1335e0 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient(hunting): authenticate will time out at 1395.340496s 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b1eb7700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37ac09fc80 0x7f37ac141b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39762/0 (socket says 172.21.15.26:39762) 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b1eb7700 1 -- 172.21.15.26:0/2438316478 learned_addr learned my addr 172.21.15.26:0/2438316478 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238222943 0 0) 0x7f37ac11e2f0 con 0x7f37ac1335e0 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): my global_id is 194172 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3790003570 con 0x7f37ac1335e0 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95209879 0 0) 0x7f37ac0a1b70 con 0x7f37ac09fc80 2024-09-15T03:54:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): my global_id is 194157 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac11e2f0 con 0x7f37ac09fc80 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212865454 0 0) 0x7f37ac120090 con 0x7f37ac0a0050 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): my global_id is 184423 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac0a1b70 con 0x7f37ac0a0050 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3390500687 0 0) 0x7f37ac11e2f0 con 0x7f37ac09fc80 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37ac120090 con 0x7f37ac09fc80 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a80017b0 con 0x7f37ac09fc80 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): dump: 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.932 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 783389239 0 0) 0x7f3790003570 con 0x7f37ac1335e0 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37ac11e2f0 con 0x7f37ac1335e0 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a4001c10 con 0x7f37ac1335e0 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient(hunting): dump: 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4032305795 0 0) 0x7f37ac11e2f0 con 0x7f37ac1335e0 2024-09-15T03:54:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.26:6791/0 conn(0x7f37ac0a0050 legacy=0x7f37ac12fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.26:6789/0 conn(0x7f37ac09fc80 legacy=0x7f37ac141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 monclient: found mon.d 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac11fdb0 con 0x7f37ac1335e0 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 5 monclient: authenticate success, global_id 194172 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient: _renew_subs 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac1428d0 con 0x7f37ac1335e0 2024-09-15T03:54:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient: _renew_subs 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac130510 con 0x7f37ac1335e0 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f37a4004bb0 con 0x7f37ac1335e0 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a4005060 con 0x7f37ac1335e0 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a3fff700 4 set_mon_vals no callback set 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37a1ffb700 10 monclient: dump: 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.910+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f378c006590 con 0x7f378c002840 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f37a4005240 con 0x7f37ac1335e0 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296439006 0 0) 0x7f378c006590 con 0x7f378c002840 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient(hunting): my global_id is 184375 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac11e2f0 con 0x7f378c002840 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(436..436 src has 224..436) v4 ==== 3943+0+0 (unknown 3288886531 0 0) 0x7f37a4044010 con 0x7f37ac1335e0 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37b16b6700 10 monclient: get_auth_request con 0x7f3790037d00 auth_method 0 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2837122935 0 0) 0x7f37ac11e2f0 con 0x7f378c002840 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f378c006590 con 0x7f378c002840 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a8004830 con 0x7f378c002840 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient: dump: 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1229137821 0 0) 0x7f378c006590 con 0x7f378c002840 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f378c00a130 con 0x7f378c002840 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f37a8007490 con 0x7f378c002840 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.122:6789/0 conn(0x7f378c002840 legacy=0x7f378c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.914+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.122:6789/0 conn(0x7f378c002840 legacy=0x7f378c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f378c009640 con 0x7f378c0090b0 2024-09-15T03:54:10.941 INFO:tasks.rados.rados.0.smithi122.stdout:1940: writing smithi12223906-26 from 615321 to 1019844 tid 1 2024-09-15T03:54:10.941 INFO:tasks.rados.rados.0.smithi122.stdout:1940: writing smithi12223906-26 from 1804606 to 2261846 tid 2 2024-09-15T03:54:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219721340 0 0) 0x7f378c009640 con 0x7f378c0090b0 2024-09-15T03:54:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient(hunting): my global_id is 184384 2024-09-15T03:54:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f378c006590 con 0x7f378c0090b0 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1521449985 0 0) 0x7f378c006590 con 0x7f378c0090b0 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f378c009640 con 0x7f378c0090b0 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f37a8005c70 con 0x7f378c0090b0 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient: dump: 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:10.943 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 132851350 0 0) 0x7f378c009640 con 0x7f378c0090b0 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f378c0098f0 con 0x7f378c0090b0 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1209126895) 0x7f37a8007160 con 0x7f378c0090b0 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:10.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.122:6789/0 conn(0x7f378c0090b0 legacy=0x7f378c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37a1ffb700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.122:6789/0 conn(0x7f378c0090b0 legacy=0x7f378c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:10.945 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.946 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.947 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:10.948 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:10.949 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:10.950 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:10.954 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:10.957 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.957 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.962 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.963 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:10.965 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:10.966 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:10.967 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:10.968 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:10.968 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.122:6812/20895 conn(0x7f3790037d00 legacy=0x7f379003a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 10 monclient: shutdown 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 >> v1:172.21.15.26:6792/0 conn(0x7f37ac1335e0 legacy=0x7f37ac149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b26b8700 1 -- 172.21.15.26:0/2438316478 reap_dead start 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b26b8700 1 -- 172.21.15.26:0/2438316478 reap_dead start 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 shutdown_connections 2024-09-15T03:54:10.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 >> 172.21.15.26:0/2438316478 conn(0x7f37ac06e6e0 msgr2=0x7f37ac1311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:10.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 shutdown_connections 2024-09-15T03:54:10.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:10.922+0000 7f37b2eb9700 1 -- 172.21.15.26:0/2438316478 wait complete. 2024-09-15T03:54:10.970 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:10.970 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:54:10.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1940: writing smithi12223906-26 from 2755658 to 3481883 tid 3 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1941: copy_from oid 22 from oid 9 current snap is 215 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1940: finishing write tid 1 to smithi12223906-26 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1940: finishing write tid 2 to smithi12223906-26 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1942: rollback oid 16 current snap is 215 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 197 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1943: copy_from oid 33 from oid 5 current snap is 215 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1944: rollback oid 32 current snap is 215 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 188 2024-09-15T03:54:10.972 INFO:tasks.rados.rados.0.smithi122.stdout:1945: snap_remove snap 200 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.074+0000 7fa957192700 10 monclient: get_monmap_and_config 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.074+0000 7fa957192700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.074+0000 7fa957192700 10 monclient: monmap: 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:11.078142+0000 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:11.078142+0000 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.081 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.074+0000 7fa957192700 1 Processor -- start 2024-09-15T03:54:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.074+0000 7fa957192700 1 -- start start 2024-09-15T03:54:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.074+0000 7fa957192700 10 monclient: init 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: _add_conns ranks=[4,3,5,8,7,1,0,6,2] 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient(hunting): picked mon.noname-c con 0x7fa950070fb0 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient(hunting): picked mon.noname-g con 0x7fa95009de00 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient(hunting): picked mon.noname-h con 0x7fa95009e8e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95004ca10 con 0x7fa95009de00 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95009b120 con 0x7fa950070fb0 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95009b400 con 0x7fa95009e8e0 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa956190700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fa950070fb0 0x7fa95009c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55154/0 (socket says 172.21.15.26:55154) 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa956190700 1 -- 172.21.15.26:0/924670287 learned_addr learned my addr 172.21.15.26:0/924670287 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient(hunting): authenticate will time out at 1395.509283s 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065982285 0 0) 0x7fa95009b120 con 0x7fa950070fb0 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient(hunting): my global_id is 184432 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93c003570 con 0x7fa950070fb0 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1416473218 0 0) 0x7fa93c003570 con 0x7fa950070fb0 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 >> v1:172.21.15.122:6791/0 conn(0x7fa95009e8e0 legacy=0x7fa950137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 >> v1:172.21.15.122:6790/0 conn(0x7fa95009de00 legacy=0x7fa95009e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 monclient: found mon.noname-c 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95013c490 con 0x7fa950070fb0 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 5 monclient: authenticate success, global_id 184432 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa94c0017b0 con 0x7fa950070fb0 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: dump: 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa94c001a60 con 0x7fa950070fb0 2024-09-15T03:54:11.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 4 set_mon_vals no callback set 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 1 -- 172.21.15.26:0/924670287 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa94c004c30 con 0x7fa950070fb0 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa95518e700 10 monclient: dump: 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: shutdown 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- 172.21.15.26:0/924670287 >> v1:172.21.15.26:6791/0 conn(0x7fa950070fb0 legacy=0x7fa95009c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- 172.21.15.26:0/924670287 shutdown_connections 2024-09-15T03:54:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- 172.21.15.26:0/924670287 >> 172.21.15.26:0/924670287 conn(0x7fa9500a2ea0 msgr2=0x7fa95009eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- 172.21.15.26:0/924670287 shutdown_connections 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- 172.21.15.26:0/924670287 wait complete. 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: monmap: 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.092 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 Processor -- start 2024-09-15T03:54:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 1 -- start start 2024-09-15T03:54:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.078+0000 7fa957192700 10 monclient: init 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient: _add_conns ranks=[8,5,3,7,0,2,4,1,6] 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient(hunting): picked mon.noname-e con 0x7fa95009de00 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient(hunting): picked mon.noname-h con 0x7fa95009e8e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient(hunting): picked mon.noname-g con 0x7fa95012adb0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95009f910 con 0x7fa95012adb0 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9501282f0 con 0x7fa95009e8e0 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa950129b40 con 0x7fa95009de00 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient(hunting): authenticate will time out at 1395.512406s 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa956190700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fa95009de00 0x7fa9501274d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51678/0 (socket says 172.21.15.26:51678) 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa956190700 1 -- 172.21.15.26:0/3959039842 learned_addr learned my addr 172.21.15.26:0/3959039842 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250890349 0 0) 0x7fa950129b40 con 0x7fa95009de00 2024-09-15T03:54:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): my global_id is 134882 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa934003570 con 0x7fa95009de00 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791032763 0 0) 0x7fa95009f910 con 0x7fa95012adb0 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): my global_id is 194124 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa950129b40 con 0x7fa95012adb0 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721964913 0 0) 0x7fa9501282f0 con 0x7fa95009e8e0 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): my global_id is 5414 2024-09-15T03:54:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa95009f910 con 0x7fa95009e8e0 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2261345961 0 0) 0x7fa934003570 con 0x7fa95009de00 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9501282f0 con 0x7fa95009de00 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa94c004b70 con 0x7fa95009de00 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): dump: 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4256005442 0 0) 0x7fa9501282f0 con 0x7fa95009de00 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6791/0 conn(0x7fa95009e8e0 legacy=0x7fa950127be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6790/0 conn(0x7fa95012adb0 legacy=0x7fa950147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 monclient: found mon.e 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa950129e20 con 0x7fa95009de00 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 5 monclient: authenticate success, global_id 134882 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient: _renew_subs 2024-09-15T03:54:11.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95006e260 con 0x7fa95009de00 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient: _renew_subs 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95006ec40 con 0x7fa95009de00 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa94c001860 con 0x7fa95009de00 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa95498d700 4 set_mon_vals no callback set 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa94c005230 con 0x7fa95009de00 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:11.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient: dump: 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.103 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa94c00fbf0 con 0x7fa95009de00 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa938006590 con 0x7fa938002840 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(436..436 src has 224..436) v4 ==== 3943+0+0 (unknown 3288886531 0 0) 0x7fa94c044150 con 0x7fa95009de00 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa956991700 10 monclient: get_auth_request con 0x7fa9340375e0 auth_method 0 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200564476 0 0) 0x7fa938006590 con 0x7fa938002840 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 10 monclient(hunting): my global_id is 194133 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.082+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9501282f0 con 0x7fa938002840 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1559612494 0 0) 0x7fa9501282f0 con 0x7fa938002840 2024-09-15T03:54:11.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa938006590 con 0x7fa938002840 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa940001970 con 0x7fa938002840 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 10 monclient: dump: 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.105 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1639491837 0 0) 0x7fa938006590 con 0x7fa938002840 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa93800a130 con 0x7fa938002840 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa940003fc0 con 0x7fa938002840 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6790/0 conn(0x7fa938002840 legacy=0x7fa938004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.086+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6790/0 conn(0x7fa938002840 legacy=0x7fa938004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:11.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.094+0000 7fa957192700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.094+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa938007520 con 0x7fa9380068d0 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360317888 0 0) 0x7fa938007520 con 0x7fa9380068d0 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient(hunting): my global_id is 194142 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa938006590 con 0x7fa9380068d0 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 480951096 0 0) 0x7fa938006590 con 0x7fa9380068d0 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa938007520 con 0x7fa9380068d0 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa940002530 con 0x7fa9380068d0 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:54:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient: dump: 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.109 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1827999165 0 0) 0x7fa938007520 con 0x7fa9380068d0 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa938008760 con 0x7fa9380068d0 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1853041266) 0x7fa940003960 con 0x7fa9380068d0 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6790/0 conn(0x7fa9380068d0 legacy=0x7fa938006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa9467fc700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6790/0 conn(0x7fa9380068d0 legacy=0x7fa938006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:11.110 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:11.111 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.112 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.113 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:11.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.115 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.117 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.118 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:11.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.122 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.125 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.126 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.128 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:11.129 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.122:6812/20895 conn(0x7fa9340375e0 legacy=0x7fa934039a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 10 monclient: shutdown 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 >> v1:172.21.15.26:6793/0 conn(0x7fa95009de00 legacy=0x7fa9501274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa956991700 1 -- 172.21.15.26:0/3959039842 reap_dead start 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 shutdown_connections 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 >> 172.21.15.26:0/3959039842 conn(0x7fa9500a2ea0 msgr2=0x7fa9500a15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 shutdown_connections 2024-09-15T03:54:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.098+0000 7fa957192700 1 -- 172.21.15.26:0/3959039842 wait complete. 2024-09-15T03:54:11.131 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:11.131 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:54:11.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:54:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient: get_monmap_and_config 2024-09-15T03:54:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient: monmap: 2024-09-15T03:54:11.244 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:11.242297+0000 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:11.242297+0000 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.245 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.246 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.246 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 1 Processor -- start 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 1 -- start start 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient: init 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient: _add_conns ranks=[3,1,6,8,4,2,5,0,7] 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient(hunting): picked mon.noname-g con 0x7f1a5c06df70 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:11.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient(hunting): picked mon.noname-f con 0x7f1a5c12f860 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient(hunting): picked mon.noname-d con 0x7f1a5c12fc50 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c04ca10 con 0x7f1a5c12f860 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c09e260 con 0x7f1a5c06df70 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c09e550 con 0x7f1a5c12fc50 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.238+0000 7f1a63eaa700 10 monclient(hunting): authenticate will time out at 1395.671925s 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a62447700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1a5c12fc50 0x7f1a5c13b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60310/0 (socket says 172.21.15.26:60310) 2024-09-15T03:54:11.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a62447700 1 -- 172.21.15.26:0/4048595212 learned_addr learned my addr 172.21.15.26:0/4048595212 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170472050 0 0) 0x7f1a5c09e550 con 0x7f1a5c12fc50 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): my global_id is 194181 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a48003570 con 0x7f1a5c12fc50 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569836570 0 0) 0x7f1a5c09e260 con 0x7f1a5c06df70 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): my global_id is 194151 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a5c09e550 con 0x7f1a5c06df70 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282881422 0 0) 0x7f1a5c04ca10 con 0x7f1a5c12f860 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient(hunting): my global_id is 184393 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a5c09e260 con 0x7f1a5c12f860 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 420302191 0 0) 0x7f1a48003570 con 0x7f1a5c12fc50 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 >> v1:172.21.15.122:6790/0 conn(0x7f1a5c06df70 legacy=0x7f1a5c09bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 >> v1:172.21.15.122:6789/0 conn(0x7f1a5c12f860 legacy=0x7f1a5c137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 monclient: found mon.noname-d 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c09eb10 con 0x7f1a5c12fc50 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a54001b30 con 0x7f1a5c12fc50 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 5 monclient: authenticate success, global_id 194181 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: dump: 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.253 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.254 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a54001da0 con 0x7f1a5c12fc50 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 1 -- 172.21.15.26:0/4048595212 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a54004fb0 con 0x7f1a5c12fc50 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 4 set_mon_vals no callback set 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a60c44700 10 monclient: dump: 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.255 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 10 monclient: shutdown 2024-09-15T03:54:11.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 1 -- 172.21.15.26:0/4048595212 >> v1:172.21.15.26:6792/0 conn(0x7f1a5c12fc50 legacy=0x7f1a5c13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 1 -- 172.21.15.26:0/4048595212 shutdown_connections 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 1 -- 172.21.15.26:0/4048595212 >> 172.21.15.26:0/4048595212 conn(0x7f1a5c0a21f0 msgr2=0x7f1a5c0a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 1 -- 172.21.15.26:0/4048595212 shutdown_connections 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 1 -- 172.21.15.26:0/4048595212 wait complete. 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.242+0000 7f1a63eaa700 10 monclient: monmap: 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.258 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 Processor -- start 2024-09-15T03:54:11.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- start start 2024-09-15T03:54:11.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: init 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: _add_conns ranks=[6,2,3,4,1,7,5,0,8] 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient(hunting): picked mon.noname-d con 0x7f1a5c06df70 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient(hunting): picked mon.noname-b con 0x7f1a5c12f860 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient(hunting): picked mon.noname-g con 0x7f1a5c12fc50 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c08b8b0 con 0x7f1a5c12f860 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c08b5d0 con 0x7f1a5c12fc50 2024-09-15T03:54:11.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c08bb90 con 0x7f1a5c06df70 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient(hunting): authenticate will time out at 1395.676989s 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a61445700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f1a5c12f860 0x7f1a5c146e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33600/0 (socket says 172.21.15.26:33600) 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a61445700 1 -- 172.21.15.26:0/1097028500 learned_addr learned my addr 172.21.15.26:0/1097028500 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116998129 0 0) 0x7f1a5c08b8b0 con 0x7f1a5c12f860 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): my global_id is 134930 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a40003570 con 0x7f1a5c12f860 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373031561 0 0) 0x7f1a5c08bb90 con 0x7f1a5c06df70 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): my global_id is 194190 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a5c08b8b0 con 0x7f1a5c06df70 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2866448829 0 0) 0x7f1a40003570 con 0x7f1a5c12f860 2024-09-15T03:54:11.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a5c08bb90 con 0x7f1a5c12f860 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a50002b90 con 0x7f1a5c12f860 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): dump: 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.270 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224831784 0 0) 0x7f1a5c08b5d0 con 0x7f1a5c12fc50 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient(hunting): my global_id is 194160 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a40003570 con 0x7f1a5c12fc50 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 461960484 0 0) 0x7f1a5c08bb90 con 0x7f1a5c12f860 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.26:6792/0 conn(0x7f1a5c06df70 legacy=0x7f1a5c098450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.122:6790/0 conn(0x7f1a5c12fc50 legacy=0x7f1a5c14a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 monclient: found mon.b 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c12d290 con 0x7f1a5c12f860 2024-09-15T03:54:11.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 5 monclient: authenticate success, global_id 134930 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: _renew_subs 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c098b60 con 0x7f1a5c12f860 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a500016a0 con 0x7f1a5c12f860 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: _renew_subs 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a50001bd0 con 0x7f1a5c12f860 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c12c4e0 con 0x7f1a5c12f860 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4ffff700 4 set_mon_vals no callback set 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a4dffb700 10 monclient: dump: 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.246+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1a5c04bbf0 con 0x7f1a5c141fc0 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1a50001db0 con 0x7f1a5c12f860 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(436..436 src has 224..436) v4 ==== 3943+0+0 (unknown 3288886531 0 0) 0x7f1a50043fb0 con 0x7f1a5c12f860 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414853411 0 0) 0x7f1a5c04bbf0 con 0x7f1a5c141fc0 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient(hunting): my global_id is 5423 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a5c08bb90 con 0x7f1a5c141fc0 2024-09-15T03:54:11.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a62447700 10 monclient: get_auth_request con 0x7f1a400375d0 auth_method 0 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 66520458 0 0) 0x7f1a5c08bb90 con 0x7f1a5c141fc0 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a5c04bbf0 con 0x7f1a5c141fc0 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a58001a40 con 0x7f1a5c141fc0 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient: dump: 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.276 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2191020241 0 0) 0x7f1a5c04bbf0 con 0x7f1a5c141fc0 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a5c098d50 con 0x7f1a5c141fc0 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1a58001f70 con 0x7f1a5c141fc0 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.122:6791/0 conn(0x7f1a5c141fc0 legacy=0x7f1a5c05e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.250+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.122:6791/0 conn(0x7f1a5c141fc0 legacy=0x7f1a5c05e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.258+0000 7f1a63eaa700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.258+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1a5c142390 con 0x7f1a5c02ae80 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473416280 0 0) 0x7f1a5c142390 con 0x7f1a5c02ae80 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient(hunting): my global_id is 5432 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a5c04bbf0 con 0x7f1a5c02ae80 2024-09-15T03:54:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3272639331 0 0) 0x7f1a5c04bbf0 con 0x7f1a5c02ae80 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a5c142390 con 0x7f1a5c02ae80 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1a58002780 con 0x7f1a5c02ae80 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient: dump: 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.279 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 456454234 0 0) 0x7f1a5c142390 con 0x7f1a5c02ae80 2024-09-15T03:54:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1a5c12cec0 con 0x7f1a5c02ae80 2024-09-15T03:54:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7740 (unknown 0 0 3900195121) 0x7f1a58002b80 con 0x7f1a5c02ae80 2024-09-15T03:54:11.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.122:6791/0 conn(0x7f1a5c02ae80 legacy=0x7f1a5c1282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a4dffb700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.122:6791/0 conn(0x7f1a5c02ae80 legacy=0x7f1a5c1282d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:11.282 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:11.282 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:11.283 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.284 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.285 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:11.286 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:11.287 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:11.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.292 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.293 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:11.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.295 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.296 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:11.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.298 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:11.300 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.301 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.302 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:54:11.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:11.304 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.122:6812/20895 conn(0x7f1a400375d0 legacy=0x7f1a40039a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 10 monclient: shutdown 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 >> v1:172.21.15.26:6790/0 conn(0x7f1a5c12f860 legacy=0x7f1a5c146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a62447700 1 -- 172.21.15.26:0/1097028500 reap_dead start 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a62447700 1 -- 172.21.15.26:0/1097028500 reap_dead start 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 shutdown_connections 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 >> 172.21.15.26:0/1097028500 conn(0x7f1a5c0a21f0 msgr2=0x7f1a5c0a0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 shutdown_connections 2024-09-15T03:54:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.262+0000 7f1a63eaa700 1 -- 172.21.15.26:0/1097028500 wait complete. 2024-09-15T03:54:11.306 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:11.306 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:54:11.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:54:11.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient: get_monmap_and_config 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient: monmap: 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:11.424357+0000 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:11.424357+0000 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.428 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.429 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.430 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 1 Processor -- start 2024-09-15T03:54:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 1 -- start start 2024-09-15T03:54:11.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient: init 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient: _add_conns ranks=[3,7,4,6,2,5,1,0,8] 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient(hunting): picked mon.noname-g con 0x7f06080a2aa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient(hunting): picked mon.noname-i con 0x7f060806e160 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient(hunting): picked mon.noname-c con 0x7f060809b310 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060804ca10 con 0x7f06080a2aa0 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0608099950 con 0x7f060809b310 2024-09-15T03:54:11.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06080995e0 con 0x7f060806e160 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060d473700 10 monclient(hunting): authenticate will time out at 1395.854727s 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060cc72700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f060809b310 0x7f0608137640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55164/0 (socket says 172.21.15.26:55164) 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0607fff700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f06080a2aa0 0x7f060809c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:34464/0 (socket says 172.21.15.26:34464) 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f060cc72700 1 -- 172.21.15.26:0/2005914119 learned_addr learned my addr 172.21.15.26:0/2005914119 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f06077fe700 1 --1- 172.21.15.26:0/2005914119 >> v1:172.21.15.122:6792/0 conn(0x7f060806e160 0x7f060809cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:42316/0 (socket says 172.21.15.26:42316) 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740613234 0 0) 0x7f060804ca10 con 0x7f06080a2aa0 2024-09-15T03:54:11.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): my global_id is 194169 2024-09-15T03:54:11.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f0003570 con 0x7f06080a2aa0 2024-09-15T03:54:11.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726710265 0 0) 0x7f0608099950 con 0x7f060809b310 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): my global_id is 184441 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f060804ca10 con 0x7f060809b310 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687469326 0 0) 0x7f06080995e0 con 0x7f060806e160 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 10 monclient(hunting): my global_id is 134818 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.422+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0608099950 con 0x7f060806e160 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205817361 0 0) 0x7f060804ca10 con 0x7f060809b310 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 >> v1:172.21.15.122:6792/0 conn(0x7f060806e160 legacy=0x7f060809cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 >> v1:172.21.15.122:6790/0 conn(0x7f06080a2aa0 legacy=0x7f060809c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 monclient: found mon.noname-c 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060809d630 con 0x7f060809b310 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 5 monclient: authenticate success, global_id 184441 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0600001b10 con 0x7f060809b310 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: dump: 2024-09-15T03:54:11.443 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.444 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0600001db0 con 0x7f060809b310 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 1 -- 172.21.15.26:0/2005914119 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0600004fb0 con 0x7f060809b310 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 4 set_mon_vals no callback set 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f0606ffd700 10 monclient: dump: 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.445 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.446 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: shutdown 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- 172.21.15.26:0/2005914119 >> v1:172.21.15.26:6791/0 conn(0x7f060809b310 legacy=0x7f0608137640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- 172.21.15.26:0/2005914119 shutdown_connections 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- 172.21.15.26:0/2005914119 >> 172.21.15.26:0/2005914119 conn(0x7f0608063cd0 msgr2=0x7f06080a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- 172.21.15.26:0/2005914119 shutdown_connections 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- 172.21.15.26:0/2005914119 wait complete. 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: monmap: 2024-09-15T03:54:11.447 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.448 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 Processor -- start 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- start start 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: init 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient: _add_conns ranks=[7,2,0,8,6,3,5,1,4] 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient(hunting): picked mon.noname-i con 0x7f060806e160 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient(hunting): picked mon.noname-b con 0x7f060809b310 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient(hunting): picked mon.noname-a con 0x7f0608122b20 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060809b9b0 con 0x7f0608122b20 2024-09-15T03:54:11.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0608121c00 con 0x7f060809b310 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060809be80 con 0x7f060806e160 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f060d473700 10 monclient(hunting): authenticate will time out at 1395.859353s 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f06077fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f060809b310 0x7f0608146f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33606/0 (socket says 172.21.15.26:33606) 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.426+0000 7f06077fe700 1 -- 172.21.15.26:0/1918398613 learned_addr learned my addr 172.21.15.26:0/1918398613 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206462711 0 0) 0x7f060809b9b0 con 0x7f0608122b20 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): my global_id is 194166 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f4003570 con 0x7f0608122b20 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374172758 0 0) 0x7f060809be80 con 0x7f060806e160 2024-09-15T03:54:11.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): my global_id is 134827 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f060809b9b0 con 0x7f060806e160 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767957777 0 0) 0x7f0608121c00 con 0x7f060809b310 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): my global_id is 134939 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f060809be80 con 0x7f060809b310 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2836150133 0 0) 0x7f05f4003570 con 0x7f0608122b20 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0608121c00 con 0x7f0608122b20 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0600004bb0 con 0x7f0608122b20 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): dump: 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.452 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2452722098 0 0) 0x7f060809b9b0 con 0x7f060806e160 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05f4003570 con 0x7f060806e160 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05f80019e0 con 0x7f060806e160 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): dump: 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.453 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.454 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2314589714 0 0) 0x7f060809be80 con 0x7f060809b310 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f060809b9b0 con 0x7f060809b310 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2605541356 0 0) 0x7f0608121c00 con 0x7f0608122b20 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.122:6792/0 conn(0x7f060806e160 legacy=0x7f0608141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.26:6790/0 conn(0x7f060809b310 legacy=0x7f0608146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 monclient: found mon.a 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060811fd60 con 0x7f0608122b20 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 5 monclient: authenticate success, global_id 194166 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 10 monclient: _renew_subs 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0608121890 con 0x7f0608122b20 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 10 monclient: _renew_subs 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f060811f360 con 0x7f0608122b20 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0600005280 con 0x7f0608122b20 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0600005730 con 0x7f0608122b20 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f0605ffb700 4 set_mon_vals no callback set 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:11.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient: dump: 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.457 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f06000100f0 con 0x7f0608122b20 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f05e8006590 con 0x7f05e8002840 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(436..436 src has 224..436) v4 ==== 3943+0+0 (unknown 3288886531 0 0) 0x7f0600044430 con 0x7f0608122b20 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f06077fe700 10 monclient: get_auth_request con 0x7f05f4037f50 auth_method 0 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338854697 0 0) 0x7f05e8006590 con 0x7f05e8002840 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 10 monclient(hunting): my global_id is 134836 2024-09-15T03:54:11.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.430+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0608121c00 con 0x7f05e8002840 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2175934338 0 0) 0x7f0608121c00 con 0x7f05e8002840 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05e8006590 con 0x7f05e8002840 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05f8004b70 con 0x7f05e8002840 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 10 monclient: dump: 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.459 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1392889347 0 0) 0x7f05e8006590 con 0x7f05e8002840 2024-09-15T03:54:11.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05e800a130 con 0x7f05e8002840 2024-09-15T03:54:11.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f05f8004fd0 con 0x7f05e8002840 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.122:6792/0 conn(0x7f05e8002840 legacy=0x7f05e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.434+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.122:6792/0 conn(0x7f05e8002840 legacy=0x7f05e8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f060d473700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f05e8009640 con 0x7f05e80090b0 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539641143 0 0) 0x7f05e8009640 con 0x7f05e80090b0 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f05effff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f05effff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f05effff700 10 monclient(hunting): my global_id is 134845 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.442+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e8006590 con 0x7f05e80090b0 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2446747018 0 0) 0x7f05e8006590 con 0x7f05e80090b0 2024-09-15T03:54:11.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05e8009640 con 0x7f05e80090b0 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f05f8005860 con 0x7f05e80090b0 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 10 monclient: dump: 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.464 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 625019592 0 0) 0x7f05e8009640 con 0x7f05e80090b0 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f05e80098f0 con 0x7f05e80090b0 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 3882195945) 0x7f05f8005a10 con 0x7f05e80090b0 2024-09-15T03:54:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.122:6792/0 conn(0x7f05e80090b0 legacy=0x7f05e8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f05effff700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.122:6792/0 conn(0x7f05e80090b0 legacy=0x7f05e8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:11.466 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:54:11.467 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.468 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:11.470 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:11.471 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.472 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.473 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.475 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.476 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:11.477 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:11.478 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:11.481 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.482 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:11.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:11.485 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:11.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:11.487 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:11.488 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.122:6812/20895 conn(0x7f05f4037f50 legacy=0x7f05f403a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 10 monclient: shutdown 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060cc72700 1 -- 172.21.15.26:0/1918398613 reap_dead start 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 >> v1:172.21.15.26:6789/0 conn(0x7f0608122b20 legacy=0x7f060814a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 shutdown_connections 2024-09-15T03:54:11.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 >> 172.21.15.26:0/1918398613 conn(0x7f0608063cd0 msgr2=0x7f060809f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 shutdown_connections 2024-09-15T03:54:11.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.446+0000 7f060d473700 1 -- 172.21.15.26:0/1918398613 wait complete. 2024-09-15T03:54:11.490 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:11.490 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T03:54:11.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T03:54:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: get_monmap_and_config 2024-09-15T03:54:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: monmap: 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:11.604482+0000 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:11.604482+0000 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.608 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.608 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 1 Processor -- start 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 1 -- start start 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: init 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: _add_conns ranks=[3,8,2,5,7,4,6,1,0] 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient(hunting): picked mon.noname-g con 0x7fcd8c0a6a00 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient(hunting): picked mon.noname-e con 0x7fcd8c09fcd0 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient(hunting): picked mon.noname-b con 0x7fcd8c0a0100 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c04ca10 con 0x7fcd8c0a0100 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0a28b0 con 0x7fcd8c0a6a00 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c1337c0 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient(hunting): authenticate will time out at 1396.033895s 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8bfff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fcd8c09fcd0 0x7fcd8c137cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51684/0 (socket says 172.21.15.26:51684) 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8bfff700 1 -- 172.21.15.26:0/1740810473 learned_addr learned my addr 172.21.15.26:0/1740810473 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502159780 0 0) 0x7fcd8c1337c0 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): my global_id is 134891 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd74003570 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938289091 0 0) 0x7fcd8c04ca10 con 0x7fcd8c0a0100 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): my global_id is 134948 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd8c1337c0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483530787 0 0) 0x7fcd8c0a28b0 con 0x7fcd8c0a6a00 2024-09-15T03:54:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient(hunting): my global_id is 194178 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd8c04ca10 con 0x7fcd8c0a6a00 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17669294 0 0) 0x7fcd74003570 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 >> v1:172.21.15.122:6790/0 conn(0x7fcd8c0a6a00 legacy=0x7fcd8c1330b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 >> v1:172.21.15.26:6790/0 conn(0x7fcd8c0a0100 legacy=0x7fcd8c13b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 monclient: found mon.noname-e 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0a0c40 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcd7c0017b0 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: dump: 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 5 monclient: authenticate success, global_id 134891 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcd7c001a20 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd91817700 4 set_mon_vals no callback set 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 1 -- 172.21.15.26:0/1740810473 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcd7c004c30 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.602+0000 7fcd8b7fe700 10 monclient: dump: 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.617 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 10 monclient: shutdown 2024-09-15T03:54:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 1 -- 172.21.15.26:0/1740810473 >> v1:172.21.15.26:6793/0 conn(0x7fcd8c09fcd0 legacy=0x7fcd8c137cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 1 -- 172.21.15.26:0/1740810473 shutdown_connections 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 1 -- 172.21.15.26:0/1740810473 >> 172.21.15.26:0/1740810473 conn(0x7fcd8c06e2a0 msgr2=0x7fcd8c0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 1 -- 172.21.15.26:0/1740810473 shutdown_connections 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 1 -- 172.21.15.26:0/1740810473 wait complete. 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 10 monclient: build_initial_monmap 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 10 monclient: monmap: 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.606+0000 7fcd91817700 1 Processor -- start 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 1 -- start start 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: init 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: _add_conns ranks=[4,2,7,8,0,1,5,6,3] 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient(hunting): picked mon.noname-c con 0x7fcd8c09fcd0 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient(hunting): picked mon.noname-b con 0x7fcd8c0a0100 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient(hunting): picked mon.noname-i con 0x7fcd8c0a6a00 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0a1850 con 0x7fcd8c0a0100 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c09cf10 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd8bfff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fcd8c0a0100 0x7fcd8c146cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33620/0 (socket says 172.21.15.26:33620) 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c11ffc0 con 0x7fcd8c0a6a00 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd8bfff700 1 -- 172.21.15.26:0/42938186 learned_addr learned my addr 172.21.15.26:0/42938186 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient(hunting): authenticate will time out at 1396.041487s 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd90815700 1 --1- 172.21.15.26:0/42938186 >> v1:172.21.15.26:6791/0 conn(0x7fcd8c09fcd0 0x7fcd8c131930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55180/0 (socket says 172.21.15.26:55180) 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308541010 0 0) 0x7fcd8c0a1850 con 0x7fcd8c0a0100 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): my global_id is 134957 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd78003570 con 0x7fcd8c0a0100 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395194616 0 0) 0x7fcd8c09cf10 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): my global_id is 184450 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd8c0a1850 con 0x7fcd8c09fcd0 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016757818 0 0) 0x7fcd8c11ffc0 con 0x7fcd8c0a6a00 2024-09-15T03:54:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): my global_id is 134854 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd8c09cf10 con 0x7fcd8c0a6a00 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 175008646 0 0) 0x7fcd78003570 con 0x7fcd8c0a0100 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcd8c11ffc0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcd7c001830 con 0x7fcd8c0a0100 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): dump: 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1184744001 0 0) 0x7fcd8c11ffc0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 >> v1:172.21.15.122:6792/0 conn(0x7fcd8c0a6a00 legacy=0x7fcd8c14a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 >> v1:172.21.15.26:6791/0 conn(0x7fcd8c09fcd0 legacy=0x7fcd8c131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 monclient: found mon.b 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c121a80 con 0x7fcd8c0a0100 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 10 monclient: _finish_auth 0 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd88ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 5 monclient: authenticate success, global_id 134957 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: _renew_subs 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c1206c0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: _renew_subs 2024-09-15T03:54:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.610+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c133e20 con 0x7fcd8c0a0100 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcd7c004c00 con 0x7fcd8c0a0100 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcd7c005240 con 0x7fcd8c0a0100 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd8affd700 4 set_mon_vals no callback set 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 10 monclient: dump: 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcd7c00fc00 con 0x7fcd8c0a0100 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(436..436 src has 224..436) v4 ==== 3943+0+0 (unknown 3288886531 0 0) 0x7fcd7c044100 con 0x7fcd8c0a0100 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd90815700 10 monclient: get_auth_request con 0x7fcd780376b0 auth_method 0 2024-09-15T03:54:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd91817700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd91817700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd91817700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.614+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c0050b0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.622+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fcd7c014bd0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.622+0000 7fcd88ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.622+0000 7fcd88ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:11.675 INFO:tasks.rados.rados.0.smithi122.stdout:1945: done (5 left) 2024-09-15T03:54:11.675 INFO:tasks.rados.rados.0.smithi122.stdout:1946: write (excl) oid 21 current snap is 215 2024-09-15T03:54:11.675 INFO:tasks.rados.rados.0.smithi122.stdout:1946: seq_num 704 ranges {632128=479772,1746735=600898,3066728=653850} 2024-09-15T03:54:11.683 INFO:tasks.rados.rados.0.smithi122.stdout:1946: writing smithi12223906-21 from 632128 to 1111900 tid 1 2024-09-15T03:54:11.691 INFO:tasks.rados.rados.0.smithi122.stdout:1946: writing smithi12223906-21 from 1746735 to 2347633 tid 2 2024-09-15T03:54:11.699 INFO:tasks.rados.rados.0.smithi122.stdout:1946: writing smithi12223906-21 from 3066728 to 3720578 tid 3 2024-09-15T03:54:11.700 INFO:tasks.rados.rados.0.smithi122.stdout:1947: write oid 27 current snap is 215 2024-09-15T03:54:11.700 INFO:tasks.rados.rados.0.smithi122.stdout:1940: finishing write tid 3 to smithi12223906-26 2024-09-15T03:54:11.700 INFO:tasks.rados.rados.0.smithi122.stdout:1947: seq_num 705 ranges {648935=555021,1805217=460909,2694151=465122,3817442=31493} 2024-09-15T03:54:11.706 INFO:tasks.rados.rados.0.smithi122.stdout:1947: writing smithi12223906-27 from 648935 to 1203956 tid 1 2024-09-15T03:54:11.711 INFO:tasks.rados.rados.0.smithi122.stdout:1947: writing smithi12223906-27 from 1805217 to 2266126 tid 2 2024-09-15T03:54:11.716 INFO:tasks.rados.rados.0.smithi122.stdout:1947: writing smithi12223906-27 from 2694151 to 3159273 tid 3 2024-09-15T03:54:11.721 INFO:tasks.rados.rados.0.smithi122.stdout:1947: writing smithi12223906-27 from 3817442 to 3848935 tid 4 2024-09-15T03:54:11.721 INFO:tasks.rados.rados.0.smithi122.stdout:1948: copy_from oid 50 from oid 47 current snap is 215 2024-09-15T03:54:11.721 INFO:tasks.rados.rados.0.smithi122.stdout:1942: finishing rollback tid 0 to smithi12223906-16 2024-09-15T03:54:11.721 INFO:tasks.rados.rados.0.smithi122.stdout:1949: snap_remove snap 195 2024-09-15T03:54:11.764 INFO:tasks.rados.rados.0.smithi122.stdout:1949: done (8 left) 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1950: read oid 3 snap -1 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1950: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1951: write oid 23 current snap is 215 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1940: finishing write tid 4 to smithi12223906-26 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1942: finishing rollback tid 1 to smithi12223906-16 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1942: finishing rollback tid 2 to smithi12223906-16 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 900 (ObjNum 625 snap 187 seq_num 625) dirty exists 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1940: finishing write tid 5 to smithi12223906-26 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 922 (ObjNum 703 snap 215 seq_num 703) dirty exists 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1940: left oid 26 (ObjNum 703 snap 215 seq_num 703) 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1941: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:54:11.765 INFO:tasks.rados.rados.0.smithi122.stdout:1943: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1944: finishing rollback tid 0 to smithi12223906-32 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1944: finishing rollback tid 1 to smithi12223906-32 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1944: finishing rollback tid 2 to smithi12223906-32 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 1054 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1943: finishing copy_from to smithi12223906-33 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 778 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1941: finishing copy_from to smithi12223906-22 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 809 (ObjNum 682 snap 210 seq_num 682) dirty exists 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1946: finishing write tid 1 to smithi12223906-21 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1946: finishing write tid 2 to smithi12223906-21 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1946: finishing write tid 3 to smithi12223906-21 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1946: finishing write tid 4 to smithi12223906-21 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:1946: finishing write tid 5 to smithi12223906-21 2024-09-15T03:54:11.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1002 (ObjNum 704 snap 215 seq_num 704) dirty exists 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1946: left oid 21 (ObjNum 704 snap 215 seq_num 704) 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: finishing write tid 1 to smithi12223906-27 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: finishing write tid 2 to smithi12223906-27 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: finishing write tid 3 to smithi12223906-27 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: finishing write tid 4 to smithi12223906-27 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: finishing write tid 5 to smithi12223906-27 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: finishing write tid 6 to smithi12223906-27 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1072 (ObjNum 705 snap 215 seq_num 705) dirty exists 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1947: left oid 27 (ObjNum 705 snap 215 seq_num 705) 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1948: finishing copy_from racing read to smithi12223906-50 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1948: finishing copy_from to smithi12223906-50 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 849 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:54:11.767 INFO:tasks.rados.rados.0.smithi122.stdout:1951: seq_num 706 ranges {665742=630270,1863699=604567,3121574=676394} 2024-09-15T03:54:11.772 INFO:tasks.rados.rados.0.smithi122.stdout:1951: writing smithi12223906-23 from 665742 to 1296012 tid 1 2024-09-15T03:54:11.780 INFO:tasks.rados.rados.0.smithi122.stdout:1951: writing smithi12223906-23 from 1863699 to 2468266 tid 2 2024-09-15T03:54:11.787 INFO:tasks.rados.rados.0.smithi122.stdout:1951: writing smithi12223906-23 from 3121574 to 3797968 tid 3 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1940: done (9 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1941: done (8 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1942: done (7 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1943: done (6 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1944: done (5 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1946: done (4 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1947: done (3 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1948: done (2 left) 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1952: write (excl) oid 8 current snap is 215 2024-09-15T03:54:11.788 INFO:tasks.rados.rados.0.smithi122.stdout:1952: seq_num 707 ranges {682549=705519,1922181=464578,3148997=487666} 2024-09-15T03:54:11.795 INFO:tasks.rados.rados.0.smithi122.stdout:1952: writing smithi12223906-8 from 682549 to 1388068 tid 1 2024-09-15T03:54:11.801 INFO:tasks.rados.rados.0.smithi122.stdout:1952: writing smithi12223906-8 from 1922181 to 2386759 tid 2 2024-09-15T03:54:11.808 INFO:tasks.rados.rados.0.smithi122.stdout:1952: writing smithi12223906-8 from 3148997 to 3636663 tid 3 2024-09-15T03:54:11.808 INFO:tasks.rados.rados.0.smithi122.stdout:1953: delete oid 37 current snap is 215 2024-09-15T03:54:11.808 INFO:tasks.rados.rados.0.smithi122.stdout:1950: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:54:11.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.894+0000 7fcd91817700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T03:54:11.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.894+0000 7fcd91817700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T03:54:11.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.894+0000 7fcd91817700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:11.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.894+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fcd6c005b70 con 0x7fcd8c0a0100 2024-09-15T03:54:11.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd88ff9700 1 -- 172.21.15.26:0/42938186 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fcd7c0149f0 con 0x7fcd8c0a0100 2024-09-15T03:54:11.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd88ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T03:54:11.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd88ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1704929984,config=3140514009,health=1012707849,kv=2519098299,logm=680654000}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=11,kv=7,logm=53} crc {auth=3174399989,config=3140514009,health=38080560,kv=2862677453,logm=387817219}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=2,health=11,kv=8,logm=51} crc {auth=2611919492,config=3140514009,health=1295987503,kv=2519098299,logm=427305531}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.906 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=2812961041,config=3140514010,health=1295987503,kv=2519098299,logm=1246715037}) 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=3112209381,config=3140514009,health=2396032787,kv=2519098299,logm=2105124874}) 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=24,config=2,health=10,kv=8,logm=56} crc {auth=1558223399,config=3140514009,health=2282538830,kv=2519098299,logm=2355410929}) 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.907 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=2611919492,config=3140514009,health=1012707849,kv=2519098299,logm=946946331}) 2024-09-15T03:54:11.908 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.908 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=27,config=2,health=12,kv=8,logm=51} crc {auth=2815813776,config=3140514009,health=1012707849,kv=2519098299,logm=2451686666}) 2024-09-15T03:54:11.908 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.900+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=2446317753,config=3140514009,health=1012707849,kv=2519098299,logm=189917062}) 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 >> v1:172.21.15.122:6812/20895 conn(0x7fcd780376b0 legacy=0x7fcd78039b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd91817700 10 monclient: shutdown 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd91817700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.898+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 >> v1:172.21.15.26:6790/0 conn(0x7fcd8c0a0100 legacy=0x7fcd8c146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.902+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 shutdown_connections 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.902+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 >> 172.21.15.26:0/42938186 conn(0x7fcd8c06e2a0 msgr2=0x7fcd8c09df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.902+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 shutdown_connections 2024-09-15T03:54:11.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:11.902+0000 7fcd91817700 1 -- 172.21.15.26:0/42938186 wait complete. 2024-09-15T03:54:11.923 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T03:54:11.928 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.929 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.929 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2599527704}) 2024-09-15T03:54:11.929 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.929 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.929 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=930993831}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1758577689}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2881689266}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1697473371}) 2024-09-15T03:54:11.930 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3473121558}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2396243165}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2370786081}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.904+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=681436156}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2581971753}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.931 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2408714220}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3908677529}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3771586565}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=902958658}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1820739184}) 2024-09-15T03:54:11.932 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3730350125}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4063295904}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.908+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2957510436}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3988761854}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4291318056}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.933 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=913341277}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1008972721}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3200558094}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=573909701}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3620566526}) 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.934 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2003617683}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1895519622}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1739777262}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943303569}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3609145702}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1451759709}) 2024-09-15T03:54:11.935 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=996048079}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2555303284}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3555923568}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1365754879}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.912+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=458104173}) 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.936 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1062259228}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=271279169}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=473332942}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4011487282}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3718939362}) 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.937 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1539345821}) 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3567612194}) 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1273922706}) 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2789589696}) 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.938 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2866901045}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3609141732}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2968501386}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3183798099}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3142206338}) 2024-09-15T03:54:11.939 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3935778322}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2789956364}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3897070567}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.916+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4271891831}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2051867301}) 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.940 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860867786}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=870741942}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3732774897}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3651590990}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2831673137}) 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.941 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2954926477}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2112271901}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.920+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1363714104}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3774892961}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1573723907}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2855481089}) 2024-09-15T03:54:11.942 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1551843266}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2510207220}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4174031438}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2408308722}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2675481131}) 2024-09-15T03:54:11.943 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1145440812}) 2024-09-15T03:54:11.994 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.994 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.994 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1019801803}) 2024-09-15T03:54:11.994 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=684144498}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1425366903}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=465973197}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2052817074}) 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.995 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=345854745}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=575092021}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=191076508}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.924+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3952211136}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=72908365}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1560077777}) 2024-09-15T03:54:11.996 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=982982372}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3240712871}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=184381718}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1954472099}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.997 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=538832343}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=194908552}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1882388588}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2440756160}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.928+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2750573995}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=572177024}) 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.998 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1285586752}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4038764603}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3241781670}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2815674714}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4233178504}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1898043561}) 2024-09-15T03:54:11.999 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=498687522}) 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3279637018}) 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3674026679}) 2024-09-15T03:54:12.000 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1941242863}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1133174181}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=740001954}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2232893388}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4283620124}) 2024-09-15T03:54:12.001 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.932+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4024070937}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=267982736}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2981925572}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3995648821}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2819269104}) 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.002 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=16214055}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3341947024}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3860931842}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2760209954}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=931085495}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3246950547}) 2024-09-15T03:54:12.003 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1723355729}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1163453987}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1452202387}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3930259908}) 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.004 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2645790046}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1121210087}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2088134315}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.936+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=392838131}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346470748}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1422090191}) 2024-09-15T03:54:12.005 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3505457576}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1376385316}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4111483409}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=633609370}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4054549905}) 2024-09-15T03:54:12.006 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567340646}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2547847903}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2005996601}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2272601224}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.940+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=13849218}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.007 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2386558037}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2411876453}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4073973907}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=716229214}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2832769667}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3269085912}) 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.008 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2987050738}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2759237325}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3317376115}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1547088483}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2202682634}) 2024-09-15T03:54:12.009 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2753301561}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3301571177}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2035042435}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1892203215}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.944+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=329014810}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.010 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=657769981}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1523290877}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1477023195}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2649580285}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4221487343}) 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.011 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1836398403}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144543603}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1331070382}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=494697227}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1251740015}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.012 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3735337521}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3009815146}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=734755109}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3931191778}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3253470344}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2438378682}) 2024-09-15T03:54:12.013 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.948+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=140505018}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2307506765}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3412580277}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=240767112}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2355636158}) 2024-09-15T03:54:12.014 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=393898920}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=723370088}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1421524962}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1952687335}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3567783188}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.015 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=165144340}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2539639094}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=866697976}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2255834940}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2182692682}) 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.016 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=413637580}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2458395061}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3006449856}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.952+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4165861247}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=374675647}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2301342704}) 2024-09-15T03:54:12.017 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4224173917}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1284927142}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2008514441}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3382341555}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.018 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3229633814}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3195044418}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3031697039}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2065178429}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3439795776}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1853390530}) 2024-09-15T03:54:12.019 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1198571978}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2424618878}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3218137616}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4054819535}) 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.020 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342494232}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.956+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1901784997}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2399427770}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=243773973}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231342256}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3728048055}) 2024-09-15T03:54:12.021 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1978583682}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2293148754}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=172129675}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1985608739}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.960+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=50330287}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.022 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2240056909}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3386970802}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1711949389}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2614486617}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4111616450}) 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.023 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4245067919}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3433761297}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1457935094}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3423466768}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2113734183}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2495596052}) 2024-09-15T03:54:12.024 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1955773894}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=653877184}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2059017219}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2971718453}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.025 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1980481340}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1792836628}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.964+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4188570407}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1949790791}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=68845460}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1693398902}) 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.026 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2980851249}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1524952991}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3040133885}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1596248730}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=906268342}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.968+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=900718335}) 2024-09-15T03:54:12.027 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=79,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=1453902622,mdsmap=3585761405,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355,mon_config_key=551709721}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=78,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=3} crc {logm=1258109242,mdsmap=1531151339,mgr=73913752,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=3553786792,mon_config_key=822606717}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=76,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=4} crc {logm=2000844767,mdsmap=3585761405,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355,mon_config_key=3784792071}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=79,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=3734949500,mdsmap=1531151339,mgr=991478979,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355,mon_config_key=551709721}) 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.028 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=79,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=1} crc {logm=494218182,mdsmap=1531151339,mgr=436553390,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355,mon_config_key=551709721}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=79,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=7,mon_config_key=1} crc {logm=276192594,mdsmap=1531151339,mgr=1925365903,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=544794869,mon_config_key=551709721}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=77,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=3} crc {logm=2797714343,mdsmap=1531151339,mgr=991478979,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355,mon_config_key=822606717}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=836579143,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.972+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5} crc {logm=596541046,mdsmap=1531151339,mgr=2796602762,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=745644355,mon_config_key=733955277}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=73,osdmap=12} crc {mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=302054337,osdmap=2002743284}) 2024-09-15T03:54:12.029 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=79,osdmap=7} crc {mon_config_key=1595646671,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=2366878485,osdmap=2357866153}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=78,osdmap=8} crc {mon_config_key=733955277,monmap=4273209975,osd_metadata=2459566930,osd_pg_creating=4225289012,osd_snap=4030532000,osdmap=1460512020}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=79,osdmap=7} crc {mon_config_key=733955277,monmap=4273209975,osd_metadata=2196848756,osd_pg_creating=4225289012,osd_snap=1448179153,osdmap=550585112}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=7} crc {mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=824224325,osdmap=413241351}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.030 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=80,osdmap=5} crc {mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=2062312923,osdmap=750226957}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=4,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=79,osdmap=8} crc {mon_config_key=2113949861,monmap=2173287178,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=2122171264,osdmap=1460512020}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=78,osdmap=9} crc {mon_config_key=733955277,monmap=4181778465,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=972052555,osdmap=3133394916}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.976+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=77,osdmap=9} crc {mon_config_key=1595646671,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4225289012,osd_snap=1748153037,osdmap=3592949193}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2938294782}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1912477198}) 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.031 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4273247037}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2690188639}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4216734428}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4245556268}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.032 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2697467827}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3772081558}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3725866257}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1781481938}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4275021921}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2577373059}) 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.033 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1313716817}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1786748067}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2757970697}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.980+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.984+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.984+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3233447197}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.984+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.984+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=805872159}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.984+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3029298024}) 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.034 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3208672700}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2808606359}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2532463649}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1355109107}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3175839310}) 2024-09-15T03:54:12.035 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1286719689}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=587055592}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3873536456}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1287754282}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=16} crc {osdmap=2525376961}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=14} crc {osdmap=766734684}) 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.036 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=16} crc {osdmap=2525376961}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=15} crc {osdmap=1466393166}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=16} crc {osdmap=2525376961}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=15} crc {osdmap=1498582109}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=16} crc {osdmap=2525376961}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=15} crc {osdmap=1901229518}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=16} crc {osdmap=2525376961}) 2024-09-15T03:54:12.037 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:54:11.988+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=15} crc {osdmap=1466393166}) 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1951: finishing write tid 1 to smithi12223906-23 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1951: finishing write tid 2 to smithi12223906-23 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1951: finishing write tid 3 to smithi12223906-23 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1951: finishing write tid 4 to smithi12223906-23 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1951: finishing write tid 5 to smithi12223906-23 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 1086 (ObjNum 706 snap 215 seq_num 706) dirty exists 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1951: left oid 23 (ObjNum 706 snap 215 seq_num 706) 2024-09-15T03:54:12.058 INFO:tasks.rados.rados.0.smithi122.stdout:1952: finishing write tid 1 to smithi12223906-8 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1952: finishing write tid 2 to smithi12223906-8 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1952: finishing write tid 3 to smithi12223906-8 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1952: finishing write tid 4 to smithi12223906-8 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1952: finishing write tid 5 to smithi12223906-8 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 926 (ObjNum 707 snap 215 seq_num 707) dirty exists 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1952: left oid 8 (ObjNum 707 snap 215 seq_num 707) 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1950: done (3 left) 2024-09-15T03:54:12.059 INFO:tasks.rados.rados.0.smithi122.stdout:1951: done (2 left) 2024-09-15T03:54:12.060 INFO:tasks.rados.rados.0.smithi122.stdout:1952: done (1 left) 2024-09-15T03:54:12.060 INFO:tasks.rados.rados.0.smithi122.stdout:1953: done (0 left) 2024-09-15T03:54:12.060 INFO:tasks.rados.rados.0.smithi122.stdout:1954: snap_create 2024-09-15T03:54:12.785 INFO:tasks.rados.rados.0.smithi122.stdout:1954: done (0 left) 2024-09-15T03:54:12.785 INFO:tasks.rados.rados.0.smithi122.stdout:1955: read oid 10 snap -1 2024-09-15T03:54:12.785 INFO:tasks.rados.rados.0.smithi122.stdout:1955: expect (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:54:12.795 INFO:tasks.rados.rados.0.smithi122.stdout:1956: write oid 42 current snap is 216 2024-09-15T03:54:12.795 INFO:tasks.rados.rados.0.smithi122.stdout:1956: seq_num 709 ranges {716163=572370,2039145=468247,3203843=626563} 2024-09-15T03:54:12.798 INFO:tasks.rados.rados.0.smithi122.stdout:1956: writing smithi12223906-42 from 716163 to 1288533 tid 1 2024-09-15T03:54:12.805 INFO:tasks.rados.rados.0.smithi122.stdout:1956: writing smithi12223906-42 from 2039145 to 2507392 tid 2 2024-09-15T03:54:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:1956: writing smithi12223906-42 from 3203843 to 3830406 tid 3 2024-09-15T03:54:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:1957: snap_create 2024-09-15T03:54:12.814 INFO:tasks.rados.rados.0.smithi122.stdout:1955: expect (ObjNum 635 snap 195 seq_num 635) 2024-09-15T03:54:12.924 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:54:12.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: get_monmap_and_config 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: monmap: 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:13.036783+0000 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:13.036783+0000 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.039 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.040 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 1 Processor -- start 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 1 -- start start 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: init 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: _add_conns ranks=[0,3,4,6,8,1,7,2,5] 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient(hunting): picked mon.noname-a con 0x7f9398063c20 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:13.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient(hunting): picked mon.noname-g con 0x7f939809aca0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient(hunting): picked mon.noname-c con 0x7f939809e100 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939804ca10 con 0x7f9398063c20 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980696a0 con 0x7f939809aca0 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980687d0 con 0x7f939809e100 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient(hunting): authenticate will time out at 1397.466225s 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939e1d5700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9398063c20 0x7f939809d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39776/0 (socket says 172.21.15.26:39776) 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939e1d5700 1 -- 172.21.15.26:0/393298894 learned_addr learned my addr 172.21.15.26:0/393298894 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054275348 0 0) 0x7f939804ca10 con 0x7f9398063c20 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): my global_id is 194175 2024-09-15T03:54:13.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9384003570 con 0x7f9398063c20 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481598045 0 0) 0x7f93980687d0 con 0x7f939809e100 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): my global_id is 184459 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f939804ca10 con 0x7f939809e100 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177815176 0 0) 0x7f93980696a0 con 0x7f939809aca0 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient(hunting): my global_id is 194187 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93980687d0 con 0x7f939809aca0 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365696648 0 0) 0x7f9384003570 con 0x7f9398063c20 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 >> v1:172.21.15.26:6791/0 conn(0x7f939809e100 legacy=0x7f939813b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 >> v1:172.21.15.122:6790/0 conn(0x7f939809aca0 legacy=0x7f9398137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 monclient: found mon.noname-a 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980a0250 con 0x7f9398063c20 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 5 monclient: authenticate success, global_id 194175 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f93880017b0 con 0x7f9398063c20 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: dump: 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.049 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9388001a20 con 0x7f9398063c20 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 1 -- 172.21.15.26:0/393298894 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9388004c30 con 0x7f9398063c20 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939f1d7700 4 set_mon_vals no callback set 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.034+0000 7f939d1d3700 10 monclient: dump: 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 10 monclient: shutdown 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 -- 172.21.15.26:0/393298894 >> v1:172.21.15.26:6789/0 conn(0x7f9398063c20 legacy=0x7f939809d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 -- 172.21.15.26:0/393298894 shutdown_connections 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 -- 172.21.15.26:0/393298894 >> 172.21.15.26:0/393298894 conn(0x7f93980a4950 msgr2=0x7f9398063340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 -- 172.21.15.26:0/393298894 shutdown_connections 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 -- 172.21.15.26:0/393298894 wait complete. 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 10 monclient: monmap: 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 Processor -- start 2024-09-15T03:54:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 1 -- start start 2024-09-15T03:54:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.038+0000 7f939f1d7700 10 monclient: init 2024-09-15T03:54:13.055 INFO:tasks.rados.rados.0.smithi122.stdout:1956: finishing write tid 1 to smithi12223906-42 2024-09-15T03:54:13.055 INFO:tasks.rados.rados.0.smithi122.stdout:1956: finishing write tid 2 to smithi12223906-42 2024-09-15T03:54:13.056 INFO:tasks.rados.rados.0.smithi122.stdout:1956: finishing write tid 3 to smithi12223906-42 2024-09-15T03:54:13.056 INFO:tasks.rados.rados.0.smithi122.stdout:1956: finishing write tid 4 to smithi12223906-42 2024-09-15T03:54:13.056 INFO:tasks.rados.rados.0.smithi122.stdout:1956: finishing write tid 5 to smithi12223906-42 2024-09-15T03:54:13.056 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1114 (ObjNum 709 snap 216 seq_num 709) dirty exists 2024-09-15T03:54:13.056 INFO:tasks.rados.rados.0.smithi122.stdout:1956: left oid 42 (ObjNum 709 snap 216 seq_num 709) 2024-09-15T03:54:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient: _add_conns ranks=[4,5,6,8,7,3,1,2,0] 2024-09-15T03:54:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient(hunting): picked mon.noname-c con 0x7f939809aca0 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient(hunting): picked mon.noname-h con 0x7f939809e100 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient(hunting): picked mon.noname-d con 0x7f9398120a10 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980678b0 con 0x7f939809aca0 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939813e3e0 con 0x7f939809e100 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939806e120 con 0x7f9398120a10 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient(hunting): authenticate will time out at 1397.472822s 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939e1d5700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f939809aca0 0x7f939813dcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55194/0 (socket says 172.21.15.26:55194) 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939e1d5700 1 -- 172.21.15.26:0/2069673876 learned_addr learned my addr 172.21.15.26:0/2069673876 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251770278 0 0) 0x7f93980678b0 con 0x7f939809aca0 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): my global_id is 184468 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f937c003570 con 0x7f939809aca0 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234707190 0 0) 0x7f939806e120 con 0x7f9398120a10 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): my global_id is 194199 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93980678b0 con 0x7f9398120a10 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342652760 0 0) 0x7f939813e3e0 con 0x7f939809e100 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): my global_id is 5441 2024-09-15T03:54:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f939806e120 con 0x7f939809e100 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3410158198 0 0) 0x7f937c003570 con 0x7f939809aca0 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f939813e3e0 con 0x7f939809aca0 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9388001830 con 0x7f939809aca0 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2950187852 0 0) 0x7f93980678b0 con 0x7f9398120a10 2024-09-15T03:54:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f937c003570 con 0x7f9398120a10 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4026026911 0 0) 0x7f939806e120 con 0x7f939809e100 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f93980678b0 con 0x7f939809e100 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9390001980 con 0x7f9398120a10 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9394001d50 con 0x7f939809e100 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.067 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2773490788 0 0) 0x7f939813e3e0 con 0x7f939809aca0 2024-09-15T03:54:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 >> v1:172.21.15.26:6792/0 conn(0x7f9398120a10 legacy=0x7f939814a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 >> v1:172.21.15.122:6791/0 conn(0x7f939809e100 legacy=0x7f9398146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 monclient: found mon.c 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398122d70 con 0x7f939809aca0 2024-09-15T03:54:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 5 monclient: authenticate success, global_id 184468 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient: _renew_subs 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9398123050 con 0x7f939809aca0 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient: _renew_subs 2024-09-15T03:54:13.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f939813f280 con 0x7f939809aca0 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f93880011d0 con 0x7f939809aca0 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9388005230 con 0x7f939809aca0 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939c9d2700 4 set_mon_vals no callback set 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f938e7fc700 10 monclient: dump: 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.071 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.042+0000 7f939f1d7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.046+0000 7f939f1d7700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.046+0000 7f939f1d7700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.046+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93800050b0 con 0x7f939809aca0 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.046+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f938800fbf0 con 0x7f939809aca0 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.046+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(439..439 src has 224..439) v4 ==== 3911+0+0 (unknown 3382902949 0 0) 0x7f93880440a0 con 0x7f939809aca0 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.046+0000 7f939d9d4700 10 monclient: get_auth_request con 0x7f937c0381a0 auth_method 0 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.054+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f9388014f30 con 0x7f939809aca0 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.054+0000 7f938e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.054+0000 7f938e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:54:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f93800058e0 con 0x7f939809aca0 2024-09-15T03:54:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f938e7fc700 1 -- 172.21.15.26:0/2069673876 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 1874299187) 0x7f9388014930 con 0x7f939809aca0 2024-09-15T03:54:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f938e7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:13.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f938e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:13.287 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":72,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:54:13.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 >> v1:172.21.15.122:6812/20895 conn(0x7f937c0381a0 legacy=0x7f937c03a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 10 monclient: shutdown 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 >> v1:172.21.15.26:6791/0 conn(0x7f939809aca0 legacy=0x7f939813dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 shutdown_connections 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.282+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 >> 172.21.15.26:0/2069673876 conn(0x7f93980a4950 msgr2=0x7f93980a10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.286+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 shutdown_connections 2024-09-15T03:54:13.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.286+0000 7f939f1d7700 1 -- 172.21.15.26:0/2069673876 wait complete. 2024-09-15T03:54:13.301 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:54:13.301 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:54:13.301 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:54:13.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:54:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient: get_monmap_and_config 2024-09-15T03:54:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient: monmap: 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:13.410497+0000 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:13.410497+0000 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.414 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 1 Processor -- start 2024-09-15T03:54:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 1 -- start start 2024-09-15T03:54:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient: init 2024-09-15T03:54:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient: _add_conns ranks=[6,1,8,7,4,0,3,2,5] 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient(hunting): picked mon.noname-d con 0x7fc778132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient(hunting): picked mon.noname-f con 0x7fc77809faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient(hunting): picked mon.noname-e con 0x7fc778138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77804ca10 con 0x7fc77809faa0 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77809d5d0 con 0x7fc778132b40 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc778133620 con 0x7fc778138d40 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.406+0000 7fc77f10e700 10 monclient(hunting): authenticate will time out at 1397.839879s 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77e10c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fc778132b40 0x7fc778132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60336/0 (socket says 172.21.15.26:60336) 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77e10c700 1 -- 172.21.15.26:0/2918261399 learned_addr learned my addr 172.21.15.26:0/2918261399 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77e90d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fc778138d40 0x7fc77813b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51694/0 (socket says 172.21.15.26:51694) 2024-09-15T03:54:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713865062 0 0) 0x7fc77809d5d0 con 0x7fc778132b40 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): my global_id is 194208 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc760003570 con 0x7fc778132b40 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588831868 0 0) 0x7fc778133620 con 0x7fc778138d40 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): my global_id is 134900 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc77809d5d0 con 0x7fc778138d40 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369504298 0 0) 0x7fc77804ca10 con 0x7fc77809faa0 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient(hunting): my global_id is 184402 2024-09-15T03:54:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc778133620 con 0x7fc77809faa0 2024-09-15T03:54:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780563462 0 0) 0x7fc77809d5d0 con 0x7fc778138d40 2024-09-15T03:54:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 >> v1:172.21.15.26:6792/0 conn(0x7fc778132b40 legacy=0x7fc778132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 >> v1:172.21.15.122:6789/0 conn(0x7fc77809faa0 legacy=0x7fc77809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 monclient: found mon.noname-e 2024-09-15T03:54:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7780a0cc0 con 0x7fc778138d40 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc774001790 con 0x7fc778138d40 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 5 monclient: authenticate success, global_id 134900 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: dump: 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.426 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc774001a40 con 0x7fc778138d40 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 1 -- 172.21.15.26:0/2918261399 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc774001c30 con 0x7fc778138d40 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 4 set_mon_vals no callback set 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77d10a700 10 monclient: dump: 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 10 monclient: shutdown 2024-09-15T03:54:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 -- 172.21.15.26:0/2918261399 >> v1:172.21.15.26:6793/0 conn(0x7fc778138d40 legacy=0x7fc77813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 -- 172.21.15.26:0/2918261399 shutdown_connections 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 -- 172.21.15.26:0/2918261399 >> 172.21.15.26:0/2918261399 conn(0x7fc77806e370 msgr2=0x7fc7780a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 -- 172.21.15.26:0/2918261399 shutdown_connections 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 -- 172.21.15.26:0/2918261399 wait complete. 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 10 monclient: monmap: 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 Processor -- start 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 1 -- start start 2024-09-15T03:54:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.410+0000 7fc77f10e700 10 monclient: init 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient: _add_conns ranks=[5,0,3,8,1,7,6,2,4] 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient(hunting): picked mon.noname-h con 0x7fc77809faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient(hunting): picked mon.noname-a con 0x7fc778132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient(hunting): picked mon.noname-g con 0x7fc778138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77809ccc0 con 0x7fc778132b40 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77809cfa0 con 0x7fc778138d40 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77809c830 con 0x7fc77809faa0 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient(hunting): authenticate will time out at 1397.844623s 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77d90b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc778132b40 0x7fc7781428b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39786/0 (socket says 172.21.15.26:39786) 2024-09-15T03:54:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77d90b700 1 -- 172.21.15.26:0/706661438 learned_addr learned my addr 172.21.15.26:0/706661438 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61708442 0 0) 0x7fc77809ccc0 con 0x7fc778132b40 2024-09-15T03:54:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): my global_id is 194184 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc764003570 con 0x7fc778132b40 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280829679 0 0) 0x7fc77809cfa0 con 0x7fc778138d40 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): my global_id is 194196 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc77809ccc0 con 0x7fc778138d40 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898405966 0 0) 0x7fc77809c830 con 0x7fc77809faa0 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): my global_id is 5450 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc77809cfa0 con 0x7fc77809faa0 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 375055742 0 0) 0x7fc764003570 con 0x7fc778132b40 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc77809c830 con 0x7fc778132b40 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc770001b20 con 0x7fc778132b40 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 28337075 0 0) 0x7fc77809c830 con 0x7fc778132b40 2024-09-15T03:54:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.122:6791/0 conn(0x7fc77809faa0 legacy=0x7fc7781421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.122:6790/0 conn(0x7fc778138d40 legacy=0x7fc77814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 monclient: found mon.a 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778142fc0 con 0x7fc778132b40 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 5 monclient: authenticate success, global_id 194184 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient: _renew_subs 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc778133c70 con 0x7fc778132b40 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient: _renew_subs 2024-09-15T03:54:13.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77809c070 con 0x7fc778132b40 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc770002040 con 0x7fc778132b40 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc770002570 con 0x7fc778132b40 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77c909700 4 set_mon_vals no callback set 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 10 monclient: dump: 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.441 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc7700026d0 con 0x7fc778132b40 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(439..439 src has 224..439) v4 ==== 3911+0+0 (unknown 3382902949 0 0) 0x7fc770043ec0 con 0x7fc778132b40 2024-09-15T03:54:13.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc758006590 con 0x7fc758002840 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.414+0000 7fc77e10c700 10 monclient: get_auth_request con 0x7fc764037650 auth_method 0 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350293520 0 0) 0x7fc758006590 con 0x7fc758002840 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient(hunting): my global_id is 194193 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc77809c830 con 0x7fc758002840 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2850873398 0 0) 0x7fc77809c830 con 0x7fc758002840 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc758006590 con 0x7fc758002840 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc774001210 con 0x7fc758002840 2024-09-15T03:54:13.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient: dump: 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.444 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1206627753 0 0) 0x7fc758006590 con 0x7fc758002840 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc75800a130 con 0x7fc758002840 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc774007120 con 0x7fc758002840 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.26:6789/0 conn(0x7fc758002840 legacy=0x7fc758004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.418+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.26:6789/0 conn(0x7fc758002840 legacy=0x7fc758004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc758009640 con 0x7fc7580090b0 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802404908 0 0) 0x7fc758009640 con 0x7fc7580090b0 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient(hunting): my global_id is 194202 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc758006590 con 0x7fc7580090b0 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 808569573 0 0) 0x7fc758006590 con 0x7fc7580090b0 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc758009640 con 0x7fc7580090b0 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc7740058c0 con 0x7fc7580090b0 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient: dump: 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.446 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.447 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 969279927 0 0) 0x7fc758009640 con 0x7fc7580090b0 2024-09-15T03:54:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc7580098f0 con 0x7fc7580090b0 2024-09-15T03:54:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 2090618372) 0x7fc774006e10 con 0x7fc7580090b0 2024-09-15T03:54:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:13.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.26:6789/0 conn(0x7fc7580090b0 legacy=0x7fc758008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc76e7fc700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.26:6789/0 conn(0x7fc7580090b0 legacy=0x7fc758008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:13.449 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:13.450 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:13.451 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.452 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:13.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.454 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.456 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.457 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.458 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.459 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:13.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.461 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:13.462 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.463 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.465 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.466 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.468 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.469 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:13.469 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.469 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.469 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:13.469 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:13.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.122:6812/20895 conn(0x7fc764037650 legacy=0x7fc764039b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 10 monclient: shutdown 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 >> v1:172.21.15.26:6789/0 conn(0x7fc778132b40 legacy=0x7fc7781428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77e90d700 1 -- 172.21.15.26:0/706661438 reap_dead start 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 shutdown_connections 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 >> 172.21.15.26:0/706661438 conn(0x7fc77806e370 msgr2=0x7fc7780a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 shutdown_connections 2024-09-15T03:54:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.430+0000 7fc77f10e700 1 -- 172.21.15.26:0/706661438 wait complete. 2024-09-15T03:54:13.471 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:13.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:54:13.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:54:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient: get_monmap_and_config 2024-09-15T03:54:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient: monmap: 2024-09-15T03:54:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:13.586346+0000 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:13.586346+0000 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.590 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 1 Processor -- start 2024-09-15T03:54:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 1 -- start start 2024-09-15T03:54:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient: init 2024-09-15T03:54:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient: _add_conns ranks=[3,4,1,8,7,6,2,5,0] 2024-09-15T03:54:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient(hunting): picked mon.noname-g con 0x7fb7dc063340 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient(hunting): picked mon.noname-c con 0x7fb7dc063a10 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient(hunting): picked mon.noname-f con 0x7fb7dc133110 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc04ca10 con 0x7fb7dc133110 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc0a0ad0 con 0x7fb7dc063340 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc09f4f0 con 0x7fb7dc063a10 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.582+0000 7fb7e0a3b700 10 monclient(hunting): authenticate will time out at 1398.015728s 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d9d9b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fb7dc063a10 0x7fb7dc137f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55202/0 (socket says 172.21.15.26:55202) 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d9d9b700 1 -- 172.21.15.26:0/2400916969 learned_addr learned my addr 172.21.15.26:0/2400916969 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439517141 0 0) 0x7fb7dc0a0ad0 con 0x7fb7dc063340 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): my global_id is 194205 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c4003570 con 0x7fb7dc063340 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6820971 0 0) 0x7fb7dc09f4f0 con 0x7fb7dc063a10 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): my global_id is 184477 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc0a0ad0 con 0x7fb7dc063a10 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115402910 0 0) 0x7fb7dc04ca10 con 0x7fb7dc133110 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient(hunting): my global_id is 184411 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc09f4f0 con 0x7fb7dc133110 2024-09-15T03:54:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992910670 0 0) 0x7fb7dc0a0ad0 con 0x7fb7dc063a10 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 >> v1:172.21.15.122:6790/0 conn(0x7fb7dc063340 legacy=0x7fb7dc0991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 >> v1:172.21.15.122:6789/0 conn(0x7fb7dc133110 legacy=0x7fb7dc13b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 monclient: found mon.noname-c 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc063710 con 0x7fb7dc063a10 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 5 monclient: authenticate success, global_id 184477 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb7d00016c0 con 0x7fb7dc063a10 2024-09-15T03:54:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: dump: 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7d0001970 con 0x7fb7dc063a10 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 4 set_mon_vals no callback set 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 1 -- 172.21.15.26:0/2400916969 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb7d00048f0 con 0x7fb7dc063a10 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7d959a700 10 monclient: dump: 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 10 monclient: shutdown 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/2400916969 >> v1:172.21.15.26:6791/0 conn(0x7fb7dc063a10 legacy=0x7fb7dc137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/2400916969 shutdown_connections 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/2400916969 >> 172.21.15.26:0/2400916969 conn(0x7fb7dc0a3cb0 msgr2=0x7fb7dc09fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/2400916969 shutdown_connections 2024-09-15T03:54:13.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/2400916969 wait complete. 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.586+0000 7fb7e0a3b700 10 monclient: monmap: 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 Processor -- start 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 -- start start 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: init 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: _add_conns ranks=[1,6,5,0,8,2,7,3,4] 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient(hunting): picked mon.noname-f con 0x7fb7dc063340 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient(hunting): picked mon.noname-d con 0x7fb7dc063a10 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient(hunting): picked mon.noname-h con 0x7fb7dc133110 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc09ecb0 con 0x7fb7dc063340 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc099f20 con 0x7fb7dc133110 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc11fdf0 con 0x7fb7dc063a10 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient(hunting): authenticate will time out at 1398.021113s 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7d9d9b700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fb7dc063a10 0x7fb7dc146fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60350/0 (socket says 172.21.15.26:60350) 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7d9d9b700 1 -- 172.21.15.26:0/86199935 learned_addr learned my addr 172.21.15.26:0/86199935 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361403045 0 0) 0x7fb7dc11fdf0 con 0x7fb7dc063a10 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): my global_id is 194217 2024-09-15T03:54:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7bc003570 con 0x7fb7dc063a10 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138306584 0 0) 0x7fb7dc09ecb0 con 0x7fb7dc063340 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): my global_id is 184420 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc11fdf0 con 0x7fb7dc063340 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2397157351 0 0) 0x7fb7bc003570 con 0x7fb7dc063a10 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7dc09ecb0 con 0x7fb7dc063a10 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303469818 0 0) 0x7fb7dc099f20 con 0x7fb7dc133110 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): my global_id is 5459 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7bc003570 con 0x7fb7dc133110 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb7d0001680 con 0x7fb7dc063a10 2024-09-15T03:54:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3931403676 0 0) 0x7fb7dc09ecb0 con 0x7fb7dc063a10 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.122:6791/0 conn(0x7fb7dc133110 legacy=0x7fb7dc14a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.122:6789/0 conn(0x7fb7dc063340 legacy=0x7fb7dc141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 monclient: found mon.d 2024-09-15T03:54:13.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc120d50 con 0x7fb7dc063a10 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 5 monclient: authenticate success, global_id 194217 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: _renew_subs 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc11f220 con 0x7fb7dc063a10 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: _renew_subs 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc1220a0 con 0x7fb7dc063a10 2024-09-15T03:54:13.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb7d0004e60 con 0x7fb7dc063a10 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb7d0005310 con 0x7fb7dc063a10 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7d8d99700 4 set_mon_vals no callback set 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.590+0000 7fb7ca7fc700 10 monclient: dump: 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.612 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.613 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7e0a3b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb7d000fcd0 con 0x7fb7dc063a10 2024-09-15T03:54:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb7c0006590 con 0x7fb7c0002840 2024-09-15T03:54:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(439..439 src has 224..439) v4 ==== 3911+0+0 (unknown 3382902949 0 0) 0x7fb7d00440d0 con 0x7fb7dc063a10 2024-09-15T03:54:13.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7dad9d700 10 monclient: get_auth_request con 0x7fb7bc0377c0 auth_method 0 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655665358 0 0) 0x7fb7c0006590 con 0x7fb7c0002840 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient(hunting): my global_id is 134966 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc09ecb0 con 0x7fb7c0002840 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4064281254 0 0) 0x7fb7dc09ecb0 con 0x7fb7c0002840 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7c0006590 con 0x7fb7c0002840 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb7d40025e0 con 0x7fb7c0002840 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient: dump: 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.617 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.618 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3457253621 0 0) 0x7fb7c0006590 con 0x7fb7c0002840 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7c000a130 con 0x7fb7c0002840 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb7d4002ba0 con 0x7fb7c0002840 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.26:6790/0 conn(0x7fb7c0002840 legacy=0x7fb7c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.594+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.26:6790/0 conn(0x7fb7c0002840 legacy=0x7fb7c0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb7c0009640 con 0x7fb7c00090b0 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672717636 0 0) 0x7fb7c0009640 con 0x7fb7c00090b0 2024-09-15T03:54:13.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient(hunting): my global_id is 134975 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c0006590 con 0x7fb7c00090b0 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4285194105 0 0) 0x7fb7c0006590 con 0x7fb7c00090b0 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7c0009640 con 0x7fb7c00090b0 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb7d40034a0 con 0x7fb7c00090b0 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient: dump: 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.620 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.621 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 474345446 0 0) 0x7fb7c0009640 con 0x7fb7c00090b0 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb7c00098f0 con 0x7fb7c00090b0 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 173509234) 0x7fb7d4005840 con 0x7fb7c00090b0 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.26:6790/0 conn(0x7fb7c00090b0 legacy=0x7fb7c0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7ca7fc700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.26:6790/0 conn(0x7fb7c00090b0 legacy=0x7fb7c0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:13.622 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:13.623 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.624 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.625 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.626 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.627 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.628 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.630 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.632 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:13.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.635 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.637 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.641 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:13.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.643 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.644 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:13.644 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:13.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.122:6812/20895 conn(0x7fb7bc0377c0 legacy=0x7fb7bc039c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 10 monclient: shutdown 2024-09-15T03:54:13.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 >> v1:172.21.15.26:6792/0 conn(0x7fb7dc063a10 legacy=0x7fb7dc146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7dad9d700 1 -- 172.21.15.26:0/86199935 reap_dead start 2024-09-15T03:54:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7dad9d700 1 -- 172.21.15.26:0/86199935 reap_dead start 2024-09-15T03:54:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 shutdown_connections 2024-09-15T03:54:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.602+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 >> 172.21.15.26:0/86199935 conn(0x7fb7dc0a3cb0 msgr2=0x7fb7dc0a1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.606+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 shutdown_connections 2024-09-15T03:54:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.606+0000 7fb7e0a3b700 1 -- 172.21.15.26:0/86199935 wait complete. 2024-09-15T03:54:13.646 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:13.646 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:54:13.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:54:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient: get_monmap_and_config 2024-09-15T03:54:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient: monmap: 2024-09-15T03:54:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:13.765252+0000 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:13.765252+0000 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.769 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 1 Processor -- start 2024-09-15T03:54:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 1 -- start start 2024-09-15T03:54:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient: init 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient: _add_conns ranks=[5,7,8,6,3,1,2,4,0] 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient(hunting): picked mon.noname-h con 0x7fce100a4740 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient(hunting): picked mon.noname-i con 0x7fce1009aca0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient(hunting): picked mon.noname-e con 0x7fce10136f40 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1004ca10 con 0x7fce100a4740 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1009c120 con 0x7fce1009aca0 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1009d7a0 con 0x7fce10136f40 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce1749b700 10 monclient(hunting): authenticate will time out at 1398.195144s 2024-09-15T03:54:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce16c9a700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fce10136f40 0x7fce10139320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51700/0 (socket says 172.21.15.26:51700) 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce16c9a700 1 -- 172.21.15.26:0/1890854572 learned_addr learned my addr 172.21.15.26:0/1890854572 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474248791 0 0) 0x7fce1009d7a0 con 0x7fce10136f40 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce15497700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce15497700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.762+0000 7fce15497700 10 monclient(hunting): my global_id is 134909 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdfc003570 con 0x7fce10136f40 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950993047 0 0) 0x7fce1009c120 con 0x7fce1009aca0 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient(hunting): my global_id is 134863 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1009d7a0 con 0x7fce1009aca0 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241794407 0 0) 0x7fce1004ca10 con 0x7fce100a4740 2024-09-15T03:54:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient(hunting): my global_id is 5468 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1009c120 con 0x7fce100a4740 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3516493013 0 0) 0x7fcdfc003570 con 0x7fce10136f40 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 >> v1:172.21.15.122:6792/0 conn(0x7fce1009aca0 legacy=0x7fce10135820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 >> v1:172.21.15.122:6791/0 conn(0x7fce100a4740 legacy=0x7fce1009dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 monclient: found mon.noname-e 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1009b510 con 0x7fce10136f40 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 5 monclient: authenticate success, global_id 134909 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce08001a60 con 0x7fce10136f40 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: dump: 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.777 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce08001cd0 con 0x7fce10136f40 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 4 set_mon_vals no callback set 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 1 -- 172.21.15.26:0/1890854572 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce08004c70 con 0x7fce10136f40 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce15497700 10 monclient: dump: 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 10 monclient: shutdown 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 -- 172.21.15.26:0/1890854572 >> v1:172.21.15.26:6793/0 conn(0x7fce10136f40 legacy=0x7fce10139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 -- 172.21.15.26:0/1890854572 shutdown_connections 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 -- 172.21.15.26:0/1890854572 >> 172.21.15.26:0/1890854572 conn(0x7fce100638b0 msgr2=0x7fce100a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 -- 172.21.15.26:0/1890854572 shutdown_connections 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 -- 172.21.15.26:0/1890854572 wait complete. 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 10 monclient: monmap: 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.781 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 Processor -- start 2024-09-15T03:54:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 1 -- start start 2024-09-15T03:54:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.766+0000 7fce1749b700 10 monclient: init 2024-09-15T03:54:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient: _add_conns ranks=[4,6,3,7,8,1,5,2,0] 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient(hunting): picked mon.noname-c con 0x7fce1009aca0 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient(hunting): picked mon.noname-d con 0x7fce100a4740 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient(hunting): picked mon.noname-g con 0x7fce10136f40 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1006e6a0 con 0x7fce10136f40 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce10133090 con 0x7fce1009aca0 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce16499700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fce1009aca0 0x7fce10097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55212/0 (socket says 172.21.15.26:55212) 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce10098880 con 0x7fce100a4740 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce16499700 1 -- 172.21.15.26:0/1216291194 learned_addr learned my addr 172.21.15.26:0/1216291194 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient(hunting): authenticate will time out at 1398.200999s 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630043741 0 0) 0x7fce10133090 con 0x7fce1009aca0 2024-09-15T03:54:13.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): my global_id is 184486 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf4003570 con 0x7fce1009aca0 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877253222 0 0) 0x7fce10098880 con 0x7fce100a4740 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): my global_id is 194226 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce10133090 con 0x7fce100a4740 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309656205 0 0) 0x7fce1006e6a0 con 0x7fce10136f40 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): my global_id is 194214 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce10098880 con 0x7fce10136f40 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 266623401 0 0) 0x7fce10133090 con 0x7fce100a4740 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce1006e6a0 con 0x7fce100a4740 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce0c001a60 con 0x7fce100a4740 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.788 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:13.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3771381476 0 0) 0x7fcdf4003570 con 0x7fce1009aca0 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce10133090 con 0x7fce1009aca0 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce00002af0 con 0x7fce1009aca0 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 259632161 0 0) 0x7fce1006e6a0 con 0x7fce100a4740 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.26:6791/0 conn(0x7fce1009aca0 legacy=0x7fce10097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.122:6790/0 conn(0x7fce10136f40 legacy=0x7fce101483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 monclient: found mon.d 2024-09-15T03:54:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10136770 con 0x7fce100a4740 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 5 monclient: authenticate success, global_id 194226 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient: _renew_subs 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce10131e20 con 0x7fce100a4740 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient: _renew_subs 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fce10099710 con 0x7fce100a4740 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce0c002d70 con 0x7fce100a4740 2024-09-15T03:54:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce0c0022f0 con 0x7fce100a4740 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce067fc700 10 monclient: dump: 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce14c96700 4 set_mon_vals no callback set 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.770+0000 7fce1749b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.774+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdf8006590 con 0x7fcdf8002840 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.774+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fce0c002450 con 0x7fce100a4740 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.774+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(439..439 src has 224..439) v4 ==== 3911+0+0 (unknown 3382902949 0 0) 0x7fce0c041470 con 0x7fce100a4740 2024-09-15T03:54:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.774+0000 7fce16c9a700 10 monclient: get_auth_request con 0x7fcdf4037c90 auth_method 0 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299347842 0 0) 0x7fcdf8006590 con 0x7fcdf8002840 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient(hunting): my global_id is 184495 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1006e6a0 con 0x7fcdf8002840 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3979535588 0 0) 0x7fce1006e6a0 con 0x7fcdf8002840 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdf8006590 con 0x7fcdf8002840 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce000018a0 con 0x7fcdf8002840 2024-09-15T03:54:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient: dump: 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1781124237 0 0) 0x7fcdf8006590 con 0x7fcdf8002840 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdf800a130 con 0x7fcdf8002840 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fce00001dc0 con 0x7fcdf8002840 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.26:6791/0 conn(0x7fcdf8002840 legacy=0x7fcdf8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.782+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.26:6791/0 conn(0x7fcdf8002840 legacy=0x7fcdf8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.799 INFO:tasks.rados.rados.0.smithi122.stdout:1955: done (2 left) 2024-09-15T03:54:13.799 INFO:tasks.rados.rados.0.smithi122.stdout:1956: done (1 left) 2024-09-15T03:54:13.799 INFO:tasks.rados.rados.0.smithi122.stdout:1957: done (0 left) 2024-09-15T03:54:13.799 INFO:tasks.rados.rados.0.smithi122.stdout:1958: rollback oid 24 current snap is 217 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 204 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:1959: read oid 9 snap -1 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:1959: expect (ObjNum 682 snap 210 seq_num 682) 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:1960: delete oid 19 current snap is 217 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:1960: done (2 left) 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:1961: write (excl) oid 23 current snap is 217 2024-09-15T03:54:13.800 INFO:tasks.rados.rados.0.smithi122.stdout:1961: seq_num 711 ranges {749777=722868,2156109=471916,3258689=649107} 2024-09-15T03:54:13.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.798+0000 7fce1749b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:13.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.798+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcdf8009640 con 0x7fcdf80090b0 2024-09-15T03:54:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585716064 0 0) 0x7fcdf8009640 con 0x7fcdf80090b0 2024-09-15T03:54:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient(hunting): my global_id is 184504 2024-09-15T03:54:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf8006590 con 0x7fcdf80090b0 2024-09-15T03:54:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 496088952 0 0) 0x7fcdf8006590 con 0x7fcdf80090b0 2024-09-15T03:54:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdf8009640 con 0x7fcdf80090b0 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce00005b50 con 0x7fcdf80090b0 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient: dump: 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4095209653 0 0) 0x7fcdf8009640 con 0x7fcdf80090b0 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcdf80098f0 con 0x7fcdf80090b0 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1257909476) 0x7fce00007da0 con 0x7fcdf80090b0 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.26:6791/0 conn(0x7fcdf80090b0 legacy=0x7fcdf8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce067fc700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.26:6791/0 conn(0x7fcdf80090b0 legacy=0x7fcdf8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:13.810 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:13.811 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.812 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:13.813 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:13.814 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:13.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.816 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.817 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.819 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:13.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.822 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.823 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.825 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.826 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.827 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.828 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:13.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:13.830 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.122:6812/20895 conn(0x7fcdf4037c90 legacy=0x7fcdf403a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 10 monclient: shutdown 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 >> v1:172.21.15.26:6792/0 conn(0x7fce100a4740 legacy=0x7fce10098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce16c9a700 1 -- 172.21.15.26:0/1216291194 reap_dead start 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce16c9a700 1 -- 172.21.15.26:0/1216291194 reap_dead start 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 shutdown_connections 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 >> 172.21.15.26:0/1216291194 conn(0x7fce100638b0 msgr2=0x7fce1012e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 shutdown_connections 2024-09-15T03:54:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.802+0000 7fce1749b700 1 -- 172.21.15.26:0/1216291194 wait complete. 2024-09-15T03:54:13.833 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:13.833 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:54:13.833 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:54:13.834 INFO:tasks.rados.rados.0.smithi122.stdout:1961: writing smithi12223906-23 from 749777 to 1472645 tid 1 2024-09-15T03:54:13.834 INFO:tasks.rados.rados.0.smithi122.stdout:1961: writing smithi12223906-23 from 2156109 to 2628025 tid 2 2024-09-15T03:54:13.834 INFO:tasks.rados.rados.0.smithi122.stdout:1961: writing smithi12223906-23 from 3258689 to 3907796 tid 3 2024-09-15T03:54:13.834 INFO:tasks.rados.rados.0.smithi122.stdout:1962: write (excl) oid 31 current snap is 217 2024-09-15T03:54:13.834 INFO:tasks.rados.rados.0.smithi122.stdout:1962: seq_num 712 ranges {766584=798117,2098238=615574,3169759=460379} 2024-09-15T03:54:13.834 INFO:tasks.rados.rados.0.smithi122.stdout:1962: writing smithi12223906-31 from 766584 to 1564701 tid 1 2024-09-15T03:54:13.837 INFO:tasks.rados.rados.0.smithi122.stdout:1962: writing smithi12223906-31 from 2098238 to 2713812 tid 2 2024-09-15T03:54:13.843 INFO:tasks.rados.rados.0.smithi122.stdout:1962: writing smithi12223906-31 from 3169759 to 3630138 tid 3 2024-09-15T03:54:13.843 INFO:tasks.rados.rados.0.smithi122.stdout:1963: delete oid 15 current snap is 217 2024-09-15T03:54:13.843 INFO:tasks.rados.rados.0.smithi122.stdout:1959: expect (ObjNum 682 snap 210 seq_num 682) 2024-09-15T03:54:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.934+0000 7f9f88a18700 10 monclient: get_monmap_and_config 2024-09-15T03:54:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.934+0000 7f9f88a18700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.934+0000 7f9f88a18700 10 monclient: monmap: 2024-09-15T03:54:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.940 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:13.938379+0000 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:13.938379+0000 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.941 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.942 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.934+0000 7f9f88a18700 1 Processor -- start 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.934+0000 7f9f88a18700 1 -- start start 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.934+0000 7f9f88a18700 10 monclient: init 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient: _add_conns ranks=[7,2,1,0,3,4,6,5,8] 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient(hunting): picked mon.noname-i con 0x7f9f840a4840 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient(hunting): picked mon.noname-b con 0x7f9f8409aa90 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient(hunting): picked mon.noname-f con 0x7f9f841392c0 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8404ca10 con 0x7f9f841392c0 2024-09-15T03:54:13.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f84068c10 con 0x7f9f8409aa90 2024-09-15T03:54:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f840a4c10 con 0x7f9f840a4840 2024-09-15T03:54:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient(hunting): authenticate will time out at 1398.369089s 2024-09-15T03:54:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f7bfff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f9f8409aa90 0x7f9f84137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33654/0 (socket says 172.21.15.26:33654) 2024-09-15T03:54:13.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f7bfff700 1 -- 172.21.15.26:0/2230726550 learned_addr learned my addr 172.21.15.26:0/2230726550 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245826727 0 0) 0x7f9f84068c10 con 0x7f9f8409aa90 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): my global_id is 134984 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f70003570 con 0x7f9f8409aa90 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040431565 0 0) 0x7f9f8404ca10 con 0x7f9f841392c0 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): my global_id is 184429 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f84068c10 con 0x7f9f841392c0 2024-09-15T03:54:13.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630311027 0 0) 0x7f9f840a4c10 con 0x7f9f840a4840 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient(hunting): my global_id is 134872 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8404ca10 con 0x7f9f840a4840 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3442820877 0 0) 0x7f9f70003570 con 0x7f9f8409aa90 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 >> v1:172.21.15.122:6792/0 conn(0x7f9f840a4840 legacy=0x7f9f8409db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 >> v1:172.21.15.122:6789/0 conn(0x7f9f841392c0 legacy=0x7f9f8413b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 monclient: found mon.noname-b 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f84068930 con 0x7f9f8409aa90 2024-09-15T03:54:13.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 5 monclient: authenticate success, global_id 134984 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f6c0017b0 con 0x7f9f8409aa90 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: dump: 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.949 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f6c001a90 con 0x7f9f8409aa90 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 1 -- 172.21.15.26:0/2230726550 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f6c004c30 con 0x7f9f8409aa90 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 4 set_mon_vals no callback set 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f81d9b700 10 monclient: dump: 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.951 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 10 monclient: shutdown 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- 172.21.15.26:0/2230726550 >> v1:172.21.15.26:6790/0 conn(0x7f9f8409aa90 legacy=0x7f9f84137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- 172.21.15.26:0/2230726550 shutdown_connections 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- 172.21.15.26:0/2230726550 >> 172.21.15.26:0/2230726550 conn(0x7f9f840639b0 msgr2=0x7f9f840a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- 172.21.15.26:0/2230726550 shutdown_connections 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.938+0000 7f9f88a18700 1 -- 172.21.15.26:0/2230726550 wait complete. 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient: build_initial_monmap 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient: monmap: 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:13.953 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 1 Processor -- start 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 1 -- start start 2024-09-15T03:54:13.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient: init 2024-09-15T03:54:13.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:13.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient: _add_conns ranks=[0,6,7,3,2,1,5,8,4] 2024-09-15T03:54:13.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient(hunting): picked mon.noname-a con 0x7f9f8409aa90 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:13.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient(hunting): picked mon.noname-d con 0x7f9f840a4840 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:13.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient(hunting): picked mon.noname-i con 0x7f9f841392c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8406e2f0 con 0x7f9f8409aa90 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f841389f0 con 0x7f9f840a4840 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8406ce50 con 0x7f9f841392c0 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f88a18700 10 monclient(hunting): authenticate will time out at 1398.374378s 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f8259c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9f8409aa90 0x7f9f841382e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39800/0 (socket says 172.21.15.26:39800) 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f8259c700 1 -- 172.21.15.26:0/1580780998 learned_addr learned my addr 172.21.15.26:0/1580780998 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778476643 0 0) 0x7f9f8406e2f0 con 0x7f9f8409aa90 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): my global_id is 194211 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f64003570 con 0x7f9f8409aa90 2024-09-15T03:54:13.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694386899 0 0) 0x7f9f8406ce50 con 0x7f9f841392c0 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): my global_id is 134881 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8406e2f0 con 0x7f9f841392c0 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838133219 0 0) 0x7f9f841389f0 con 0x7f9f840a4840 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): my global_id is 194235 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8406ce50 con 0x7f9f840a4840 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3916825511 0 0) 0x7f9f64003570 con 0x7f9f8409aa90 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f841389f0 con 0x7f9f8409aa90 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f74002b10 con 0x7f9f8409aa90 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.962 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 745163222 0 0) 0x7f9f8406e2f0 con 0x7f9f841392c0 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f64003570 con 0x7f9f841392c0 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3356539806 0 0) 0x7f9f8406ce50 con 0x7f9f840a4840 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.942+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f8406e2f0 con 0x7f9f840a4840 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f7c001c00 con 0x7f9f841392c0 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.963 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.964 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f6c004b70 con 0x7f9f840a4840 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.965 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2298863718 0 0) 0x7f9f841389f0 con 0x7f9f8409aa90 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.122:6792/0 conn(0x7f9f841392c0 legacy=0x7f9f84149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.26:6792/0 conn(0x7f9f840a4840 legacy=0x7f9f84141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 monclient: found mon.a 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:13.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f84135500 con 0x7f9f8409aa90 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 5 monclient: authenticate success, global_id 194211 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 10 monclient: _renew_subs 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f841357e0 con 0x7f9f8409aa90 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 10 monclient: _renew_subs 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f84142110 con 0x7f9f8409aa90 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9f740016a0 con 0x7f9f8409aa90 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f74001bd0 con 0x7f9f8409aa90 2024-09-15T03:54:13.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f8159a700 4 set_mon_vals no callback set 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: dump: 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.968 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9f74001d30 con 0x7f9f8409aa90 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9f841359d0 con 0x7f9f840a2710 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(440..440 src has 224..440) v4 ==== 3911+0+0 (unknown 3815124011 0 0) 0x7f9f74043ee0 con 0x7f9f8409aa90 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7bfff700 10 monclient: get_auth_request con 0x7f9f64038110 auth_method 0 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742776103 0 0) 0x7f9f841359d0 con 0x7f9f840a2710 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient(hunting): my global_id is 194244 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f841389f0 con 0x7f9f840a2710 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2555428388 0 0) 0x7f9f841389f0 con 0x7f9f840a2710 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f841359d0 con 0x7f9f840a2710 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f7c0026d0 con 0x7f9f840a2710 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: dump: 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.970 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1706753907 0 0) 0x7f9f841359d0 con 0x7f9f840a2710 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f84142930 con 0x7f9f840a2710 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9f7c004a80 con 0x7f9f840a2710 2024-09-15T03:54:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:13.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:13.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.26:6792/0 conn(0x7f9f840a2710 legacy=0x7f9f840a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.946+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.26:6792/0 conn(0x7f9f840a2710 legacy=0x7f9f840a2ae0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9f8404bbf0 con 0x7f9f840a31f0 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475631728 0 0) 0x7f9f8404bbf0 con 0x7f9f840a31f0 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient(hunting): my global_id is 194253 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f841359d0 con 0x7f9f840a31f0 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3695082903 0 0) 0x7f9f841359d0 con 0x7f9f840a31f0 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9f8404bbf0 con 0x7f9f840a31f0 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9f7c003270 con 0x7f9f840a31f0 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient: dump: 2024-09-15T03:54:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1133619784 0 0) 0x7f9f8404bbf0 con 0x7f9f840a31f0 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9f8406f230 con 0x7f9f840a31f0 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3399339162) 0x7f9f7c004830 con 0x7f9f840a31f0 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.26:6792/0 conn(0x7f9f840a31f0 legacy=0x7f9f8402ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f7a7fc700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.26:6792/0 conn(0x7f9f840a31f0 legacy=0x7f9f8402ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:13.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:13.977 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.978 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.979 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:13.980 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:13.981 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:13.982 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:13.983 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.984 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.986 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:13.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.988 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.989 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.990 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:13.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.992 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.994 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:13.995 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.122:6812/20895 conn(0x7f9f64038110 legacy=0x7f9f6403a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 10 monclient: shutdown 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 >> v1:172.21.15.26:6789/0 conn(0x7f9f8409aa90 legacy=0x7f9f841382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f82d9d700 1 -- 172.21.15.26:0/1580780998 reap_dead start 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 shutdown_connections 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 >> 172.21.15.26:0/1580780998 conn(0x7f9f840639b0 msgr2=0x7f9f8406f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 shutdown_connections 2024-09-15T03:54:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:13.958+0000 7f9f88a18700 1 -- 172.21.15.26:0/1580780998 wait complete. 2024-09-15T03:54:13.998 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:13.998 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:54:13.998 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:54:14.061 INFO:tasks.rados.rados.0.smithi122.stdout:1958: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1958: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1958: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 1118 (ObjNum 669 snap 204 seq_num 669) dirty exists 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1961: finishing write tid 1 to smithi12223906-23 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1961: finishing write tid 2 to smithi12223906-23 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1961: finishing write tid 3 to smithi12223906-23 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1961: finishing write tid 4 to smithi12223906-23 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:1961: finishing write tid 5 to smithi12223906-23 2024-09-15T03:54:14.062 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 1095 (ObjNum 711 snap 217 seq_num 711) dirty exists 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1961: left oid 23 (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: finishing write tid 1 to smithi12223906-31 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: finishing write tid 2 to smithi12223906-31 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: finishing write tid 3 to smithi12223906-31 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: finishing write tid 4 to smithi12223906-31 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: finishing write tid 5 to smithi12223906-31 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 799 (ObjNum 712 snap 217 seq_num 712) dirty exists 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: left oid 31 (ObjNum 712 snap 217 seq_num 712) 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1958: done (4 left) 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1959: done (3 left) 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1961: done (2 left) 2024-09-15T03:54:14.063 INFO:tasks.rados.rados.0.smithi122.stdout:1962: done (1 left) 2024-09-15T03:54:14.064 INFO:tasks.rados.rados.0.smithi122.stdout:1963: done (0 left) 2024-09-15T03:54:14.064 INFO:tasks.rados.rados.0.smithi122.stdout:1964: write oid 19 current snap is 217 2024-09-15T03:54:14.064 INFO:tasks.rados.rados.0.smithi122.stdout:1964: seq_num 714 ranges {400198=548615,1415202=584996} 2024-09-15T03:54:14.075 INFO:tasks.rados.rados.0.smithi122.stdout:1964: writing smithi12223906-19 from 400198 to 948813 tid 1 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1964: writing smithi12223906-19 from 1415202 to 2000198 tid 2 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1965: read oid 11 snap -1 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1965: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1966: read oid 4 snap -1 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1966: expect (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1967: read oid 35 snap -1 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1967: expect (ObjNum 616 snap 183 seq_num 616) 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1968: read oid 8 snap -1 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1968: expect (ObjNum 707 snap 215 seq_num 707) 2024-09-15T03:54:14.082 INFO:tasks.rados.rados.0.smithi122.stdout:1969: write (excl) oid 5 current snap is 217 2024-09-15T03:54:14.083 INFO:tasks.rados.rados.0.smithi122.stdout:1969: seq_num 715 ranges {417005=740217,1590037=426968} 2024-09-15T03:54:14.089 INFO:tasks.rados.rados.0.smithi122.stdout:1969: writing smithi12223906-5 from 417005 to 1157222 tid 1 2024-09-15T03:54:14.095 INFO:tasks.rados.rados.0.smithi122.stdout:1969: writing smithi12223906-5 from 1590037 to 2017005 tid 2 2024-09-15T03:54:14.095 INFO:tasks.rados.rados.0.smithi122.stdout:1970: read oid 25 snap 201 2024-09-15T03:54:14.095 INFO:tasks.rados.rados.0.smithi122.stdout:1970: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:54:14.096 INFO:tasks.rados.rados.0.smithi122.stdout:1971: delete oid 34 current snap is 217 2024-09-15T03:54:14.096 INFO:tasks.rados.rados.0.smithi122.stdout:1964: finishing write tid 1 to smithi12223906-19 2024-09-15T03:54:14.096 INFO:tasks.rados.rados.0.smithi122.stdout:1965: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.110+0000 7f160f59e700 10 monclient: get_monmap_and_config 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.110+0000 7f160f59e700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.110+0000 7f160f59e700 10 monclient: monmap: 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:14.115293+0000 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:14.115293+0000 2024-09-15T03:54:14.117 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.118 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 Processor -- start 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- start start 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient: init 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient: _add_conns ranks=[6,3,1,5,0,2,7,4,8] 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient(hunting): picked mon.noname-d con 0x7f161009ff60 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient(hunting): picked mon.noname-g con 0x7f161009dda0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient(hunting): picked mon.noname-f con 0x7f1610136b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161004ca10 con 0x7f1610136b40 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16101314b0 con 0x7f161009dda0 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16100a0350 con 0x7f161009ff60 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient(hunting): authenticate will time out at 1398.545185s 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160e59c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f161009ff60 0x7f1610130da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60394/0 (socket says 172.21.15.26:60394) 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160e59c700 1 -- 172.21.15.26:0/1685631998 learned_addr learned my addr 172.21.15.26:0/1685631998 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5411870 0 0) 0x7f16100a0350 con 0x7f161009ff60 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): my global_id is 194262 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f8003570 con 0x7f161009ff60 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155463432 0 0) 0x7f16101314b0 con 0x7f161009dda0 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): my global_id is 194223 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16100a0350 con 0x7f161009dda0 2024-09-15T03:54:14.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233542742 0 0) 0x7f161004ca10 con 0x7f1610136b40 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient(hunting): my global_id is 184438 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16101314b0 con 0x7f1610136b40 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966258832 0 0) 0x7f15f8003570 con 0x7f161009ff60 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 >> v1:172.21.15.122:6790/0 conn(0x7f161009dda0 legacy=0x7f161009ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 >> v1:172.21.15.122:6789/0 conn(0x7f1610136b40 legacy=0x7f1610138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 monclient: found mon.noname-d 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161009f7d0 con 0x7f161009ff60 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f16000017b0 con 0x7f161009ff60 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 5 monclient: authenticate success, global_id 194262 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: dump: 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.126 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1600001a90 con 0x7f161009ff60 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 1 -- 172.21.15.26:0/1685631998 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1600004c30 con 0x7f161009ff60 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 4 set_mon_vals no callback set 2024-09-15T03:54:14.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160d59a700 10 monclient: dump: 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.128 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 10 monclient: shutdown 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- 172.21.15.26:0/1685631998 >> v1:172.21.15.26:6792/0 conn(0x7f161009ff60 legacy=0x7f1610130da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- 172.21.15.26:0/1685631998 shutdown_connections 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- 172.21.15.26:0/1685631998 >> 172.21.15.26:0/1685631998 conn(0x7f16100a3430 msgr2=0x7f16100a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- 172.21.15.26:0/1685631998 shutdown_connections 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.114+0000 7f160f59e700 1 -- 172.21.15.26:0/1685631998 wait complete. 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: monmap: 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.130 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 Processor -- start 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 -- start start 2024-09-15T03:54:14.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: init 2024-09-15T03:54:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: _add_conns ranks=[3,2,1,4,5,0,7,8,6] 2024-09-15T03:54:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient(hunting): picked mon.noname-g con 0x7f161009dda0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient(hunting): picked mon.noname-b con 0x7f161009ff60 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient(hunting): picked mon.noname-f con 0x7f1610136b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16101363a0 con 0x7f1610136b40 2024-09-15T03:54:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161011b260 con 0x7f161009ff60 2024-09-15T03:54:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161009e7e0 con 0x7f161009dda0 2024-09-15T03:54:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient(hunting): authenticate will time out at 1398.550126s 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160dd9b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f161009ff60 0x7f1610134790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33668/0 (socket says 172.21.15.26:33668) 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160dd9b700 1 -- 172.21.15.26:0/2947578277 learned_addr learned my addr 172.21.15.26:0/2947578277 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829312412 0 0) 0x7f161011b260 con 0x7f161009ff60 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): my global_id is 134993 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f0003570 con 0x7f161009ff60 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298117247 0 0) 0x7f161009e7e0 con 0x7f161009dda0 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): my global_id is 194232 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161011b260 con 0x7f161009dda0 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672294354 0 0) 0x7f16101363a0 con 0x7f1610136b40 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): my global_id is 184447 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f161009e7e0 con 0x7f1610136b40 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1856386129 0 0) 0x7f15f0003570 con 0x7f161009ff60 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16101363a0 con 0x7f161009ff60 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3686956225 0 0) 0x7f161011b260 con 0x7f161009dda0 2024-09-15T03:54:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15f0003570 con 0x7f161009dda0 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1604002b50 con 0x7f161009ff60 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2101172947 0 0) 0x7f161009e7e0 con 0x7f1610136b40 2024-09-15T03:54:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f161011b260 con 0x7f1610136b40 2024-09-15T03:54:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1600004b70 con 0x7f161009dda0 2024-09-15T03:54:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:54:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1608001cc0 con 0x7f1610136b40 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3670938418 0 0) 0x7f16101363a0 con 0x7f161009ff60 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.122:6790/0 conn(0x7f161009dda0 legacy=0x7f1610134080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.122:6789/0 conn(0x7f1610136b40 legacy=0x7f1610134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 monclient: found mon.b 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161006e550 con 0x7f161009ff60 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 5 monclient: authenticate success, global_id 134993 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: _renew_subs 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161011a5a0 con 0x7f161009ff60 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: _renew_subs 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1610135be0 con 0x7f161009ff60 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1604002d40 con 0x7f161009ff60 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1604001a60 con 0x7f161009ff60 2024-09-15T03:54:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fffff700 4 set_mon_vals no callback set 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.118+0000 7f15fe7fc700 10 monclient: dump: 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f160f59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f15f4006590 con 0x7f15f4002840 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1604001c40 con 0x7f161009ff60 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(440..440 src has 224..440) v4 ==== 3911+0+0 (unknown 3815124011 0 0) 0x7f1604044000 con 0x7f161009ff60 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033571301 0 0) 0x7f15f4006590 con 0x7f15f4002840 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient(hunting): my global_id is 134918 2024-09-15T03:54:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16101363a0 con 0x7f15f4002840 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f160ed9d700 10 monclient: get_auth_request con 0x7f15f0038230 auth_method 0 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1377561463 0 0) 0x7f16101363a0 con 0x7f15f4002840 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15f4006590 con 0x7f15f4002840 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1600001250 con 0x7f15f4002840 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient: dump: 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 310232706 0 0) 0x7f15f4006590 con 0x7f15f4002840 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15f400a130 con 0x7f15f4002840 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1600005340 con 0x7f15f4002840 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.26:6793/0 conn(0x7f15f4002840 legacy=0x7f15f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.122+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.26:6793/0 conn(0x7f15f4002840 legacy=0x7f15f4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.130+0000 7f160f59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f15f4009640 con 0x7f15f40090b0 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461392499 0 0) 0x7f15f4009640 con 0x7f15f40090b0 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient(hunting): my global_id is 134927 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f4006590 con 0x7f15f40090b0 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2935097845 0 0) 0x7f15f4006590 con 0x7f15f40090b0 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15f4009640 con 0x7f15f40090b0 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1600007d10 con 0x7f15f40090b0 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient: dump: 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1315145337 0 0) 0x7f15f4009640 con 0x7f15f40090b0 2024-09-15T03:54:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f15f40098f0 con 0x7f15f40090b0 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 4117084199) 0x7f1600008330 con 0x7f15f40090b0 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.26:6793/0 conn(0x7f15f40090b0 legacy=0x7f15f4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f15fe7fc700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.26:6793/0 conn(0x7f15f40090b0 legacy=0x7f15f4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:14.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:14.162 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:54:14.163 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.164 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.165 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:14.166 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.167 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.168 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.170 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:14.171 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.174 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.174 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.174 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.174 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:14.176 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.177 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:14.178 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:14.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.180 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.181 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.182 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.183 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.184 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:14.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:14.186 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.122:6812/20895 conn(0x7f15f0038230 legacy=0x7f15f003a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 10 monclient: shutdown 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 >> v1:172.21.15.26:6790/0 conn(0x7f161009ff60 legacy=0x7f1610134790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160ed9d700 1 -- 172.21.15.26:0/2947578277 reap_dead start 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160ed9d700 1 -- 172.21.15.26:0/2947578277 reap_dead start 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 shutdown_connections 2024-09-15T03:54:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 >> 172.21.15.26:0/2947578277 conn(0x7f16100a3430 msgr2=0x7f161009c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 shutdown_connections 2024-09-15T03:54:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.134+0000 7f160f59e700 1 -- 172.21.15.26:0/2947578277 wait complete. 2024-09-15T03:54:14.189 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:14.189 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:54:14.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:54:14.240 INFO:tasks.rados.rados.0.smithi122.stdout:1964: finishing write tid 2 to smithi12223906-19 2024-09-15T03:54:14.240 INFO:tasks.rados.rados.0.smithi122.stdout:1964: finishing write tid 3 to smithi12223906-19 2024-09-15T03:54:14.240 INFO:tasks.rados.rados.0.smithi122.stdout:1964: finishing write tid 4 to smithi12223906-19 2024-09-15T03:54:14.240 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 794 (ObjNum 714 snap 217 seq_num 714) dirty exists 2024-09-15T03:54:14.241 INFO:tasks.rados.rados.0.smithi122.stdout:1964: left oid 19 (ObjNum 714 snap 217 seq_num 714) 2024-09-15T03:54:14.241 INFO:tasks.rados.rados.0.smithi122.stdout:1967: expect (ObjNum 616 snap 183 seq_num 616) 2024-09-15T03:54:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient: get_monmap_and_config 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient: monmap: 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:14.297830+0000 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:14.297830+0000 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.301 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 1 Processor -- start 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 1 -- start start 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient: init 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient: _add_conns ranks=[0,6,8,3,1,5,2,7,4] 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient(hunting): picked mon.noname-a con 0x7f52280a4920 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient(hunting): picked mon.noname-d con 0x7f5228132dc0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient(hunting): picked mon.noname-e con 0x7f52281338d0 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522804ca10 con 0x7f52280a4920 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522806a1d0 con 0x7f5228132dc0 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5228069300 con 0x7f52281338d0 2024-09-15T03:54:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ee17700 10 monclient(hunting): authenticate will time out at 1398.727162s 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522de15700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52280a4920 0x7f5228068bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39810/0 (socket says 172.21.15.26:39810) 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522e616700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f52281338d0 0x7f522813b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51732/0 (socket says 172.21.15.26:51732) 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522de15700 1 -- 172.21.15.26:0/2132683997 learned_addr learned my addr 172.21.15.26:0/2132683997 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873423407 0 0) 0x7f5228069300 con 0x7f52281338d0 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 10 monclient(hunting): my global_id is 134936 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5214003570 con 0x7f52281338d0 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718603734 0 0) 0x7f522804ca10 con 0x7f52280a4920 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 10 monclient(hunting): my global_id is 194220 2024-09-15T03:54:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.294+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5228069300 con 0x7f52280a4920 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099969662 0 0) 0x7f522806a1d0 con 0x7f5228132dc0 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient(hunting): my global_id is 194271 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f522804ca10 con 0x7f5228132dc0 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372932331 0 0) 0x7f5214003570 con 0x7f52281338d0 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 >> v1:172.21.15.26:6792/0 conn(0x7f5228132dc0 legacy=0x7f52281331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 >> v1:172.21.15.26:6789/0 conn(0x7f52280a4920 legacy=0x7f5228068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 monclient: found mon.noname-e 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:54:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280a4cf0 con 0x7f52281338d0 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5220001b30 con 0x7f52281338d0 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: dump: 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.315 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.316 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 5 monclient: authenticate success, global_id 134936 2024-09-15T03:54:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5220001da0 con 0x7f52281338d0 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 1 -- 172.21.15.26:0/2132683997 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5220001f90 con 0x7f52281338d0 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 4 set_mon_vals no callback set 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ce13700 10 monclient: dump: 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.318 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: shutdown 2024-09-15T03:54:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- 172.21.15.26:0/2132683997 >> v1:172.21.15.26:6793/0 conn(0x7f52281338d0 legacy=0x7f522813b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- 172.21.15.26:0/2132683997 shutdown_connections 2024-09-15T03:54:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- 172.21.15.26:0/2132683997 >> 172.21.15.26:0/2132683997 conn(0x7f5228069a70 msgr2=0x7f52280a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- 172.21.15.26:0/2132683997 shutdown_connections 2024-09-15T03:54:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- 172.21.15.26:0/2132683997 wait complete. 2024-09-15T03:54:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: monmap: 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.321 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 Processor -- start 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- start start 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: init 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient: _add_conns ranks=[3,2,6,4,0,5,7,1,8] 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient(hunting): picked mon.noname-g con 0x7f52280a4920 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient(hunting): picked mon.noname-b con 0x7f5228132dc0 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient(hunting): picked mon.noname-d con 0x7f52281338d0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52280a01b0 con 0x7f5228132dc0 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5228069530 con 0x7f52280a4920 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5228121de0 con 0x7f52281338d0 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522ee17700 10 monclient(hunting): authenticate will time out at 1398.731215s 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522d614700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f5228132dc0 0x7f52281470f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33676/0 (socket says 172.21.15.26:33676) 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f522d614700 1 -- 172.21.15.26:0/391197721 learned_addr learned my addr 172.21.15.26:0/391197721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302903709 0 0) 0x7f52280a01b0 con 0x7f5228132dc0 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f521dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f521dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f521dffb700 10 monclient(hunting): my global_id is 135002 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f520c003570 con 0x7f5228132dc0 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.298+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550617840 0 0) 0x7f5228069530 con 0x7f52280a4920 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): my global_id is 194241 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52280a01b0 con 0x7f52280a4920 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059433839 0 0) 0x7f5228121de0 con 0x7f52281338d0 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): my global_id is 194280 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5228069530 con 0x7f52281338d0 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3781680757 0 0) 0x7f520c003570 con 0x7f5228132dc0 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5228121de0 con 0x7f5228132dc0 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5218001aa0 con 0x7f5228132dc0 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:54:14.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): dump: 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 598931332 0 0) 0x7f52280a01b0 con 0x7f52280a4920 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f520c003570 con 0x7f52280a4920 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 193974094 0 0) 0x7f5228069530 con 0x7f52281338d0 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f52280a01b0 con 0x7f52281338d0 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5224001910 con 0x7f52280a4920 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): dump: 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.327 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1938877312 0 0) 0x7f5228121de0 con 0x7f5228132dc0 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.26:6792/0 conn(0x7f52281338d0 legacy=0x7f522814a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.122:6790/0 conn(0x7f52280a4920 legacy=0x7f5228131c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 monclient: found mon.b 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228120320 con 0x7f5228132dc0 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 5 monclient: authenticate success, global_id 135002 2024-09-15T03:54:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 10 monclient: _renew_subs 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5228120c20 con 0x7f5228132dc0 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 10 monclient: _renew_subs 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f52281342a0 con 0x7f5228132dc0 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5218002d50 con 0x7f5228132dc0 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f52180022b0 con 0x7f5228132dc0 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521ffff700 4 set_mon_vals no callback set 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: dump: 2024-09-15T03:54:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5208006590 con 0x7f5208002840 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5218002410 con 0x7f5228132dc0 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(440..440 src has 224..440) v4 ==== 3911+0+0 (unknown 3815124011 0 0) 0x7f5218043d30 con 0x7f5228132dc0 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542712432 0 0) 0x7f5208006590 con 0x7f5208002840 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient(hunting): my global_id is 184456 2024-09-15T03:54:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5228121de0 con 0x7f5208002840 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f522e616700 10 monclient: get_auth_request con 0x7f520c037f40 auth_method 0 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2719504681 0 0) 0x7f5228121de0 con 0x7f5208002840 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5208006590 con 0x7f5208002840 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5224004b70 con 0x7f5208002840 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.302+0000 7f521dffb700 10 monclient: dump: 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 64386341 0 0) 0x7f5208006590 con 0x7f5208002840 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f520800a130 con 0x7f5208002840 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5224005920 con 0x7f5208002840 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.122:6789/0 conn(0x7f5208002840 legacy=0x7f5208004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.306+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.122:6789/0 conn(0x7f5208002840 legacy=0x7f5208004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.314+0000 7f522ee17700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.314+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5208009640 con 0x7f52080090b0 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267364332 0 0) 0x7f5208009640 con 0x7f52080090b0 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient(hunting): my global_id is 184465 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5208006590 con 0x7f52080090b0 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 196968505 0 0) 0x7f5208006590 con 0x7f52080090b0 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5208009640 con 0x7f52080090b0 2024-09-15T03:54:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5224005440 con 0x7f52080090b0 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient: dump: 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 136821606 0 0) 0x7f5208009640 con 0x7f52080090b0 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f52080098f0 con 0x7f52080090b0 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1054479546) 0x7f5224007cf0 con 0x7f52080090b0 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.122:6789/0 conn(0x7f52080090b0 legacy=0x7f5208008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f521dffb700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.122:6789/0 conn(0x7f52080090b0 legacy=0x7f5208008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:14.342 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.344 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:14.345 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.346 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.347 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.348 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.350 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.351 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.352 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:14.353 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:14.354 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.355 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.356 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.357 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:14.358 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.359 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.360 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:14.361 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.122:6812/20895 conn(0x7f520c037f40 legacy=0x7f520c03a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 10 monclient: shutdown 2024-09-15T03:54:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 >> v1:172.21.15.26:6790/0 conn(0x7f5228132dc0 legacy=0x7f52281470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522e616700 1 -- 172.21.15.26:0/391197721 reap_dead start 2024-09-15T03:54:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522e616700 1 -- 172.21.15.26:0/391197721 reap_dead start 2024-09-15T03:54:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 shutdown_connections 2024-09-15T03:54:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 >> 172.21.15.26:0/391197721 conn(0x7f5228069a70 msgr2=0x7f522806df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 shutdown_connections 2024-09-15T03:54:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.318+0000 7f522ee17700 1 -- 172.21.15.26:0/391197721 wait complete. 2024-09-15T03:54:14.363 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:14.363 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:54:14.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:54:14.366 INFO:tasks.rados.rados.0.smithi122.stdout:1966: expect (ObjNum 650 snap 200 seq_num 650) 2024-09-15T03:54:14.455 INFO:tasks.rados.rados.0.smithi122.stdout:1968: expect (ObjNum 707 snap 215 seq_num 707) 2024-09-15T03:54:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.474+0000 7f06421a0700 10 monclient: get_monmap_and_config 2024-09-15T03:54:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.474+0000 7f06421a0700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.474+0000 7f06421a0700 10 monclient: monmap: 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:14.479620+0000 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:14.479620+0000 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.483 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 1 Processor -- start 2024-09-15T03:54:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 1 -- start start 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient: init 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient: _add_conns ranks=[7,2,6,1,0,8,5,3,4] 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient(hunting): picked mon.noname-i con 0x7f063c0990c0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient(hunting): picked mon.noname-b con 0x7f063c0a1310 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient(hunting): picked mon.noname-d con 0x7f063c1335a0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c04ca10 con 0x7f063c0a1310 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c0a1b20 con 0x7f063c1335a0 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c0a10b0 con 0x7f063c0990c0 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient(hunting): authenticate will time out at 1398.909603s 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063affd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f063c0a1310 0x7f063c099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:33678/0 (socket says 172.21.15.26:33678) 2024-09-15T03:54:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063affd700 1 -- 172.21.15.26:0/456243325 learned_addr learned my addr 172.21.15.26:0/456243325 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063bfff700 1 --1- 172.21.15.26:0/456243325 >> v1:172.21.15.26:6792/0 conn(0x7f063c1335a0 0x7f063c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60420/0 (socket says 172.21.15.26:60420) 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822901796 0 0) 0x7f063c04ca10 con 0x7f063c0a1310 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): my global_id is 135011 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0628003570 con 0x7f063c0a1310 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674639059 0 0) 0x7f063c0a1b20 con 0x7f063c1335a0 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): my global_id is 194289 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c04ca10 con 0x7f063c1335a0 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058648920 0 0) 0x7f063c0a10b0 con 0x7f063c0990c0 2024-09-15T03:54:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient(hunting): my global_id is 134890 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c0a1b20 con 0x7f063c0990c0 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4160186505 0 0) 0x7f063c04ca10 con 0x7f063c1335a0 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 >> v1:172.21.15.122:6792/0 conn(0x7f063c0990c0 legacy=0x7f063c132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 >> v1:172.21.15.26:6790/0 conn(0x7f063c0a1310 legacy=0x7f063c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 monclient: found mon.noname-d 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c099ba0 con 0x7f063c1335a0 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0634001790 con 0x7f063c1335a0 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: dump: 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 5 monclient: authenticate success, global_id 194289 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0634001a30 con 0x7f063c1335a0 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 1 -- 172.21.15.26:0/456243325 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0634004b70 con 0x7f063c1335a0 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f063a7fc700 10 monclient: dump: 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 4 set_mon_vals no callback set 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 10 monclient: shutdown 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.478+0000 7f06421a0700 1 -- 172.21.15.26:0/456243325 >> v1:172.21.15.26:6792/0 conn(0x7f063c1335a0 legacy=0x7f063c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- 172.21.15.26:0/456243325 shutdown_connections 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- 172.21.15.26:0/456243325 >> 172.21.15.26:0/456243325 conn(0x7f063c06e370 msgr2=0x7f063c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- 172.21.15.26:0/456243325 shutdown_connections 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- 172.21.15.26:0/456243325 wait complete. 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: monmap: 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.495 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.495 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 Processor -- start 2024-09-15T03:54:14.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- start start 2024-09-15T03:54:14.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: init 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: _add_conns ranks=[0,4,5,1,3,7,8,6,2] 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient(hunting): picked mon.noname-a con 0x7f063c0990c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient(hunting): picked mon.noname-c con 0x7f063c0a1310 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient(hunting): picked mon.noname-h con 0x7f063c1335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c09a010 con 0x7f063c0990c0 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c122270 con 0x7f063c0a1310 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063c120180 con 0x7f063c1335a0 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient(hunting): authenticate will time out at 1398.914381s 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f063affd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f063c0a1310 0x7f063c147030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55240/0 (socket says 172.21.15.26:55240) 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f063affd700 1 -- 172.21.15.26:0/3882503650 learned_addr learned my addr 172.21.15.26:0/3882503650 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696094369 0 0) 0x7f0624001a70 con 0x7f063c0a1310 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): my global_id is 184513 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061c003570 con 0x7f063c0a1310 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463821911 0 0) 0x7f063c09a010 con 0x7f063c0990c0 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): my global_id is 194229 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0624001a70 con 0x7f063c0990c0 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611654240 0 0) 0x7f063c120180 con 0x7f063c1335a0 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): my global_id is 5477 2024-09-15T03:54:14.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c09a010 con 0x7f063c1335a0 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 635395337 0 0) 0x7f061c003570 con 0x7f063c0a1310 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f063c120180 con 0x7f063c0a1310 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0624001d00 con 0x7f063c0a1310 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): dump: 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.503 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4172263498 0 0) 0x7f0624001a70 con 0x7f063c0990c0 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f061c003570 con 0x7f063c0990c0 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c001d60 con 0x7f063c0990c0 2024-09-15T03:54:14.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): dump: 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.505 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.506 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2193473788 0 0) 0x7f063c09a010 con 0x7f063c1335a0 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0624001a70 con 0x7f063c1335a0 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0634001410 con 0x7f063c1335a0 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient(hunting): dump: 2024-09-15T03:54:14.507 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.508 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 791137743 0 0) 0x7f063c120180 con 0x7f063c0a1310 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.122:6791/0 conn(0x7f063c1335a0 legacy=0x7f063c14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.26:6789/0 conn(0x7f063c0990c0 legacy=0x7f063c131af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 monclient: found mon.c 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c1208d0 con 0x7f063c0a1310 2024-09-15T03:54:14.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 5 monclient: authenticate success, global_id 184513 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: _renew_subs 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c121c40 con 0x7f063c0a1310 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: _renew_subs 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c120d50 con 0x7f063c0a1310 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0624004b70 con 0x7f063c0a1310 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0624005020 con 0x7f063c0a1310 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06397fa700 4 set_mon_vals no callback set 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.482+0000 7f06337fe700 10 monclient: dump: 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.511 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06421a0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0620006590 con 0x7f0620002840 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f062400fa60 con 0x7f063c0a1310 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(440..440 src has 224..440) v4 ==== 3911+0+0 (unknown 3815124011 0 0) 0x7f0624043f20 con 0x7f063c0a1310 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f063bfff700 10 monclient: get_auth_request con 0x7f061c038150 auth_method 0 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039032760 0 0) 0x7f0620006590 con 0x7f0620002840 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient(hunting): my global_id is 194250 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c120180 con 0x7f0620002840 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 33060061 0 0) 0x7f063c120180 con 0x7f0620002840 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0620006590 con 0x7f0620002840 2024-09-15T03:54:14.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c0022c0 con 0x7f0620002840 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient: dump: 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.514 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1498016724 0 0) 0x7f0620006590 con 0x7f0620002840 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f062000a130 con 0x7f0620002840 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f062c002810 con 0x7f0620002840 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.122:6790/0 conn(0x7f0620002840 legacy=0x7f0620004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.486+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.122:6790/0 conn(0x7f0620002840 legacy=0x7f0620004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06421a0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:14.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0620007520 con 0x7f06200068d0 2024-09-15T03:54:14.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836050793 0 0) 0x7f0620007520 con 0x7f06200068d0 2024-09-15T03:54:14.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 10 monclient(hunting): my global_id is 194259 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0620006590 con 0x7f06200068d0 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1296144583 0 0) 0x7f0620006590 con 0x7f06200068d0 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0620007520 con 0x7f06200068d0 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f062c0035f0 con 0x7f06200068d0 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.494+0000 7f06337fe700 10 monclient: dump: 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.519 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2004031208 0 0) 0x7f0620007520 con 0x7f06200068d0 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0620008760 con 0x7f06200068d0 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 414773543) 0x7f062c0062a0 con 0x7f06200068d0 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.122:6790/0 conn(0x7f06200068d0 legacy=0x7f0620006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06337fe700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.122:6790/0 conn(0x7f06200068d0 legacy=0x7f0620006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:14.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:14.521 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.522 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.523 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:14.524 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.525 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:14.526 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.529 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.531 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.532 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.533 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.534 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.535 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.536 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.537 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:14.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:14.539 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.122:6812/20895 conn(0x7f061c038150 legacy=0x7f061c03a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 10 monclient: shutdown 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 >> v1:172.21.15.26:6791/0 conn(0x7f063c0a1310 legacy=0x7f063c147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f063bfff700 1 -- 172.21.15.26:0/3882503650 reap_dead start 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f063bfff700 1 -- 172.21.15.26:0/3882503650 reap_dead start 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 shutdown_connections 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 >> 172.21.15.26:0/3882503650 conn(0x7f063c06e370 msgr2=0x7f063c0a4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 shutdown_connections 2024-09-15T03:54:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.498+0000 7f06421a0700 1 -- 172.21.15.26:0/3882503650 wait complete. 2024-09-15T03:54:14.541 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:14.541 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:54:14.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:54:14.575 INFO:tasks.rados.rados.0.smithi122.stdout:1970: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: get_monmap_and_config 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: monmap: 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:14.656795+0000 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:14.656795+0000 2024-09-15T03:54:14.666 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 1 Processor -- start 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 1 -- start start 2024-09-15T03:54:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: init 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: _add_conns ranks=[2,6,3,1,7,4,8,0,5] 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient(hunting): picked mon.noname-b con 0x7fc1d8070de0 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient(hunting): picked mon.noname-d con 0x7fc1d809bdd0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient(hunting): picked mon.noname-g con 0x7fc1d8136b20 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d804ca10 con 0x7fc1d8070de0 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d80999f0 con 0x7fc1d8136b20 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d809c8b0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient(hunting): authenticate will time out at 1399.086666s 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1dcc43700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fc1d809bdd0 0x7fc1d809c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60424/0 (socket says 172.21.15.26:60424) 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1dcc43700 1 -- 172.21.15.26:0/3020238446 learned_addr learned my addr 172.21.15.26:0/3020238446 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564935081 0 0) 0x7fc1d809c8b0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): my global_id is 194298 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c4003570 con 0x7fc1d809bdd0 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477126206 0 0) 0x7fc1d804ca10 con 0x7fc1d8070de0 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): my global_id is 135020 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d809c8b0 con 0x7fc1d8070de0 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407421652 0 0) 0x7fc1d80999f0 con 0x7fc1d8136b20 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient(hunting): my global_id is 194268 2024-09-15T03:54:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d804ca10 con 0x7fc1d8136b20 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834708385 0 0) 0x7fc1c4003570 con 0x7fc1d809bdd0 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 >> v1:172.21.15.122:6790/0 conn(0x7fc1d8136b20 legacy=0x7fc1d8138f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 >> v1:172.21.15.26:6790/0 conn(0x7fc1d8070de0 legacy=0x7fc1d8130d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 monclient: found mon.noname-d 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8098d10 con 0x7fc1d809bdd0 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 5 monclient: authenticate success, global_id 194298 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1d4001b30 con 0x7fc1d809bdd0 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: dump: 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc1d4001da0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 1 -- 172.21.15.26:0/3020238446 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1d4004fb0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1de446700 4 set_mon_vals no callback set 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.654+0000 7fc1cffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1cffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1cffff700 10 monclient: dump: 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient: shutdown 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- 172.21.15.26:0/3020238446 >> v1:172.21.15.26:6792/0 conn(0x7fc1d809bdd0 legacy=0x7fc1d809c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- 172.21.15.26:0/3020238446 shutdown_connections 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- 172.21.15.26:0/3020238446 >> 172.21.15.26:0/3020238446 conn(0x7fc1d80a2ea0 msgr2=0x7fc1d809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- 172.21.15.26:0/3020238446 shutdown_connections 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- 172.21.15.26:0/3020238446 wait complete. 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient: monmap: 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 Processor -- start 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- start start 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient: init 2024-09-15T03:54:14.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient: _add_conns ranks=[0,5,4,3,6,2,7,8,1] 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient(hunting): picked mon.noname-a con 0x7fc1d8070de0 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient(hunting): picked mon.noname-h con 0x7fc1d809bdd0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient(hunting): picked mon.noname-c con 0x7fc1d8136b20 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d812cce0 con 0x7fc1d8070de0 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d812ca00 con 0x7fc1d8136b20 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d812b8e0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1de446700 10 monclient(hunting): authenticate will time out at 1399.091369s 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1dd444700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1d8070de0 0x7fc1d8096660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39834/0 (socket says 172.21.15.26:39834) 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1dd444700 1 -- 172.21.15.26:0/3782957021 learned_addr learned my addr 172.21.15.26:0/3782957021 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1ddc45700 1 --1- 172.21.15.26:0/3782957021 >> v1:172.21.15.26:6791/0 conn(0x7fc1d8136b20 0x7fc1d8149ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55244/0 (socket says 172.21.15.26:55244) 2024-09-15T03:54:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.658+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035051439 0 0) 0x7fc1d812ca00 con 0x7fc1d8136b20 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): my global_id is 184522 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1bc003570 con 0x7fc1d8136b20 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653615988 0 0) 0x7fc1d812cce0 con 0x7fc1d8070de0 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): my global_id is 194238 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d812ca00 con 0x7fc1d8070de0 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254516488 0 0) 0x7fc1d812b8e0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): my global_id is 5486 2024-09-15T03:54:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d812cce0 con 0x7fc1d809bdd0 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 80985820 0 0) 0x7fc1bc003570 con 0x7fc1d8136b20 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1d812b8e0 con 0x7fc1d8136b20 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1d00019c0 con 0x7fc1d8136b20 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): dump: 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2482552837 0 0) 0x7fc1d812ca00 con 0x7fc1d8070de0 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1bc003570 con 0x7fc1d8070de0 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1c8001aa0 con 0x7fc1d8070de0 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): dump: 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.682 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.682 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.682 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 461282982 0 0) 0x7fc1d812b8e0 con 0x7fc1d8136b20 2024-09-15T03:54:14.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.122:6791/0 conn(0x7fc1d809bdd0 legacy=0x7fc1d8096d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.26:6789/0 conn(0x7fc1d8070de0 legacy=0x7fc1d8096660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 monclient: found mon.c 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d812c720 con 0x7fc1d8136b20 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 5 monclient: authenticate success, global_id 184522 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 10 monclient: _renew_subs 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d812c310 con 0x7fc1d8136b20 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 10 monclient: _renew_subs 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:54:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d812da50 con 0x7fc1d8136b20 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc1d0002d80 con 0x7fc1d8136b20 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1d0002310 con 0x7fc1d8136b20 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cf7fe700 4 set_mon_vals no callback set 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: dump: 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc1d804bbf0 con 0x7fc1d802ae80 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103231968 0 0) 0x7fc1d804bbf0 con 0x7fc1d802ae80 2024-09-15T03:54:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient(hunting): my global_id is 5495 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d812b8e0 con 0x7fc1d802ae80 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc1d0002510 con 0x7fc1d8136b20 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(440..440 src has 224..440) v4 ==== 3911+0+0 (unknown 3815124011 0 0) 0x7fc1d0043d30 con 0x7fc1d8136b20 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1105889901 0 0) 0x7fc1d812b8e0 con 0x7fc1d802ae80 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1d804bbf0 con 0x7fc1d802ae80 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1c8002620 con 0x7fc1d802ae80 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 10 monclient: dump: 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1dcc43700 10 monclient: get_auth_request con 0x7fc1bc037d00 auth_method 0 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2699545785 0 0) 0x7fc1d804bbf0 con 0x7fc1d802ae80 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.662+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1d812c4a0 con 0x7fc1d802ae80 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.666+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fc1c8002a70 con 0x7fc1d802ae80 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.666+0000 7fc1cd7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.666+0000 7fc1cd7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.666+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.122:6791/0 conn(0x7fc1d802ae80 legacy=0x7fc1d805e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.666+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.122:6791/0 conn(0x7fc1d802ae80 legacy=0x7fc1d805e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.674+0000 7fc1de446700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.674+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fc1d806fac0 con 0x7fc1d8097480 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012173381 0 0) 0x7fc1d806fac0 con 0x7fc1d8097480 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient(hunting): my global_id is 5504 2024-09-15T03:54:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d804bbf0 con 0x7fc1d8097480 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1276782150 0 0) 0x7fc1d804bbf0 con 0x7fc1d8097480 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1d806fac0 con 0x7fc1d8097480 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc1c8005d40 con 0x7fc1d8097480 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient: dump: 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.692 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 915148249 0 0) 0x7fc1d806fac0 con 0x7fc1d8097480 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc1d802b250 con 0x7fc1d8097480 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7740 (unknown 0 0 1997802074) 0x7fc1c80060a0 con 0x7fc1d8097480 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.122:6791/0 conn(0x7fc1d8097480 legacy=0x7fc1d8132d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1cd7fa700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.122:6791/0 conn(0x7fc1d8097480 legacy=0x7fc1d8132d70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:14.694 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.695 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.696 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:14.697 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:14.698 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.698 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.699 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.700 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.701 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.702 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.703 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:14.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.706 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.707 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.708 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:14.709 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.711 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:14.712 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:14.713 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.714 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:14.715 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.122:6812/20895 conn(0x7fc1bc037d00 legacy=0x7fc1bc03a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 10 monclient: shutdown 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 >> v1:172.21.15.26:6791/0 conn(0x7fc1d8136b20 legacy=0x7fc1d8149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1ddc45700 1 -- 172.21.15.26:0/3782957021 reap_dead start 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1ddc45700 1 -- 172.21.15.26:0/3782957021 reap_dead start 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 shutdown_connections 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 >> 172.21.15.26:0/3782957021 conn(0x7fc1d80a2ea0 msgr2=0x7fc1d8136270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 shutdown_connections 2024-09-15T03:54:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.678+0000 7fc1de446700 1 -- 172.21.15.26:0/3782957021 wait complete. 2024-09-15T03:54:14.717 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:14.717 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:54:14.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:54:14.725 INFO:tasks.rados.rados.0.smithi122.stdout:1969: finishing write tid 1 to smithi12223906-5 2024-09-15T03:54:14.725 INFO:tasks.rados.rados.0.smithi122.stdout:1969: finishing write tid 2 to smithi12223906-5 2024-09-15T03:54:14.725 INFO:tasks.rados.rados.0.smithi122.stdout:1969: finishing write tid 3 to smithi12223906-5 2024-09-15T03:54:14.725 INFO:tasks.rados.rados.0.smithi122.stdout:1969: finishing write tid 4 to smithi12223906-5 2024-09-15T03:54:14.725 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 853 (ObjNum 715 snap 217 seq_num 715) dirty exists 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1969: left oid 5 (ObjNum 715 snap 217 seq_num 715) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1964: done (7 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1965: done (6 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1966: done (5 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1967: done (4 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1968: done (3 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1969: done (2 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1970: done (1 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1971: done (0 left) 2024-09-15T03:54:14.726 INFO:tasks.rados.rados.0.smithi122.stdout:1972: snap_remove snap 213 2024-09-15T03:54:14.808 INFO:tasks.rados.rados.0.smithi122.stdout:1972: done (0 left) 2024-09-15T03:54:14.808 INFO:tasks.rados.rados.0.smithi122.stdout:1973: rollback oid 9 current snap is 217 2024-09-15T03:54:14.808 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 197 2024-09-15T03:54:14.808 INFO:tasks.rados.rados.0.smithi122.stdout:1974: delete oid 29 current snap is 217 2024-09-15T03:54:14.811 INFO:tasks.rados.rados.0.smithi122.stdout:1974: done (1 left) 2024-09-15T03:54:14.811 INFO:tasks.rados.rados.0.smithi122.stdout:1975: snap_remove snap 191 2024-09-15T03:54:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: get_monmap_and_config 2024-09-15T03:54:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: monmap: 2024-09-15T03:54:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:14.832543+0000 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:14.832543+0000 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.836 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 1 Processor -- start 2024-09-15T03:54:14.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 1 -- start start 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: init 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: _add_conns ranks=[8,6,0,1,5,3,2,4,7] 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient(hunting): picked mon.noname-e con 0x7f2314070de0 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient(hunting): picked mon.noname-d con 0x7f2314099020 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient(hunting): picked mon.noname-a con 0x7f2314132f70 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231404ca10 con 0x7f2314132f70 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2314096140 con 0x7f2314099020 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231406e620 con 0x7f2314070de0 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient(hunting): authenticate will time out at 1399.262345s 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2313fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f2314099020 0x7f2314137d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60430/0 (socket says 172.21.15.26:60430) 2024-09-15T03:54:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2313fff700 1 -- 172.21.15.26:0/3275373623 learned_addr learned my addr 172.21.15.26:0/3275373623 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150073129 0 0) 0x7f2314096140 con 0x7f2314099020 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): my global_id is 194307 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22fc003570 con 0x7f2314099020 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777199842 0 0) 0x7f231404ca10 con 0x7f2314132f70 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): my global_id is 194247 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2314096140 con 0x7f2314132f70 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740904781 0 0) 0x7f231406e620 con 0x7f2314070de0 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient(hunting): my global_id is 134945 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231404ca10 con 0x7f2314070de0 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138733891 0 0) 0x7f22fc003570 con 0x7f2314099020 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 >> v1:172.21.15.26:6793/0 conn(0x7f2314070de0 legacy=0x7f2314098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 >> v1:172.21.15.26:6789/0 conn(0x7f2314132f70 legacy=0x7f231413b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 monclient: found mon.noname-d 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314096440 con 0x7f2314099020 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 5 monclient: authenticate success, global_id 194307 2024-09-15T03:54:14.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f23040017b0 con 0x7f2314099020 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: dump: 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.848 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2304001a20 con 0x7f2314099020 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:14.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 1 -- 172.21.15.26:0/3275373623 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2304004c30 con 0x7f2314099020 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f2319c6c700 4 set_mon_vals no callback set 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.830+0000 7f23137fe700 10 monclient: dump: 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.850 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: shutdown 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/3275373623 >> v1:172.21.15.26:6792/0 conn(0x7f2314099020 legacy=0x7f2314137d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/3275373623 shutdown_connections 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/3275373623 >> 172.21.15.26:0/3275373623 conn(0x7f23140a2e80 msgr2=0x7f231409eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/3275373623 shutdown_connections 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/3275373623 wait complete. 2024-09-15T03:54:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: build_initial_monmap 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: monmap: 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 Processor -- start 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- start start 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: init 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: _add_conns ranks=[8,3,4,0,6,2,7,1,5] 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient(hunting): picked mon.noname-e con 0x7f2314070de0 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient(hunting): picked mon.noname-g con 0x7f2314132f70 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient(hunting): picked mon.noname-c con 0x7f231412a260 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231409fb20 con 0x7f2314132f70 2024-09-15T03:54:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231412acf0 con 0x7f231412a260 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231412db90 con 0x7f2314070de0 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient(hunting): authenticate will time out at 1399.266395s 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2318c6a700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f2314070de0 0x7f2314141bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51740/0 (socket says 172.21.15.26:51740) 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2318c6a700 1 -- 172.21.15.26:0/439842562 learned_addr learned my addr 172.21.15.26:0/439842562 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388314477 0 0) 0x7f231412db90 con 0x7f2314070de0 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): my global_id is 134954 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2300003570 con 0x7f2314070de0 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689225198 0 0) 0x7f231412acf0 con 0x7f231412a260 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): my global_id is 184531 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231412db90 con 0x7f231412a260 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792937860 0 0) 0x7f231409fb20 con 0x7f2314132f70 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): my global_id is 194277 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231412acf0 con 0x7f2314132f70 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2844666494 0 0) 0x7f2300003570 con 0x7f2314070de0 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f231409fb20 con 0x7f2314070de0 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2308001aa0 con 0x7f2314070de0 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): dump: 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2758728437 0 0) 0x7f231412db90 con 0x7f231412a260 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2300003570 con 0x7f231412a260 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f230c001c00 con 0x7f231412a260 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient(hunting): dump: 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3870071393 0 0) 0x7f231409fb20 con 0x7f2314070de0 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.26:6791/0 conn(0x7f231412a260 legacy=0x7f2314149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.122:6790/0 conn(0x7f2314132f70 legacy=0x7f23141422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 monclient: found mon.e 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23141429e0 con 0x7f2314070de0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 10 monclient: _finish_auth 0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2310ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 5 monclient: authenticate success, global_id 134954 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: _renew_subs 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231409f8f0 con 0x7f2314070de0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: _renew_subs 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.834+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f231412b8c0 con 0x7f2314070de0 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2308002080 con 0x7f2314070de0 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f23080025b0 con 0x7f2314070de0 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2312ffd700 4 set_mon_vals no callback set 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: dump: 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2319c6c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22f4006590 con 0x7f22f4002840 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2308002710 con 0x7f2314070de0 2024-09-15T03:54:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(441..441 src has 224..441) v4 ==== 3955+0+0 (unknown 391162800 0 0) 0x7f2308043eb0 con 0x7f2314070de0 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982584061 0 0) 0x7f22f4006590 con 0x7f22f4002840 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f231946b700 10 monclient: get_auth_request con 0x7f2300037be0 auth_method 0 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient(hunting): my global_id is 134899 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231409fb20 con 0x7f22f4002840 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3889877765 0 0) 0x7f231409fb20 con 0x7f22f4002840 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22f4006590 con 0x7f22f4002840 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2304004b30 con 0x7f22f4002840 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: dump: 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3336852548 0 0) 0x7f22f4006590 con 0x7f22f4002840 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22f400a130 con 0x7f22f4002840 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2304007280 con 0x7f22f4002840 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.122:6792/0 conn(0x7f22f4002840 legacy=0x7f22f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.838+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.122:6792/0 conn(0x7f22f4002840 legacy=0x7f22f4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.846+0000 7f2319c6c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.846+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f22f4009640 con 0x7f22f40090b0 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702381582 0 0) 0x7f22f4009640 con 0x7f22f40090b0 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient(hunting): my global_id is 134908 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f4006590 con 0x7f22f40090b0 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2747683923 0 0) 0x7f22f4006590 con 0x7f22f40090b0 2024-09-15T03:54:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22f4009640 con 0x7f22f40090b0 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2304005a30 con 0x7f22f40090b0 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient: dump: 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4075116879 0 0) 0x7f22f4009640 con 0x7f22f40090b0 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f22f40098f0 con 0x7f22f40090b0 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 4291593457) 0x7f230400a270 con 0x7f22f40090b0 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.122:6792/0 conn(0x7f22f40090b0 legacy=0x7f22f4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2310ff9700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.122:6792/0 conn(0x7f22f40090b0 legacy=0x7f22f4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 72, 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:54:14.870 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.871 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.872 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:14.873 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:14.874 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:14.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.876 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:14.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.880 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.882 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:14.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.884 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:14.886 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:14.888 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:14.889 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.122:6812/20895 conn(0x7f2300037be0 legacy=0x7f230003a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 10 monclient: shutdown 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 >> v1:172.21.15.26:6793/0 conn(0x7f2314070de0 legacy=0x7f2314141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f231946b700 1 -- 172.21.15.26:0/439842562 reap_dead start 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f231946b700 1 -- 172.21.15.26:0/439842562 reap_dead start 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 shutdown_connections 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 >> 172.21.15.26:0/439842562 conn(0x7f23140a2e80 msgr2=0x7f231409e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 shutdown_connections 2024-09-15T03:54:14.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:14.850+0000 7f2319c6c700 1 -- 172.21.15.26:0/439842562 wait complete. 2024-09-15T03:54:14.891 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:54:14.892 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b', 'f', 'c', 'g'] 2024-09-15T03:54:14.892 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['h'] 2024-09-15T03:54:14.892 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T03:54:14.892 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T03:54:14.892 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T03:54:14.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:54:14.914 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T03:54:14.914 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T03:54:14.914 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T03:54:14.914 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T03:54:14.914 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:54:14.927 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T03:54:14.927 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-15T03:54:14.927 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T03:54:14.928 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T03:54:14.928 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:54:14.944 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T03:54:14.945 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T03:54:14.945 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:54:14.945 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T03:54:14.945 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:54:14.991 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T03:54:14.992 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-09-15T03:54:14.992 INFO:tasks.ceph.mon.h:Sent signal 19 2024-09-15T03:54:14.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:54:29.992 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-09-15T03:54:29.993 INFO:tasks.ceph.mon.h:Sent signal 18 2024-09-15T03:54:29.993 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-15T03:54:29.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient: get_monmap_and_config 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient: build_initial_monmap 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient: monmap: 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:30.104922+0000 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:30.104922+0000 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:30.108 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 1 Processor -- start 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 1 -- start start 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient: init 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient: _add_conns ranks=[2,3,7,6,1,0,8,4,5] 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient(hunting): picked mon.noname-b con 0x7fbad0070f80 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient(hunting): picked mon.noname-g con 0x7fbad009dee0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient(hunting): picked mon.noname-i con 0x7fbad01391e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad004ca10 con 0x7fbad0070f80 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad6365700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fbad0070f80 legacy=0x7fbad00950b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:30.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad009edc0 con 0x7fbad009dee0 2024-09-15T03:54:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad009eae0 con 0x7fbad01391e0 2024-09-15T03:54:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.101+0000 7fbad7367700 10 monclient(hunting): authenticate will time out at 1414.535974s 2024-09-15T03:54:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.105+0000 7fbad5b64700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fbad009dee0 legacy=0x7fbad00957c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.105+0000 7fbad6b66700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fbad01391e0 0x7fbad013b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:58574/0 (socket says 172.21.15.26:58574) 2024-09-15T03:54:30.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.105+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 learned_addr learned my addr 172.21.15.26:0/995712846 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.305+0000 7fbad6365700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6790/0 conn(0x7fbad0070f80 legacy=0x7fbad00950b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.309+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbad009dee0 legacy=0x7fbad00957c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.709+0000 7fbad6365700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6790/0 conn(0x7fbad0070f80 legacy=0x7fbad00950b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:30.713+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbad009dee0 legacy=0x7fbad00957c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:31.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:31.513+0000 7fbad6365700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6790/0 conn(0x7fbad0070f80 legacy=0x7fbad00950b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:31.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:31.517+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbad009dee0 legacy=0x7fbad00957c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient(hunting): tick 2024-09-15T03:54:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 monclient(hunting): continuing hunt 2024-09-15T03:54:33.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6792/0 conn(0x7fbad01391e0 legacy=0x7fbad013b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbad009dee0 legacy=0x7fbad00957c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6790/0 conn(0x7fbad0070f80 legacy=0x7fbad00950b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient: _add_conns ranks=[8,5,1,6,0,4] 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-e con 0x7fbab40037f0 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-h con 0x7fbab4007470 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-f con 0x7fbab400b1d0 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad009e800 con 0x7fbab400b1d0 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad004ca10 con 0x7fbab4007470 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad009edc0 con 0x7fbab40037f0 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbac7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:33.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.105+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6789/0 conn(0x7fbab400b1d0 legacy=0x7fbab400d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.309+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6789/0 conn(0x7fbab400b1d0 legacy=0x7fbab400d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:33.713+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6789/0 conn(0x7fbab400b1d0 legacy=0x7fbab400d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:34.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:34.517+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6789/0 conn(0x7fbab400b1d0 legacy=0x7fbab400d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:36.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient(hunting): tick 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 monclient(hunting): continuing hunt 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6793/0 conn(0x7fbab40037f0 legacy=0x7fbab4005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6791/0 conn(0x7fbab4007470 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6789/0 conn(0x7fbab400b1d0 legacy=0x7fbab400d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 reap_dead start 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient: _add_conns ranks=[6,4,0] 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 reap_dead start 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-d con 0x7fbab4010000 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-c con 0x7fbab40104a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-a con 0x7fbab4010a50 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4012050 con 0x7fbab4010a50 2024-09-15T03:54:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad009e800 con 0x7fbab40104a0 2024-09-15T03:54:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4011080 con 0x7fbab4010000 2024-09-15T03:54:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbac7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.105+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab40104a0 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.309+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab40104a0 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:36.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:36.713+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab40104a0 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:37.517+0000 7fbad5b64700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab40104a0 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient(hunting): tick 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 monclient(hunting): continuing hunt 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6792/0 conn(0x7fbab4010000 legacy=0x7fbab4005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab40104a0 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6789/0 conn(0x7fbab4010a50 legacy=0x7fbab400d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient: _add_conns ranks=[4,7,3,1,6,2,8,5,0] 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-c con 0x7fbab4011570 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-i con 0x7fbab4011a40 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient(hunting): picked mon.noname-g con 0x7fbab400e560 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab40112c0 con 0x7fbab400e560 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbad6365700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab4011570 legacy=0x7fbab400de50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad009e800 con 0x7fbab4011570 2024-09-15T03:54:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab400ea00 con 0x7fbab4011a40 2024-09-15T03:54:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbac7fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.105+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbab400e560 legacy=0x7fbab4017370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.309+0000 7fbad6365700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab4011570 legacy=0x7fbab400de50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.309+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbab400e560 legacy=0x7fbab4017370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.713+0000 7fbad6365700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab4011570 legacy=0x7fbab400de50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.713+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbab400e560 legacy=0x7fbab4017370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.829+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649261338 0 0) 0x7fbab400ea00 con 0x7fbab4011a40 2024-09-15T03:54:39.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.829+0000 7fbad5363700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.829+0000 7fbad5363700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.829+0000 7fbad5363700 10 monclient(hunting): my global_id is 134917 2024-09-15T03:54:39.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.829+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbabc003570 con 0x7fbab4011a40 2024-09-15T03:54:39.845 INFO:tasks.rados.rados.0.smithi122.stdout:1975: done (1 left) 2024-09-15T03:54:39.845 INFO:tasks.rados.rados.0.smithi122.stdout:1976: rollback oid 36 current snap is 217 2024-09-15T03:54:39.845 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 36 to 197 2024-09-15T03:54:39.845 INFO:tasks.rados.rados.0.smithi122.stdout:1973: finishing rollback tid 0 to smithi12223906-9 2024-09-15T03:54:39.845 INFO:tasks.rados.rados.0.smithi122.stdout:1973: finishing rollback tid 1 to smithi12223906-9 2024-09-15T03:54:39.845 INFO:tasks.rados.rados.0.smithi122.stdout:1973: finishing rollback tid 2 to smithi12223906-9 2024-09-15T03:54:39.846 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 816 (ObjNum 534 snap 160 seq_num 534) dirty exists 2024-09-15T03:54:39.846 INFO:tasks.rados.rados.0.smithi122.stdout:1973: done (1 left) 2024-09-15T03:54:39.846 INFO:tasks.rados.rados.0.smithi122.stdout:1977: delete oid 46 current snap is 217 2024-09-15T03:54:39.848 INFO:tasks.rados.rados.0.smithi122.stdout:1977: done (1 left) 2024-09-15T03:54:39.848 INFO:tasks.rados.rados.0.smithi122.stdout:1978: snap_create 2024-09-15T03:54:39.853 INFO:tasks.rados.rados.0.smithi122.stdout:1976: finishing rollback tid 0 to smithi12223906-36 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1768049385 0 0) 0x7fbabc003570 con 0x7fbab4011a40 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.26:6791/0 conn(0x7fbab4011570 legacy=0x7fbab400de50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6790/0 conn(0x7fbab400e560 legacy=0x7fbab4017370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: _finish_hunting 0 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 monclient: found mon.noname-i 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab400eb90 con 0x7fbab4011a40 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: _finish_auth 0 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbac00041f0 con 0x7fbab4011a40 2024-09-15T03:54:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: dump: 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:39.857 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 5 monclient: authenticate success, global_id 134917 2024-09-15T03:54:39.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:1976: finishing rollback tid 1 to smithi12223906-36 2024-09-15T03:54:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:1976: finishing rollback tid 2 to smithi12223906-36 2024-09-15T03:54:39.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 1132 (ObjNum 633 snap 192 seq_num 633) dirty exists 2024-09-15T03:54:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbac0004410 con 0x7fbab4011a40 2024-09-15T03:54:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 1 -- 172.21.15.26:0/995712846 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbac00049c0 con 0x7fbab4011a40 2024-09-15T03:54:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 4 set_mon_vals no callback set 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad5363700 10 monclient: dump: 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 10 monclient: shutdown 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 1 -- 172.21.15.26:0/995712846 >> v1:172.21.15.122:6792/0 conn(0x7fbab4011a40 legacy=0x7fbab4009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad6b66700 1 -- 172.21.15.26:0/995712846 reap_dead start 2024-09-15T03:54:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 1 -- 172.21.15.26:0/995712846 shutdown_connections 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 1 -- 172.21.15.26:0/995712846 >> 172.21.15.26:0/995712846 conn(0x7fbad00a24e0 msgr2=0x7fbad00a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 1 -- 172.21.15.26:0/995712846 shutdown_connections 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.849+0000 7fbad7367700 1 -- 172.21.15.26:0/995712846 wait complete. 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: build_initial_monmap 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: monmap: 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 Processor -- start 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- start start 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: init 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _add_conns ranks=[7,3,1,4,0,8,6,5,2] 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient(hunting): picked mon.noname-i con 0x7fbad0098db0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient(hunting): picked mon.noname-g con 0x7fbad012a5e0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient(hunting): picked mon.noname-f con 0x7fbad0097c10 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad012c210 con 0x7fbad0097c10 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad012c4f0 con 0x7fbad012a5e0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad5b64700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fbad012a5e0 legacy=0x7fbad0097500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad6b66700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fbad0097c10 legacy=0x7fbad014a690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad012b010 con 0x7fbad0098db0 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient(hunting): authenticate will time out at 1424.285676s 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad6365700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fbad0098db0 0x7fbad0138570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:37342/0 (socket says 172.21.15.26:37342) 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad6365700 1 -- 172.21.15.26:0/2801076438 learned_addr learned my addr 172.21.15.26:0/2801076438 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369190820 0 0) 0x7fbad012b010 con 0x7fbad0098db0 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): my global_id is 134926 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab4010e20 con 0x7fbad0098db0 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1378096389 0 0) 0x7fbab4010e20 con 0x7fbad0098db0 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbad012b010 con 0x7fbad0098db0 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbacc0043d0 con 0x7fbad0098db0 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): dump: 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3393134107 0 0) 0x7fbad012b010 con 0x7fbad0098db0 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 >> v1:172.21.15.122:6790/0 conn(0x7fbad012a5e0 legacy=0x7fbad0097500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 >> v1:172.21.15.122:6789/0 conn(0x7fbad0097c10 legacy=0x7fbad014a690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 monclient: found mon.i 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0099180 con 0x7fbad0098db0 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 5 monclient: authenticate success, global_id 134926 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _renew_subs 2024-09-15T03:54:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad0098050 con 0x7fbad0098db0 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _renew_subs 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad014bdb0 con 0x7fbad0098db0 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbacc004960 con 0x7fbad0098db0 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbacc004e90 con 0x7fbad0098db0 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac7fff700 4 set_mon_vals no callback set 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 10 monclient: dump: 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:39.874 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:39.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbacc005070 con 0x7fbad0098db0 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab80050b0 con 0x7fbad0098db0 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.853+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(442..442 src has 224..442) v4 ==== 3971+0+0 (unknown 2340481024 0 0) 0x7fbacc043ee0 con 0x7fbad0098db0 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.857+0000 7fbad5b64700 10 monclient: get_auth_request con 0x7fbab4039f50 auth_method 0 2024-09-15T03:54:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.861+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fbacc014cd0 con 0x7fbad0098db0 2024-09-15T03:54:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.861+0000 7fbac67fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:39.861+0000 7fbac67fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:40.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbad7367700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:54:40.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbad7367700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:40.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbad7367700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T03:54:40.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbab80058e0 con 0x7fbad0098db0 2024-09-15T03:54:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbac67fc700 1 -- 172.21.15.26:0/2801076438 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2676 (unknown 254746657 0 3373991479) 0x7fbacc0146d0 con 0x7fbad0098db0 2024-09-15T03:54:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbac67fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:54:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.085+0000 7fbac67fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:40.092 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":80,"quorum":[0,5,6,7,8],"quorum_names":["a","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:54:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 >> v1:172.21.15.122:6812/20895 conn(0x7fbab4039f50 legacy=0x7fbab403c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 10 monclient: shutdown 2024-09-15T03:54:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 >> v1:172.21.15.122:6792/0 conn(0x7fbad0098db0 legacy=0x7fbad0138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 shutdown_connections 2024-09-15T03:54:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 >> 172.21.15.26:0/2801076438 conn(0x7fbad00a24e0 msgr2=0x7fbad00923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 shutdown_connections 2024-09-15T03:54:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.089+0000 7fbad7367700 1 -- 172.21.15.26:0/2801076438 wait complete. 2024-09-15T03:54:40.107 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-15T03:54:40.107 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-15T03:54:40.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:54:40.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.205+0000 7f5caf09b700 10 monclient: get_monmap_and_config 2024-09-15T03:54:40.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.205+0000 7f5caf09b700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.205+0000 7f5caf09b700 10 monclient: monmap: 2024-09-15T03:54:40.213 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:40.211289+0000 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:40.211289+0000 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.214 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.215 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.215 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.215 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.205+0000 7f5caf09b700 1 Processor -- start 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- start start 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: init 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: _add_conns ranks=[3,0,2,8,6,5,4,7,1] 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient(hunting): picked mon.noname-g con 0x7f5ca80a2a70 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient(hunting): picked mon.noname-a con 0x7f5ca809e380 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient(hunting): picked mon.noname-b con 0x7f5ca809bfc0 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca804ca10 con 0x7f5ca809e380 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca809d3e0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca8098880 con 0x7f5ca80a2a70 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient(hunting): authenticate will time out at 1424.640610s 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cae89a700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7f5ca809bfc0 legacy=0x7f5ca8138d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad898700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ca809e380 0x7f5ca8098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53672/0 (socket says 172.21.15.26:53672) 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad898700 1 -- 172.21.15.26:0/2733373326 learned_addr learned my addr 172.21.15.26:0/2733373326 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cae099700 1 -- 172.21.15.26:0/2733373326 >> v1:172.21.15.122:6790/0 conn(0x7f5ca80a2a70 legacy=0x7f5ca8097a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020967188 0 0) 0x7f5ca804ca10 con 0x7f5ca809e380 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient(hunting): my global_id is 194256 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c94003570 con 0x7f5ca809e380 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1389121684 0 0) 0x7f5c94003570 con 0x7f5ca809e380 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 >> v1:172.21.15.122:6790/0 conn(0x7f5ca80a2a70 legacy=0x7f5ca8097a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 >> v1:172.21.15.26:6790/0 conn(0x7f5ca809bfc0 legacy=0x7f5ca8138d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 monclient: found mon.noname-a 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca809d100 con 0x7f5ca809e380 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 5 monclient: authenticate success, global_id 194256 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5c9c0017b0 con 0x7f5ca809e380 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: dump: 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.226 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5c9c001ad0 con 0x7f5ca809e380 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:40.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 4 set_mon_vals no callback set 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 1 -- 172.21.15.26:0/2733373326 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5c9c004c30 con 0x7f5ca809e380 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5cad097700 10 monclient: dump: 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.228 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: shutdown 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- 172.21.15.26:0/2733373326 >> v1:172.21.15.26:6789/0 conn(0x7f5ca809e380 legacy=0x7f5ca8098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- 172.21.15.26:0/2733373326 shutdown_connections 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- 172.21.15.26:0/2733373326 >> 172.21.15.26:0/2733373326 conn(0x7f5ca8063cd0 msgr2=0x7f5ca80a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- 172.21.15.26:0/2733373326 shutdown_connections 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- 172.21.15.26:0/2733373326 wait complete. 2024-09-15T03:54:40.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: monmap: 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.230 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 Processor -- start 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 1 -- start start 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.209+0000 7f5caf09b700 10 monclient: init 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient: _add_conns ranks=[5,3,2,0,7,4,1,8,6] 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient(hunting): picked mon.noname-h con 0x7f5ca809bfc0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient(hunting): picked mon.noname-g con 0x7f5ca809e380 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient(hunting): picked mon.noname-b con 0x7f5ca80a2a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca806df30 con 0x7f5ca80a2a70 2024-09-15T03:54:40.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca809cad0 con 0x7f5ca809e380 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca811f3e0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient(hunting): authenticate will time out at 1424.644438s 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5cae89a700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7f5ca80a2a70 legacy=0x7f5ca811c940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5cad898700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f5ca809e380 legacy=0x7f5ca811c230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5cae099700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f5ca809bfc0 0x7f5ca811bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:49440/0 (socket says 172.21.15.26:49440) 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5cae099700 1 -- 172.21.15.26:0/2385491209 learned_addr learned my addr 172.21.15.26:0/2385491209 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025611898 0 0) 0x7f5ca811f3e0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): my global_id is 5513 2024-09-15T03:54:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c8c003570 con 0x7f5ca809bfc0 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2498145369 0 0) 0x7f5c8c003570 con 0x7f5ca809bfc0 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5ca811f3e0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca4004380 con 0x7f5ca809bfc0 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): dump: 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3555798296 0 0) 0x7f5ca811f3e0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.122:6790/0 conn(0x7f5ca809e380 legacy=0x7f5ca811c230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.26:6790/0 conn(0x7f5ca80a2a70 legacy=0x7f5ca811c940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 monclient: found mon.h 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca811d5d0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 5 monclient: authenticate success, global_id 5513 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient: _renew_subs 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:54:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca811f100 con 0x7f5ca809bfc0 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient: _renew_subs 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca811e1a0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5ca4004ae0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5ca4004f90 con 0x7f5ca809bfc0 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9bfff700 4 set_mon_vals no callback set 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5c9a7fc700 10 monclient: dump: 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.213+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c90006590 con 0x7f5c90002840 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339271148 0 0) 0x7f5c90006590 con 0x7f5c90002840 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient(hunting): my global_id is 194265 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca811f3e0 con 0x7f5c90002840 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5ca400f9d0 con 0x7f5ca809bfc0 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(442..442 src has 224..442) v4 ==== 3971+0+0 (unknown 2340481024 0 0) 0x7f5ca4044060 con 0x7f5ca809bfc0 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4199953853 0 0) 0x7f5ca811f3e0 con 0x7f5c90002840 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c90006590 con 0x7f5c90002840 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5c9c004c30 con 0x7f5c90002840 2024-09-15T03:54:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5cae89a700 10 monclient: get_auth_request con 0x7f5c8c037560 auth_method 0 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient: dump: 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3219334854 0 0) 0x7f5c90006590 con 0x7f5c90002840 2024-09-15T03:54:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c9000a130 con 0x7f5c90002840 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5c9c001430 con 0x7f5c90002840 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.26:6789/0 conn(0x7f5c90002840 legacy=0x7f5c90004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.217+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.26:6789/0 conn(0x7f5c90002840 legacy=0x7f5c90004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5caf09b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5c90009620 con 0x7f5c90009090 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462828978 0 0) 0x7f5c90009620 con 0x7f5c90009090 2024-09-15T03:54:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 10 monclient(hunting): my global_id is 194274 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90006590 con 0x7f5c90009090 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1415493695 0 0) 0x7f5c90006590 con 0x7f5c90009090 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c90009620 con 0x7f5c90009090 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5c9c005340 con 0x7f5c90009090 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 10 monclient: dump: 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 356698459 0 0) 0x7f5c90009620 con 0x7f5c90009090 2024-09-15T03:54:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.229+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5c900098d0 con 0x7f5c90009090 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7354 (unknown 0 0 1631752631) 0x7f5c9c0076b0 con 0x7f5c90009090 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5c9a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5c9a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.26:6789/0 conn(0x7f5c90009090 legacy=0x7f5c90008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5c9a7fc700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.26:6789/0 conn(0x7f5c90009090 legacy=0x7f5c90008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:40.246 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 80, 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:40.247 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:40.248 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:40.249 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:40.250 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.251 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.254 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.255 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:40.259 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:40.266 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.122:6812/20895 conn(0x7f5c8c037560 legacy=0x7f5c8c039a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 10 monclient: shutdown 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 >> v1:172.21.15.122:6791/0 conn(0x7f5ca809bfc0 legacy=0x7f5ca811bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5cae89a700 1 -- 172.21.15.26:0/2385491209 reap_dead start 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5cae89a700 1 -- 172.21.15.26:0/2385491209 reap_dead start 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 shutdown_connections 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 >> 172.21.15.26:0/2385491209 conn(0x7f5ca8063cd0 msgr2=0x7f5ca8130e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 shutdown_connections 2024-09-15T03:54:40.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.233+0000 7f5caf09b700 1 -- 172.21.15.26:0/2385491209 wait complete. 2024-09-15T03:54:40.270 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:54:40.270 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-09-15T03:54:40.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient: get_monmap_and_config 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient: monmap: 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:40.384729+0000 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:40.384729+0000 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.387 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.388 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 1 Processor -- start 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 1 -- start start 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient: init 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient: _add_conns ranks=[8,6,3,7,4,5,1,2,0] 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient(hunting): picked mon.noname-e con 0x7fde98132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient(hunting): picked mon.noname-d con 0x7fde9809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient(hunting): picked mon.noname-g con 0x7fde98138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9804ca10 con 0x7fde98138d40 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9809d5d0 con 0x7fde9809faa0 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98133620 con 0x7fde98132b40 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9d6b2700 10 monclient(hunting): authenticate will time out at 1424.814826s 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde9ceb1700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fde98138d40 legacy=0x7fde9813b120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde977fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fde9809faa0 0x7fde9809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:40534/0 (socket says 172.21.15.26:40534) 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde97fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fde98132b40 0x7fde98132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33438/0 (socket says 172.21.15.26:33438) 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde977fe700 1 -- 172.21.15.26:0/1061963910 learned_addr learned my addr 172.21.15.26:0/1061963910 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023961368 0 0) 0x7fde9809d5d0 con 0x7fde9809faa0 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 10 monclient(hunting): my global_id is 194316 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde80003570 con 0x7fde9809faa0 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804423162 0 0) 0x7fde98133620 con 0x7fde98132b40 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 10 monclient(hunting): my global_id is 134963 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde9809d5d0 con 0x7fde98132b40 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.381+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1346771711 0 0) 0x7fde80003570 con 0x7fde9809faa0 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 >> v1:172.21.15.26:6793/0 conn(0x7fde98132b40 legacy=0x7fde98132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 >> v1:172.21.15.122:6790/0 conn(0x7fde98138d40 legacy=0x7fde9813b120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 monclient: found mon.noname-d 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde980a0cc0 con 0x7fde9809faa0 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde880017b0 con 0x7fde9809faa0 2024-09-15T03:54:40.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 5 monclient: authenticate success, global_id 194316 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: dump: 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.395 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fde880019a0 con 0x7fde9809faa0 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 1 -- 172.21.15.26:0/1061963910 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde880048f0 con 0x7fde9809faa0 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 4 set_mon_vals no callback set 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde96ffd700 10 monclient: dump: 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 10 monclient: shutdown 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 -- 172.21.15.26:0/1061963910 >> v1:172.21.15.26:6792/0 conn(0x7fde9809faa0 legacy=0x7fde9809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 -- 172.21.15.26:0/1061963910 shutdown_connections 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 -- 172.21.15.26:0/1061963910 >> 172.21.15.26:0/1061963910 conn(0x7fde9806e370 msgr2=0x7fde980a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 -- 172.21.15.26:0/1061963910 shutdown_connections 2024-09-15T03:54:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 -- 172.21.15.26:0/1061963910 wait complete. 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 10 monclient: monmap: 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 Processor -- start 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 1 -- start start 2024-09-15T03:54:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.385+0000 7fde9d6b2700 10 monclient: init 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient: _add_conns ranks=[6,2,1,7,0,3,5,4,8] 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient(hunting): picked mon.noname-d con 0x7fde9809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient(hunting): picked mon.noname-b con 0x7fde98132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient(hunting): picked mon.noname-f con 0x7fde98138d40 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9809cfa0 con 0x7fde98138d40 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9809ccc0 con 0x7fde98132b40 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98098880 con 0x7fde9809faa0 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde977fe700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fde98132b40 legacy=0x7fde98098170 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient(hunting): authenticate will time out at 1424.820422s 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde97fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fde9809faa0 0x7fde98097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:40540/0 (socket says 172.21.15.26:40540) 2024-09-15T03:54:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde97fff700 1 -- 172.21.15.26:0/2379908613 learned_addr learned my addr 172.21.15.26:0/2379908613 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9ceb1700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.122:6789/0 conn(0x7fde98138d40 legacy=0x7fde9814a1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206724863 0 0) 0x7fde98098880 con 0x7fde9809faa0 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): my global_id is 194325 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde84003570 con 0x7fde9809faa0 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2023439711 0 0) 0x7fde84003570 con 0x7fde9809faa0 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde98098880 con 0x7fde9809faa0 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde8c001550 con 0x7fde9809faa0 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:54:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): dump: 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 179128030 0 0) 0x7fde98098880 con 0x7fde9809faa0 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.26:6790/0 conn(0x7fde98132b40 legacy=0x7fde98098170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.122:6789/0 conn(0x7fde98138d40 legacy=0x7fde9814a1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 monclient: found mon.d 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9809a830 con 0x7fde9809faa0 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 5 monclient: authenticate success, global_id 194325 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient: _renew_subs 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde98098d70 con 0x7fde9809faa0 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient: _renew_subs 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9809c770 con 0x7fde9809faa0 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fde8c004ae0 con 0x7fde9809faa0 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde8c004f90 con 0x7fde9809faa0 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde967fc700 4 set_mon_vals no callback set 2024-09-15T03:54:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient: dump: 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.410 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fde78006590 con 0x7fde78002840 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fde8c00f950 con 0x7fde9809faa0 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(442..442 src has 224..442) v4 ==== 3971+0+0 (unknown 2340481024 0 0) 0x7fde8c043ff0 con 0x7fde9809faa0 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142511827 0 0) 0x7fde78006590 con 0x7fde78002840 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.389+0000 7fde7ffff700 10 monclient(hunting): my global_id is 194334 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde98098880 con 0x7fde78002840 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde9ceb1700 10 monclient: get_auth_request con 0x7fde840374d0 auth_method 0 2024-09-15T03:54:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1596412 0 0) 0x7fde98098880 con 0x7fde78002840 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde78006590 con 0x7fde78002840 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde88001430 con 0x7fde78002840 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 10 monclient: dump: 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2552599200 0 0) 0x7fde78006590 con 0x7fde78002840 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde7800a130 con 0x7fde78002840 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fde88004bc0 con 0x7fde78002840 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.26:6792/0 conn(0x7fde78002840 legacy=0x7fde78004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.393+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.26:6792/0 conn(0x7fde78002840 legacy=0x7fde78004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde9d6b2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fde78009640 con 0x7fde780090b0 2024-09-15T03:54:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170396039 0 0) 0x7fde78009640 con 0x7fde780090b0 2024-09-15T03:54:40.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient(hunting): my global_id is 194343 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde78006590 con 0x7fde780090b0 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 634476929 0 0) 0x7fde78006590 con 0x7fde780090b0 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde78009640 con 0x7fde780090b0 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde88005340 con 0x7fde780090b0 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient: dump: 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1235770752 0 0) 0x7fde78009640 con 0x7fde780090b0 2024-09-15T03:54:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde780098f0 con 0x7fde780090b0 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7354 (unknown 0 0 2379049648) 0x7fde880076a0 con 0x7fde780090b0 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.26:6792/0 conn(0x7fde780090b0 legacy=0x7fde78008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.405+0000 7fde7ffff700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.26:6792/0 conn(0x7fde780090b0 legacy=0x7fde78008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:40.418 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 80, 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.419 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.420 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:40.421 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:40.422 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:40.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:40.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.427 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:40.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.430 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:40.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.436 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:40.437 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.122:6812/20895 conn(0x7fde840374d0 legacy=0x7fde84039980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 10 monclient: shutdown 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 >> v1:172.21.15.26:6792/0 conn(0x7fde9809faa0 legacy=0x7fde98097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9ceb1700 1 -- 172.21.15.26:0/2379908613 reap_dead start 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 shutdown_connections 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 >> 172.21.15.26:0/2379908613 conn(0x7fde9806e370 msgr2=0x7fde980669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 shutdown_connections 2024-09-15T03:54:40.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.409+0000 7fde9d6b2700 1 -- 172.21.15.26:0/2379908613 wait complete. 2024-09-15T03:54:40.440 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:54:40.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-09-15T03:54:40.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.549+0000 7f69a20f2700 10 monclient: get_monmap_and_config 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.549+0000 7f69a20f2700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.549+0000 7f69a20f2700 10 monclient: monmap: 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:40.555730+0000 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:40.555730+0000 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.559 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 Processor -- start 2024-09-15T03:54:40.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- start start 2024-09-15T03:54:40.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient: init 2024-09-15T03:54:40.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient: _add_conns ranks=[0,3,1,7,5,4,2,8,6] 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient(hunting): picked mon.noname-a con 0x7f699c09e440 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient(hunting): picked mon.noname-g con 0x7f699c12ffc0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient(hunting): picked mon.noname-f con 0x7f699c130390 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c04ca10 con 0x7f699c09e440 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c09a0f0 con 0x7f699c130390 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c09e810 con 0x7f699c12ffc0 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient(hunting): authenticate will time out at 1424.985590s 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699affd700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f699c12ffc0 legacy=0x7f699c134be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699b7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f699c09e440 0x7f699c12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53700/0 (socket says 172.21.15.26:53700) 2024-09-15T03:54:40.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699b7fe700 1 -- 172.21.15.26:0/4164113798 learned_addr learned my addr 172.21.15.26:0/4164113798 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699bfff700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f699c130390 legacy=0x7f699c138310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971897075 0 0) 0x7f699c04ca10 con 0x7f699c09e440 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient(hunting): my global_id is 194283 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f697c003570 con 0x7f699c09e440 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2599331083 0 0) 0x7f697c003570 con 0x7f699c09e440 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 >> v1:172.21.15.122:6790/0 conn(0x7f699c12ffc0 legacy=0x7f699c134be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 >> v1:172.21.15.122:6789/0 conn(0x7f699c130390 legacy=0x7f699c138310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 monclient: found mon.noname-a 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c09c2a0 con 0x7f699c09e440 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 5 monclient: authenticate success, global_id 194283 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f69840017b0 con 0x7f699c09e440 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: dump: 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.568 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6984001ad0 con 0x7f699c09e440 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 1 -- 172.21.15.26:0/4164113798 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6984004c30 con 0x7f699c09e440 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 4 set_mon_vals no callback set 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f699a7fc700 10 monclient: dump: 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.570 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 10 monclient: shutdown 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- 172.21.15.26:0/4164113798 >> v1:172.21.15.26:6789/0 conn(0x7f699c09e440 legacy=0x7f699c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- 172.21.15.26:0/4164113798 shutdown_connections 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- 172.21.15.26:0/4164113798 >> 172.21.15.26:0/4164113798 conn(0x7f699c063f30 msgr2=0x7f699c0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- 172.21.15.26:0/4164113798 shutdown_connections 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.553+0000 7f69a20f2700 1 -- 172.21.15.26:0/4164113798 wait complete. 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: monmap: 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:40.572 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 Processor -- start 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 -- start start 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: init 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: _add_conns ranks=[1,7,8,0,4,3,6,2,5] 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient(hunting): picked mon.noname-f con 0x7f699c09e440 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient(hunting): picked mon.noname-i con 0x7f699c12ffc0 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient(hunting): picked mon.noname-e con 0x7f699c130390 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c119d90 con 0x7f699c09e440 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c0994f0 con 0x7f699c12ffc0 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c119480 con 0x7f699c130390 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient(hunting): authenticate will time out at 1424.989220s 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f699b7fe700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f699c09e440 legacy=0x7f699c118660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f699bfff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f699c130390 0x7f699c145cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33450/0 (socket says 172.21.15.26:33450) 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f699bfff700 1 -- 172.21.15.26:0/4090716595 learned_addr learned my addr 172.21.15.26:0/4090716595 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712774939 0 0) 0x7f699c119480 con 0x7f699c130390 2024-09-15T03:54:40.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): my global_id is 134972 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6980003570 con 0x7f699c130390 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3132042002 0 0) 0x7f6980003570 con 0x7f699c130390 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f699c119480 con 0x7f699c130390 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39301339 0 0) 0x7f699c0994f0 con 0x7f699c12ffc0 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): my global_id is 134935 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6980003570 con 0x7f699c12ffc0 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f69940042c0 con 0x7f699c130390 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): dump: 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.576 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1325094385 0 0) 0x7f699c119480 con 0x7f699c130390 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.122:6792/0 conn(0x7f699c12ffc0 legacy=0x7f699c118d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.122:6789/0 conn(0x7f699c09e440 legacy=0x7f699c118660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 monclient: found mon.e 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c099720 con 0x7f699c130390 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 5 monclient: authenticate success, global_id 134972 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: _renew_subs 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c11b2d0 con 0x7f699c130390 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f69940044f0 con 0x7f699c130390 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: _renew_subs 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c12d380 con 0x7f699c130390 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6994004a20 con 0x7f699c130390 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f6999ffb700 4 set_mon_vals no callback set 2024-09-15T03:54:40.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.557+0000 7f69937fe700 10 monclient: dump: 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.579 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.580 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.580 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.580 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69a20f2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f699c06dbc0 con 0x7f699c06d7f0 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476605754 0 0) 0x7f699c06dbc0 con 0x7f699c06d7f0 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient(hunting): my global_id is 134981 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699c119480 con 0x7f699c06d7f0 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1959305195 0 0) 0x7f699c119480 con 0x7f699c06d7f0 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f699c06dbc0 con 0x7f699c06d7f0 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6984004c30 con 0x7f699c06d7f0 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:40.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient: dump: 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6994004b80 con 0x7f699c130390 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(442..442 src has 224..442) v4 ==== 3971+0+0 (unknown 2340481024 0 0) 0x7f6994043e10 con 0x7f699c130390 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1773209682 0 0) 0x7f699c06dbc0 con 0x7f699c06d7f0 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f699c066fc0 con 0x7f699c06d7f0 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f699affd700 10 monclient: get_auth_request con 0x7f6980037c30 auth_method 0 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6984001630 con 0x7f699c06d7f0 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.26:6793/0 conn(0x7f699c06d7f0 legacy=0x7f699c0a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.561+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.26:6793/0 conn(0x7f699c06d7f0 legacy=0x7f699c0a0340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f699c04bbf0 con 0x7f699c02ad10 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091364508 0 0) 0x7f699c04bbf0 con 0x7f699c02ad10 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient(hunting): my global_id is 134990 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699c06dbc0 con 0x7f699c02ad10 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3918297113 0 0) 0x7f699c06dbc0 con 0x7f699c02ad10 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f699c04bbf0 con 0x7f699c02ad10 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6984005530 con 0x7f699c02ad10 2024-09-15T03:54:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient: dump: 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 846293236 0 0) 0x7f699c04bbf0 con 0x7f699c02ad10 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f699c06f230 con 0x7f699c02ad10 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7524 (unknown 0 0 2320935962) 0x7f6984007790 con 0x7f699c02ad10 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.26:6793/0 conn(0x7f699c02ad10 legacy=0x7f699c0a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69937fe700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.26:6793/0 conn(0x7f699c02ad10 legacy=0x7f699c0a0a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 80, 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.592 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.593 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.594 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:40.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.596 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:40.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.600 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:40.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.604 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.608 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:40.609 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:40.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.122:6812/20895 conn(0x7f6980037c30 legacy=0x7f698003a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 10 monclient: shutdown 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f699bfff700 1 -- 172.21.15.26:0/4090716595 reap_dead start 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 >> v1:172.21.15.26:6793/0 conn(0x7f699c130390 legacy=0x7f699c145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 shutdown_connections 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 >> 172.21.15.26:0/4090716595 conn(0x7f699c063f30 msgr2=0x7f699c12efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 shutdown_connections 2024-09-15T03:54:40.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.569+0000 7f69a20f2700 1 -- 172.21.15.26:0/4090716595 wait complete. 2024-09-15T03:54:40.612 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:54:40.612 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-09-15T03:54:40.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:54:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.721+0000 7f31abf2d700 10 monclient: get_monmap_and_config 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.721+0000 7f31abf2d700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.721+0000 7f31abf2d700 10 monclient: monmap: 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:40.726734+0000 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:40.726734+0000 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.730 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.721+0000 7f31abf2d700 1 Processor -- start 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.721+0000 7f31abf2d700 1 -- start start 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.721+0000 7f31abf2d700 10 monclient: init 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient: _add_conns ranks=[4,2,0,8,7,6,1,3,5] 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient(hunting): picked mon.noname-c con 0x7f31a409e440 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient(hunting): picked mon.noname-b con 0x7f31a412ffc0 addr v1:172.21.15.26:6790/0 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient(hunting): picked mon.noname-a con 0x7f31a4130390 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a404ca10 con 0x7f31a4130390 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a409a0f0 con 0x7f31a412ffc0 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a409e810 con 0x7f31a409e440 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient(hunting): authenticate will time out at 1425.156694s 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a9cc9700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f31a409e440 legacy=0x7f31a412f8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a94c8700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7f31a412ffc0 legacy=0x7f31a4134be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31aa4ca700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f31a4130390 0x7f31a4138310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53714/0 (socket says 172.21.15.26:53714) 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31aa4ca700 1 -- 172.21.15.26:0/2440623944 learned_addr learned my addr 172.21.15.26:0/2440623944 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355914774 0 0) 0x7f31a404ca10 con 0x7f31a4130390 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient(hunting): my global_id is 194292 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3190003570 con 0x7f31a4130390 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304545301 0 0) 0x7f3190003570 con 0x7f31a4130390 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 >> v1:172.21.15.26:6791/0 conn(0x7f31a409e440 legacy=0x7f31a412f8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 >> v1:172.21.15.26:6790/0 conn(0x7f31a412ffc0 legacy=0x7f31a4134be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 monclient: found mon.noname-a 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a409c2a0 con 0x7f31a4130390 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 5 monclient: authenticate success, global_id 194292 2024-09-15T03:54:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f319c001b30 con 0x7f31a4130390 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: dump: 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f319c001e50 con 0x7f31a4130390 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 4 set_mon_vals no callback set 2024-09-15T03:54:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 1 -- 172.21.15.26:0/2440623944 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f319c004fb0 con 0x7f31a4130390 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31a8cc7700 10 monclient: dump: 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient: shutdown 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- 172.21.15.26:0/2440623944 >> v1:172.21.15.26:6789/0 conn(0x7f31a4130390 legacy=0x7f31a4138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- 172.21.15.26:0/2440623944 shutdown_connections 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- 172.21.15.26:0/2440623944 >> 172.21.15.26:0/2440623944 conn(0x7f31a4063f30 msgr2=0x7f31a40a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- 172.21.15.26:0/2440623944 shutdown_connections 2024-09-15T03:54:40.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 1 -- 172.21.15.26:0/2440623944 wait complete. 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.725+0000 7f31abf2d700 10 monclient: monmap: 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 Processor -- start 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 -- start start 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: init 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: _add_conns ranks=[4,1,8,5,0,7,2,3,6] 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient(hunting): picked mon.noname-c con 0x7f31a409e440 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient(hunting): picked mon.noname-f con 0x7f31a412ffc0 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient(hunting): picked mon.noname-e con 0x7f31a4130390 addr v1:172.21.15.26:6793/0 2024-09-15T03:54:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a406cee0 con 0x7f31a412ffc0 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a4099720 con 0x7f31a409e440 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a406c9f0 con 0x7f31a4130390 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31a9cc9700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f31a409e440 legacy=0x7f31a406ab60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient(hunting): authenticate will time out at 1425.161618s 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31a94c8700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f31a412ffc0 legacy=0x7f31a406b270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31aa4ca700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f31a4130390 0x7f31a4145c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33496/0 (socket says 172.21.15.26:33496) 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31aa4ca700 1 -- 172.21.15.26:0/636120673 learned_addr learned my addr 172.21.15.26:0/636120673 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994211286 0 0) 0x7f31a406c9f0 con 0x7f31a4130390 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): my global_id is 134999 2024-09-15T03:54:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3188003570 con 0x7f31a4130390 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2615225958 0 0) 0x7f3188003570 con 0x7f31a4130390 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31a406c9f0 con 0x7f31a4130390 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f319c001af0 con 0x7f31a4130390 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): dump: 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 807664417 0 0) 0x7f31a406c9f0 con 0x7f31a4130390 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.26:6791/0 conn(0x7f31a409e440 legacy=0x7f31a406ab60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.122:6789/0 conn(0x7f31a412ffc0 legacy=0x7f31a406b270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 monclient: found mon.e 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a406c4e0 con 0x7f31a4130390 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 5 monclient: authenticate success, global_id 134999 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: _renew_subs 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a412a490 con 0x7f31a4130390 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: _renew_subs 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:54:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a406b980 con 0x7f31a4130390 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f319c0051e0 con 0x7f31a4130390 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f319c005690 con 0x7f31a4130390 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f319bfff700 4 set_mon_vals no callback set 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.729+0000 7f3199ffb700 10 monclient: dump: 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f31abf2d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f319c010050 con 0x7f31a4130390 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3184006590 con 0x7f3184002840 2024-09-15T03:54:40.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(442..442 src has 224..442) v4 ==== 3971+0+0 (unknown 2340481024 0 0) 0x7f319c044570 con 0x7f31a4130390 2024-09-15T03:54:40.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f31a94c8700 10 monclient: get_auth_request con 0x7f3188037380 auth_method 0 2024-09-15T03:54:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851433895 0 0) 0x7f3184006590 con 0x7f3184002840 2024-09-15T03:54:40.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient(hunting): my global_id is 5522 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a406c9f0 con 0x7f3184002840 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1239309890 0 0) 0x7f31a406c9f0 con 0x7f3184002840 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3184006590 con 0x7f3184002840 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3194004150 con 0x7f3184002840 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient: dump: 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.757 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3859351302 0 0) 0x7f3184006590 con 0x7f3184002840 2024-09-15T03:54:40.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f318400a130 con 0x7f3184002840 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3194006470 con 0x7f3184002840 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.122:6791/0 conn(0x7f3184002840 legacy=0x7f3184004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.733+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.122:6791/0 conn(0x7f3184002840 legacy=0x7f3184004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f31abf2d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3184009640 con 0x7f31840090b0 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744273453 0 0) 0x7f3184009640 con 0x7f31840090b0 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f3199ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f3199ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f3199ffb700 10 monclient(hunting): my global_id is 5531 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3184006590 con 0x7f31840090b0 2024-09-15T03:54:40.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.745+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2990916529 0 0) 0x7f3184006590 con 0x7f31840090b0 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3184009640 con 0x7f31840090b0 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3194004ca0 con 0x7f31840090b0 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 10 monclient: dump: 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.760 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2981737990 0 0) 0x7f3184009640 con 0x7f31840090b0 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f31840098f0 con 0x7f31840090b0 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7696 (unknown 0 0 2097010681) 0x7f31940061f0 con 0x7f31840090b0 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:40.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.122:6791/0 conn(0x7f31840090b0 legacy=0x7f3184008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f3199ffb700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.122:6791/0 conn(0x7f31840090b0 legacy=0x7f3184008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 80, 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:40.762 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.763 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.764 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:40.765 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:40.766 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.767 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.768 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:40.769 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:40.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:40.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.775 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.776 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.779 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:54:40.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.122:6812/20895 conn(0x7f3188037380 legacy=0x7f3188039830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 10 monclient: shutdown 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 >> v1:172.21.15.26:6793/0 conn(0x7f31a4130390 legacy=0x7f31a4145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31aa4ca700 1 -- 172.21.15.26:0/636120673 reap_dead start 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31aa4ca700 1 -- 172.21.15.26:0/636120673 reap_dead start 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 shutdown_connections 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 >> 172.21.15.26:0/636120673 conn(0x7f31a4063f30 msgr2=0x7f31a4097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 shutdown_connections 2024-09-15T03:54:40.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.749+0000 7f31abf2d700 1 -- 172.21.15.26:0/636120673 wait complete. 2024-09-15T03:54:40.785 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:54:40.785 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon i 2024-09-15T03:54:40.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:54:40.835 INFO:tasks.rados.rados.0.smithi122.stdout:1976: done (1 left) 2024-09-15T03:54:40.836 INFO:tasks.rados.rados.0.smithi122.stdout:1978: done (0 left) 2024-09-15T03:54:40.836 INFO:tasks.rados.rados.0.smithi122.stdout:1979: write oid 12 current snap is 218 2024-09-15T03:54:40.836 INFO:tasks.rados.rados.0.smithi122.stdout:1979: seq_num 719 ranges {484233=641213,1707612=376621} 2024-09-15T03:54:40.844 INFO:tasks.rados.rados.0.smithi122.stdout:1979: writing smithi12223906-12 from 484233 to 1125446 tid 1 2024-09-15T03:54:40.852 INFO:tasks.rados.rados.0.smithi122.stdout:1979: writing smithi12223906-12 from 1707612 to 2084233 tid 2 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1980: read oid 11 snap 217 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1980: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1981: read oid 7 snap -1 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1981: expect (ObjNum 687 snap 211 seq_num 687) 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1982: read oid 44 snap -1 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1982: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1983: read oid 19 snap -1 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1983: expect (ObjNum 714 snap 217 seq_num 714) 2024-09-15T03:54:40.853 INFO:tasks.rados.rados.0.smithi122.stdout:1984: snap_create 2024-09-15T03:54:40.857 INFO:tasks.rados.rados.0.smithi122.stdout:1980: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:54:40.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient: get_monmap_and_config 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient: monmap: 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:54:40.909618+0000 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:54:40.909618+0000 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.913 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 1 Processor -- start 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 1 -- start start 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient: init 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient: _add_conns ranks=[1,7,5,4,2,0,8,6,3] 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient(hunting): picked mon.noname-f con 0x7f03bc09f600 addr v1:172.21.15.122:6789/0 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient(hunting): picked mon.noname-i con 0x7f03bc09c640 addr v1:172.21.15.122:6792/0 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient(hunting): picked mon.noname-h con 0x7f03bc09b2c0 addr v1:172.21.15.122:6791/0 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc04ca10 con 0x7f03bc09f600 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc09af10 con 0x7f03bc09b2c0 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc09f9d0 con 0x7f03bc09c640 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c26b3700 10 monclient(hunting): authenticate will time out at 1425.339174s 2024-09-15T03:54:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c16b1700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f03bc09f600 legacy=0x7f03bc09de00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c0eb0700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f03bc09c640 0x7f03bc09e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:37364/0 (socket says 172.21.15.26:37364) 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03c0eb0700 1 -- 172.21.15.26:0/327067021 learned_addr learned my addr 172.21.15.26:0/327067021 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143865232 0 0) 0x7f03bc09af10 con 0x7f03bc09b2c0 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03bbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.905+0000 7f03bbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient(hunting): my global_id is 5540 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a4004680 con 0x7f03bc09b2c0 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234487196 0 0) 0x7f03bc09f9d0 con 0x7f03bc09c640 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient(hunting): my global_id is 134944 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03bc09af10 con 0x7f03bc09c640 2024-09-15T03:54:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1308217558 0 0) 0x7f03bc09af10 con 0x7f03bc09c640 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 >> v1:172.21.15.122:6791/0 conn(0x7f03bc09b2c0 legacy=0x7f03bc1375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 >> v1:172.21.15.122:6789/0 conn(0x7f03bc09f600 legacy=0x7f03bc09de00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 monclient: found mon.noname-i 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc09ec20 con 0x7f03bc09c640 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f03b0001790 con 0x7f03bc09c640 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 5 monclient: authenticate success, global_id 134944 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: dump: 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03b0001a40 con 0x7f03bc09c640 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: get_monmap_and_config success 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 1 -- 172.21.15.26:0/327067021 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f03b0004c30 con 0x7f03bc09c640 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 4 set_mon_vals no callback set 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03bbfff700 10 monclient: dump: 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.922 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: shutdown 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- 172.21.15.26:0/327067021 >> v1:172.21.15.122:6792/0 conn(0x7f03bc09c640 legacy=0x7f03bc09e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- 172.21.15.26:0/327067021 shutdown_connections 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- 172.21.15.26:0/327067021 >> 172.21.15.26:0/327067021 conn(0x7f03bc0a2ea0 msgr2=0x7f03bc09eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- 172.21.15.26:0/327067021 shutdown_connections 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- 172.21.15.26:0/327067021 wait complete. 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: build_initial_monmap 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: monmap: 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:54:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 Processor -- start 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- start start 2024-09-15T03:54:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: init 2024-09-15T03:54:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: _reopen_session rank -1 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient: _add_conns ranks=[3,0,4,7,8,5,1,6,2] 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient(hunting): picked mon.noname-g con 0x7f03bc09b2c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient(hunting): picked mon.noname-a con 0x7f03bc09c640 addr v1:172.21.15.26:6789/0 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient(hunting): picked mon.noname-c con 0x7f03bc09f600 addr v1:172.21.15.26:6791/0 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc06e300 con 0x7f03bc09c640 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc1295c0 con 0x7f03bc09b2c0 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc128430 con 0x7f03bc09f600 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient(hunting): _renew_subs 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c16b1700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f03bc09b2c0 legacy=0x7f03bc06cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c26b3700 10 monclient(hunting): authenticate will time out at 1425.343748s 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c1eb2700 1 -- >> v1:172.21.15.26:6791/0 conn(0x7f03bc09f600 legacy=0x7f03bc145890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6791/0 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c0eb0700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03bc09c640 0x7f03bc06d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53722/0 (socket says 172.21.15.26:53722) 2024-09-15T03:54:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.909+0000 7f03c0eb0700 1 -- 172.21.15.26:0/2204326215 learned_addr learned my addr 172.21.15.26:0/2204326215 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638405283 0 0) 0x7f03bc06e300 con 0x7f03bc09c640 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): my global_id is 194301 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a8003570 con 0x7f03bc09c640 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 474846638 0 0) 0x7f03a8003570 con 0x7f03bc09c640 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f03bc06e300 con 0x7f03bc09c640 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f03b0001750 con 0x7f03bc09c640 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): dump: 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.931 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2714487883 0 0) 0x7f03bc06e300 con 0x7f03bc09c640 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.26:6791/0 conn(0x7f03bc09f600 legacy=0x7f03bc145890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.122:6790/0 conn(0x7f03bc09b2c0 legacy=0x7f03bc06cb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: _finish_hunting 0 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 monclient: found mon.a 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc1289f0 con 0x7f03bc09c640 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: _finish_auth 0 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 5 monclient: authenticate success, global_id 194301 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 10 monclient: _renew_subs 2024-09-15T03:54:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc128770 con 0x7f03bc09c640 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 10 monclient: _renew_subs 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc06e720 con 0x7f03bc09c640 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f03b0004f70 con 0x7f03bc09c640 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f03b00054a0 con 0x7f03bc09c640 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03bb7fe700 4 set_mon_vals no callback set 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: dump: 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f039c006590 con 0x7f039c002840 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f03b000fee0 con 0x7f03bc09c640 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(443..443 src has 224..443) v4 ==== 3943+0+0 (unknown 1546806804 0 0) 0x7f03b0044250 con 0x7f03bc09c640 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367658939 0 0) 0x7f039c006590 con 0x7f039c002840 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient(hunting): my global_id is 134953 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03bc06e300 con 0x7f039c002840 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03c1eb2700 10 monclient: get_auth_request con 0x7f03a80373c0 auth_method 0 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2443689592 0 0) 0x7f03bc06e300 con 0x7f039c002840 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f039c006590 con 0x7f039c002840 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f03ac004110 con 0x7f039c002840 2024-09-15T03:54:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 10 monclient: dump: 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 236448821 0 0) 0x7f039c006590 con 0x7f039c002840 2024-09-15T03:54:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.913+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f039c00a130 con 0x7f039c002840 2024-09-15T03:54:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.917+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f03ac006410 con 0x7f039c002840 2024-09-15T03:54:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.917+0000 7f03b97fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:54:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.917+0000 7f03b97fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:54:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.917+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.122:6792/0 conn(0x7f039c002840 legacy=0x7f039c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.917+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.122:6792/0 conn(0x7f039c002840 legacy=0x7f039c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03c26b3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:54:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f039c007520 con 0x7f039c0068d0 2024-09-15T03:54:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329223399 0 0) 0x7f039c007520 con 0x7f039c0068d0 2024-09-15T03:54:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:54:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 10 monclient(hunting): my global_id is 134962 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f039c006590 con 0x7f039c0068d0 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2280876376 0 0) 0x7f039c006590 con 0x7f039c0068d0 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f039c007520 con 0x7f039c0068d0 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f03ac004ca0 con 0x7f039c0068d0 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 10 monclient: dump: 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:54:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1055165054 0 0) 0x7f039c007520 con 0x7f039c0068d0 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.925+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f039c008760 con 0x7f039c0068d0 2024-09-15T03:54:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7524 (unknown 0 0 2075810726) 0x7f03ac0061f0 con 0x7f039c0068d0 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03b97fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03b97fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.122:6792/0 conn(0x7f039c0068d0 legacy=0x7f039c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03b97fa700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.122:6792/0 conn(0x7f039c0068d0 legacy=0x7f039c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:54:40.943 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 80, 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.944 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.945 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:54:40.946 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:54:40.947 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.948 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.950 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.953 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.954 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.956 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.957 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.958 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.961 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:54:40.962 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:54:40.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:54:40.963 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:54:40.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:54:40.963 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:54:40.963 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:54:40.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.122:6812/20895 conn(0x7f03a80373c0 legacy=0x7f03a8039870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 10 monclient: shutdown 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 >> v1:172.21.15.26:6789/0 conn(0x7f03bc09c640 legacy=0x7f03bc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c1eb2700 1 -- 172.21.15.26:0/2204326215 reap_dead start 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 shutdown_connections 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 >> 172.21.15.26:0/2204326215 conn(0x7f03bc0a2ea0 msgr2=0x7f03bc094600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 shutdown_connections 2024-09-15T03:54:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:54:40.929+0000 7f03c26b3700 1 -- 172.21.15.26:0/2204326215 wait complete. 2024-09-15T03:54:40.965 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T03:54:40.966 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:54:40.997 INFO:tasks.rados.rados.0.smithi122.stdout:1981: expect (ObjNum 687 snap 211 seq_num 687) 2024-09-15T03:54:41.204 INFO:tasks.rados.rados.0.smithi122.stdout:1979: finishing write tid 1 to smithi12223906-12 2024-09-15T03:54:41.205 INFO:tasks.rados.rados.0.smithi122.stdout:1979: finishing write tid 2 to smithi12223906-12 2024-09-15T03:54:41.205 INFO:tasks.rados.rados.0.smithi122.stdout:1979: finishing write tid 3 to smithi12223906-12 2024-09-15T03:54:41.205 INFO:tasks.rados.rados.0.smithi122.stdout:1979: finishing write tid 4 to smithi12223906-12 2024-09-15T03:54:41.205 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 732 (ObjNum 719 snap 218 seq_num 719) dirty exists 2024-09-15T03:54:41.205 INFO:tasks.rados.rados.0.smithi122.stdout:1979: left oid 12 (ObjNum 719 snap 218 seq_num 719) 2024-09-15T03:54:41.205 INFO:tasks.rados.rados.0.smithi122.stdout:1982: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:54:41.390 INFO:tasks.rados.rados.0.smithi122.stdout:1983: expect (ObjNum 714 snap 217 seq_num 714) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1979: done (5 left) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1980: done (4 left) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1981: done (3 left) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1982: done (2 left) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1983: done (1 left) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1984: done (0 left) 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1985: copy_from oid 25 from oid 40 current snap is 219 2024-09-15T03:54:41.858 INFO:tasks.rados.rados.0.smithi122.stdout:1986: delete oid 28 current snap is 219 2024-09-15T03:54:41.859 INFO:tasks.rados.rados.0.smithi122.stdout:1985: finishing copy_from to smithi12223906-25 2024-09-15T03:54:41.859 INFO:tasks.rados.rados.0.smithi122.stdout:1985: got expected ENOENT (src dne) 2024-09-15T03:54:41.859 INFO:tasks.rados.rados.0.smithi122.stdout:1985: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:54:41.864 INFO:tasks.rados.rados.0.smithi122.stdout:1985: done (1 left) 2024-09-15T03:54:41.864 INFO:tasks.rados.rados.0.smithi122.stdout:1986: done (0 left) 2024-09-15T03:54:41.864 INFO:tasks.rados.rados.0.smithi122.stdout:1987: read oid 29 snap -1 2024-09-15T03:54:41.864 INFO:tasks.rados.rados.0.smithi122.stdout:1987: expect deleted 2024-09-15T03:54:41.864 INFO:tasks.rados.rados.0.smithi122.stdout:1988: write (excl) oid 35 current snap is 219 2024-09-15T03:54:41.864 INFO:tasks.rados.rados.0.smithi122.stdout:1988: seq_num 721 ranges {517847=791711,2108223=9624} 2024-09-15T03:54:41.873 INFO:tasks.rados.rados.0.smithi122.stdout:1988: writing smithi12223906-35 from 517847 to 1309558 tid 1 2024-09-15T03:54:41.879 INFO:tasks.rados.rados.0.smithi122.stdout:1988: writing smithi12223906-35 from 2108223 to 2117847 tid 2 2024-09-15T03:54:41.879 INFO:tasks.rados.rados.0.smithi122.stdout:1989: snap_create 2024-09-15T03:54:41.887 INFO:tasks.rados.rados.0.smithi122.stdout:1988: finishing write tid 1 to smithi12223906-35 2024-09-15T03:54:41.887 INFO:tasks.rados.rados.0.smithi122.stdout:1988: finishing write tid 2 to smithi12223906-35 2024-09-15T03:54:41.888 INFO:tasks.rados.rados.0.smithi122.stdout:1988: finishing write tid 3 to smithi12223906-35 2024-09-15T03:54:41.888 INFO:tasks.rados.rados.0.smithi122.stdout:1988: finishing write tid 4 to smithi12223906-35 2024-09-15T03:54:41.888 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 586 (ObjNum 721 snap 219 seq_num 721) dirty exists 2024-09-15T03:54:41.888 INFO:tasks.rados.rados.0.smithi122.stdout:1988: left oid 35 (ObjNum 721 snap 219 seq_num 721) 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:1987: done (2 left) 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:1988: done (1 left) 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:1989: done (0 left) 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:1990: rollback oid 28 current snap is 220 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 28 to 207 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:1991: write (excl) oid 8 current snap is 220 2024-09-15T03:54:42.871 INFO:tasks.rados.rados.0.smithi122.stdout:1991: seq_num 722 ranges {534654=466960,1766705=367949} 2024-09-15T03:54:42.881 INFO:tasks.rados.rados.0.smithi122.stdout:1991: writing smithi12223906-8 from 534654 to 1001614 tid 1 2024-09-15T03:54:42.887 INFO:tasks.rados.rados.0.smithi122.stdout:1991: writing smithi12223906-8 from 1766705 to 2134654 tid 2 2024-09-15T03:54:42.887 INFO:tasks.rados.rados.0.smithi122.stdout:1992: rollback oid 40 current snap is 220 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 218 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1990: finishing rollback tid 1 to smithi12223906-28 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1990: finishing rollback tid 2 to smithi12223906-28 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 816 (ObjNum 674 snap 204 seq_num 674) dirty exists 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1990: done (2 left) 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1993: read oid 12 snap -1 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1993: expect (ObjNum 719 snap 218 seq_num 719) 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1994: snap_create 2024-09-15T03:54:42.888 INFO:tasks.rados.rados.0.smithi122.stdout:1991: finishing write tid 1 to smithi12223906-8 2024-09-15T03:54:42.889 INFO:tasks.rados.rados.0.smithi122.stdout:1992: finishing rollback tid 1 to smithi12223906-40 2024-09-15T03:54:42.889 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 0 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:54:42.892 INFO:tasks.rados.rados.0.smithi122.stdout:1991: finishing write tid 2 to smithi12223906-8 2024-09-15T03:54:42.893 INFO:tasks.rados.rados.0.smithi122.stdout:1991: finishing write tid 3 to smithi12223906-8 2024-09-15T03:54:42.894 INFO:tasks.rados.rados.0.smithi122.stdout:1991: finishing write tid 4 to smithi12223906-8 2024-09-15T03:54:42.894 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 929 (ObjNum 722 snap 220 seq_num 722) dirty exists 2024-09-15T03:54:42.894 INFO:tasks.rados.rados.0.smithi122.stdout:1991: left oid 8 (ObjNum 722 snap 220 seq_num 722) 2024-09-15T03:54:42.898 INFO:tasks.rados.rados.0.smithi122.stdout:1993: expect (ObjNum 719 snap 218 seq_num 719) 2024-09-15T03:54:43.880 INFO:tasks.rados.rados.0.smithi122.stdout:1991: done (3 left) 2024-09-15T03:54:43.880 INFO:tasks.rados.rados.0.smithi122.stdout:1992: done (2 left) 2024-09-15T03:54:43.880 INFO:tasks.rados.rados.0.smithi122.stdout:1993: done (1 left) 2024-09-15T03:54:43.881 INFO:tasks.rados.rados.0.smithi122.stdout:1994: done (0 left) 2024-09-15T03:54:43.881 INFO:tasks.rados.rados.0.smithi122.stdout:1995: read oid 38 snap -1 2024-09-15T03:54:43.881 INFO:tasks.rados.rados.0.smithi122.stdout:1995: expect (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:54:43.881 INFO:tasks.rados.rados.0.smithi122.stdout:1996: write (excl) oid 30 current snap is 221 2024-09-15T03:54:43.881 INFO:tasks.rados.rados.0.smithi122.stdout:1996: seq_num 723 ranges {551461=658562,1941540=209921} 2024-09-15T03:54:43.891 INFO:tasks.rados.rados.0.smithi122.stdout:1996: writing smithi12223906-30 from 551461 to 1210023 tid 1 2024-09-15T03:54:43.899 INFO:tasks.rados.rados.0.smithi122.stdout:1996: writing smithi12223906-30 from 1941540 to 2151461 tid 2 2024-09-15T03:54:43.899 INFO:tasks.rados.rados.0.smithi122.stdout:1997: delete oid 27 current snap is 221 2024-09-15T03:54:43.900 INFO:tasks.rados.rados.0.smithi122.stdout:1995: expect (ObjNum 588 snap 176 seq_num 588) 2024-09-15T03:54:44.028 INFO:tasks.rados.rados.0.smithi122.stdout:1996: finishing write tid 1 to smithi12223906-30 2024-09-15T03:54:44.028 INFO:tasks.rados.rados.0.smithi122.stdout:1996: finishing write tid 2 to smithi12223906-30 2024-09-15T03:54:44.028 INFO:tasks.rados.rados.0.smithi122.stdout:1996: finishing write tid 3 to smithi12223906-30 2024-09-15T03:54:44.028 INFO:tasks.rados.rados.0.smithi122.stdout:1996: finishing write tid 4 to smithi12223906-30 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 807 (ObjNum 723 snap 221 seq_num 723) dirty exists 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:1996: left oid 30 (ObjNum 723 snap 221 seq_num 723) 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:1995: done (2 left) 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:1996: done (1 left) 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:1997: done (0 left) 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:1998: write oid 4 current snap is 221 2024-09-15T03:54:44.029 INFO:tasks.rados.rados.0.smithi122.stdout:1998: seq_num 725 ranges {585075=409060,1542151=642924} 2024-09-15T03:54:44.042 INFO:tasks.rados.rados.0.smithi122.stdout:1998: writing smithi12223906-4 from 585075 to 994135 tid 1 2024-09-15T03:54:44.049 INFO:tasks.rados.rados.0.smithi122.stdout:1998: writing smithi12223906-4 from 1542151 to 2185075 tid 2 2024-09-15T03:54:44.049 INFO:tasks.rados.rados.0.smithi122.stdout:1999: write oid 38 current snap is 221 2024-09-15T03:54:44.050 INFO:tasks.rados.rados.0.smithi122.stdout:1999: seq_num 726 ranges {601882=484309,1600633=524904} 2024-09-15T03:54:44.055 INFO:tasks.rados.rados.0.smithi122.stdout:1999: writing smithi12223906-38 from 601882 to 1086191 tid 1 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1999: writing smithi12223906-38 from 1600633 to 2125537 tid 2 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:2000: delete oid 37 current snap is 221 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1998: finishing write tid 1 to smithi12223906-4 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1998: finishing write tid 2 to smithi12223906-4 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1998: finishing write tid 3 to smithi12223906-4 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1998: finishing write tid 4 to smithi12223906-4 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 802 (ObjNum 725 snap 221 seq_num 725) dirty exists 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1998: left oid 4 (ObjNum 725 snap 221 seq_num 725) 2024-09-15T03:54:44.062 INFO:tasks.rados.rados.0.smithi122.stdout:1999: finishing write tid 1 to smithi12223906-38 2024-09-15T03:54:44.063 INFO:tasks.rados.rados.0.smithi122.stdout:1998: done (2 left) 2024-09-15T03:54:44.064 INFO:tasks.rados.rados.0.smithi122.stdout:2000: done (1 left) 2024-09-15T03:54:44.064 INFO:tasks.rados.rados.0.smithi122.stdout:2001: snap_create 2024-09-15T03:54:44.066 INFO:tasks.rados.rados.0.smithi122.stdout:1999: finishing write tid 2 to smithi12223906-38 2024-09-15T03:54:44.067 INFO:tasks.rados.rados.0.smithi122.stdout:1999: finishing write tid 3 to smithi12223906-38 2024-09-15T03:54:44.067 INFO:tasks.rados.rados.0.smithi122.stdout:1999: finishing write tid 4 to smithi12223906-38 2024-09-15T03:54:44.067 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 932 (ObjNum 726 snap 221 seq_num 726) dirty exists 2024-09-15T03:54:44.067 INFO:tasks.rados.rados.0.smithi122.stdout:1999: left oid 38 (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:1999: done (1 left) 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:2001: done (0 left) 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:2002: rollback oid 29 current snap is 222 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 29 to 219 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:2003: rollback oid 44 current snap is 222 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 204 2024-09-15T03:54:44.885 INFO:tasks.rados.rados.0.smithi122.stdout:2004: delete oid 14 current snap is 222 2024-09-15T03:54:44.887 INFO:tasks.rados.rados.0.smithi122.stdout:2002: finishing rollback tid 1 to smithi12223906-29 2024-09-15T03:54:44.887 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 809 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-15T03:54:44.888 INFO:tasks.rados.rados.0.smithi122.stdout:2002: done (2 left) 2024-09-15T03:54:44.888 INFO:tasks.rados.rados.0.smithi122.stdout:2004: done (1 left) 2024-09-15T03:54:44.888 INFO:tasks.rados.rados.0.smithi122.stdout:2005: delete oid 31 current snap is 222 2024-09-15T03:54:44.894 INFO:tasks.rados.rados.0.smithi122.stdout:2003: finishing rollback tid 0 to smithi12223906-44 2024-09-15T03:54:44.894 INFO:tasks.rados.rados.0.smithi122.stdout:2003: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:54:44.895 INFO:tasks.rados.rados.0.smithi122.stdout:2003: finishing rollback tid 2 to smithi12223906-44 2024-09-15T03:54:44.895 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1125 (ObjNum 636 snap 195 seq_num 636) dirty exists 2024-09-15T03:54:44.895 INFO:tasks.rados.rados.0.smithi122.stdout:2003: done (1 left) 2024-09-15T03:54:44.895 INFO:tasks.rados.rados.0.smithi122.stdout:2005: done (0 left) 2024-09-15T03:54:44.895 INFO:tasks.rados.rados.0.smithi122.stdout:2006: snap_remove snap 201 2024-09-15T03:54:45.007 INFO:tasks.rados.rados.0.smithi122.stdout:2006: done (0 left) 2024-09-15T03:54:45.008 INFO:tasks.rados.rados.0.smithi122.stdout:2007: delete oid 17 current snap is 222 2024-09-15T03:54:45.032 INFO:tasks.rados.rados.0.smithi122.stdout:2007: done (0 left) 2024-09-15T03:54:45.032 INFO:tasks.rados.rados.0.smithi122.stdout:2008: write (excl) oid 22 current snap is 222 2024-09-15T03:54:45.032 INFO:tasks.rados.rados.0.smithi122.stdout:2008: seq_num 731 ranges {685917=576907,1893043=392874} 2024-09-15T03:54:45.041 INFO:tasks.rados.rados.0.smithi122.stdout:2008: writing smithi12223906-22 from 685917 to 1262824 tid 1 2024-09-15T03:54:45.048 INFO:tasks.rados.rados.0.smithi122.stdout:2008: writing smithi12223906-22 from 1893043 to 2285917 tid 2 2024-09-15T03:54:45.048 INFO:tasks.rados.rados.0.smithi122.stdout:2009: rollback oid 43 current snap is 222 2024-09-15T03:54:45.048 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 204 2024-09-15T03:54:45.048 INFO:tasks.rados.rados.0.smithi122.stdout:2008: finishing write tid 1 to smithi12223906-22 2024-09-15T03:54:45.048 INFO:tasks.rados.rados.0.smithi122.stdout:2010: write oid 46 current snap is 222 2024-09-15T03:54:45.048 INFO:tasks.rados.rados.0.smithi122.stdout:2010: seq_num 732 ranges {702724=652156,1951525=351199} 2024-09-15T03:54:45.055 INFO:tasks.rados.rados.0.smithi122.stdout:2010: writing smithi12223906-46 from 702724 to 1354880 tid 1 2024-09-15T03:54:45.061 INFO:tasks.rados.rados.0.smithi122.stdout:2010: writing smithi12223906-46 from 1951525 to 2302724 tid 2 2024-09-15T03:54:45.061 INFO:tasks.rados.rados.0.smithi122.stdout:2011: write (excl) oid 9 current snap is 222 2024-09-15T03:54:45.061 INFO:tasks.rados.rados.0.smithi122.stdout:2009: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:54:45.061 INFO:tasks.rados.rados.0.smithi122.stdout:2009: finishing rollback tid 2 to smithi12223906-43 2024-09-15T03:54:45.061 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 1151 (ObjNum 664 snap 203 seq_num 664) dirty exists 2024-09-15T03:54:45.061 INFO:tasks.rados.rados.0.smithi122.stdout:2008: finishing write tid 2 to smithi12223906-22 2024-09-15T03:54:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2008: finishing write tid 3 to smithi12223906-22 2024-09-15T03:54:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2011: seq_num 733 ranges {719531=727405,1893654=425877} 2024-09-15T03:54:45.069 INFO:tasks.rados.rados.0.smithi122.stdout:2011: writing smithi12223906-9 from 719531 to 1446936 tid 1 2024-09-15T03:54:45.074 INFO:tasks.rados.rados.0.smithi122.stdout:2011: writing smithi12223906-9 from 1893654 to 2319531 tid 2 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2009: done (3 left) 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2012: snap_create 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2008: finishing write tid 4 to smithi12223906-22 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1141 (ObjNum 731 snap 222 seq_num 731) dirty exists 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2008: left oid 22 (ObjNum 731 snap 222 seq_num 731) 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2010: finishing write tid 1 to smithi12223906-46 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2010: finishing write tid 2 to smithi12223906-46 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2010: finishing write tid 3 to smithi12223906-46 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:2010: finishing write tid 4 to smithi12223906-46 2024-09-15T03:54:45.075 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 797 (ObjNum 732 snap 222 seq_num 732) dirty exists 2024-09-15T03:54:45.076 INFO:tasks.rados.rados.0.smithi122.stdout:2010: left oid 46 (ObjNum 732 snap 222 seq_num 732) 2024-09-15T03:54:45.076 INFO:tasks.rados.rados.0.smithi122.stdout:2011: finishing write tid 1 to smithi12223906-9 2024-09-15T03:54:45.078 INFO:tasks.rados.rados.0.smithi122.stdout:2011: finishing write tid 2 to smithi12223906-9 2024-09-15T03:54:45.079 INFO:tasks.rados.rados.0.smithi122.stdout:2011: finishing write tid 3 to smithi12223906-9 2024-09-15T03:54:45.079 INFO:tasks.rados.rados.0.smithi122.stdout:2011: finishing write tid 4 to smithi12223906-9 2024-09-15T03:54:45.079 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 819 (ObjNum 733 snap 222 seq_num 733) dirty exists 2024-09-15T03:54:45.079 INFO:tasks.rados.rados.0.smithi122.stdout:2011: left oid 9 (ObjNum 733 snap 222 seq_num 733) 2024-09-15T03:54:46.033 INFO:tasks.rados.rados.0.smithi122.stdout:2008: done (3 left) 2024-09-15T03:54:46.034 INFO:tasks.rados.rados.0.smithi122.stdout:2010: done (2 left) 2024-09-15T03:54:46.034 INFO:tasks.rados.rados.0.smithi122.stdout:2011: done (1 left) 2024-09-15T03:54:46.034 INFO:tasks.rados.rados.0.smithi122.stdout:2012: done (0 left) 2024-09-15T03:54:46.034 INFO:tasks.rados.rados.0.smithi122.stdout:2013: write (excl) oid 20 current snap is 223 2024-09-15T03:54:46.034 INFO:tasks.rados.rados.0.smithi122.stdout:2013: seq_num 734 ranges {736338=402654,1552136=539580} 2024-09-15T03:54:46.046 INFO:tasks.rados.rados.0.smithi122.stdout:2013: writing smithi12223906-20 from 736338 to 1138992 tid 1 2024-09-15T03:54:46.052 INFO:tasks.rados.rados.0.smithi122.stdout:2013: writing smithi12223906-20 from 1552136 to 2091716 tid 2 2024-09-15T03:54:46.052 INFO:tasks.rados.rados.0.smithi122.stdout:2014: read oid 42 snap -1 2024-09-15T03:54:46.052 INFO:tasks.rados.rados.0.smithi122.stdout:2014: expect (ObjNum 709 snap 216 seq_num 709) 2024-09-15T03:54:46.052 INFO:tasks.rados.rados.0.smithi122.stdout:2015: delete oid 35 current snap is 223 2024-09-15T03:54:46.052 INFO:tasks.rados.rados.0.smithi122.stdout:2013: finishing write tid 1 to smithi12223906-20 2024-09-15T03:54:46.057 INFO:tasks.rados.rados.0.smithi122.stdout:2013: finishing write tid 2 to smithi12223906-20 2024-09-15T03:54:46.058 INFO:tasks.rados.rados.0.smithi122.stdout:2015: done (2 left) 2024-09-15T03:54:46.058 INFO:tasks.rados.rados.0.smithi122.stdout:2016: read oid 32 snap -1 2024-09-15T03:54:46.058 INFO:tasks.rados.rados.0.smithi122.stdout:2016: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:54:46.058 INFO:tasks.rados.rados.0.smithi122.stdout:2017: snap_remove snap 218 2024-09-15T03:54:47.046 INFO:tasks.rados.rados.0.smithi122.stdout:2013: finishing write tid 3 to smithi12223906-20 2024-09-15T03:54:47.046 INFO:tasks.rados.rados.0.smithi122.stdout:2013: finishing write tid 4 to smithi12223906-20 2024-09-15T03:54:47.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 661 (ObjNum 734 snap 223 seq_num 734) dirty exists 2024-09-15T03:54:47.046 INFO:tasks.rados.rados.0.smithi122.stdout:2013: left oid 20 (ObjNum 734 snap 223 seq_num 734) 2024-09-15T03:54:47.046 INFO:tasks.rados.rados.0.smithi122.stdout:2014: expect (ObjNum 709 snap 216 seq_num 709) 2024-09-15T03:54:47.166 INFO:tasks.rados.rados.0.smithi122.stdout:2016: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:54:47.337 INFO:tasks.rados.rados.0.smithi122.stdout:2013: done (3 left) 2024-09-15T03:54:47.337 INFO:tasks.rados.rados.0.smithi122.stdout:2014: done (2 left) 2024-09-15T03:54:47.337 INFO:tasks.rados.rados.0.smithi122.stdout:2016: done (1 left) 2024-09-15T03:54:47.338 INFO:tasks.rados.rados.0.smithi122.stdout:2017: done (0 left) 2024-09-15T03:54:47.338 INFO:tasks.rados.rados.0.smithi122.stdout:2018: write (excl) oid 40 current snap is 223 2024-09-15T03:54:47.338 INFO:tasks.rados.rados.0.smithi122.stdout:2018: seq_num 736 ranges {769952=553152,2069100=300852} 2024-09-15T03:54:47.353 INFO:tasks.rados.rados.0.smithi122.stdout:2018: writing smithi12223906-40 from 769952 to 1323104 tid 1 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:2018: writing smithi12223906-40 from 2069100 to 2369952 tid 2 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:2019: rollback oid 14 current snap is 223 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 197 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:2018: finishing write tid 1 to smithi12223906-40 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:2020: read oid 3 snap 215 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:2020: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:54:47.360 INFO:tasks.rados.rados.0.smithi122.stdout:2021: snap_remove snap 188 2024-09-15T03:54:48.053 INFO:tasks.rados.rados.0.smithi122.stdout:2021: done (3 left) 2024-09-15T03:54:48.053 INFO:tasks.rados.rados.0.smithi122.stdout:2022: read oid 42 snap -1 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2022: expect (ObjNum 709 snap 216 seq_num 709) 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2023: read oid 37 snap -1 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2023: expect deleted 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2024: snap_create 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2019: finishing rollback tid 1 to smithi12223906-14 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 0 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2018: finishing write tid 2 to smithi12223906-40 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2018: finishing write tid 3 to smithi12223906-40 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2018: finishing write tid 4 to smithi12223906-40 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 810 (ObjNum 736 snap 223 seq_num 736) dirty exists 2024-09-15T03:54:48.054 INFO:tasks.rados.rados.0.smithi122.stdout:2018: left oid 40 (ObjNum 736 snap 223 seq_num 736) 2024-09-15T03:54:48.055 INFO:tasks.rados.rados.0.smithi122.stdout:2020: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:54:48.276 INFO:tasks.rados.rados.0.smithi122.stdout:2022: expect (ObjNum 709 snap 216 seq_num 709) 2024-09-15T03:54:49.055 INFO:tasks.rados.rados.0.smithi122.stdout:2018: done (5 left) 2024-09-15T03:54:49.055 INFO:tasks.rados.rados.0.smithi122.stdout:2019: done (4 left) 2024-09-15T03:54:49.055 INFO:tasks.rados.rados.0.smithi122.stdout:2020: done (3 left) 2024-09-15T03:54:49.055 INFO:tasks.rados.rados.0.smithi122.stdout:2022: done (2 left) 2024-09-15T03:54:49.055 INFO:tasks.rados.rados.0.smithi122.stdout:2023: done (1 left) 2024-09-15T03:54:49.056 INFO:tasks.rados.rados.0.smithi122.stdout:2024: done (0 left) 2024-09-15T03:54:49.056 INFO:tasks.rados.rados.0.smithi122.stdout:2025: write (excl) oid 14 current snap is 224 2024-09-15T03:54:49.056 INFO:tasks.rados.rados.0.smithi122.stdout:2025: seq_num 737 ranges {786759=628401,2011229=375530} 2024-09-15T03:54:49.066 INFO:tasks.rados.rados.0.smithi122.stdout:2025: writing smithi12223906-14 from 786759 to 1415160 tid 1 2024-09-15T03:54:49.072 INFO:tasks.rados.rados.0.smithi122.stdout:2025: writing smithi12223906-14 from 2011229 to 2386759 tid 2 2024-09-15T03:54:49.073 INFO:tasks.rados.rados.0.smithi122.stdout:2026: delete oid 15 current snap is 224 2024-09-15T03:54:49.076 INFO:tasks.rados.rados.0.smithi122.stdout:2026: done (1 left) 2024-09-15T03:54:49.077 INFO:tasks.rados.rados.0.smithi122.stdout:2027: write (excl) oid 48 current snap is 224 2024-09-15T03:54:49.077 INFO:tasks.rados.rados.0.smithi122.stdout:2027: seq_num 739 ranges {420373=495252,1444546=690576} 2024-09-15T03:54:49.082 INFO:tasks.rados.rados.0.smithi122.stdout:2027: writing smithi12223906-48 from 420373 to 915625 tid 1 2024-09-15T03:54:49.089 INFO:tasks.rados.rados.0.smithi122.stdout:2027: writing smithi12223906-48 from 1444546 to 2135122 tid 2 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:2028: copy_from oid 29 from oid 16 current snap is 224 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:2025: finishing write tid 1 to smithi12223906-14 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:2025: finishing write tid 2 to smithi12223906-14 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:2025: finishing write tid 3 to smithi12223906-14 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:2025: finishing write tid 4 to smithi12223906-14 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1168 (ObjNum 737 snap 224 seq_num 737) dirty exists 2024-09-15T03:54:49.090 INFO:tasks.rados.rados.0.smithi122.stdout:2025: left oid 14 (ObjNum 737 snap 224 seq_num 737) 2024-09-15T03:54:49.091 INFO:tasks.rados.rados.0.smithi122.stdout:2025: done (2 left) 2024-09-15T03:54:49.091 INFO:tasks.rados.rados.0.smithi122.stdout:2029: delete oid 42 current snap is 224 2024-09-15T03:54:49.096 INFO:tasks.rados.rados.0.smithi122.stdout:2027: finishing write tid 1 to smithi12223906-48 2024-09-15T03:54:49.098 INFO:tasks.rados.rados.0.smithi122.stdout:2029: done (2 left) 2024-09-15T03:54:49.098 INFO:tasks.rados.rados.0.smithi122.stdout:2030: write oid 31 current snap is 224 2024-09-15T03:54:49.098 INFO:tasks.rados.rados.0.smithi122.stdout:2030: seq_num 741 ranges {453987=645750,1845157=608830} 2024-09-15T03:54:49.106 INFO:tasks.rados.rados.0.smithi122.stdout:2030: writing smithi12223906-31 from 453987 to 1099737 tid 1 2024-09-15T03:54:49.114 INFO:tasks.rados.rados.0.smithi122.stdout:2030: writing smithi12223906-31 from 1845157 to 2453987 tid 2 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2031: copy_from oid 10 from oid 42 current snap is 224 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2027: finishing write tid 2 to smithi12223906-48 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2027: finishing write tid 3 to smithi12223906-48 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2027: finishing write tid 4 to smithi12223906-48 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1006 (ObjNum 739 snap 224 seq_num 739) dirty exists 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2027: left oid 48 (ObjNum 739 snap 224 seq_num 739) 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2028: finishing copy_from racing read to smithi12223906-29 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2028: finishing copy_from to smithi12223906-29 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 900 (ObjNum 625 snap 187 seq_num 625) dirty exists 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2030: finishing write tid 1 to smithi12223906-31 2024-09-15T03:54:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2027: done (3 left) 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:2028: done (2 left) 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:2032: read oid 8 snap -1 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:2032: expect (ObjNum 722 snap 220 seq_num 722) 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:2033: rollback oid 20 current snap is 224 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 197 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:2034: snap_create 2024-09-15T03:54:49.116 INFO:tasks.rados.rados.0.smithi122.stdout:2031: finishing copy_from to smithi12223906-10 2024-09-15T03:54:49.117 INFO:tasks.rados.rados.0.smithi122.stdout:2031: got expected ENOENT (src dne) 2024-09-15T03:54:49.117 INFO:tasks.rados.rados.0.smithi122.stdout:2031: finishing copy_from racing read to smithi12223906-10 2024-09-15T03:54:49.117 INFO:tasks.rados.rados.0.smithi122.stdout:2033: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:54:49.118 INFO:tasks.rados.rados.0.smithi122.stdout:2033: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:54:49.118 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 663 (ObjNum 0 snap 0 seq_num 304) dirty dne 2024-09-15T03:54:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2030: finishing write tid 2 to smithi12223906-31 2024-09-15T03:54:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2030: finishing write tid 3 to smithi12223906-31 2024-09-15T03:54:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2030: finishing write tid 4 to smithi12223906-31 2024-09-15T03:54:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 806 (ObjNum 741 snap 224 seq_num 741) dirty exists 2024-09-15T03:54:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2030: left oid 31 (ObjNum 741 snap 224 seq_num 741) 2024-09-15T03:54:49.124 INFO:tasks.rados.rados.0.smithi122.stdout:2032: expect (ObjNum 722 snap 220 seq_num 722) 2024-09-15T03:54:50.013 INFO:tasks.rados.rados.0.smithi122.stdout:2030: done (4 left) 2024-09-15T03:54:50.014 INFO:tasks.rados.rados.0.smithi122.stdout:2031: done (3 left) 2024-09-15T03:54:50.014 INFO:tasks.rados.rados.0.smithi122.stdout:2032: done (2 left) 2024-09-15T03:54:50.014 INFO:tasks.rados.rados.0.smithi122.stdout:2033: done (1 left) 2024-09-15T03:54:50.014 INFO:tasks.rados.rados.0.smithi122.stdout:2034: done (0 left) 2024-09-15T03:54:50.014 INFO:tasks.rados.rados.0.smithi122.stdout:2035: snap_remove snap 211 2024-09-15T03:54:51.018 INFO:tasks.rados.rados.0.smithi122.stdout:2035: done (0 left) 2024-09-15T03:54:51.018 INFO:tasks.rados.rados.0.smithi122.stdout:2036: snap_create 2024-09-15T03:54:52.028 INFO:tasks.rados.rados.0.smithi122.stdout:2036: done (0 left) 2024-09-15T03:54:52.028 INFO:tasks.rados.rados.0.smithi122.stdout:2037: delete oid 48 current snap is 226 2024-09-15T03:54:52.035 INFO:tasks.rados.rados.0.smithi122.stdout:2037: done (0 left) 2024-09-15T03:54:52.035 INFO:tasks.rados.rados.0.smithi122.stdout:2038: copy_from oid 2 from oid 21 current snap is 226 2024-09-15T03:54:52.035 INFO:tasks.rados.rados.0.smithi122.stdout:2039: delete oid 36 current snap is 226 2024-09-15T03:54:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2039: done (1 left) 2024-09-15T03:54:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2040: read oid 25 snap -1 2024-09-15T03:54:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2040: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:54:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2041: read oid 32 snap -1 2024-09-15T03:54:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2041: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:54:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2042: read oid 44 snap 224 2024-09-15T03:54:52.040 INFO:tasks.rados.rados.0.smithi122.stdout:2042: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:54:52.040 INFO:tasks.rados.rados.0.smithi122.stdout:2043: snap_remove snap 186 2024-09-15T03:54:53.046 INFO:tasks.rados.rados.0.smithi122.stdout:2043: done (4 left) 2024-09-15T03:54:53.046 INFO:tasks.rados.rados.0.smithi122.stdout:2044: snap_remove snap 207 2024-09-15T03:54:54.055 INFO:tasks.rados.rados.0.smithi122.stdout:2044: done (4 left) 2024-09-15T03:54:54.055 INFO:tasks.rados.rados.0.smithi122.stdout:2045: copy_from oid 42 from oid 50 current snap is 226 2024-09-15T03:54:54.055 INFO:tasks.rados.rados.0.smithi122.stdout:2046: write oid 6 current snap is 226 2024-09-15T03:54:54.055 INFO:tasks.rados.rados.0.smithi122.stdout:2046: seq_num 744 ranges {504408=471497,1620603=557925} 2024-09-15T03:54:54.064 INFO:tasks.rados.rados.0.smithi122.stdout:2046: writing smithi12223906-6 from 504408 to 975905 tid 1 2024-09-15T03:54:54.072 INFO:tasks.rados.rados.0.smithi122.stdout:2046: writing smithi12223906-6 from 1620603 to 2178528 tid 2 2024-09-15T03:54:54.072 INFO:tasks.rados.rados.0.smithi122.stdout:2047: write (excl) oid 46 current snap is 226 2024-09-15T03:54:54.072 INFO:tasks.rados.rados.0.smithi122.stdout:2047: seq_num 745 ranges {521215=546746,1562732=701583} 2024-09-15T03:54:54.078 INFO:tasks.rados.rados.0.smithi122.stdout:2047: writing smithi12223906-46 from 521215 to 1067961 tid 1 2024-09-15T03:54:54.085 INFO:tasks.rados.rados.0.smithi122.stdout:2047: writing smithi12223906-46 from 1562732 to 2264315 tid 2 2024-09-15T03:54:54.085 INFO:tasks.rados.rados.0.smithi122.stdout:2048: write (excl) oid 30 current snap is 226 2024-09-15T03:54:54.085 INFO:tasks.rados.rados.0.smithi122.stdout:2048: seq_num 746 ranges {538022=738348,1737567=561594} 2024-09-15T03:54:54.092 INFO:tasks.rados.rados.0.smithi122.stdout:2048: writing smithi12223906-30 from 538022 to 1276370 tid 1 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2048: writing smithi12223906-30 from 1737567 to 2299161 tid 2 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2049: read oid 19 snap -1 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2049: expect (ObjNum 714 snap 217 seq_num 714) 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2050: read oid 45 snap -1 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2050: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2051: write (excl) oid 15 current snap is 226 2024-09-15T03:54:54.098 INFO:tasks.rados.rados.0.smithi122.stdout:2051: seq_num 747 ranges {554829=413597,1396049=705252} 2024-09-15T03:54:54.103 INFO:tasks.rados.rados.0.smithi122.stdout:2051: writing smithi12223906-15 from 554829 to 968426 tid 1 2024-09-15T03:54:54.110 INFO:tasks.rados.rados.0.smithi122.stdout:2051: writing smithi12223906-15 from 1396049 to 2101301 tid 2 2024-09-15T03:54:54.110 INFO:tasks.rados.rados.0.smithi122.stdout:2052: snap_create 2024-09-15T03:54:54.111 INFO:tasks.rados.rados.0.smithi122.stdout:2040: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:54:54.275 INFO:tasks.rados.rados.0.smithi122.stdout:2038: finishing copy_from racing read to smithi12223906-2 2024-09-15T03:54:54.275 INFO:tasks.rados.rados.0.smithi122.stdout:2038: finishing copy_from to smithi12223906-2 2024-09-15T03:54:54.275 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1002 (ObjNum 704 snap 215 seq_num 704) dirty exists 2024-09-15T03:54:54.275 INFO:tasks.rados.rados.0.smithi122.stdout:2041: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T03:54:54.456 INFO:tasks.rados.rados.0.smithi122.stdout:2042: expect (ObjNum 636 snap 195 seq_num 636) 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:2045: finishing copy_from racing read to smithi12223906-42 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:2045: finishing copy_from to smithi12223906-42 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 849 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:2046: finishing write tid 1 to smithi12223906-6 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:2046: finishing write tid 2 to smithi12223906-6 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:2046: finishing write tid 3 to smithi12223906-6 2024-09-15T03:54:54.644 INFO:tasks.rados.rados.0.smithi122.stdout:2046: finishing write tid 4 to smithi12223906-6 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 822 (ObjNum 744 snap 226 seq_num 744) dirty exists 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2046: left oid 6 (ObjNum 744 snap 226 seq_num 744) 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2047: finishing write tid 1 to smithi12223906-46 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2047: finishing write tid 2 to smithi12223906-46 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2047: finishing write tid 3 to smithi12223906-46 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2047: finishing write tid 4 to smithi12223906-46 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 800 (ObjNum 745 snap 226 seq_num 745) dirty exists 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2047: left oid 46 (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2048: finishing write tid 1 to smithi12223906-30 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2048: finishing write tid 2 to smithi12223906-30 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2048: finishing write tid 3 to smithi12223906-30 2024-09-15T03:54:54.645 INFO:tasks.rados.rados.0.smithi122.stdout:2049: expect (ObjNum 714 snap 217 seq_num 714) 2024-09-15T03:54:54.701 INFO:tasks.rados.rados.0.smithi122.stdout:2048: finishing write tid 4 to smithi12223906-30 2024-09-15T03:54:54.701 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 813 (ObjNum 746 snap 226 seq_num 746) dirty exists 2024-09-15T03:54:54.701 INFO:tasks.rados.rados.0.smithi122.stdout:2048: left oid 30 (ObjNum 746 snap 226 seq_num 746) 2024-09-15T03:54:54.701 INFO:tasks.rados.rados.0.smithi122.stdout:2050: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:54:54.801 INFO:tasks.rados.rados.0.smithi122.stdout:2051: finishing write tid 1 to smithi12223906-15 2024-09-15T03:54:54.801 INFO:tasks.rados.rados.0.smithi122.stdout:2051: finishing write tid 2 to smithi12223906-15 2024-09-15T03:54:54.801 INFO:tasks.rados.rados.0.smithi122.stdout:2051: finishing write tid 3 to smithi12223906-15 2024-09-15T03:54:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:2051: finishing write tid 4 to smithi12223906-15 2024-09-15T03:54:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 904 (ObjNum 747 snap 226 seq_num 747) dirty exists 2024-09-15T03:54:54.802 INFO:tasks.rados.rados.0.smithi122.stdout:2051: left oid 15 (ObjNum 747 snap 226 seq_num 747) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2038: done (11 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2040: done (10 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2041: done (9 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2042: done (8 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2045: done (7 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2046: done (6 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2047: done (5 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2048: done (4 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2049: done (3 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2050: done (2 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2051: done (1 left) 2024-09-15T03:54:55.017 INFO:tasks.rados.rados.0.smithi122.stdout:2052: done (0 left) 2024-09-15T03:54:55.018 INFO:tasks.rados.rados.0.smithi122.stdout:2053: snap_create 2024-09-15T03:54:56.018 INFO:tasks.rados.rados.0.smithi122.stdout:2053: done (0 left) 2024-09-15T03:54:56.018 INFO:tasks.rados.rados.0.smithi122.stdout:2054: read oid 2 snap 224 2024-09-15T03:54:56.018 INFO:tasks.rados.rados.0.smithi122.stdout:2054: expect deleted 2024-09-15T03:54:56.018 INFO:tasks.rados.rados.0.smithi122.stdout:2055: copy_from oid 7 from oid 36 current snap is 228 2024-09-15T03:54:56.018 INFO:tasks.rados.rados.0.smithi122.stdout:2056: write oid 9 current snap is 228 2024-09-15T03:54:56.019 INFO:tasks.rados.rados.0.smithi122.stdout:2056: seq_num 748 ranges {571636=488846,1854531=565263} 2024-09-15T03:54:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:2056: writing smithi12223906-9 from 571636 to 1060482 tid 1 2024-09-15T03:54:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2056: writing smithi12223906-9 from 1854531 to 2419794 tid 2 2024-09-15T03:54:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2057: rollback oid 19 current snap is 228 2024-09-15T03:54:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 227 2024-09-15T03:54:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2055: finishing copy_from to smithi12223906-7 2024-09-15T03:54:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:2055: got expected ENOENT (src dne) 2024-09-15T03:54:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:2055: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:54:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:2054: done (3 left) 2024-09-15T03:54:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:2055: done (2 left) 2024-09-15T03:54:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:2058: delete oid 28 current snap is 228 2024-09-15T03:54:56.042 INFO:tasks.rados.rados.0.smithi122.stdout:2056: finishing write tid 1 to smithi12223906-9 2024-09-15T03:54:56.044 INFO:tasks.rados.rados.0.smithi122.stdout:2056: finishing write tid 2 to smithi12223906-9 2024-09-15T03:54:56.045 INFO:tasks.rados.rados.0.smithi122.stdout:2056: finishing write tid 3 to smithi12223906-9 2024-09-15T03:54:56.045 INFO:tasks.rados.rados.0.smithi122.stdout:2058: done (2 left) 2024-09-15T03:54:56.045 INFO:tasks.rados.rados.0.smithi122.stdout:2059: snap_create 2024-09-15T03:54:56.045 INFO:tasks.rados.rados.0.smithi122.stdout:2056: finishing write tid 4 to smithi12223906-9 2024-09-15T03:54:56.045 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 825 (ObjNum 748 snap 228 seq_num 748) dirty exists 2024-09-15T03:54:56.045 INFO:tasks.rados.rados.0.smithi122.stdout:2056: left oid 9 (ObjNum 748 snap 228 seq_num 748) 2024-09-15T03:54:56.046 INFO:tasks.rados.rados.0.smithi122.stdout:2057: finishing rollback tid 0 to smithi12223906-19 2024-09-15T03:54:56.046 INFO:tasks.rados.rados.0.smithi122.stdout:2057: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:54:56.046 INFO:tasks.rados.rados.0.smithi122.stdout:2057: finishing rollback tid 2 to smithi12223906-19 2024-09-15T03:54:56.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 803 (ObjNum 714 snap 217 seq_num 714) dirty exists 2024-09-15T03:54:57.026 INFO:tasks.rados.rados.0.smithi122.stdout:2056: done (2 left) 2024-09-15T03:54:57.026 INFO:tasks.rados.rados.0.smithi122.stdout:2057: done (1 left) 2024-09-15T03:54:57.026 INFO:tasks.rados.rados.0.smithi122.stdout:2059: done (0 left) 2024-09-15T03:54:57.026 INFO:tasks.rados.rados.0.smithi122.stdout:2060: write (excl) oid 4 current snap is 229 2024-09-15T03:54:57.026 INFO:tasks.rados.rados.0.smithi122.stdout:2060: seq_num 750 ranges {605250=639344,1855142=452579} 2024-09-15T03:54:57.035 INFO:tasks.rados.rados.0.smithi122.stdout:2060: writing smithi12223906-4 from 605250 to 1244594 tid 1 2024-09-15T03:54:57.044 INFO:tasks.rados.rados.0.smithi122.stdout:2060: writing smithi12223906-4 from 1855142 to 2307721 tid 2 2024-09-15T03:54:57.044 INFO:tasks.rados.rados.0.smithi122.stdout:2061: delete oid 34 current snap is 229 2024-09-15T03:54:57.044 INFO:tasks.rados.rados.0.smithi122.stdout:2060: finishing write tid 1 to smithi12223906-4 2024-09-15T03:54:57.047 INFO:tasks.rados.rados.0.smithi122.stdout:2061: done (1 left) 2024-09-15T03:54:57.047 INFO:tasks.rados.rados.0.smithi122.stdout:2062: delete oid 44 current snap is 229 2024-09-15T03:54:57.052 INFO:tasks.rados.rados.0.smithi122.stdout:2060: finishing write tid 2 to smithi12223906-4 2024-09-15T03:54:57.053 INFO:tasks.rados.rados.0.smithi122.stdout:2060: finishing write tid 3 to smithi12223906-4 2024-09-15T03:54:57.053 INFO:tasks.rados.rados.0.smithi122.stdout:2060: finishing write tid 4 to smithi12223906-4 2024-09-15T03:54:57.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 809 (ObjNum 750 snap 229 seq_num 750) dirty exists 2024-09-15T03:54:57.053 INFO:tasks.rados.rados.0.smithi122.stdout:2060: left oid 4 (ObjNum 750 snap 229 seq_num 750) 2024-09-15T03:54:57.054 INFO:tasks.rados.rados.0.smithi122.stdout:2060: done (1 left) 2024-09-15T03:54:57.054 INFO:tasks.rados.rados.0.smithi122.stdout:2062: done (0 left) 2024-09-15T03:54:57.054 INFO:tasks.rados.rados.0.smithi122.stdout:2063: delete oid 2 current snap is 229 2024-09-15T03:54:57.063 INFO:tasks.rados.rados.0.smithi122.stdout:2063: done (0 left) 2024-09-15T03:54:57.063 INFO:tasks.rados.rados.0.smithi122.stdout:2064: snap_remove snap 224 2024-09-15T03:54:58.035 INFO:tasks.rados.rados.0.smithi122.stdout:2064: done (0 left) 2024-09-15T03:54:58.035 INFO:tasks.rados.rados.0.smithi122.stdout:2065: copy_from oid 16 from oid 6 current snap is 229 2024-09-15T03:54:58.035 INFO:tasks.rados.rados.0.smithi122.stdout:2066: snap_remove snap 225 2024-09-15T03:54:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2066: done (1 left) 2024-09-15T03:54:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2067: snap_create 2024-09-15T03:54:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2065: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:54:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2065: finishing copy_from to smithi12223906-16 2024-09-15T03:54:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 822 (ObjNum 744 snap 226 seq_num 744) dirty exists 2024-09-15T03:55:00.019 INFO:tasks.rados.rados.0.smithi122.stdout:2065: done (1 left) 2024-09-15T03:55:00.020 INFO:tasks.rados.rados.0.smithi122.stdout:2067: done (0 left) 2024-09-15T03:55:00.020 INFO:tasks.rados.rados.0.smithi122.stdout:2068: snap_remove snap 204 2024-09-15T03:55:01.019 INFO:tasks.rados.rados.0.smithi122.stdout:2068: done (0 left) 2024-09-15T03:55:01.019 INFO:tasks.rados.rados.0.smithi122.stdout:2069: snap_remove snap 222 2024-09-15T03:55:02.022 INFO:tasks.rados.rados.0.smithi122.stdout:2069: done (0 left) 2024-09-15T03:55:02.022 INFO:tasks.rados.rados.0.smithi122.stdout:2070: read oid 17 snap 229 2024-09-15T03:55:02.022 INFO:tasks.rados.rados.0.smithi122.stdout:2070: expect deleted 2024-09-15T03:55:02.022 INFO:tasks.rados.rados.0.smithi122.stdout:2071: snap_create 2024-09-15T03:55:03.029 INFO:tasks.rados.rados.0.smithi122.stdout:2070: done (1 left) 2024-09-15T03:55:03.030 INFO:tasks.rados.rados.0.smithi122.stdout:2071: done (0 left) 2024-09-15T03:55:03.030 INFO:tasks.rados.rados.0.smithi122.stdout:2072: copy_from oid 49 from oid 16 current snap is 231 2024-09-15T03:55:03.030 INFO:tasks.rados.rados.0.smithi122.stdout:2073: rollback oid 13 current snap is 231 2024-09-15T03:55:03.030 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 192 2024-09-15T03:55:03.030 INFO:tasks.rados.rados.0.smithi122.stdout:2074: snap_remove snap 190 2024-09-15T03:55:04.038 INFO:tasks.rados.rados.0.smithi122.stdout:2074: done (2 left) 2024-09-15T03:55:04.038 INFO:tasks.rados.rados.0.smithi122.stdout:2075: snap_remove snap 197 2024-09-15T03:55:05.019 INFO:tasks.rados.rados.0.smithi122.stdout:2075: done (2 left) 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:2076: delete oid 19 current snap is 231 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:2073: finishing rollback tid 0 to smithi12223906-13 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:2073: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:2073: finishing rollback tid 2 to smithi12223906-13 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 806 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:2072: finishing copy_from racing read to smithi12223906-49 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:2072: finishing copy_from to smithi12223906-49 2024-09-15T03:55:05.020 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 822 (ObjNum 744 snap 226 seq_num 744) dirty exists 2024-09-15T03:55:05.028 INFO:tasks.rados.rados.0.smithi122.stdout:2072: done (2 left) 2024-09-15T03:55:05.028 INFO:tasks.rados.rados.0.smithi122.stdout:2073: done (1 left) 2024-09-15T03:55:05.028 INFO:tasks.rados.rados.0.smithi122.stdout:2076: done (0 left) 2024-09-15T03:55:05.028 INFO:tasks.rados.rados.0.smithi122.stdout:2077: snap_create 2024-09-15T03:55:06.021 INFO:tasks.rados.rados.0.smithi122.stdout:2077: done (0 left) 2024-09-15T03:55:06.021 INFO:tasks.rados.rados.0.smithi122.stdout:2078: delete oid 43 current snap is 232 2024-09-15T03:55:06.027 INFO:tasks.rados.rados.0.smithi122.stdout:2078: done (0 left) 2024-09-15T03:55:06.028 INFO:tasks.rados.rados.0.smithi122.stdout:2079: read oid 48 snap -1 2024-09-15T03:55:06.028 INFO:tasks.rados.rados.0.smithi122.stdout:2079: expect deleted 2024-09-15T03:55:06.028 INFO:tasks.rados.rados.0.smithi122.stdout:2080: write (excl) oid 34 current snap is 232 2024-09-15T03:55:06.028 INFO:tasks.rados.rados.0.smithi122.stdout:2080: seq_num 756 ranges {706092=407191,1806034=463586} 2024-09-15T03:55:06.037 INFO:tasks.rados.rados.0.smithi122.stdout:2080: writing smithi12223906-34 from 706092 to 1113283 tid 1 2024-09-15T03:55:06.045 INFO:tasks.rados.rados.0.smithi122.stdout:2080: writing smithi12223906-34 from 1806034 to 2269620 tid 2 2024-09-15T03:55:06.045 INFO:tasks.rados.rados.0.smithi122.stdout:2081: read oid 11 snap -1 2024-09-15T03:55:06.045 INFO:tasks.rados.rados.0.smithi122.stdout:2081: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:55:06.045 INFO:tasks.rados.rados.0.smithi122.stdout:2082: write oid 29 current snap is 232 2024-09-15T03:55:06.045 INFO:tasks.rados.rados.0.smithi122.stdout:2082: seq_num 757 ranges {722899=482440,1748163=723597} 2024-09-15T03:55:06.052 INFO:tasks.rados.rados.0.smithi122.stdout:2082: writing smithi12223906-29 from 722899 to 1205339 tid 1 2024-09-15T03:55:06.060 INFO:tasks.rados.rados.0.smithi122.stdout:2082: writing smithi12223906-29 from 1748163 to 2471760 tid 2 2024-09-15T03:55:06.060 INFO:tasks.rados.rados.0.smithi122.stdout:2083: copy_from oid 50 from oid 5 current snap is 232 2024-09-15T03:55:06.060 INFO:tasks.rados.rados.0.smithi122.stdout:2080: finishing write tid 1 to smithi12223906-34 2024-09-15T03:55:06.060 INFO:tasks.rados.rados.0.smithi122.stdout:2081: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2080: finishing write tid 2 to smithi12223906-34 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2080: finishing write tid 3 to smithi12223906-34 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2080: finishing write tid 4 to smithi12223906-34 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 735 (ObjNum 756 snap 232 seq_num 756) dirty exists 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2080: left oid 34 (ObjNum 756 snap 232 seq_num 756) 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2082: finishing write tid 1 to smithi12223906-29 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2082: finishing write tid 2 to smithi12223906-29 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2082: finishing write tid 3 to smithi12223906-29 2024-09-15T03:55:06.203 INFO:tasks.rados.rados.0.smithi122.stdout:2082: finishing write tid 4 to smithi12223906-29 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 904 (ObjNum 757 snap 232 seq_num 757) dirty exists 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2082: left oid 29 (ObjNum 757 snap 232 seq_num 757) 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2083: finishing copy_from racing read to smithi12223906-50 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2083: finishing copy_from to smithi12223906-50 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 853 (ObjNum 715 snap 217 seq_num 715) dirty exists 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2079: done (4 left) 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2080: done (3 left) 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2081: done (2 left) 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2082: done (1 left) 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2083: done (0 left) 2024-09-15T03:55:06.204 INFO:tasks.rados.rados.0.smithi122.stdout:2084: snap_create 2024-09-15T03:55:07.045 INFO:tasks.rados.rados.0.smithi122.stdout:2084: done (0 left) 2024-09-15T03:55:07.045 INFO:tasks.rados.rados.0.smithi122.stdout:2085: snap_create 2024-09-15T03:55:08.054 INFO:tasks.rados.rados.0.smithi122.stdout:2085: done (0 left) 2024-09-15T03:55:08.054 INFO:tasks.rados.rados.0.smithi122.stdout:2086: read oid 24 snap -1 2024-09-15T03:55:08.054 INFO:tasks.rados.rados.0.smithi122.stdout:2086: expect (ObjNum 669 snap 204 seq_num 669) 2024-09-15T03:55:08.054 INFO:tasks.rados.rados.0.smithi122.stdout:2087: write (excl) oid 4 current snap is 234 2024-09-15T03:55:08.054 INFO:tasks.rados.rados.0.smithi122.stdout:2087: seq_num 758 ranges {739706=557689,1806645=467255} 2024-09-15T03:55:08.061 INFO:tasks.rados.rados.0.smithi122.stdout:2087: writing smithi12223906-4 from 739706 to 1297395 tid 1 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2087: writing smithi12223906-4 from 1806645 to 2273900 tid 2 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2088: read oid 49 snap 198 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2088: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2089: rollback oid 16 current snap is 234 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 219 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2090: read oid 7 snap 232 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2090: expect (ObjNum 687 snap 211 seq_num 687) 2024-09-15T03:55:08.068 INFO:tasks.rados.rados.0.smithi122.stdout:2091: delete oid 1 current snap is 234 2024-09-15T03:55:08.072 INFO:tasks.rados.rados.0.smithi122.stdout:2086: expect (ObjNum 669 snap 204 seq_num 669) 2024-09-15T03:55:08.258 INFO:tasks.rados.rados.0.smithi122.stdout:2090: expect (ObjNum 687 snap 211 seq_num 687) 2024-09-15T03:55:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:2087: finishing write tid 1 to smithi12223906-4 2024-09-15T03:55:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:2087: finishing write tid 2 to smithi12223906-4 2024-09-15T03:55:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:2088: expect (ObjNum 564 snap 167 seq_num 564) 2024-09-15T03:55:08.647 INFO:tasks.rados.rados.0.smithi122.stdout:2089: finishing rollback tid 0 to smithi12223906-16 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2087: finishing write tid 3 to smithi12223906-4 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2089: finishing rollback tid 1 to smithi12223906-16 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2089: finishing rollback tid 2 to smithi12223906-16 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 907 (ObjNum 625 snap 187 seq_num 625) dirty exists 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2087: finishing write tid 4 to smithi12223906-4 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 812 (ObjNum 758 snap 234 seq_num 758) dirty exists 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2087: left oid 4 (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2086: done (5 left) 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2087: done (4 left) 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2088: done (3 left) 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2089: done (2 left) 2024-09-15T03:55:08.648 INFO:tasks.rados.rados.0.smithi122.stdout:2090: done (1 left) 2024-09-15T03:55:08.649 INFO:tasks.rados.rados.0.smithi122.stdout:2091: done (0 left) 2024-09-15T03:55:08.649 INFO:tasks.rados.rados.0.smithi122.stdout:2092: snap_create 2024-09-15T03:55:09.057 INFO:tasks.rados.rados.0.smithi122.stdout:2092: done (0 left) 2024-09-15T03:55:09.057 INFO:tasks.rados.rados.0.smithi122.stdout:2093: rollback oid 24 current snap is 235 2024-09-15T03:55:09.057 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 192 2024-09-15T03:55:09.057 INFO:tasks.rados.rados.0.smithi122.stdout:2094: snap_remove snap 234 2024-09-15T03:55:10.022 INFO:tasks.rados.rados.0.smithi122.stdout:2094: done (1 left) 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2095: rollback oid 18 current snap is 235 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 221 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2093: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2093: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2093: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 1253 (ObjNum 551 snap 163 seq_num 551) dirty exists 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2093: done (1 left) 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2096: write (excl) oid 14 current snap is 235 2024-09-15T03:55:10.023 INFO:tasks.rados.rados.0.smithi122.stdout:2096: seq_num 760 ranges {773320=708187,1923609=470924} 2024-09-15T03:55:10.032 INFO:tasks.rados.rados.0.smithi122.stdout:2096: writing smithi12223906-14 from 773320 to 1481507 tid 1 2024-09-15T03:55:10.038 INFO:tasks.rados.rados.0.smithi122.stdout:2096: writing smithi12223906-14 from 1923609 to 2394533 tid 2 2024-09-15T03:55:10.038 INFO:tasks.rados.rados.0.smithi122.stdout:2097: write oid 35 current snap is 235 2024-09-15T03:55:10.038 INFO:tasks.rados.rados.0.smithi122.stdout:2095: finishing rollback tid 0 to smithi12223906-18 2024-09-15T03:55:10.038 INFO:tasks.rados.rados.0.smithi122.stdout:2097: seq_num 761 ranges {790127=499789,1698444=730935} 2024-09-15T03:55:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:2097: writing smithi12223906-35 from 790127 to 1289916 tid 1 2024-09-15T03:55:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:2097: writing smithi12223906-35 from 1698444 to 2429379 tid 2 2024-09-15T03:55:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:2098: snap_create 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2095: finishing rollback tid 1 to smithi12223906-18 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2095: finishing rollback tid 2 to smithi12223906-18 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1245 (ObjNum 698 snap 214 seq_num 698) dirty exists 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2096: finishing write tid 1 to smithi12223906-14 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2096: finishing write tid 2 to smithi12223906-14 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2096: finishing write tid 3 to smithi12223906-14 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2096: finishing write tid 4 to smithi12223906-14 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1259 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2096: left oid 14 (ObjNum 760 snap 235 seq_num 760) 2024-09-15T03:55:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2097: finishing write tid 1 to smithi12223906-35 2024-09-15T03:55:10.055 INFO:tasks.rados.rados.0.smithi122.stdout:2097: finishing write tid 2 to smithi12223906-35 2024-09-15T03:55:10.055 INFO:tasks.rados.rados.0.smithi122.stdout:2097: finishing write tid 3 to smithi12223906-35 2024-09-15T03:55:10.056 INFO:tasks.rados.rados.0.smithi122.stdout:2097: finishing write tid 4 to smithi12223906-35 2024-09-15T03:55:10.056 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 590 (ObjNum 761 snap 235 seq_num 761) dirty exists 2024-09-15T03:55:10.056 INFO:tasks.rados.rados.0.smithi122.stdout:2097: left oid 35 (ObjNum 761 snap 235 seq_num 761) 2024-09-15T03:55:11.032 INFO:tasks.rados.rados.0.smithi122.stdout:2095: done (3 left) 2024-09-15T03:55:11.032 INFO:tasks.rados.rados.0.smithi122.stdout:2096: done (2 left) 2024-09-15T03:55:11.032 INFO:tasks.rados.rados.0.smithi122.stdout:2097: done (1 left) 2024-09-15T03:55:11.032 INFO:tasks.rados.rados.0.smithi122.stdout:2098: done (0 left) 2024-09-15T03:55:11.032 INFO:tasks.rados.rados.0.smithi122.stdout:2099: snap_remove snap 232 2024-09-15T03:55:12.041 INFO:tasks.rados.rados.0.smithi122.stdout:2099: done (0 left) 2024-09-15T03:55:12.041 INFO:tasks.rados.rados.0.smithi122.stdout:2100: snap_create 2024-09-15T03:55:13.043 INFO:tasks.rados.rados.0.smithi122.stdout:2100: done (0 left) 2024-09-15T03:55:13.043 INFO:tasks.rados.rados.0.smithi122.stdout:2101: write oid 5 current snap is 237 2024-09-15T03:55:13.043 INFO:tasks.rados.rados.0.smithi122.stdout:2101: seq_num 762 ranges {406934=575038,1640573=474593,2642791=164143} 2024-09-15T03:55:13.051 INFO:tasks.rados.rados.0.smithi122.stdout:2101: writing smithi12223906-5 from 406934 to 981972 tid 1 2024-09-15T03:55:13.058 INFO:tasks.rados.rados.0.smithi122.stdout:2101: writing smithi12223906-5 from 1640573 to 2115166 tid 2 2024-09-15T03:55:13.062 INFO:tasks.rados.rados.0.smithi122.stdout:2101: writing smithi12223906-5 from 2642791 to 2806934 tid 3 2024-09-15T03:55:13.062 INFO:tasks.rados.rados.0.smithi122.stdout:2102: delete oid 22 current snap is 237 2024-09-15T03:55:13.062 INFO:tasks.rados.rados.0.smithi122.stdout:2101: finishing write tid 1 to smithi12223906-5 2024-09-15T03:55:13.064 INFO:tasks.rados.rados.0.smithi122.stdout:2101: finishing write tid 2 to smithi12223906-5 2024-09-15T03:55:13.065 INFO:tasks.rados.rados.0.smithi122.stdout:2101: finishing write tid 3 to smithi12223906-5 2024-09-15T03:55:13.067 INFO:tasks.rados.rados.0.smithi122.stdout:2101: finishing write tid 4 to smithi12223906-5 2024-09-15T03:55:13.067 INFO:tasks.rados.rados.0.smithi122.stdout:2101: finishing write tid 5 to smithi12223906-5 2024-09-15T03:55:13.067 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 857 (ObjNum 762 snap 237 seq_num 762) dirty exists 2024-09-15T03:55:13.067 INFO:tasks.rados.rados.0.smithi122.stdout:2101: left oid 5 (ObjNum 762 snap 237 seq_num 762) 2024-09-15T03:55:13.068 INFO:tasks.rados.rados.0.smithi122.stdout:2101: done (1 left) 2024-09-15T03:55:13.068 INFO:tasks.rados.rados.0.smithi122.stdout:2102: done (0 left) 2024-09-15T03:55:13.069 INFO:tasks.rados.rados.0.smithi122.stdout:2103: read oid 50 snap -1 2024-09-15T03:55:13.069 INFO:tasks.rados.rados.0.smithi122.stdout:2103: expect (ObjNum 715 snap 217 seq_num 715) 2024-09-15T03:55:13.069 INFO:tasks.rados.rados.0.smithi122.stdout:2104: write (excl) oid 37 current snap is 237 2024-09-15T03:55:13.069 INFO:tasks.rados.rados.0.smithi122.stdout:2104: seq_num 764 ranges {440548=725536,1757537=478262,2813990=26558} 2024-09-15T03:55:13.075 INFO:tasks.rados.rados.0.smithi122.stdout:2104: writing smithi12223906-37 from 440548 to 1166084 tid 1 2024-09-15T03:55:13.083 INFO:tasks.rados.rados.0.smithi122.stdout:2104: writing smithi12223906-37 from 1757537 to 2235799 tid 2 2024-09-15T03:55:13.087 INFO:tasks.rados.rados.0.smithi122.stdout:2104: writing smithi12223906-37 from 2813990 to 2840548 tid 3 2024-09-15T03:55:13.087 INFO:tasks.rados.rados.0.smithi122.stdout:2105: snap_create 2024-09-15T03:55:13.087 INFO:tasks.rados.rados.0.smithi122.stdout:2103: expect (ObjNum 715 snap 217 seq_num 715) 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2104: finishing write tid 1 to smithi12223906-37 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2104: finishing write tid 2 to smithi12223906-37 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2104: finishing write tid 3 to smithi12223906-37 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2104: finishing write tid 4 to smithi12223906-37 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2104: finishing write tid 5 to smithi12223906-37 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1011 (ObjNum 764 snap 237 seq_num 764) dirty exists 2024-09-15T03:55:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2104: left oid 37 (ObjNum 764 snap 237 seq_num 764) 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:2103: done (2 left) 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:2104: done (1 left) 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:2105: done (0 left) 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:2106: rollback oid 49 current snap is 238 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 230 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:2107: write oid 29 current snap is 238 2024-09-15T03:55:14.065 INFO:tasks.rados.rados.0.smithi122.stdout:2107: seq_num 765 ranges {457355=400785,1416019=621920,2725060=132295} 2024-09-15T03:55:14.072 INFO:tasks.rados.rados.0.smithi122.stdout:2107: writing smithi12223906-29 from 457355 to 858140 tid 1 2024-09-15T03:55:14.079 INFO:tasks.rados.rados.0.smithi122.stdout:2107: writing smithi12223906-29 from 1416019 to 2037939 tid 2 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2107: writing smithi12223906-29 from 2725060 to 2857355 tid 3 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2108: read oid 10 snap 192 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2108: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2109: snap_create 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2106: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2106: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:55:14.084 INFO:tasks.rados.rados.0.smithi122.stdout:2106: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:55:14.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1006 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:55:14.085 INFO:tasks.rados.rados.0.smithi122.stdout:2107: finishing write tid 1 to smithi12223906-29 2024-09-15T03:55:14.085 INFO:tasks.rados.rados.0.smithi122.stdout:2107: finishing write tid 2 to smithi12223906-29 2024-09-15T03:55:14.087 INFO:tasks.rados.rados.0.smithi122.stdout:2107: finishing write tid 3 to smithi12223906-29 2024-09-15T03:55:14.088 INFO:tasks.rados.rados.0.smithi122.stdout:2108: expect (ObjNum 493 snap 149 seq_num 493) 2024-09-15T03:55:14.266 INFO:tasks.rados.rados.0.smithi122.stdout:2107: finishing write tid 4 to smithi12223906-29 2024-09-15T03:55:14.266 INFO:tasks.rados.rados.0.smithi122.stdout:2107: finishing write tid 5 to smithi12223906-29 2024-09-15T03:55:14.266 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 908 (ObjNum 765 snap 238 seq_num 765) dirty exists 2024-09-15T03:55:14.266 INFO:tasks.rados.rados.0.smithi122.stdout:2107: left oid 29 (ObjNum 765 snap 238 seq_num 765) 2024-09-15T03:55:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:2106: done (3 left) 2024-09-15T03:55:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:2107: done (2 left) 2024-09-15T03:55:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:2108: done (1 left) 2024-09-15T03:55:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:2109: done (0 left) 2024-09-15T03:55:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:2110: copy_from oid 28 from oid 41 current snap is 239 2024-09-15T03:55:15.024 INFO:tasks.rados.rados.0.smithi122.stdout:2111: snap_create 2024-09-15T03:55:15.027 INFO:tasks.rados.rados.0.smithi122.stdout:2110: finishing copy_from to smithi12223906-28 2024-09-15T03:55:15.027 INFO:tasks.rados.rados.0.smithi122.stdout:2110: got expected ENOENT (src dne) 2024-09-15T03:55:15.027 INFO:tasks.rados.rados.0.smithi122.stdout:2110: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:55:16.026 INFO:tasks.rados.rados.0.smithi122.stdout:2110: done (1 left) 2024-09-15T03:55:16.026 INFO:tasks.rados.rados.0.smithi122.stdout:2111: done (0 left) 2024-09-15T03:55:16.027 INFO:tasks.rados.rados.0.smithi122.stdout:2112: write oid 44 current snap is 240 2024-09-15T03:55:16.027 INFO:tasks.rados.rados.0.smithi122.stdout:2112: seq_num 766 ranges {474162=476034,1358148=481931,2352483=465303} 2024-09-15T03:55:16.036 INFO:tasks.rados.rados.0.smithi122.stdout:2112: writing smithi12223906-44 from 474162 to 950196 tid 1 2024-09-15T03:55:16.041 INFO:tasks.rados.rados.0.smithi122.stdout:2112: writing smithi12223906-44 from 1358148 to 1840079 tid 2 2024-09-15T03:55:16.047 INFO:tasks.rados.rados.0.smithi122.stdout:2112: writing smithi12223906-44 from 2352483 to 2817786 tid 3 2024-09-15T03:55:16.047 INFO:tasks.rados.rados.0.smithi122.stdout:2113: delete oid 40 current snap is 240 2024-09-15T03:55:16.047 INFO:tasks.rados.rados.0.smithi122.stdout:2112: finishing write tid 1 to smithi12223906-44 2024-09-15T03:55:16.047 INFO:tasks.rados.rados.0.smithi122.stdout:2112: finishing write tid 2 to smithi12223906-44 2024-09-15T03:55:16.053 INFO:tasks.rados.rados.0.smithi122.stdout:2112: finishing write tid 3 to smithi12223906-44 2024-09-15T03:55:16.054 INFO:tasks.rados.rados.0.smithi122.stdout:2112: finishing write tid 4 to smithi12223906-44 2024-09-15T03:55:16.054 INFO:tasks.rados.rados.0.smithi122.stdout:2112: finishing write tid 5 to smithi12223906-44 2024-09-15T03:55:16.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1253 (ObjNum 766 snap 240 seq_num 766) dirty exists 2024-09-15T03:55:16.054 INFO:tasks.rados.rados.0.smithi122.stdout:2112: left oid 44 (ObjNum 766 snap 240 seq_num 766) 2024-09-15T03:55:16.055 INFO:tasks.rados.rados.0.smithi122.stdout:2112: done (1 left) 2024-09-15T03:55:16.055 INFO:tasks.rados.rados.0.smithi122.stdout:2113: done (0 left) 2024-09-15T03:55:16.055 INFO:tasks.rados.rados.0.smithi122.stdout:2114: write (excl) oid 6 current snap is 240 2024-09-15T03:55:16.056 INFO:tasks.rados.rados.0.smithi122.stdout:2114: seq_num 768 ranges {507776=742885,1991465=485600} 2024-09-15T03:55:16.063 INFO:tasks.rados.rados.0.smithi122.stdout:2114: writing smithi12223906-6 from 507776 to 1250661 tid 1 2024-09-15T03:55:16.071 INFO:tasks.rados.rados.0.smithi122.stdout:2114: writing smithi12223906-6 from 1991465 to 2477065 tid 2 2024-09-15T03:55:16.071 INFO:tasks.rados.rados.0.smithi122.stdout:2115: read oid 46 snap -1 2024-09-15T03:55:16.072 INFO:tasks.rados.rados.0.smithi122.stdout:2115: expect (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:55:16.072 INFO:tasks.rados.rados.0.smithi122.stdout:2116: snap_create 2024-09-15T03:55:16.072 INFO:tasks.rados.rados.0.smithi122.stdout:2114: finishing write tid 1 to smithi12223906-6 2024-09-15T03:55:16.076 INFO:tasks.rados.rados.0.smithi122.stdout:2114: finishing write tid 2 to smithi12223906-6 2024-09-15T03:55:16.076 INFO:tasks.rados.rados.0.smithi122.stdout:2114: finishing write tid 3 to smithi12223906-6 2024-09-15T03:55:16.077 INFO:tasks.rados.rados.0.smithi122.stdout:2114: finishing write tid 4 to smithi12223906-6 2024-09-15T03:55:16.077 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 828 (ObjNum 768 snap 240 seq_num 768) dirty exists 2024-09-15T03:55:16.077 INFO:tasks.rados.rados.0.smithi122.stdout:2114: left oid 6 (ObjNum 768 snap 240 seq_num 768) 2024-09-15T03:55:16.082 INFO:tasks.rados.rados.0.smithi122.stdout:2115: expect (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:55:17.061 INFO:tasks.rados.rados.0.smithi122.stdout:2114: done (2 left) 2024-09-15T03:55:17.061 INFO:tasks.rados.rados.0.smithi122.stdout:2115: done (1 left) 2024-09-15T03:55:17.061 INFO:tasks.rados.rados.0.smithi122.stdout:2116: done (0 left) 2024-09-15T03:55:17.061 INFO:tasks.rados.rados.0.smithi122.stdout:2117: delete oid 35 current snap is 241 2024-09-15T03:55:17.066 INFO:tasks.rados.rados.0.smithi122.stdout:2117: done (0 left) 2024-09-15T03:55:17.066 INFO:tasks.rados.rados.0.smithi122.stdout:2118: snap_create 2024-09-15T03:55:18.071 INFO:tasks.rados.rados.0.smithi122.stdout:2118: done (0 left) 2024-09-15T03:55:18.071 INFO:tasks.rados.rados.0.smithi122.stdout:2119: snap_create 2024-09-15T03:55:19.077 INFO:tasks.rados.rados.0.smithi122.stdout:2119: done (0 left) 2024-09-15T03:55:19.077 INFO:tasks.rados.rados.0.smithi122.stdout:2120: snap_create 2024-09-15T03:55:20.030 INFO:tasks.rados.rados.0.smithi122.stdout:2120: done (0 left) 2024-09-15T03:55:20.030 INFO:tasks.rados.rados.0.smithi122.stdout:2121: read oid 41 snap -1 2024-09-15T03:55:20.030 INFO:tasks.rados.rados.0.smithi122.stdout:2121: expect deleted 2024-09-15T03:55:20.030 INFO:tasks.rados.rados.0.smithi122.stdout:2122: rollback oid 30 current snap is 244 2024-09-15T03:55:20.031 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 30 to 237 2024-09-15T03:55:20.031 INFO:tasks.rados.rados.0.smithi122.stdout:2123: copy_from oid 32 from oid 17 current snap is 244 2024-09-15T03:55:20.031 INFO:tasks.rados.rados.0.smithi122.stdout:2124: write oid 28 current snap is 244 2024-09-15T03:55:20.031 INFO:tasks.rados.rados.0.smithi122.stdout:2124: seq_num 770 ranges {541390=493383,1592076=489269,2862175=79215} 2024-09-15T03:55:20.036 INFO:tasks.rados.rados.0.smithi122.stdout:2124: writing smithi12223906-28 from 541390 to 1034773 tid 1 2024-09-15T03:55:20.043 INFO:tasks.rados.rados.0.smithi122.stdout:2124: writing smithi12223906-28 from 1592076 to 2081345 tid 2 2024-09-15T03:55:20.048 INFO:tasks.rados.rados.0.smithi122.stdout:2124: writing smithi12223906-28 from 2862175 to 2941390 tid 3 2024-09-15T03:55:20.048 INFO:tasks.rados.rados.0.smithi122.stdout:2125: snap_create 2024-09-15T03:55:20.048 INFO:tasks.rados.rados.0.smithi122.stdout:2123: finishing copy_from to smithi12223906-32 2024-09-15T03:55:20.048 INFO:tasks.rados.rados.0.smithi122.stdout:2123: got expected ENOENT (src dne) 2024-09-15T03:55:20.048 INFO:tasks.rados.rados.0.smithi122.stdout:2123: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:55:20.048 INFO:tasks.rados.rados.0.smithi122.stdout:2124: finishing write tid 1 to smithi12223906-28 2024-09-15T03:55:20.051 INFO:tasks.rados.rados.0.smithi122.stdout:2124: finishing write tid 2 to smithi12223906-28 2024-09-15T03:55:20.052 INFO:tasks.rados.rados.0.smithi122.stdout:2124: finishing write tid 3 to smithi12223906-28 2024-09-15T03:55:20.052 INFO:tasks.rados.rados.0.smithi122.stdout:2122: finishing rollback tid 0 to smithi12223906-30 2024-09-15T03:55:20.052 INFO:tasks.rados.rados.0.smithi122.stdout:2122: finishing rollback tid 1 to smithi12223906-30 2024-09-15T03:55:20.053 INFO:tasks.rados.rados.0.smithi122.stdout:2122: finishing rollback tid 2 to smithi12223906-30 2024-09-15T03:55:20.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 817 (ObjNum 746 snap 226 seq_num 746) dirty exists 2024-09-15T03:55:20.053 INFO:tasks.rados.rados.0.smithi122.stdout:2124: finishing write tid 4 to smithi12223906-28 2024-09-15T03:55:20.054 INFO:tasks.rados.rados.0.smithi122.stdout:2124: finishing write tid 5 to smithi12223906-28 2024-09-15T03:55:20.054 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 912 (ObjNum 770 snap 244 seq_num 770) dirty exists 2024-09-15T03:55:20.054 INFO:tasks.rados.rados.0.smithi122.stdout:2124: left oid 28 (ObjNum 770 snap 244 seq_num 770) 2024-09-15T03:55:21.043 INFO:tasks.rados.rados.0.smithi122.stdout:2121: done (4 left) 2024-09-15T03:55:21.043 INFO:tasks.rados.rados.0.smithi122.stdout:2122: done (3 left) 2024-09-15T03:55:21.043 INFO:tasks.rados.rados.0.smithi122.stdout:2123: done (2 left) 2024-09-15T03:55:21.043 INFO:tasks.rados.rados.0.smithi122.stdout:2124: done (1 left) 2024-09-15T03:55:21.043 INFO:tasks.rados.rados.0.smithi122.stdout:2125: done (0 left) 2024-09-15T03:55:21.043 INFO:tasks.rados.rados.0.smithi122.stdout:2126: snap_create 2024-09-15T03:55:22.046 INFO:tasks.rados.rados.0.smithi122.stdout:2126: done (0 left) 2024-09-15T03:55:22.047 INFO:tasks.rados.rados.0.smithi122.stdout:2127: copy_from oid 16 from oid 21 current snap is 246 2024-09-15T03:55:22.047 INFO:tasks.rados.rados.0.smithi122.stdout:2128: snap_create 2024-09-15T03:55:22.069 INFO:tasks.rados.rados.0.smithi122.stdout:2127: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:55:22.072 INFO:tasks.rados.rados.0.smithi122.stdout:2127: finishing copy_from to smithi12223906-16 2024-09-15T03:55:22.072 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1002 (ObjNum 704 snap 215 seq_num 704) dirty exists 2024-09-15T03:55:23.063 INFO:tasks.rados.rados.0.smithi122.stdout:2127: done (1 left) 2024-09-15T03:55:23.064 INFO:tasks.rados.rados.0.smithi122.stdout:2128: done (0 left) 2024-09-15T03:55:23.064 INFO:tasks.rados.rados.0.smithi122.stdout:2129: snap_create 2024-09-15T03:55:24.075 INFO:tasks.rados.rados.0.smithi122.stdout:2129: done (0 left) 2024-09-15T03:55:24.075 INFO:tasks.rados.rados.0.smithi122.stdout:2130: read oid 35 snap -1 2024-09-15T03:55:24.075 INFO:tasks.rados.rados.0.smithi122.stdout:2130: expect deleted 2024-09-15T03:55:24.075 INFO:tasks.rados.rados.0.smithi122.stdout:2131: snap_create 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2130: done (1 left) 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2131: done (0 left) 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2132: copy_from oid 36 from oid 19 current snap is 249 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2133: rollback oid 24 current snap is 249 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 24 to 237 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2134: read oid 25 snap 247 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2134: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:55:25.032 INFO:tasks.rados.rados.0.smithi122.stdout:2135: snap_remove snap 202 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2135: done (3 left) 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2136: copy_from oid 7 from oid 3 current snap is 249 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2132: finishing copy_from to smithi12223906-36 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2132: got expected ENOENT (src dne) 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2132: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2133: finishing rollback tid 0 to smithi12223906-24 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2133: finishing rollback tid 1 to smithi12223906-24 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2133: finishing rollback tid 2 to smithi12223906-24 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 1267 (ObjNum 551 snap 163 seq_num 551) dirty exists 2024-09-15T03:55:26.036 INFO:tasks.rados.rados.0.smithi122.stdout:2134: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:55:26.203 INFO:tasks.rados.rados.0.smithi122.stdout:2136: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:55:26.203 INFO:tasks.rados.rados.0.smithi122.stdout:2136: finishing copy_from to smithi12223906-7 2024-09-15T03:55:26.204 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 939 (ObjNum 659 snap 201 seq_num 659) dirty exists 2024-09-15T03:55:26.204 INFO:tasks.rados.rados.0.smithi122.stdout:2132: done (3 left) 2024-09-15T03:55:26.204 INFO:tasks.rados.rados.0.smithi122.stdout:2133: done (2 left) 2024-09-15T03:55:26.204 INFO:tasks.rados.rados.0.smithi122.stdout:2134: done (1 left) 2024-09-15T03:55:26.204 INFO:tasks.rados.rados.0.smithi122.stdout:2136: done (0 left) 2024-09-15T03:55:26.204 INFO:tasks.rados.rados.0.smithi122.stdout:2137: snap_create 2024-09-15T03:55:27.078 INFO:tasks.rados.rados.0.smithi122.stdout:2137: done (0 left) 2024-09-15T03:55:27.078 INFO:tasks.rados.rados.0.smithi122.stdout:2138: write oid 18 current snap is 250 2024-09-15T03:55:27.078 INFO:tasks.rados.rados.0.smithi122.stdout:2138: seq_num 771 ranges {558197=568632,1650558=632927,2773245=184952} 2024-09-15T03:55:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2138: writing smithi12223906-18 from 558197 to 1126829 tid 1 2024-09-15T03:55:27.097 INFO:tasks.rados.rados.0.smithi122.stdout:2138: writing smithi12223906-18 from 1650558 to 2283485 tid 2 2024-09-15T03:55:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:2138: writing smithi12223906-18 from 2773245 to 2958197 tid 3 2024-09-15T03:55:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:2139: copy_from oid 4 from oid 2 current snap is 250 2024-09-15T03:55:27.101 INFO:tasks.rados.rados.0.smithi122.stdout:2140: delete oid 30 current snap is 250 2024-09-15T03:55:27.107 INFO:tasks.rados.rados.0.smithi122.stdout:2140: done (2 left) 2024-09-15T03:55:27.107 INFO:tasks.rados.rados.0.smithi122.stdout:2141: rollback oid 37 current snap is 250 2024-09-15T03:55:27.107 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 37 to 220 2024-09-15T03:55:27.107 INFO:tasks.rados.rados.0.smithi122.stdout:2142: write oid 46 current snap is 250 2024-09-15T03:55:27.107 INFO:tasks.rados.rados.0.smithi122.stdout:2142: seq_num 773 ranges {591811=719130,1767522=636596,2828091=163720} 2024-09-15T03:55:27.117 INFO:tasks.rados.rados.0.smithi122.stdout:2142: writing smithi12223906-46 from 591811 to 1310941 tid 1 2024-09-15T03:55:27.120 INFO:tasks.rados.rados.0.smithi122.stdout:2142: writing smithi12223906-46 from 1767522 to 2404118 tid 2 2024-09-15T03:55:27.124 INFO:tasks.rados.rados.0.smithi122.stdout:2142: writing smithi12223906-46 from 2828091 to 2991811 tid 3 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2143: delete oid 31 current snap is 250 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2139: finishing copy_from to smithi12223906-4 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2139: got expected ENOENT (src dne) 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2139: finishing copy_from racing read to smithi12223906-4 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2138: finishing write tid 1 to smithi12223906-18 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2141: finishing rollback tid 0 to smithi12223906-37 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2138: finishing write tid 2 to smithi12223906-18 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2138: finishing write tid 3 to smithi12223906-18 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2138: finishing write tid 4 to smithi12223906-18 2024-09-15T03:55:27.125 INFO:tasks.rados.rados.0.smithi122.stdout:2138: finishing write tid 5 to smithi12223906-18 2024-09-15T03:55:27.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1270 (ObjNum 771 snap 250 seq_num 771) dirty exists 2024-09-15T03:55:27.126 INFO:tasks.rados.rados.0.smithi122.stdout:2138: left oid 18 (ObjNum 771 snap 250 seq_num 771) 2024-09-15T03:55:27.126 INFO:tasks.rados.rados.0.smithi122.stdout:2141: finishing rollback tid 1 to smithi12223906-37 2024-09-15T03:55:27.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1013 (ObjNum 353 snap 0 seq_num 1842453392) dirty dne 2024-09-15T03:55:27.126 INFO:tasks.rados.rados.0.smithi122.stdout:2142: finishing write tid 1 to smithi12223906-46 2024-09-15T03:55:27.127 INFO:tasks.rados.rados.0.smithi122.stdout:2142: finishing write tid 2 to smithi12223906-46 2024-09-15T03:55:27.128 INFO:tasks.rados.rados.0.smithi122.stdout:2142: finishing write tid 3 to smithi12223906-46 2024-09-15T03:55:27.128 INFO:tasks.rados.rados.0.smithi122.stdout:2142: finishing write tid 4 to smithi12223906-46 2024-09-15T03:55:27.129 INFO:tasks.rados.rados.0.smithi122.stdout:2138: done (4 left) 2024-09-15T03:55:27.129 INFO:tasks.rados.rados.0.smithi122.stdout:2139: done (3 left) 2024-09-15T03:55:27.129 INFO:tasks.rados.rados.0.smithi122.stdout:2141: done (2 left) 2024-09-15T03:55:27.129 INFO:tasks.rados.rados.0.smithi122.stdout:2143: done (1 left) 2024-09-15T03:55:27.129 INFO:tasks.rados.rados.0.smithi122.stdout:2144: write (excl) oid 47 current snap is 250 2024-09-15T03:55:27.129 INFO:tasks.rados.rados.0.smithi122.stdout:2144: seq_num 775 ranges {625425=469628,1768133=640265} 2024-09-15T03:55:27.134 INFO:tasks.rados.rados.0.smithi122.stdout:2144: writing smithi12223906-47 from 625425 to 1095053 tid 1 2024-09-15T03:55:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:2144: writing smithi12223906-47 from 1768133 to 2408398 tid 2 2024-09-15T03:55:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:2145: rollback oid 9 current snap is 250 2024-09-15T03:55:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 231 2024-09-15T03:55:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:2142: finishing write tid 5 to smithi12223906-46 2024-09-15T03:55:27.143 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 811 (ObjNum 773 snap 250 seq_num 773) dirty exists 2024-09-15T03:55:27.143 INFO:tasks.rados.rados.0.smithi122.stdout:2142: left oid 46 (ObjNum 773 snap 250 seq_num 773) 2024-09-15T03:55:27.143 INFO:tasks.rados.rados.0.smithi122.stdout:2142: done (2 left) 2024-09-15T03:55:27.143 INFO:tasks.rados.rados.0.smithi122.stdout:2146: snap_create 2024-09-15T03:55:27.143 INFO:tasks.rados.rados.0.smithi122.stdout:2144: finishing write tid 1 to smithi12223906-47 2024-09-15T03:55:27.148 INFO:tasks.rados.rados.0.smithi122.stdout:2144: finishing write tid 2 to smithi12223906-47 2024-09-15T03:55:27.148 INFO:tasks.rados.rados.0.smithi122.stdout:2145: finishing rollback tid 0 to smithi12223906-9 2024-09-15T03:55:27.149 INFO:tasks.rados.rados.0.smithi122.stdout:2145: finishing rollback tid 1 to smithi12223906-9 2024-09-15T03:55:27.149 INFO:tasks.rados.rados.0.smithi122.stdout:2144: finishing write tid 3 to smithi12223906-47 2024-09-15T03:55:27.149 INFO:tasks.rados.rados.0.smithi122.stdout:2144: finishing write tid 4 to smithi12223906-47 2024-09-15T03:55:27.149 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 860 (ObjNum 775 snap 250 seq_num 775) dirty exists 2024-09-15T03:55:27.149 INFO:tasks.rados.rados.0.smithi122.stdout:2144: left oid 47 (ObjNum 775 snap 250 seq_num 775) 2024-09-15T03:55:27.150 INFO:tasks.rados.rados.0.smithi122.stdout:2145: finishing rollback tid 2 to smithi12223906-9 2024-09-15T03:55:27.150 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 831 (ObjNum 748 snap 228 seq_num 748) dirty exists 2024-09-15T03:55:28.088 INFO:tasks.rados.rados.0.smithi122.stdout:2144: done (2 left) 2024-09-15T03:55:28.088 INFO:tasks.rados.rados.0.smithi122.stdout:2145: done (1 left) 2024-09-15T03:55:28.088 INFO:tasks.rados.rados.0.smithi122.stdout:2146: done (0 left) 2024-09-15T03:55:28.088 INFO:tasks.rados.rados.0.smithi122.stdout:2147: write (excl) oid 42 current snap is 251 2024-09-15T03:55:28.088 INFO:tasks.rados.rados.0.smithi122.stdout:2147: seq_num 776 ranges {642232=661230,1942968=783923} 2024-09-15T03:55:28.100 INFO:tasks.rados.rados.0.smithi122.stdout:2147: writing smithi12223906-42 from 642232 to 1303462 tid 1 2024-09-15T03:55:28.108 INFO:tasks.rados.rados.0.smithi122.stdout:2147: writing smithi12223906-42 from 1942968 to 2726891 tid 2 2024-09-15T03:55:28.108 INFO:tasks.rados.rados.0.smithi122.stdout:2148: read oid 7 snap -1 2024-09-15T03:55:28.108 INFO:tasks.rados.rados.0.smithi122.stdout:2148: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:55:28.108 INFO:tasks.rados.rados.0.smithi122.stdout:2149: copy_from oid 2 from oid 39 current snap is 251 2024-09-15T03:55:28.108 INFO:tasks.rados.rados.0.smithi122.stdout:2150: copy_from oid 8 from oid 46 current snap is 251 2024-09-15T03:55:28.108 INFO:tasks.rados.rados.0.smithi122.stdout:2151: write (excl) oid 5 current snap is 251 2024-09-15T03:55:28.109 INFO:tasks.rados.rados.0.smithi122.stdout:2151: seq_num 777 ranges {659039=736479,2001450=643934} 2024-09-15T03:55:28.116 INFO:tasks.rados.rados.0.smithi122.stdout:2151: writing smithi12223906-5 from 659039 to 1395518 tid 1 2024-09-15T03:55:28.123 INFO:tasks.rados.rados.0.smithi122.stdout:2151: writing smithi12223906-5 from 2001450 to 2645384 tid 2 2024-09-15T03:55:28.123 INFO:tasks.rados.rados.0.smithi122.stdout:2152: snap_remove snap 228 2024-09-15T03:55:29.096 INFO:tasks.rados.rados.0.smithi122.stdout:2152: done (5 left) 2024-09-15T03:55:29.096 INFO:tasks.rados.rados.0.smithi122.stdout:2153: snap_create 2024-09-15T03:55:29.096 INFO:tasks.rados.rados.0.smithi122.stdout:2148: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:55:29.337 INFO:tasks.rados.rados.0.smithi122.stdout:2147: finishing write tid 1 to smithi12223906-42 2024-09-15T03:55:29.337 INFO:tasks.rados.rados.0.smithi122.stdout:2147: finishing write tid 2 to smithi12223906-42 2024-09-15T03:55:29.337 INFO:tasks.rados.rados.0.smithi122.stdout:2147: finishing write tid 3 to smithi12223906-42 2024-09-15T03:55:29.337 INFO:tasks.rados.rados.0.smithi122.stdout:2147: finishing write tid 4 to smithi12223906-42 2024-09-15T03:55:29.337 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1281 (ObjNum 776 snap 251 seq_num 776) dirty exists 2024-09-15T03:55:29.337 INFO:tasks.rados.rados.0.smithi122.stdout:2147: left oid 42 (ObjNum 776 snap 251 seq_num 776) 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2150: finishing copy_from racing read to smithi12223906-8 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2151: finishing write tid 1 to smithi12223906-5 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2151: finishing write tid 2 to smithi12223906-5 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2149: finishing copy_from racing read to smithi12223906-2 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2151: finishing write tid 3 to smithi12223906-5 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2151: finishing write tid 4 to smithi12223906-5 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 863 (ObjNum 777 snap 251 seq_num 777) dirty exists 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2151: left oid 5 (ObjNum 777 snap 251 seq_num 777) 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2149: finishing copy_from to smithi12223906-2 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 778 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:2150: finishing copy_from to smithi12223906-8 2024-09-15T03:55:29.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 811 (ObjNum 773 snap 250 seq_num 773) dirty exists 2024-09-15T03:55:30.034 INFO:tasks.rados.rados.0.smithi122.stdout:2147: done (5 left) 2024-09-15T03:55:30.034 INFO:tasks.rados.rados.0.smithi122.stdout:2148: done (4 left) 2024-09-15T03:55:30.034 INFO:tasks.rados.rados.0.smithi122.stdout:2149: done (3 left) 2024-09-15T03:55:30.034 INFO:tasks.rados.rados.0.smithi122.stdout:2150: done (2 left) 2024-09-15T03:55:30.035 INFO:tasks.rados.rados.0.smithi122.stdout:2151: done (1 left) 2024-09-15T03:55:30.035 INFO:tasks.rados.rados.0.smithi122.stdout:2153: done (0 left) 2024-09-15T03:55:30.035 INFO:tasks.rados.rados.0.smithi122.stdout:2154: write (excl) oid 43 current snap is 252 2024-09-15T03:55:30.035 INFO:tasks.rados.rados.0.smithi122.stdout:2154: seq_num 778 ranges {675846=411728,1543579=787592,2848853=226993} 2024-09-15T03:55:30.040 INFO:tasks.rados.rados.0.smithi122.stdout:2154: writing smithi12223906-43 from 675846 to 1087574 tid 1 2024-09-15T03:55:30.048 INFO:tasks.rados.rados.0.smithi122.stdout:2154: writing smithi12223906-43 from 1543579 to 2331171 tid 2 2024-09-15T03:55:30.052 INFO:tasks.rados.rados.0.smithi122.stdout:2154: writing smithi12223906-43 from 2848853 to 3075846 tid 3 2024-09-15T03:55:30.052 INFO:tasks.rados.rados.0.smithi122.stdout:2155: read oid 36 snap -1 2024-09-15T03:55:30.052 INFO:tasks.rados.rados.0.smithi122.stdout:2155: expect deleted 2024-09-15T03:55:30.053 INFO:tasks.rados.rados.0.smithi122.stdout:2156: snap_create 2024-09-15T03:55:30.053 INFO:tasks.rados.rados.0.smithi122.stdout:2154: finishing write tid 1 to smithi12223906-43 2024-09-15T03:55:30.053 INFO:tasks.rados.rados.0.smithi122.stdout:2154: finishing write tid 2 to smithi12223906-43 2024-09-15T03:55:30.056 INFO:tasks.rados.rados.0.smithi122.stdout:2154: finishing write tid 3 to smithi12223906-43 2024-09-15T03:55:30.057 INFO:tasks.rados.rados.0.smithi122.stdout:2154: finishing write tid 4 to smithi12223906-43 2024-09-15T03:55:30.057 INFO:tasks.rados.rados.0.smithi122.stdout:2154: finishing write tid 5 to smithi12223906-43 2024-09-15T03:55:30.057 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 1293 (ObjNum 778 snap 252 seq_num 778) dirty exists 2024-09-15T03:55:30.057 INFO:tasks.rados.rados.0.smithi122.stdout:2154: left oid 43 (ObjNum 778 snap 252 seq_num 778) 2024-09-15T03:55:31.042 INFO:tasks.rados.rados.0.smithi122.stdout:2154: done (2 left) 2024-09-15T03:55:31.042 INFO:tasks.rados.rados.0.smithi122.stdout:2155: done (1 left) 2024-09-15T03:55:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2156: done (0 left) 2024-09-15T03:55:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2157: copy_from oid 15 from oid 27 current snap is 253 2024-09-15T03:55:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2158: read oid 11 snap -1 2024-09-15T03:55:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2158: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:55:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2159: write (excl) oid 18 current snap is 253 2024-09-15T03:55:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2159: seq_num 779 ranges {692653=486977,1602061=647603,2992629=100024} 2024-09-15T03:55:31.051 INFO:tasks.rados.rados.0.smithi122.stdout:2159: writing smithi12223906-18 from 692653 to 1179630 tid 1 2024-09-15T03:55:31.057 INFO:tasks.rados.rados.0.smithi122.stdout:2159: writing smithi12223906-18 from 1602061 to 2249664 tid 2 2024-09-15T03:55:31.062 INFO:tasks.rados.rados.0.smithi122.stdout:2159: writing smithi12223906-18 from 2992629 to 3092653 tid 3 2024-09-15T03:55:31.062 INFO:tasks.rados.rados.0.smithi122.stdout:2160: rollback oid 20 current snap is 253 2024-09-15T03:55:31.062 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 198 2024-09-15T03:55:31.063 INFO:tasks.rados.rados.0.smithi122.stdout:2157: finishing copy_from to smithi12223906-15 2024-09-15T03:55:31.063 INFO:tasks.rados.rados.0.smithi122.stdout:2157: got expected ENOENT (src dne) 2024-09-15T03:55:31.063 INFO:tasks.rados.rados.0.smithi122.stdout:2161: snap_create 2024-09-15T03:55:31.063 INFO:tasks.rados.rados.0.smithi122.stdout:2157: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:55:31.063 INFO:tasks.rados.rados.0.smithi122.stdout:2158: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:55:31.204 INFO:tasks.rados.rados.0.smithi122.stdout:2159: finishing write tid 1 to smithi12223906-18 2024-09-15T03:55:31.204 INFO:tasks.rados.rados.0.smithi122.stdout:2160: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:55:31.204 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 663 (ObjNum 33 snap 0 seq_num 1842358576) dirty dne 2024-09-15T03:55:31.205 INFO:tasks.rados.rados.0.smithi122.stdout:2159: finishing write tid 2 to smithi12223906-18 2024-09-15T03:55:31.205 INFO:tasks.rados.rados.0.smithi122.stdout:2159: finishing write tid 3 to smithi12223906-18 2024-09-15T03:55:31.205 INFO:tasks.rados.rados.0.smithi122.stdout:2159: finishing write tid 4 to smithi12223906-18 2024-09-15T03:55:31.205 INFO:tasks.rados.rados.0.smithi122.stdout:2159: finishing write tid 5 to smithi12223906-18 2024-09-15T03:55:31.205 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1283 (ObjNum 779 snap 253 seq_num 779) dirty exists 2024-09-15T03:55:31.205 INFO:tasks.rados.rados.0.smithi122.stdout:2159: left oid 18 (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2157: done (4 left) 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2158: done (3 left) 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2159: done (2 left) 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2160: done (1 left) 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2161: done (0 left) 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2162: write oid 5 current snap is 254 2024-09-15T03:55:32.050 INFO:tasks.rados.rados.0.smithi122.stdout:2162: seq_num 780 ranges {709460=562226,2060543=791261} 2024-09-15T03:55:32.057 INFO:tasks.rados.rados.0.smithi122.stdout:2162: writing smithi12223906-5 from 709460 to 1271686 tid 1 2024-09-15T03:55:32.069 INFO:tasks.rados.rados.0.smithi122.stdout:2162: writing smithi12223906-5 from 2060543 to 2851804 tid 2 2024-09-15T03:55:32.069 INFO:tasks.rados.rados.0.smithi122.stdout:2163: rollback oid 45 current snap is 254 2024-09-15T03:55:32.069 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 193 2024-09-15T03:55:32.069 INFO:tasks.rados.rados.0.smithi122.stdout:2162: finishing write tid 1 to smithi12223906-5 2024-09-15T03:55:32.069 INFO:tasks.rados.rados.0.smithi122.stdout:2164: write oid 42 current snap is 254 2024-09-15T03:55:32.069 INFO:tasks.rados.rados.0.smithi122.stdout:2164: seq_num 781 ranges {726267=637475,2119025=651272} 2024-09-15T03:55:32.076 INFO:tasks.rados.rados.0.smithi122.stdout:2164: writing smithi12223906-42 from 726267 to 1363742 tid 1 2024-09-15T03:55:32.084 INFO:tasks.rados.rados.0.smithi122.stdout:2164: writing smithi12223906-42 from 2119025 to 2770297 tid 2 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2165: read oid 46 snap 249 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2165: expect (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2166: snap_create 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2162: finishing write tid 2 to smithi12223906-5 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2163: finishing rollback tid 0 to smithi12223906-45 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2163: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2162: finishing write tid 3 to smithi12223906-5 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2163: finishing rollback tid 2 to smithi12223906-45 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 814 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2162: finishing write tid 4 to smithi12223906-5 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 866 (ObjNum 780 snap 254 seq_num 780) dirty exists 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2162: left oid 5 (ObjNum 780 snap 254 seq_num 780) 2024-09-15T03:55:32.085 INFO:tasks.rados.rados.0.smithi122.stdout:2164: finishing write tid 1 to smithi12223906-42 2024-09-15T03:55:32.090 INFO:tasks.rados.rados.0.smithi122.stdout:2164: finishing write tid 2 to smithi12223906-42 2024-09-15T03:55:32.090 INFO:tasks.rados.rados.0.smithi122.stdout:2164: finishing write tid 3 to smithi12223906-42 2024-09-15T03:55:32.090 INFO:tasks.rados.rados.0.smithi122.stdout:2164: finishing write tid 4 to smithi12223906-42 2024-09-15T03:55:32.091 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1297 (ObjNum 781 snap 254 seq_num 781) dirty exists 2024-09-15T03:55:32.091 INFO:tasks.rados.rados.0.smithi122.stdout:2164: left oid 42 (ObjNum 781 snap 254 seq_num 781) 2024-09-15T03:55:32.095 INFO:tasks.rados.rados.0.smithi122.stdout:2165: expect (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2162: done (4 left) 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2163: done (3 left) 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2164: done (2 left) 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2165: done (1 left) 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2166: done (0 left) 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2167: write oid 35 current snap is 255 2024-09-15T03:55:33.053 INFO:tasks.rados.rados.0.smithi122.stdout:2167: seq_num 782 ranges {743074=712724,2061154=794930} 2024-09-15T03:55:33.065 INFO:tasks.rados.rados.0.smithi122.stdout:2167: writing smithi12223906-35 from 743074 to 1455798 tid 1 2024-09-15T03:55:33.074 INFO:tasks.rados.rados.0.smithi122.stdout:2167: writing smithi12223906-35 from 2061154 to 2856084 tid 2 2024-09-15T03:55:33.074 INFO:tasks.rados.rados.0.smithi122.stdout:2168: snap_remove snap 249 2024-09-15T03:55:34.074 INFO:tasks.rados.rados.0.smithi122.stdout:2168: done (1 left) 2024-09-15T03:55:34.074 INFO:tasks.rados.rados.0.smithi122.stdout:2169: snap_create 2024-09-15T03:55:34.074 INFO:tasks.rados.rados.0.smithi122.stdout:2167: finishing write tid 1 to smithi12223906-35 2024-09-15T03:55:34.074 INFO:tasks.rados.rados.0.smithi122.stdout:2167: finishing write tid 2 to smithi12223906-35 2024-09-15T03:55:34.075 INFO:tasks.rados.rados.0.smithi122.stdout:2167: finishing write tid 3 to smithi12223906-35 2024-09-15T03:55:34.075 INFO:tasks.rados.rados.0.smithi122.stdout:2167: finishing write tid 4 to smithi12223906-35 2024-09-15T03:55:34.075 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 594 (ObjNum 782 snap 255 seq_num 782) dirty exists 2024-09-15T03:55:34.075 INFO:tasks.rados.rados.0.smithi122.stdout:2167: left oid 35 (ObjNum 782 snap 255 seq_num 782) 2024-09-15T03:55:35.039 INFO:tasks.rados.rados.0.smithi122.stdout:2167: done (1 left) 2024-09-15T03:55:35.039 INFO:tasks.rados.rados.0.smithi122.stdout:2169: done (0 left) 2024-09-15T03:55:35.039 INFO:tasks.rados.rados.0.smithi122.stdout:2170: rollback oid 7 current snap is 256 2024-09-15T03:55:35.039 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 227 2024-09-15T03:55:35.039 INFO:tasks.rados.rados.0.smithi122.stdout:2171: rollback oid 14 current snap is 256 2024-09-15T03:55:35.039 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 244 2024-09-15T03:55:35.040 INFO:tasks.rados.rados.0.smithi122.stdout:2172: snap_remove snap 229 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2172: done (2 left) 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2173: rollback oid 22 current snap is 256 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 22 to 233 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2174: delete oid 8 current snap is 256 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2171: finishing rollback tid 0 to smithi12223906-14 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2171: finishing rollback tid 1 to smithi12223906-14 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2171: finishing rollback tid 2 to smithi12223906-14 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1303 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:55:36.038 INFO:tasks.rados.rados.0.smithi122.stdout:2170: finishing rollback tid 0 to smithi12223906-7 2024-09-15T03:55:36.039 INFO:tasks.rados.rados.0.smithi122.stdout:2170: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:55:36.039 INFO:tasks.rados.rados.0.smithi122.stdout:2170: finishing rollback tid 2 to smithi12223906-7 2024-09-15T03:55:36.039 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 942 (ObjNum 687 snap 211 seq_num 687) dirty exists 2024-09-15T03:55:36.044 INFO:tasks.rados.rados.0.smithi122.stdout:2170: done (3 left) 2024-09-15T03:55:36.044 INFO:tasks.rados.rados.0.smithi122.stdout:2171: done (2 left) 2024-09-15T03:55:36.045 INFO:tasks.rados.rados.0.smithi122.stdout:2174: done (1 left) 2024-09-15T03:55:36.045 INFO:tasks.rados.rados.0.smithi122.stdout:2175: read oid 1 snap -1 2024-09-15T03:55:36.045 INFO:tasks.rados.rados.0.smithi122.stdout:2175: expect deleted 2024-09-15T03:55:36.045 INFO:tasks.rados.rados.0.smithi122.stdout:2176: rollback oid 47 current snap is 256 2024-09-15T03:55:36.045 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 47 to 245 2024-09-15T03:55:36.045 INFO:tasks.rados.rados.0.smithi122.stdout:2177: delete oid 6 current snap is 256 2024-09-15T03:55:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2175: done (3 left) 2024-09-15T03:55:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2177: done (2 left) 2024-09-15T03:55:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2178: rollback oid 3 current snap is 256 2024-09-15T03:55:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 227 2024-09-15T03:55:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2179: snap_remove snap 246 2024-09-15T03:55:37.041 INFO:tasks.rados.rados.0.smithi122.stdout:2179: done (3 left) 2024-09-15T03:55:37.041 INFO:tasks.rados.rados.0.smithi122.stdout:2180: snap_remove snap 216 2024-09-15T03:55:38.042 INFO:tasks.rados.rados.0.smithi122.stdout:2180: done (3 left) 2024-09-15T03:55:38.042 INFO:tasks.rados.rados.0.smithi122.stdout:2181: rollback oid 2 current snap is 256 2024-09-15T03:55:38.042 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 227 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2176: finishing rollback tid 0 to smithi12223906-47 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2176: finishing rollback tid 1 to smithi12223906-47 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2173: finishing rollback tid 1 to smithi12223906-22 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2173: finishing rollback tid 2 to smithi12223906-22 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1291 (ObjNum 731 snap 222 seq_num 731) dirty exists 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2176: finishing rollback tid 2 to smithi12223906-47 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 869 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2178: finishing rollback tid 0 to smithi12223906-3 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2178: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2178: finishing rollback tid 2 to smithi12223906-3 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 1315 (ObjNum 659 snap 201 seq_num 659) dirty exists 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2173: done (3 left) 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2176: done (2 left) 2024-09-15T03:55:38.043 INFO:tasks.rados.rados.0.smithi122.stdout:2178: done (1 left) 2024-09-15T03:55:38.044 INFO:tasks.rados.rados.0.smithi122.stdout:2182: snap_create 2024-09-15T03:55:38.053 INFO:tasks.rados.rados.0.smithi122.stdout:2181: finishing rollback tid 0 to smithi12223906-2 2024-09-15T03:55:38.053 INFO:tasks.rados.rados.0.smithi122.stdout:2181: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:55:38.053 INFO:tasks.rados.rados.0.smithi122.stdout:2181: finishing rollback tid 2 to smithi12223906-2 2024-09-15T03:55:38.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1009 (ObjNum 704 snap 215 seq_num 704) dirty exists 2024-09-15T03:55:39.047 INFO:tasks.rados.rados.0.smithi122.stdout:2181: done (1 left) 2024-09-15T03:55:39.047 INFO:tasks.rados.rados.0.smithi122.stdout:2182: done (0 left) 2024-09-15T03:55:39.048 INFO:tasks.rados.rados.0.smithi122.stdout:2183: snap_remove snap 214 2024-09-15T03:55:40.037 INFO:tasks.rados.rados.0.smithi122.stdout:2183: done (0 left) 2024-09-15T03:55:40.037 INFO:tasks.rados.rados.0.smithi122.stdout:2184: snap_remove snap 248 2024-09-15T03:55:41.044 INFO:tasks.rados.rados.0.smithi122.stdout:2184: done (0 left) 2024-09-15T03:55:41.102 INFO:tasks.rados.rados.0.smithi122.stdout:2185: snap_create 2024-09-15T03:55:42.046 INFO:tasks.rados.rados.0.smithi122.stdout:2185: done (0 left) 2024-09-15T03:55:42.046 INFO:tasks.rados.rados.0.smithi122.stdout:2186: read oid 1 snap -1 2024-09-15T03:55:42.046 INFO:tasks.rados.rados.0.smithi122.stdout:2186: expect deleted 2024-09-15T03:55:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:2187: rollback oid 32 current snap is 258 2024-09-15T03:55:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 217 2024-09-15T03:55:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:2188: read oid 34 snap -1 2024-09-15T03:55:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:2188: expect (ObjNum 756 snap 232 seq_num 756) 2024-09-15T03:55:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:2189: read oid 49 snap -1 2024-09-15T03:55:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:2189: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:55:42.048 INFO:tasks.rados.rados.0.smithi122.stdout:2190: delete oid 15 current snap is 258 2024-09-15T03:55:42.054 INFO:tasks.rados.rados.0.smithi122.stdout:2188: expect (ObjNum 756 snap 232 seq_num 756) 2024-09-15T03:55:42.149 INFO:tasks.rados.rados.0.smithi122.stdout:2187: finishing rollback tid 0 to smithi12223906-32 2024-09-15T03:55:42.149 INFO:tasks.rados.rados.0.smithi122.stdout:2189: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:55:42.256 INFO:tasks.rados.rados.0.smithi122.stdout:2187: finishing rollback tid 1 to smithi12223906-32 2024-09-15T03:55:42.256 INFO:tasks.rados.rados.0.smithi122.stdout:2187: finishing rollback tid 2 to smithi12223906-32 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 1325 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2186: done (4 left) 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2187: done (3 left) 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2188: done (2 left) 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2189: done (1 left) 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2190: done (0 left) 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2191: write (excl) oid 2 current snap is 258 2024-09-15T03:55:42.257 INFO:tasks.rados.rados.0.smithi122.stdout:2191: seq_num 786 ranges {410302=730073,1895082=402268,2951884=258418} 2024-09-15T03:55:42.269 INFO:tasks.rados.rados.0.smithi122.stdout:2191: writing smithi12223906-2 from 410302 to 1140375 tid 1 2024-09-15T03:55:42.277 INFO:tasks.rados.rados.0.smithi122.stdout:2191: writing smithi12223906-2 from 1895082 to 2297350 tid 2 2024-09-15T03:55:42.283 INFO:tasks.rados.rados.0.smithi122.stdout:2191: writing smithi12223906-2 from 2951884 to 3210302 tid 3 2024-09-15T03:55:42.283 INFO:tasks.rados.rados.0.smithi122.stdout:2192: copy_from oid 32 from oid 9 current snap is 258 2024-09-15T03:55:42.283 INFO:tasks.rados.rados.0.smithi122.stdout:2191: finishing write tid 1 to smithi12223906-2 2024-09-15T03:55:42.283 INFO:tasks.rados.rados.0.smithi122.stdout:2193: write (excl) oid 21 current snap is 258 2024-09-15T03:55:42.283 INFO:tasks.rados.rados.0.smithi122.stdout:2193: seq_num 787 ranges {427109=405322,1553564=545926,2579307=647802} 2024-09-15T03:55:42.288 INFO:tasks.rados.rados.0.smithi122.stdout:2193: writing smithi12223906-21 from 427109 to 832431 tid 1 2024-09-15T03:55:42.295 INFO:tasks.rados.rados.0.smithi122.stdout:2193: writing smithi12223906-21 from 1553564 to 2099490 tid 2 2024-09-15T03:55:42.302 INFO:tasks.rados.rados.0.smithi122.stdout:2193: writing smithi12223906-21 from 2579307 to 3227109 tid 3 2024-09-15T03:55:42.302 INFO:tasks.rados.rados.0.smithi122.stdout:2194: snap_remove snap 253 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2194: done (3 left) 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2195: rollback oid 29 current snap is 258 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2191: finishing write tid 2 to smithi12223906-2 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2191: finishing write tid 3 to smithi12223906-2 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2191: finishing write tid 4 to smithi12223906-2 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2191: finishing write tid 5 to smithi12223906-2 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1013 (ObjNum 786 snap 258 seq_num 786) dirty exists 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2191: left oid 2 (ObjNum 786 snap 258 seq_num 786) 2024-09-15T03:55:43.072 INFO:tasks.rados.rados.0.smithi122.stdout:2193: finishing write tid 1 to smithi12223906-21 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2193: finishing write tid 2 to smithi12223906-21 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2192: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2192: finishing copy_from to smithi12223906-32 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 831 (ObjNum 748 snap 228 seq_num 748) dirty exists 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2193: finishing write tid 3 to smithi12223906-21 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2193: finishing write tid 4 to smithi12223906-21 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2193: finishing write tid 5 to smithi12223906-21 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1017 (ObjNum 787 snap 258 seq_num 787) dirty exists 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2193: left oid 21 (ObjNum 787 snap 258 seq_num 787) 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 29 to 219 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2191: done (3 left) 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2192: done (2 left) 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2193: done (1 left) 2024-09-15T03:55:43.073 INFO:tasks.rados.rados.0.smithi122.stdout:2196: write oid 3 current snap is 258 2024-09-15T03:55:43.074 INFO:tasks.rados.rados.0.smithi122.stdout:2196: seq_num 788 ranges {443916=480571,1612046=405937,2606730=495052} 2024-09-15T03:55:43.078 INFO:tasks.rados.rados.0.smithi122.stdout:2196: writing smithi12223906-3 from 443916 to 924487 tid 1 2024-09-15T03:55:43.086 INFO:tasks.rados.rados.0.smithi122.stdout:2196: writing smithi12223906-3 from 1612046 to 2017983 tid 2 2024-09-15T03:55:43.092 INFO:tasks.rados.rados.0.smithi122.stdout:2196: writing smithi12223906-3 from 2606730 to 3101782 tid 3 2024-09-15T03:55:43.092 INFO:tasks.rados.rados.0.smithi122.stdout:2195: finishing rollback tid 0 to smithi12223906-29 2024-09-15T03:55:43.092 INFO:tasks.rados.rados.0.smithi122.stdout:2195: finishing rollback tid 1 to smithi12223906-29 2024-09-15T03:55:43.092 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 944 (ObjNum 33 snap 0 seq_num 1842175600) dirty dne 2024-09-15T03:55:43.092 INFO:tasks.rados.rados.0.smithi122.stdout:2196: finishing write tid 1 to smithi12223906-3 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:2196: finishing write tid 2 to smithi12223906-3 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:2195: done (1 left) 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:2197: copy_from oid 44 from oid 13 current snap is 258 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:2198: rollback oid 40 current snap is 258 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 251 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:2199: snap_create 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:2198: finishing rollback tid 1 to smithi12223906-40 2024-09-15T03:55:43.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 814 (ObjNum 33 snap 0 seq_num 1817658064) dirty dne 2024-09-15T03:55:43.098 INFO:tasks.rados.rados.0.smithi122.stdout:2196: finishing write tid 3 to smithi12223906-3 2024-09-15T03:55:43.098 INFO:tasks.rados.rados.0.smithi122.stdout:2196: finishing write tid 4 to smithi12223906-3 2024-09-15T03:55:43.098 INFO:tasks.rados.rados.0.smithi122.stdout:2196: finishing write tid 5 to smithi12223906-3 2024-09-15T03:55:43.098 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 1356 (ObjNum 788 snap 258 seq_num 788) dirty exists 2024-09-15T03:55:43.099 INFO:tasks.rados.rados.0.smithi122.stdout:2196: left oid 3 (ObjNum 788 snap 258 seq_num 788) 2024-09-15T03:55:43.107 INFO:tasks.rados.rados.0.smithi122.stdout:2197: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:55:43.108 INFO:tasks.rados.rados.0.smithi122.stdout:2197: finishing copy_from to smithi12223906-44 2024-09-15T03:55:43.108 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 806 (ObjNum 457 snap 133 seq_num 457) dirty exists 2024-09-15T03:55:44.084 INFO:tasks.rados.rados.0.smithi122.stdout:2196: done (3 left) 2024-09-15T03:55:44.084 INFO:tasks.rados.rados.0.smithi122.stdout:2197: done (2 left) 2024-09-15T03:55:44.084 INFO:tasks.rados.rados.0.smithi122.stdout:2198: done (1 left) 2024-09-15T03:55:44.084 INFO:tasks.rados.rados.0.smithi122.stdout:2199: done (0 left) 2024-09-15T03:55:44.084 INFO:tasks.rados.rados.0.smithi122.stdout:2200: read oid 47 snap -1 2024-09-15T03:55:44.084 INFO:tasks.rados.rados.0.smithi122.stdout:2200: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:55:44.085 INFO:tasks.rados.rados.0.smithi122.stdout:2201: read oid 11 snap -1 2024-09-15T03:55:44.085 INFO:tasks.rados.rados.0.smithi122.stdout:2201: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:55:44.085 INFO:tasks.rados.rados.0.smithi122.stdout:2202: read oid 37 snap 239 2024-09-15T03:55:44.085 INFO:tasks.rados.rados.0.smithi122.stdout:2202: expect (ObjNum 764 snap 237 seq_num 764) 2024-09-15T03:55:44.085 INFO:tasks.rados.rados.0.smithi122.stdout:2203: delete oid 36 current snap is 259 2024-09-15T03:55:44.091 INFO:tasks.rados.rados.0.smithi122.stdout:2201: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:55:44.226 INFO:tasks.rados.rados.0.smithi122.stdout:2200: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:55:44.382 INFO:tasks.rados.rados.0.smithi122.stdout:2202: expect (ObjNum 764 snap 237 seq_num 764) 2024-09-15T03:55:44.468 INFO:tasks.rados.rados.0.smithi122.stdout:2200: done (3 left) 2024-09-15T03:55:44.468 INFO:tasks.rados.rados.0.smithi122.stdout:2201: done (2 left) 2024-09-15T03:55:44.469 INFO:tasks.rados.rados.0.smithi122.stdout:2202: done (1 left) 2024-09-15T03:55:44.469 INFO:tasks.rados.rados.0.smithi122.stdout:2203: done (0 left) 2024-09-15T03:55:44.469 INFO:tasks.rados.rados.0.smithi122.stdout:2204: read oid 46 snap 235 2024-09-15T03:55:44.469 INFO:tasks.rados.rados.0.smithi122.stdout:2204: expect (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:55:44.469 INFO:tasks.rados.rados.0.smithi122.stdout:2205: delete oid 50 current snap is 259 2024-09-15T03:55:44.479 INFO:tasks.rados.rados.0.smithi122.stdout:2204: expect (ObjNum 745 snap 226 seq_num 745) 2024-09-15T03:55:44.604 INFO:tasks.rados.rados.0.smithi122.stdout:2204: done (1 left) 2024-09-15T03:55:44.604 INFO:tasks.rados.rados.0.smithi122.stdout:2205: done (0 left) 2024-09-15T03:55:44.604 INFO:tasks.rados.rados.0.smithi122.stdout:2206: snap_remove snap 219 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2206: done (0 left) 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2207: copy_from oid 2 from oid 12 current snap is 259 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2208: read oid 45 snap -1 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2208: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2209: read oid 47 snap -1 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2209: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:55:45.044 INFO:tasks.rados.rados.0.smithi122.stdout:2210: snap_create 2024-09-15T03:55:45.054 INFO:tasks.rados.rados.0.smithi122.stdout:2207: finishing copy_from racing read to smithi12223906-2 2024-09-15T03:55:45.055 INFO:tasks.rados.rados.0.smithi122.stdout:2209: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:55:45.215 INFO:tasks.rados.rados.0.smithi122.stdout:2207: finishing copy_from to smithi12223906-2 2024-09-15T03:55:45.215 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 732 (ObjNum 719 snap 218 seq_num 719) dirty exists 2024-09-15T03:55:45.215 INFO:tasks.rados.rados.0.smithi122.stdout:2208: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:55:46.045 INFO:tasks.rados.rados.0.smithi122.stdout:2207: done (3 left) 2024-09-15T03:55:46.045 INFO:tasks.rados.rados.0.smithi122.stdout:2208: done (2 left) 2024-09-15T03:55:46.045 INFO:tasks.rados.rados.0.smithi122.stdout:2209: done (1 left) 2024-09-15T03:55:46.045 INFO:tasks.rados.rados.0.smithi122.stdout:2210: done (0 left) 2024-09-15T03:55:46.045 INFO:tasks.rados.rados.0.smithi122.stdout:2211: write (excl) oid 49 current snap is 260 2024-09-15T03:55:46.045 INFO:tasks.rados.rados.0.smithi122.stdout:2211: seq_num 791 ranges {494337=422671,1387492=553264,2688999=445221} 2024-09-15T03:55:46.047 INFO:tasks.rados.rados.0.smithi122.stdout:2211: writing smithi12223906-49 from 494337 to 917008 tid 1 2024-09-15T03:55:46.054 INFO:tasks.rados.rados.0.smithi122.stdout:2211: writing smithi12223906-49 from 1387492 to 1940756 tid 2 2024-09-15T03:55:46.061 INFO:tasks.rados.rados.0.smithi122.stdout:2211: writing smithi12223906-49 from 2688999 to 3134220 tid 3 2024-09-15T03:55:46.061 INFO:tasks.rados.rados.0.smithi122.stdout:2212: copy_from oid 26 from oid 18 current snap is 260 2024-09-15T03:55:46.061 INFO:tasks.rados.rados.0.smithi122.stdout:2211: finishing write tid 1 to smithi12223906-49 2024-09-15T03:55:46.061 INFO:tasks.rados.rados.0.smithi122.stdout:2211: finishing write tid 2 to smithi12223906-49 2024-09-15T03:55:46.062 INFO:tasks.rados.rados.0.smithi122.stdout:2213: read oid 32 snap -1 2024-09-15T03:55:46.062 INFO:tasks.rados.rados.0.smithi122.stdout:2213: expect (ObjNum 748 snap 228 seq_num 748) 2024-09-15T03:55:46.062 INFO:tasks.rados.rados.0.smithi122.stdout:2214: snap_create 2024-09-15T03:55:46.066 INFO:tasks.rados.rados.0.smithi122.stdout:2211: finishing write tid 3 to smithi12223906-49 2024-09-15T03:55:46.066 INFO:tasks.rados.rados.0.smithi122.stdout:2211: finishing write tid 4 to smithi12223906-49 2024-09-15T03:55:46.067 INFO:tasks.rados.rados.0.smithi122.stdout:2211: finishing write tid 5 to smithi12223906-49 2024-09-15T03:55:46.067 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1017 (ObjNum 791 snap 260 seq_num 791) dirty exists 2024-09-15T03:55:46.067 INFO:tasks.rados.rados.0.smithi122.stdout:2211: left oid 49 (ObjNum 791 snap 260 seq_num 791) 2024-09-15T03:55:46.070 INFO:tasks.rados.rados.0.smithi122.stdout:2213: expect (ObjNum 748 snap 228 seq_num 748) 2024-09-15T03:55:46.216 INFO:tasks.rados.rados.0.smithi122.stdout:2212: finishing copy_from racing read to smithi12223906-26 2024-09-15T03:55:46.216 INFO:tasks.rados.rados.0.smithi122.stdout:2212: finishing copy_from to smithi12223906-26 2024-09-15T03:55:46.216 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1283 (ObjNum 779 snap 253 seq_num 779) dirty exists 2024-09-15T03:55:47.042 INFO:tasks.rados.rados.0.smithi122.stdout:2211: done (3 left) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2212: done (2 left) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2213: done (1 left) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2214: done (0 left) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2215: read oid 13 snap -1 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2215: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2216: rollback oid 44 current snap is 261 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 242 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2217: read oid 3 snap -1 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2217: expect (ObjNum 788 snap 258 seq_num 788) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2218: read oid 26 snap -1 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2218: expect (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:55:47.043 INFO:tasks.rados.rados.0.smithi122.stdout:2219: delete oid 36 current snap is 261 2024-09-15T03:55:47.065 INFO:tasks.rados.rados.0.smithi122.stdout:2215: expect (ObjNum 457 snap 133 seq_num 457) 2024-09-15T03:55:47.280 INFO:tasks.rados.rados.0.smithi122.stdout:2216: finishing rollback tid 0 to smithi12223906-44 2024-09-15T03:55:47.280 INFO:tasks.rados.rados.0.smithi122.stdout:2216: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:55:47.281 INFO:tasks.rados.rados.0.smithi122.stdout:2216: finishing rollback tid 2 to smithi12223906-44 2024-09-15T03:55:47.281 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1348 (ObjNum 766 snap 240 seq_num 766) dirty exists 2024-09-15T03:55:47.281 INFO:tasks.rados.rados.0.smithi122.stdout:2217: expect (ObjNum 788 snap 258 seq_num 788) 2024-09-15T03:55:47.472 INFO:tasks.rados.rados.0.smithi122.stdout:2218: expect (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:55:47.742 INFO:tasks.rados.rados.0.smithi122.stdout:2215: done (4 left) 2024-09-15T03:55:47.743 INFO:tasks.rados.rados.0.smithi122.stdout:2216: done (3 left) 2024-09-15T03:55:47.743 INFO:tasks.rados.rados.0.smithi122.stdout:2217: done (2 left) 2024-09-15T03:55:47.743 INFO:tasks.rados.rados.0.smithi122.stdout:2218: done (1 left) 2024-09-15T03:55:47.743 INFO:tasks.rados.rados.0.smithi122.stdout:2219: done (0 left) 2024-09-15T03:55:47.743 INFO:tasks.rados.rados.0.smithi122.stdout:2220: write (excl) oid 42 current snap is 261 2024-09-15T03:55:47.743 INFO:tasks.rados.rados.0.smithi122.stdout:2220: seq_num 793 ranges {527951=573169,1504456=556933,2743845=467765} 2024-09-15T03:55:47.756 INFO:tasks.rados.rados.0.smithi122.stdout:2220: writing smithi12223906-42 from 527951 to 1101120 tid 1 2024-09-15T03:55:47.761 INFO:tasks.rados.rados.0.smithi122.stdout:2220: writing smithi12223906-42 from 1504456 to 2061389 tid 2 2024-09-15T03:55:47.768 INFO:tasks.rados.rados.0.smithi122.stdout:2220: writing smithi12223906-42 from 2743845 to 3211610 tid 3 2024-09-15T03:55:47.768 INFO:tasks.rados.rados.0.smithi122.stdout:2221: rollback oid 31 current snap is 261 2024-09-15T03:55:47.768 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 31 to 252 2024-09-15T03:55:47.768 INFO:tasks.rados.rados.0.smithi122.stdout:2222: read oid 14 snap -1 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:2222: expect (ObjNum 760 snap 235 seq_num 760) 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:2223: read oid 19 snap -1 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:2223: expect deleted 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:2224: rollback oid 27 current snap is 261 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 208 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:2225: write oid 22 current snap is 261 2024-09-15T03:55:47.769 INFO:tasks.rados.rados.0.smithi122.stdout:2225: seq_num 794 ranges {544758=648418,1846585=416944,2771268=573490} 2024-09-15T03:55:47.775 INFO:tasks.rados.rados.0.smithi122.stdout:2225: writing smithi12223906-22 from 544758 to 1193176 tid 1 2024-09-15T03:55:47.781 INFO:tasks.rados.rados.0.smithi122.stdout:2225: writing smithi12223906-22 from 1846585 to 2263529 tid 2 2024-09-15T03:55:47.787 INFO:tasks.rados.rados.0.smithi122.stdout:2225: writing smithi12223906-22 from 2771268 to 3344758 tid 3 2024-09-15T03:55:47.787 INFO:tasks.rados.rados.0.smithi122.stdout:2226: snap_remove snap 223 2024-09-15T03:55:48.061 INFO:tasks.rados.rados.0.smithi122.stdout:2226: done (6 left) 2024-09-15T03:55:48.061 INFO:tasks.rados.rados.0.smithi122.stdout:2227: write oid 44 current snap is 261 2024-09-15T03:55:48.061 INFO:tasks.rados.rados.0.smithi122.stdout:2227: seq_num 795 ranges {561565=723667,1905067=560602,3082338=279227} 2024-09-15T03:55:48.074 INFO:tasks.rados.rados.0.smithi122.stdout:2227: writing smithi12223906-44 from 561565 to 1285232 tid 1 2024-09-15T03:55:48.082 INFO:tasks.rados.rados.0.smithi122.stdout:2227: writing smithi12223906-44 from 1905067 to 2465669 tid 2 2024-09-15T03:55:48.087 INFO:tasks.rados.rados.0.smithi122.stdout:2227: writing smithi12223906-44 from 3082338 to 3361565 tid 3 2024-09-15T03:55:48.087 INFO:tasks.rados.rados.0.smithi122.stdout:2228: snap_remove snap 245 2024-09-15T03:55:49.078 INFO:tasks.rados.rados.0.smithi122.stdout:2224: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:55:49.078 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1121 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T03:55:49.078 INFO:tasks.rados.rados.0.smithi122.stdout:2220: finishing write tid 1 to smithi12223906-42 2024-09-15T03:55:49.078 INFO:tasks.rados.rados.0.smithi122.stdout:2220: finishing write tid 2 to smithi12223906-42 2024-09-15T03:55:49.078 INFO:tasks.rados.rados.0.smithi122.stdout:2221: finishing rollback tid 1 to smithi12223906-31 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 813 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:2220: finishing write tid 3 to smithi12223906-42 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:2220: finishing write tid 4 to smithi12223906-42 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:2220: finishing write tid 5 to smithi12223906-42 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1371 (ObjNum 793 snap 261 seq_num 793) dirty exists 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:2220: left oid 42 (ObjNum 793 snap 261 seq_num 793) 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:2225: finishing write tid 1 to smithi12223906-22 2024-09-15T03:55:49.079 INFO:tasks.rados.rados.0.smithi122.stdout:2222: expect (ObjNum 760 snap 235 seq_num 760) 2024-09-15T03:55:49.207 INFO:tasks.rados.rados.0.smithi122.stdout:2225: finishing write tid 2 to smithi12223906-22 2024-09-15T03:55:49.207 INFO:tasks.rados.rados.0.smithi122.stdout:2225: finishing write tid 3 to smithi12223906-22 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:2225: finishing write tid 4 to smithi12223906-22 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:2225: finishing write tid 5 to smithi12223906-22 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1354 (ObjNum 794 snap 261 seq_num 794) dirty exists 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:2225: left oid 22 (ObjNum 794 snap 261 seq_num 794) 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:2227: finishing write tid 1 to smithi12223906-44 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:2227: finishing write tid 2 to smithi12223906-44 2024-09-15T03:55:49.208 INFO:tasks.rados.rados.0.smithi122.stdout:2227: finishing write tid 3 to smithi12223906-44 2024-09-15T03:55:49.209 INFO:tasks.rados.rados.0.smithi122.stdout:2227: finishing write tid 4 to smithi12223906-44 2024-09-15T03:55:49.209 INFO:tasks.rados.rados.0.smithi122.stdout:2227: finishing write tid 5 to smithi12223906-44 2024-09-15T03:55:49.209 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1366 (ObjNum 795 snap 261 seq_num 795) dirty exists 2024-09-15T03:55:49.209 INFO:tasks.rados.rados.0.smithi122.stdout:2227: left oid 44 (ObjNum 795 snap 261 seq_num 795) 2024-09-15T03:55:49.209 INFO:tasks.rados.rados.0.smithi122.stdout:2220: done (7 left) 2024-09-15T03:55:49.209 INFO:tasks.rados.rados.0.smithi122.stdout:2221: done (6 left) 2024-09-15T03:55:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:2222: done (5 left) 2024-09-15T03:55:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:2223: done (4 left) 2024-09-15T03:55:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:2224: done (3 left) 2024-09-15T03:55:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:2225: done (2 left) 2024-09-15T03:55:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:2227: done (1 left) 2024-09-15T03:55:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:2228: done (0 left) 2024-09-15T03:55:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:2229: read oid 45 snap 239 2024-09-15T03:55:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:2229: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:55:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:2230: read oid 27 snap -1 2024-09-15T03:55:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:2230: expect deleted 2024-09-15T03:55:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:2231: rollback oid 37 current snap is 261 2024-09-15T03:55:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 37 to 241 2024-09-15T03:55:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:2232: snap_create 2024-09-15T03:55:49.213 INFO:tasks.rados.rados.0.smithi122.stdout:2231: finishing rollback tid 1 to smithi12223906-37 2024-09-15T03:55:49.213 INFO:tasks.rados.rados.0.smithi122.stdout:2231: finishing rollback tid 2 to smithi12223906-37 2024-09-15T03:55:49.213 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1019 (ObjNum 764 snap 237 seq_num 764) dirty exists 2024-09-15T03:55:49.216 INFO:tasks.rados.rados.0.smithi122.stdout:2229: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:55:50.040 INFO:tasks.rados.rados.0.smithi122.stdout:2229: done (3 left) 2024-09-15T03:55:50.040 INFO:tasks.rados.rados.0.smithi122.stdout:2230: done (2 left) 2024-09-15T03:55:50.040 INFO:tasks.rados.rados.0.smithi122.stdout:2231: done (1 left) 2024-09-15T03:55:50.040 INFO:tasks.rados.rados.0.smithi122.stdout:2232: done (0 left) 2024-09-15T03:55:50.041 INFO:tasks.rados.rados.0.smithi122.stdout:2233: copy_from oid 10 from oid 20 current snap is 262 2024-09-15T03:55:50.041 INFO:tasks.rados.rados.0.smithi122.stdout:2234: write (excl) oid 18 current snap is 262 2024-09-15T03:55:50.041 INFO:tasks.rados.rados.0.smithi122.stdout:2234: seq_num 796 ranges {578372=798916,1963549=420613,2826114=417934} 2024-09-15T03:55:50.050 INFO:tasks.rados.rados.0.smithi122.stdout:2234: writing smithi12223906-18 from 578372 to 1377288 tid 1 2024-09-15T03:55:50.057 INFO:tasks.rados.rados.0.smithi122.stdout:2234: writing smithi12223906-18 from 1963549 to 2384162 tid 2 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2234: writing smithi12223906-18 from 2826114 to 3244048 tid 3 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2235: rollback oid 13 current snap is 262 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 215 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2233: finishing copy_from to smithi12223906-10 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2233: got expected ENOENT (src dne) 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2233: finishing copy_from racing read to smithi12223906-10 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2233: done (2 left) 2024-09-15T03:55:50.063 INFO:tasks.rados.rados.0.smithi122.stdout:2236: delete oid 16 current snap is 262 2024-09-15T03:55:50.064 INFO:tasks.rados.rados.0.smithi122.stdout:2234: finishing write tid 1 to smithi12223906-18 2024-09-15T03:55:50.070 INFO:tasks.rados.rados.0.smithi122.stdout:2235: finishing rollback tid 0 to smithi12223906-13 2024-09-15T03:55:50.073 INFO:tasks.rados.rados.0.smithi122.stdout:2235: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:55:50.073 INFO:tasks.rados.rados.0.smithi122.stdout:2234: finishing write tid 2 to smithi12223906-18 2024-09-15T03:55:50.073 INFO:tasks.rados.rados.0.smithi122.stdout:2234: finishing write tid 3 to smithi12223906-18 2024-09-15T03:55:50.073 INFO:tasks.rados.rados.0.smithi122.stdout:2235: finishing rollback tid 2 to smithi12223906-13 2024-09-15T03:55:50.073 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 817 (ObjNum 661 snap 201 seq_num 661) dirty exists 2024-09-15T03:55:50.074 INFO:tasks.rados.rados.0.smithi122.stdout:2234: finishing write tid 4 to smithi12223906-18 2024-09-15T03:55:50.074 INFO:tasks.rados.rados.0.smithi122.stdout:2235: done (2 left) 2024-09-15T03:55:50.074 INFO:tasks.rados.rados.0.smithi122.stdout:2236: done (1 left) 2024-09-15T03:55:50.074 INFO:tasks.rados.rados.0.smithi122.stdout:2237: snap_create 2024-09-15T03:55:50.074 INFO:tasks.rados.rados.0.smithi122.stdout:2234: finishing write tid 5 to smithi12223906-18 2024-09-15T03:55:50.074 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1381 (ObjNum 796 snap 262 seq_num 796) dirty exists 2024-09-15T03:55:50.075 INFO:tasks.rados.rados.0.smithi122.stdout:2234: left oid 18 (ObjNum 796 snap 262 seq_num 796) 2024-09-15T03:55:51.080 INFO:tasks.rados.rados.0.smithi122.stdout:2234: done (1 left) 2024-09-15T03:55:51.080 INFO:tasks.rados.rados.0.smithi122.stdout:2237: done (0 left) 2024-09-15T03:55:51.080 INFO:tasks.rados.rados.0.smithi122.stdout:2238: write oid 20 current snap is 263 2024-09-15T03:55:51.080 INFO:tasks.rados.rados.0.smithi122.stdout:2238: seq_num 798 ranges {611986=549414,1564160=424282,2764607=440478} 2024-09-15T03:55:51.091 INFO:tasks.rados.rados.0.smithi122.stdout:2238: writing smithi12223906-20 from 611986 to 1161400 tid 1 2024-09-15T03:55:51.096 INFO:tasks.rados.rados.0.smithi122.stdout:2238: writing smithi12223906-20 from 1564160 to 1988442 tid 2 2024-09-15T03:55:51.105 INFO:tasks.rados.rados.0.smithi122.stdout:2238: writing smithi12223906-20 from 2764607 to 3205085 tid 3 2024-09-15T03:55:51.105 INFO:tasks.rados.rados.0.smithi122.stdout:2239: delete oid 17 current snap is 263 2024-09-15T03:55:51.105 INFO:tasks.rados.rados.0.smithi122.stdout:2238: finishing write tid 1 to smithi12223906-20 2024-09-15T03:55:51.105 INFO:tasks.rados.rados.0.smithi122.stdout:2238: finishing write tid 2 to smithi12223906-20 2024-09-15T03:55:51.108 INFO:tasks.rados.rados.0.smithi122.stdout:2239: done (1 left) 2024-09-15T03:55:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:2240: write (excl) oid 12 current snap is 263 2024-09-15T03:55:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:2240: seq_num 800 ranges {645600=416265,1797477=711598,3219453=226147} 2024-09-15T03:55:51.114 INFO:tasks.rados.rados.0.smithi122.stdout:2240: writing smithi12223906-12 from 645600 to 1061865 tid 1 2024-09-15T03:55:51.122 INFO:tasks.rados.rados.0.smithi122.stdout:2240: writing smithi12223906-12 from 1797477 to 2509075 tid 2 2024-09-15T03:55:51.128 INFO:tasks.rados.rados.0.smithi122.stdout:2240: writing smithi12223906-12 from 3219453 to 3445600 tid 3 2024-09-15T03:55:51.129 INFO:tasks.rados.rados.0.smithi122.stdout:2241: write oid 37 current snap is 263 2024-09-15T03:55:51.129 INFO:tasks.rados.rados.0.smithi122.stdout:2238: finishing write tid 3 to smithi12223906-20 2024-09-15T03:55:51.129 INFO:tasks.rados.rados.0.smithi122.stdout:2241: seq_num 801 ranges {662407=491514,1855959=571609,2846876=615531} 2024-09-15T03:55:51.135 INFO:tasks.rados.rados.0.smithi122.stdout:2241: writing smithi12223906-37 from 662407 to 1153921 tid 1 2024-09-15T03:55:51.142 INFO:tasks.rados.rados.0.smithi122.stdout:2241: writing smithi12223906-37 from 1855959 to 2427568 tid 2 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2241: writing smithi12223906-37 from 2846876 to 3462407 tid 3 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2242: delete oid 33 current snap is 263 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2238: finishing write tid 4 to smithi12223906-20 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2238: finishing write tid 5 to smithi12223906-20 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 667 (ObjNum 798 snap 263 seq_num 798) dirty exists 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2238: left oid 20 (ObjNum 798 snap 263 seq_num 798) 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2240: finishing write tid 1 to smithi12223906-12 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2240: finishing write tid 2 to smithi12223906-12 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2240: finishing write tid 3 to smithi12223906-12 2024-09-15T03:55:51.149 INFO:tasks.rados.rados.0.smithi122.stdout:2240: finishing write tid 4 to smithi12223906-12 2024-09-15T03:55:51.150 INFO:tasks.rados.rados.0.smithi122.stdout:2240: finishing write tid 5 to smithi12223906-12 2024-09-15T03:55:51.150 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 739 (ObjNum 800 snap 263 seq_num 800) dirty exists 2024-09-15T03:55:51.150 INFO:tasks.rados.rados.0.smithi122.stdout:2240: left oid 12 (ObjNum 800 snap 263 seq_num 800) 2024-09-15T03:55:51.150 INFO:tasks.rados.rados.0.smithi122.stdout:2241: finishing write tid 1 to smithi12223906-37 2024-09-15T03:55:51.150 INFO:tasks.rados.rados.0.smithi122.stdout:2241: finishing write tid 2 to smithi12223906-37 2024-09-15T03:55:51.155 INFO:tasks.rados.rados.0.smithi122.stdout:2241: finishing write tid 3 to smithi12223906-37 2024-09-15T03:55:51.155 INFO:tasks.rados.rados.0.smithi122.stdout:2241: finishing write tid 4 to smithi12223906-37 2024-09-15T03:55:51.155 INFO:tasks.rados.rados.0.smithi122.stdout:2241: finishing write tid 5 to smithi12223906-37 2024-09-15T03:55:51.155 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1023 (ObjNum 801 snap 263 seq_num 801) dirty exists 2024-09-15T03:55:51.155 INFO:tasks.rados.rados.0.smithi122.stdout:2241: left oid 37 (ObjNum 801 snap 263 seq_num 801) 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2238: done (3 left) 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2240: done (2 left) 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2241: done (1 left) 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2242: done (0 left) 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2243: read oid 1 snap 226 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2243: expect deleted 2024-09-15T03:55:51.157 INFO:tasks.rados.rados.0.smithi122.stdout:2244: copy_from oid 27 from oid 38 current snap is 263 2024-09-15T03:55:51.158 INFO:tasks.rados.rados.0.smithi122.stdout:2245: delete oid 35 current snap is 263 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2243: done (2 left) 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2245: done (1 left) 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2246: copy_from oid 42 from oid 29 current snap is 263 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2247: rollback oid 45 current snap is 263 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 247 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2248: read oid 3 snap -1 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2248: expect (ObjNum 788 snap 258 seq_num 788) 2024-09-15T03:55:51.163 INFO:tasks.rados.rados.0.smithi122.stdout:2249: snap_remove snap 193 2024-09-15T03:55:52.082 INFO:tasks.rados.rados.0.smithi122.stdout:2249: done (4 left) 2024-09-15T03:55:52.082 INFO:tasks.rados.rados.0.smithi122.stdout:2250: delete oid 41 current snap is 263 2024-09-15T03:55:52.084 INFO:tasks.rados.rados.0.smithi122.stdout:2244: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2246: finishing copy_from to smithi12223906-42 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2246: got expected ENOENT (src dne) 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2246: finishing copy_from racing read to smithi12223906-42 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2247: finishing rollback tid 0 to smithi12223906-45 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2247: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2244: finishing copy_from to smithi12223906-27 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 932 (ObjNum 726 snap 221 seq_num 726) dirty exists 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2247: finishing rollback tid 2 to smithi12223906-45 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 820 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:55:52.085 INFO:tasks.rados.rados.0.smithi122.stdout:2248: expect (ObjNum 788 snap 258 seq_num 788) 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2244: done (4 left) 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2246: done (3 left) 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2247: done (2 left) 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2248: done (1 left) 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2250: done (0 left) 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2251: write (excl) oid 10 current snap is 263 2024-09-15T03:55:52.287 INFO:tasks.rados.rados.0.smithi122.stdout:2251: seq_num 805 ranges {729635=792510,1973534=578947,3240215=289420} 2024-09-15T03:55:52.295 INFO:tasks.rados.rados.0.smithi122.stdout:2251: writing smithi12223906-10 from 729635 to 1522145 tid 1 2024-09-15T03:55:52.302 INFO:tasks.rados.rados.0.smithi122.stdout:2251: writing smithi12223906-10 from 1973534 to 2552481 tid 2 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2251: writing smithi12223906-10 from 3240215 to 3529635 tid 3 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2252: read oid 38 snap -1 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2252: expect (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2253: read oid 20 snap 258 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2253: expect deleted 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2254: write (excl) oid 18 current snap is 263 2024-09-15T03:55:52.309 INFO:tasks.rados.rados.0.smithi122.stdout:2254: seq_num 806 ranges {746442=584112,1748369=722605,2983991=562451} 2024-09-15T03:55:52.316 INFO:tasks.rados.rados.0.smithi122.stdout:2254: writing smithi12223906-18 from 746442 to 1330554 tid 1 2024-09-15T03:55:52.323 INFO:tasks.rados.rados.0.smithi122.stdout:2254: writing smithi12223906-18 from 1748369 to 2470974 tid 2 2024-09-15T03:55:52.329 INFO:tasks.rados.rados.0.smithi122.stdout:2254: writing smithi12223906-18 from 2983991 to 3546442 tid 3 2024-09-15T03:55:52.329 INFO:tasks.rados.rados.0.smithi122.stdout:2255: snap_create 2024-09-15T03:55:52.329 INFO:tasks.rados.rados.0.smithi122.stdout:2251: finishing write tid 1 to smithi12223906-10 2024-09-15T03:55:52.329 INFO:tasks.rados.rados.0.smithi122.stdout:2251: finishing write tid 2 to smithi12223906-10 2024-09-15T03:55:52.329 INFO:tasks.rados.rados.0.smithi122.stdout:2251: finishing write tid 3 to smithi12223906-10 2024-09-15T03:55:52.329 INFO:tasks.rados.rados.0.smithi122.stdout:2251: finishing write tid 4 to smithi12223906-10 2024-09-15T03:55:52.330 INFO:tasks.rados.rados.0.smithi122.stdout:2251: finishing write tid 5 to smithi12223906-10 2024-09-15T03:55:52.330 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1027 (ObjNum 805 snap 263 seq_num 805) dirty exists 2024-09-15T03:55:52.330 INFO:tasks.rados.rados.0.smithi122.stdout:2251: left oid 10 (ObjNum 805 snap 263 seq_num 805) 2024-09-15T03:55:52.330 INFO:tasks.rados.rados.0.smithi122.stdout:2252: expect (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:55:52.435 INFO:tasks.rados.rados.0.smithi122.stdout:2254: finishing write tid 1 to smithi12223906-18 2024-09-15T03:55:52.436 INFO:tasks.rados.rados.0.smithi122.stdout:2254: finishing write tid 2 to smithi12223906-18 2024-09-15T03:55:52.436 INFO:tasks.rados.rados.0.smithi122.stdout:2254: finishing write tid 3 to smithi12223906-18 2024-09-15T03:55:52.436 INFO:tasks.rados.rados.0.smithi122.stdout:2254: finishing write tid 4 to smithi12223906-18 2024-09-15T03:55:52.436 INFO:tasks.rados.rados.0.smithi122.stdout:2254: finishing write tid 5 to smithi12223906-18 2024-09-15T03:55:52.436 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1401 (ObjNum 806 snap 263 seq_num 806) dirty exists 2024-09-15T03:55:52.436 INFO:tasks.rados.rados.0.smithi122.stdout:2254: left oid 18 (ObjNum 806 snap 263 seq_num 806) 2024-09-15T03:55:53.102 INFO:tasks.rados.rados.0.smithi122.stdout:2251: done (4 left) 2024-09-15T03:55:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:2252: done (3 left) 2024-09-15T03:55:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:2253: done (2 left) 2024-09-15T03:55:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:2254: done (1 left) 2024-09-15T03:55:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:2255: done (0 left) 2024-09-15T03:55:53.103 INFO:tasks.rados.rados.0.smithi122.stdout:2256: snap_remove snap 242 2024-09-15T03:55:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:2256: done (0 left) 2024-09-15T03:55:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:2257: write (excl) oid 11 current snap is 264 2024-09-15T03:55:54.119 INFO:tasks.rados.rados.0.smithi122.stdout:2257: seq_num 807 ranges {763249=659361,2090498=582616,3295061=268188} 2024-09-15T03:55:54.128 INFO:tasks.rados.rados.0.smithi122.stdout:2257: writing smithi12223906-11 from 763249 to 1422610 tid 1 2024-09-15T03:55:54.135 INFO:tasks.rados.rados.0.smithi122.stdout:2257: writing smithi12223906-11 from 2090498 to 2673114 tid 2 2024-09-15T03:55:54.140 INFO:tasks.rados.rados.0.smithi122.stdout:2257: writing smithi12223906-11 from 3295061 to 3563249 tid 3 2024-09-15T03:55:54.140 INFO:tasks.rados.rados.0.smithi122.stdout:2258: snap_remove snap 241 2024-09-15T03:55:55.041 INFO:tasks.rados.rados.0.smithi122.stdout:2258: done (1 left) 2024-09-15T03:55:55.042 INFO:tasks.rados.rados.0.smithi122.stdout:2259: snap_remove snap 256 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:2257: finishing write tid 1 to smithi12223906-11 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:2257: finishing write tid 2 to smithi12223906-11 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:2257: finishing write tid 3 to smithi12223906-11 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:2257: finishing write tid 4 to smithi12223906-11 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:2257: finishing write tid 5 to smithi12223906-11 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1008 (ObjNum 807 snap 264 seq_num 807) dirty exists 2024-09-15T03:55:56.053 INFO:tasks.rados.rados.0.smithi122.stdout:2257: left oid 11 (ObjNum 807 snap 264 seq_num 807) 2024-09-15T03:55:56.054 INFO:tasks.rados.rados.0.smithi122.stdout:2257: done (1 left) 2024-09-15T03:55:56.054 INFO:tasks.rados.rados.0.smithi122.stdout:2259: done (0 left) 2024-09-15T03:55:56.054 INFO:tasks.rados.rados.0.smithi122.stdout:2260: write (excl) oid 3 current snap is 264 2024-09-15T03:55:56.054 INFO:tasks.rados.rados.0.smithi122.stdout:2260: seq_num 808 ranges {780056=734610,2148980=726274,3322484=257572} 2024-09-15T03:55:56.063 INFO:tasks.rados.rados.0.smithi122.stdout:2260: writing smithi12223906-3 from 780056 to 1514666 tid 1 2024-09-15T03:55:56.071 INFO:tasks.rados.rados.0.smithi122.stdout:2260: writing smithi12223906-3 from 2148980 to 2875254 tid 2 2024-09-15T03:55:56.075 INFO:tasks.rados.rados.0.smithi122.stdout:2260: writing smithi12223906-3 from 3322484 to 3580056 tid 3 2024-09-15T03:55:56.075 INFO:tasks.rados.rados.0.smithi122.stdout:2261: snap_create 2024-09-15T03:55:56.076 INFO:tasks.rados.rados.0.smithi122.stdout:2260: finishing write tid 1 to smithi12223906-3 2024-09-15T03:55:56.076 INFO:tasks.rados.rados.0.smithi122.stdout:2260: finishing write tid 2 to smithi12223906-3 2024-09-15T03:55:56.079 INFO:tasks.rados.rados.0.smithi122.stdout:2260: finishing write tid 3 to smithi12223906-3 2024-09-15T03:55:56.079 INFO:tasks.rados.rados.0.smithi122.stdout:2260: finishing write tid 4 to smithi12223906-3 2024-09-15T03:55:56.079 INFO:tasks.rados.rados.0.smithi122.stdout:2260: finishing write tid 5 to smithi12223906-3 2024-09-15T03:55:56.080 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 1428 (ObjNum 808 snap 264 seq_num 808) dirty exists 2024-09-15T03:55:56.080 INFO:tasks.rados.rados.0.smithi122.stdout:2260: left oid 3 (ObjNum 808 snap 264 seq_num 808) 2024-09-15T03:55:57.049 INFO:tasks.rados.rados.0.smithi122.stdout:2260: done (1 left) 2024-09-15T03:55:57.049 INFO:tasks.rados.rados.0.smithi122.stdout:2261: done (0 left) 2024-09-15T03:55:57.049 INFO:tasks.rados.rados.0.smithi122.stdout:2262: read oid 36 snap -1 2024-09-15T03:55:57.049 INFO:tasks.rados.rados.0.smithi122.stdout:2262: expect deleted 2024-09-15T03:55:57.049 INFO:tasks.rados.rados.0.smithi122.stdout:2263: write oid 34 current snap is 265 2024-09-15T03:55:57.050 INFO:tasks.rados.rados.0.smithi122.stdout:2263: seq_num 809 ranges {796863=409859,1807462=586285,3066260=530603} 2024-09-15T03:55:57.060 INFO:tasks.rados.rados.0.smithi122.stdout:2263: writing smithi12223906-34 from 796863 to 1206722 tid 1 2024-09-15T03:55:57.067 INFO:tasks.rados.rados.0.smithi122.stdout:2263: writing smithi12223906-34 from 1807462 to 2393747 tid 2 2024-09-15T03:55:57.077 INFO:tasks.rados.rados.0.smithi122.stdout:2263: writing smithi12223906-34 from 3066260 to 3596863 tid 3 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2264: read oid 20 snap -1 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2264: expect (ObjNum 798 snap 263 seq_num 798) 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2265: rollback oid 27 current snap is 265 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 198 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2263: finishing write tid 1 to smithi12223906-34 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2263: finishing write tid 2 to smithi12223906-34 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2262: done (3 left) 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2266: read oid 23 snap -1 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2266: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:55:57.078 INFO:tasks.rados.rados.0.smithi122.stdout:2267: delete oid 32 current snap is 265 2024-09-15T03:55:57.079 INFO:tasks.rados.rados.0.smithi122.stdout:2263: finishing write tid 3 to smithi12223906-34 2024-09-15T03:55:57.082 INFO:tasks.rados.rados.0.smithi122.stdout:2263: finishing write tid 4 to smithi12223906-34 2024-09-15T03:55:57.082 INFO:tasks.rados.rados.0.smithi122.stdout:2263: finishing write tid 5 to smithi12223906-34 2024-09-15T03:55:57.082 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 743 (ObjNum 809 snap 265 seq_num 809) dirty exists 2024-09-15T03:55:57.083 INFO:tasks.rados.rados.0.smithi122.stdout:2263: left oid 34 (ObjNum 809 snap 265 seq_num 809) 2024-09-15T03:55:57.086 INFO:tasks.rados.rados.0.smithi122.stdout:2264: expect (ObjNum 798 snap 263 seq_num 798) 2024-09-15T03:55:57.202 INFO:tasks.rados.rados.0.smithi122.stdout:2265: finishing rollback tid 0 to smithi12223906-27 2024-09-15T03:55:57.202 INFO:tasks.rados.rados.0.smithi122.stdout:2265: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:55:57.202 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1436 (ObjNum 18369 snap 0 seq_num 1842515568) dirty dne 2024-09-15T03:55:57.202 INFO:tasks.rados.rados.0.smithi122.stdout:2266: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:55:57.465 INFO:tasks.rados.rados.0.smithi122.stdout:2263: done (4 left) 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2264: done (3 left) 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2265: done (2 left) 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2266: done (1 left) 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2267: done (0 left) 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2268: read oid 40 snap -1 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2268: expect deleted 2024-09-15T03:55:57.466 INFO:tasks.rados.rados.0.smithi122.stdout:2269: delete oid 37 current snap is 265 2024-09-15T03:55:57.474 INFO:tasks.rados.rados.0.smithi122.stdout:2268: done (1 left) 2024-09-15T03:55:57.475 INFO:tasks.rados.rados.0.smithi122.stdout:2269: done (0 left) 2024-09-15T03:55:57.475 INFO:tasks.rados.rados.0.smithi122.stdout:2270: snap_create 2024-09-15T03:55:58.064 INFO:tasks.rados.rados.0.smithi122.stdout:2270: done (0 left) 2024-09-15T03:55:58.064 INFO:tasks.rados.rados.0.smithi122.stdout:2271: copy_from oid 28 from oid 40 current snap is 266 2024-09-15T03:55:58.064 INFO:tasks.rados.rados.0.smithi122.stdout:2272: copy_from oid 47 from oid 27 current snap is 266 2024-09-15T03:55:58.064 INFO:tasks.rados.rados.0.smithi122.stdout:2273: delete oid 12 current snap is 266 2024-09-15T03:55:58.065 INFO:tasks.rados.rados.0.smithi122.stdout:2272: finishing copy_from to smithi12223906-47 2024-09-15T03:55:58.065 INFO:tasks.rados.rados.0.smithi122.stdout:2272: got expected ENOENT (src dne) 2024-09-15T03:55:58.065 INFO:tasks.rados.rados.0.smithi122.stdout:2272: finishing copy_from racing read to smithi12223906-47 2024-09-15T03:55:58.070 INFO:tasks.rados.rados.0.smithi122.stdout:2272: done (2 left) 2024-09-15T03:55:58.070 INFO:tasks.rados.rados.0.smithi122.stdout:2273: done (1 left) 2024-09-15T03:55:58.070 INFO:tasks.rados.rados.0.smithi122.stdout:2274: read oid 29 snap -1 2024-09-15T03:55:58.070 INFO:tasks.rados.rados.0.smithi122.stdout:2274: expect deleted 2024-09-15T03:55:58.070 INFO:tasks.rados.rados.0.smithi122.stdout:2275: snap_remove snap 192 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2271: finishing copy_from to smithi12223906-28 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2271: got expected ENOENT (src dne) 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2271: finishing copy_from racing read to smithi12223906-28 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2271: done (2 left) 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2274: done (1 left) 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2275: done (0 left) 2024-09-15T03:55:59.074 INFO:tasks.rados.rados.0.smithi122.stdout:2276: write oid 21 current snap is 266 2024-09-15T03:55:59.075 INFO:tasks.rados.rados.0.smithi122.stdout:2276: seq_num 813 ranges {464091=710855,1925037=477270,2943246=474970} 2024-09-15T03:55:59.087 INFO:tasks.rados.rados.0.smithi122.stdout:2276: writing smithi12223906-21 from 464091 to 1174946 tid 1 2024-09-15T03:55:59.095 INFO:tasks.rados.rados.0.smithi122.stdout:2276: writing smithi12223906-21 from 1925037 to 2402307 tid 2 2024-09-15T03:55:59.101 INFO:tasks.rados.rados.0.smithi122.stdout:2276: writing smithi12223906-21 from 2943246 to 3418216 tid 3 2024-09-15T03:55:59.101 INFO:tasks.rados.rados.0.smithi122.stdout:2277: snap_create 2024-09-15T03:55:59.101 INFO:tasks.rados.rados.0.smithi122.stdout:2276: finishing write tid 1 to smithi12223906-21 2024-09-15T03:55:59.101 INFO:tasks.rados.rados.0.smithi122.stdout:2276: finishing write tid 2 to smithi12223906-21 2024-09-15T03:55:59.104 INFO:tasks.rados.rados.0.smithi122.stdout:2276: finishing write tid 3 to smithi12223906-21 2024-09-15T03:55:59.104 INFO:tasks.rados.rados.0.smithi122.stdout:2276: finishing write tid 4 to smithi12223906-21 2024-09-15T03:55:59.104 INFO:tasks.rados.rados.0.smithi122.stdout:2276: finishing write tid 5 to smithi12223906-21 2024-09-15T03:55:59.104 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1032 (ObjNum 813 snap 266 seq_num 813) dirty exists 2024-09-15T03:55:59.104 INFO:tasks.rados.rados.0.smithi122.stdout:2276: left oid 21 (ObjNum 813 snap 266 seq_num 813) 2024-09-15T03:56:00.050 INFO:tasks.rados.rados.0.smithi122.stdout:2276: done (1 left) 2024-09-15T03:56:00.050 INFO:tasks.rados.rados.0.smithi122.stdout:2277: done (0 left) 2024-09-15T03:56:00.050 INFO:tasks.rados.rados.0.smithi122.stdout:2278: delete oid 5 current snap is 267 2024-09-15T03:56:00.055 INFO:tasks.rados.rados.0.smithi122.stdout:2278: done (0 left) 2024-09-15T03:56:00.055 INFO:tasks.rados.rados.0.smithi122.stdout:2279: rollback oid 50 current snap is 267 2024-09-15T03:56:00.055 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 266 2024-09-15T03:56:00.055 INFO:tasks.rados.rados.0.smithi122.stdout:2280: write oid 10 current snap is 267 2024-09-15T03:56:00.055 INFO:tasks.rados.rados.0.smithi122.stdout:2280: seq_num 815 ranges {497705=577706,1642001=480939,2598092=497514} 2024-09-15T03:56:00.064 INFO:tasks.rados.rados.0.smithi122.stdout:2280: writing smithi12223906-10 from 497705 to 1075411 tid 1 2024-09-15T03:56:00.071 INFO:tasks.rados.rados.0.smithi122.stdout:2280: writing smithi12223906-10 from 1642001 to 2122940 tid 2 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:2280: writing smithi12223906-10 from 2598092 to 3095606 tid 3 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:2281: copy_from oid 1 from oid 43 current snap is 267 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:2279: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1332 (ObjNum 129 snap 0 seq_num 1842481008) dirty dne 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:2279: done (2 left) 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:2282: delete oid 25 current snap is 267 2024-09-15T03:56:00.078 INFO:tasks.rados.rados.0.smithi122.stdout:2280: finishing write tid 1 to smithi12223906-10 2024-09-15T03:56:00.082 INFO:tasks.rados.rados.0.smithi122.stdout:2280: finishing write tid 2 to smithi12223906-10 2024-09-15T03:56:00.083 INFO:tasks.rados.rados.0.smithi122.stdout:2280: finishing write tid 3 to smithi12223906-10 2024-09-15T03:56:00.083 INFO:tasks.rados.rados.0.smithi122.stdout:2280: finishing write tid 4 to smithi12223906-10 2024-09-15T03:56:00.084 INFO:tasks.rados.rados.0.smithi122.stdout:2280: finishing write tid 5 to smithi12223906-10 2024-09-15T03:56:00.084 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1036 (ObjNum 815 snap 267 seq_num 815) dirty exists 2024-09-15T03:56:00.084 INFO:tasks.rados.rados.0.smithi122.stdout:2280: left oid 10 (ObjNum 815 snap 267 seq_num 815) 2024-09-15T03:56:00.086 INFO:tasks.rados.rados.0.smithi122.stdout:2280: done (2 left) 2024-09-15T03:56:00.086 INFO:tasks.rados.rados.0.smithi122.stdout:2282: done (1 left) 2024-09-15T03:56:00.086 INFO:tasks.rados.rados.0.smithi122.stdout:2283: write oid 9 current snap is 267 2024-09-15T03:56:00.086 INFO:tasks.rados.rados.0.smithi122.stdout:2283: seq_num 817 ranges {531319=728204,1758965=484608,2652938=520058} 2024-09-15T03:56:00.096 INFO:tasks.rados.rados.0.smithi122.stdout:2283: writing smithi12223906-9 from 531319 to 1259523 tid 1 2024-09-15T03:56:00.103 INFO:tasks.rados.rados.0.smithi122.stdout:2283: writing smithi12223906-9 from 1758965 to 2243573 tid 2 2024-09-15T03:56:00.109 INFO:tasks.rados.rados.0.smithi122.stdout:2283: writing smithi12223906-9 from 2652938 to 3172996 tid 3 2024-09-15T03:56:00.109 INFO:tasks.rados.rados.0.smithi122.stdout:2284: copy_from oid 34 from oid 50 current snap is 267 2024-09-15T03:56:00.109 INFO:tasks.rados.rados.0.smithi122.stdout:2281: finishing copy_from racing read to smithi12223906-1 2024-09-15T03:56:00.109 INFO:tasks.rados.rados.0.smithi122.stdout:2281: finishing copy_from to smithi12223906-1 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1293 (ObjNum 778 snap 252 seq_num 778) dirty exists 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2283: finishing write tid 1 to smithi12223906-9 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2283: finishing write tid 2 to smithi12223906-9 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2281: done (2 left) 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2285: read oid 2 snap -1 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2285: expect (ObjNum 719 snap 218 seq_num 719) 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2286: read oid 4 snap 238 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2286: expect (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2287: write oid 35 current snap is 267 2024-09-15T03:56:00.110 INFO:tasks.rados.rados.0.smithi122.stdout:2287: seq_num 818 ranges {548126=403453,1417447=744619,2680361=447683,3680450=67676} 2024-09-15T03:56:00.114 INFO:tasks.rados.rados.0.smithi122.stdout:2287: writing smithi12223906-35 from 548126 to 951579 tid 1 2024-09-15T03:56:00.121 INFO:tasks.rados.rados.0.smithi122.stdout:2287: writing smithi12223906-35 from 1417447 to 2162066 tid 2 2024-09-15T03:56:00.127 INFO:tasks.rados.rados.0.smithi122.stdout:2287: writing smithi12223906-35 from 2680361 to 3128044 tid 3 2024-09-15T03:56:00.130 INFO:tasks.rados.rados.0.smithi122.stdout:2287: writing smithi12223906-35 from 3680450 to 3748126 tid 4 2024-09-15T03:56:00.130 INFO:tasks.rados.rados.0.smithi122.stdout:2288: write oid 22 current snap is 267 2024-09-15T03:56:00.130 INFO:tasks.rados.rados.0.smithi122.stdout:2288: seq_num 819 ranges {564933=478702,1759576=488277,2991431=658955} 2024-09-15T03:56:00.136 INFO:tasks.rados.rados.0.smithi122.stdout:2288: writing smithi12223906-22 from 564933 to 1043635 tid 1 2024-09-15T03:56:00.143 INFO:tasks.rados.rados.0.smithi122.stdout:2288: writing smithi12223906-22 from 1759576 to 2247853 tid 2 2024-09-15T03:56:00.151 INFO:tasks.rados.rados.0.smithi122.stdout:2288: writing smithi12223906-22 from 2991431 to 3650386 tid 3 2024-09-15T03:56:00.151 INFO:tasks.rados.rados.0.smithi122.stdout:2289: write oid 49 current snap is 267 2024-09-15T03:56:00.151 INFO:tasks.rados.rados.0.smithi122.stdout:2289: seq_num 820 ranges {581740=553951,1818058=748288,3018854=470227} 2024-09-15T03:56:00.157 INFO:tasks.rados.rados.0.smithi122.stdout:2289: writing smithi12223906-49 from 581740 to 1135691 tid 1 2024-09-15T03:56:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2289: writing smithi12223906-49 from 1818058 to 2566346 tid 2 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2289: writing smithi12223906-49 from 3018854 to 3489081 tid 3 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2290: read oid 45 snap -1 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2290: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2291: snap_create 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2284: finishing copy_from to smithi12223906-34 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2284: got expected ENOENT (src dne) 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2284: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2283: finishing write tid 3 to smithi12223906-9 2024-09-15T03:56:00.171 INFO:tasks.rados.rados.0.smithi122.stdout:2285: expect (ObjNum 719 snap 218 seq_num 719) 2024-09-15T03:56:00.282 INFO:tasks.rados.rados.0.smithi122.stdout:2283: finishing write tid 4 to smithi12223906-9 2024-09-15T03:56:00.282 INFO:tasks.rados.rados.0.smithi122.stdout:2283: finishing write tid 5 to smithi12223906-9 2024-09-15T03:56:00.282 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 836 (ObjNum 817 snap 267 seq_num 817) dirty exists 2024-09-15T03:56:00.282 INFO:tasks.rados.rados.0.smithi122.stdout:2283: left oid 9 (ObjNum 817 snap 267 seq_num 817) 2024-09-15T03:56:00.283 INFO:tasks.rados.rados.0.smithi122.stdout:2286: expect (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:56:00.472 INFO:tasks.rados.rados.0.smithi122.stdout:2287: finishing write tid 1 to smithi12223906-35 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2287: finishing write tid 2 to smithi12223906-35 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2287: finishing write tid 3 to smithi12223906-35 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2287: finishing write tid 4 to smithi12223906-35 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2287: finishing write tid 5 to smithi12223906-35 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2287: finishing write tid 6 to smithi12223906-35 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 600 (ObjNum 818 snap 267 seq_num 818) dirty exists 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2287: left oid 35 (ObjNum 818 snap 267 seq_num 818) 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2288: finishing write tid 1 to smithi12223906-22 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2288: finishing write tid 2 to smithi12223906-22 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2288: finishing write tid 3 to smithi12223906-22 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2288: finishing write tid 4 to smithi12223906-22 2024-09-15T03:56:00.473 INFO:tasks.rados.rados.0.smithi122.stdout:2288: finishing write tid 5 to smithi12223906-22 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1456 (ObjNum 819 snap 267 seq_num 819) dirty exists 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2288: left oid 22 (ObjNum 819 snap 267 seq_num 819) 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2289: finishing write tid 1 to smithi12223906-49 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2289: finishing write tid 2 to smithi12223906-49 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2289: finishing write tid 3 to smithi12223906-49 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2289: finishing write tid 4 to smithi12223906-49 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2289: finishing write tid 5 to smithi12223906-49 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1021 (ObjNum 820 snap 267 seq_num 820) dirty exists 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2289: left oid 49 (ObjNum 820 snap 267 seq_num 820) 2024-09-15T03:56:00.474 INFO:tasks.rados.rados.0.smithi122.stdout:2290: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:56:01.091 INFO:tasks.rados.rados.0.smithi122.stdout:2283: done (8 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2284: done (7 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2285: done (6 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2286: done (5 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2287: done (4 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2288: done (3 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2289: done (2 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2290: done (1 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2291: done (0 left) 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2292: read oid 17 snap 240 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2292: expect deleted 2024-09-15T03:56:01.092 INFO:tasks.rados.rados.0.smithi122.stdout:2293: snap_remove snap 236 2024-09-15T03:56:02.101 INFO:tasks.rados.rados.0.smithi122.stdout:2293: done (1 left) 2024-09-15T03:56:02.101 INFO:tasks.rados.rados.0.smithi122.stdout:2294: snap_remove snap 243 2024-09-15T03:56:03.119 INFO:tasks.rados.rados.0.smithi122.stdout:2294: done (1 left) 2024-09-15T03:56:03.119 INFO:tasks.rados.rados.0.smithi122.stdout:2295: snap_remove snap 255 2024-09-15T03:56:04.124 INFO:tasks.rados.rados.0.smithi122.stdout:2295: done (1 left) 2024-09-15T03:56:04.124 INFO:tasks.rados.rados.0.smithi122.stdout:2296: read oid 11 snap -1 2024-09-15T03:56:04.124 INFO:tasks.rados.rados.0.smithi122.stdout:2296: expect (ObjNum 807 snap 264 seq_num 807) 2024-09-15T03:56:04.124 INFO:tasks.rados.rados.0.smithi122.stdout:2297: read oid 25 snap 260 2024-09-15T03:56:04.124 INFO:tasks.rados.rados.0.smithi122.stdout:2297: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:56:04.126 INFO:tasks.rados.rados.0.smithi122.stdout:2298: write (excl) oid 13 current snap is 268 2024-09-15T03:56:04.126 INFO:tasks.rados.rados.0.smithi122.stdout:2298: seq_num 821 ranges {598547=629200,1876540=491946,3046277=681499} 2024-09-15T03:56:04.132 INFO:tasks.rados.rados.0.smithi122.stdout:2298: writing smithi12223906-13 from 598547 to 1227747 tid 1 2024-09-15T03:56:04.139 INFO:tasks.rados.rados.0.smithi122.stdout:2298: writing smithi12223906-13 from 1876540 to 2368486 tid 2 2024-09-15T03:56:04.146 INFO:tasks.rados.rados.0.smithi122.stdout:2298: writing smithi12223906-13 from 3046277 to 3727776 tid 3 2024-09-15T03:56:04.147 INFO:tasks.rados.rados.0.smithi122.stdout:2299: copy_from oid 39 from oid 34 current snap is 268 2024-09-15T03:56:04.147 INFO:tasks.rados.rados.0.smithi122.stdout:2296: expect (ObjNum 807 snap 264 seq_num 807) 2024-09-15T03:56:04.323 INFO:tasks.rados.rados.0.smithi122.stdout:2297: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2298: finishing write tid 1 to smithi12223906-13 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2298: finishing write tid 2 to smithi12223906-13 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2292: done (4 left) 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2296: done (3 left) 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2297: done (2 left) 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2300: copy_from oid 17 from oid 16 current snap is 268 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2298: finishing write tid 3 to smithi12223906-13 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2298: finishing write tid 4 to smithi12223906-13 2024-09-15T03:56:04.483 INFO:tasks.rados.rados.0.smithi122.stdout:2298: finishing write tid 5 to smithi12223906-13 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 824 (ObjNum 821 snap 268 seq_num 821) dirty exists 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:2298: left oid 13 (ObjNum 821 snap 268 seq_num 821) 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:2299: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:2299: finishing copy_from to smithi12223906-39 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 743 (ObjNum 809 snap 265 seq_num 809) dirty exists 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:2298: done (2 left) 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:2299: done (1 left) 2024-09-15T03:56:04.484 INFO:tasks.rados.rados.0.smithi122.stdout:2301: snap_remove snap 244 2024-09-15T03:56:05.048 INFO:tasks.rados.rados.0.smithi122.stdout:2301: done (1 left) 2024-09-15T03:56:05.049 INFO:tasks.rados.rados.0.smithi122.stdout:2302: delete oid 8 current snap is 268 2024-09-15T03:56:05.049 INFO:tasks.rados.rados.0.smithi122.stdout:2300: finishing copy_from to smithi12223906-17 2024-09-15T03:56:05.049 INFO:tasks.rados.rados.0.smithi122.stdout:2300: got expected ENOENT (src dne) 2024-09-15T03:56:05.049 INFO:tasks.rados.rados.0.smithi122.stdout:2300: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2300: done (1 left) 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2302: done (0 left) 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2303: copy_from oid 6 from oid 12 current snap is 268 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2304: read oid 3 snap -1 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2304: expect (ObjNum 808 snap 264 seq_num 808) 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2305: copy_from oid 17 from oid 13 current snap is 268 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2306: snap_create 2024-09-15T03:56:05.058 INFO:tasks.rados.rados.0.smithi122.stdout:2303: finishing copy_from to smithi12223906-6 2024-09-15T03:56:05.059 INFO:tasks.rados.rados.0.smithi122.stdout:2303: got expected ENOENT (src dne) 2024-09-15T03:56:05.059 INFO:tasks.rados.rados.0.smithi122.stdout:2303: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:56:05.075 INFO:tasks.rados.rados.0.smithi122.stdout:2304: expect (ObjNum 808 snap 264 seq_num 808) 2024-09-15T03:56:05.345 INFO:tasks.rados.rados.0.smithi122.stdout:2305: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:56:05.345 INFO:tasks.rados.rados.0.smithi122.stdout:2305: finishing copy_from to smithi12223906-17 2024-09-15T03:56:05.345 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 824 (ObjNum 821 snap 268 seq_num 821) dirty exists 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2303: done (3 left) 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2304: done (2 left) 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2305: done (1 left) 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2306: done (0 left) 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2307: read oid 13 snap 217 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2307: expect (ObjNum 661 snap 201 seq_num 661) 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2308: write (excl) oid 15 current snap is 269 2024-09-15T03:56:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2308: seq_num 823 ranges {632161=496051,1593504=495615,2701123=420396,3611645=220516} 2024-09-15T03:56:06.061 INFO:tasks.rados.rados.0.smithi122.stdout:2308: writing smithi12223906-15 from 632161 to 1128212 tid 1 2024-09-15T03:56:06.067 INFO:tasks.rados.rados.0.smithi122.stdout:2308: writing smithi12223906-15 from 1593504 to 2089119 tid 2 2024-09-15T03:56:06.073 INFO:tasks.rados.rados.0.smithi122.stdout:2308: writing smithi12223906-15 from 2701123 to 3121519 tid 3 2024-09-15T03:56:06.077 INFO:tasks.rados.rados.0.smithi122.stdout:2308: writing smithi12223906-15 from 3611645 to 3832161 tid 4 2024-09-15T03:56:06.077 INFO:tasks.rados.rados.0.smithi122.stdout:2309: rollback oid 4 current snap is 269 2024-09-15T03:56:06.077 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 240 2024-09-15T03:56:06.078 INFO:tasks.rados.rados.0.smithi122.stdout:2307: expect (ObjNum 661 snap 201 seq_num 661) 2024-09-15T03:56:06.293 INFO:tasks.rados.rados.0.smithi122.stdout:2308: finishing write tid 1 to smithi12223906-15 2024-09-15T03:56:06.293 INFO:tasks.rados.rados.0.smithi122.stdout:2308: finishing write tid 2 to smithi12223906-15 2024-09-15T03:56:06.293 INFO:tasks.rados.rados.0.smithi122.stdout:2307: done (2 left) 2024-09-15T03:56:06.293 INFO:tasks.rados.rados.0.smithi122.stdout:2310: copy_from oid 20 from oid 32 current snap is 269 2024-09-15T03:56:06.293 INFO:tasks.rados.rados.0.smithi122.stdout:2308: finishing write tid 3 to smithi12223906-15 2024-09-15T03:56:06.294 INFO:tasks.rados.rados.0.smithi122.stdout:2308: finishing write tid 4 to smithi12223906-15 2024-09-15T03:56:06.294 INFO:tasks.rados.rados.0.smithi122.stdout:2308: finishing write tid 5 to smithi12223906-15 2024-09-15T03:56:06.294 INFO:tasks.rados.rados.0.smithi122.stdout:2308: finishing write tid 6 to smithi12223906-15 2024-09-15T03:56:06.294 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1298 (ObjNum 823 snap 269 seq_num 823) dirty exists 2024-09-15T03:56:06.294 INFO:tasks.rados.rados.0.smithi122.stdout:2308: left oid 15 (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:56:06.294 INFO:tasks.rados.rados.0.smithi122.stdout:2309: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:56:06.295 INFO:tasks.rados.rados.0.smithi122.stdout:2309: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:56:06.295 INFO:tasks.rados.rados.0.smithi122.stdout:2309: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:56:06.295 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 816 (ObjNum 758 snap 234 seq_num 758) dirty exists 2024-09-15T03:56:06.295 INFO:tasks.rados.rados.0.smithi122.stdout:2308: done (2 left) 2024-09-15T03:56:06.295 INFO:tasks.rados.rados.0.smithi122.stdout:2309: done (1 left) 2024-09-15T03:56:06.295 INFO:tasks.rados.rados.0.smithi122.stdout:2311: write (excl) oid 18 current snap is 269 2024-09-15T03:56:06.296 INFO:tasks.rados.rados.0.smithi122.stdout:2311: seq_num 824 ranges {648968=571300,1651986=639273,2728546=631668} 2024-09-15T03:56:06.299 INFO:tasks.rados.rados.0.smithi122.stdout:2311: writing smithi12223906-18 from 648968 to 1220268 tid 1 2024-09-15T03:56:06.305 INFO:tasks.rados.rados.0.smithi122.stdout:2311: writing smithi12223906-18 from 1651986 to 2291259 tid 2 2024-09-15T03:56:06.311 INFO:tasks.rados.rados.0.smithi122.stdout:2311: writing smithi12223906-18 from 2728546 to 3360214 tid 3 2024-09-15T03:56:06.311 INFO:tasks.rados.rados.0.smithi122.stdout:2312: delete oid 47 current snap is 269 2024-09-15T03:56:06.311 INFO:tasks.rados.rados.0.smithi122.stdout:2310: finishing copy_from to smithi12223906-20 2024-09-15T03:56:06.311 INFO:tasks.rados.rados.0.smithi122.stdout:2310: got expected ENOENT (src dne) 2024-09-15T03:56:06.311 INFO:tasks.rados.rados.0.smithi122.stdout:2310: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:56:06.320 INFO:tasks.rados.rados.0.smithi122.stdout:2311: finishing write tid 1 to smithi12223906-18 2024-09-15T03:56:06.322 INFO:tasks.rados.rados.0.smithi122.stdout:2311: finishing write tid 2 to smithi12223906-18 2024-09-15T03:56:06.322 INFO:tasks.rados.rados.0.smithi122.stdout:2311: finishing write tid 3 to smithi12223906-18 2024-09-15T03:56:06.322 INFO:tasks.rados.rados.0.smithi122.stdout:2310: done (2 left) 2024-09-15T03:56:06.322 INFO:tasks.rados.rados.0.smithi122.stdout:2312: done (1 left) 2024-09-15T03:56:06.322 INFO:tasks.rados.rados.0.smithi122.stdout:2313: snap_remove snap 254 2024-09-15T03:56:07.130 INFO:tasks.rados.rados.0.smithi122.stdout:2313: done (1 left) 2024-09-15T03:56:07.130 INFO:tasks.rados.rados.0.smithi122.stdout:2314: write (excl) oid 12 current snap is 269 2024-09-15T03:56:07.130 INFO:tasks.rados.rados.0.smithi122.stdout:2311: finishing write tid 4 to smithi12223906-18 2024-09-15T03:56:07.130 INFO:tasks.rados.rados.0.smithi122.stdout:2311: finishing write tid 5 to smithi12223906-18 2024-09-15T03:56:07.130 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1501 (ObjNum 824 snap 269 seq_num 824) dirty exists 2024-09-15T03:56:07.130 INFO:tasks.rados.rados.0.smithi122.stdout:2311: left oid 18 (ObjNum 824 snap 269 seq_num 824) 2024-09-15T03:56:07.131 INFO:tasks.rados.rados.0.smithi122.stdout:2314: seq_num 826 ranges {682582=721798,2168950=642942,3583392=299190} 2024-09-15T03:56:07.139 INFO:tasks.rados.rados.0.smithi122.stdout:2314: writing smithi12223906-12 from 682582 to 1404380 tid 1 2024-09-15T03:56:07.148 INFO:tasks.rados.rados.0.smithi122.stdout:2314: writing smithi12223906-12 from 2168950 to 2811892 tid 2 2024-09-15T03:56:07.154 INFO:tasks.rados.rados.0.smithi122.stdout:2314: writing smithi12223906-12 from 3583392 to 3882582 tid 3 2024-09-15T03:56:07.154 INFO:tasks.rados.rados.0.smithi122.stdout:2311: done (1 left) 2024-09-15T03:56:07.155 INFO:tasks.rados.rados.0.smithi122.stdout:2315: snap_remove snap 258 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2315: done (1 left) 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2316: delete oid 47 current snap is 269 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2314: finishing write tid 1 to smithi12223906-12 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2314: finishing write tid 2 to smithi12223906-12 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2314: finishing write tid 3 to smithi12223906-12 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2314: finishing write tid 4 to smithi12223906-12 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2314: finishing write tid 5 to smithi12223906-12 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 748 (ObjNum 826 snap 269 seq_num 826) dirty exists 2024-09-15T03:56:08.132 INFO:tasks.rados.rados.0.smithi122.stdout:2314: left oid 12 (ObjNum 826 snap 269 seq_num 826) 2024-09-15T03:56:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:2314: done (1 left) 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2316: done (0 left) 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2317: copy_from oid 17 from oid 10 current snap is 269 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2318: read oid 16 snap -1 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2318: expect deleted 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2319: rollback oid 38 current snap is 269 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 266 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2320: read oid 20 snap -1 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2320: expect (ObjNum 798 snap 263 seq_num 798) 2024-09-15T03:56:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2321: snap_create 2024-09-15T03:56:08.145 INFO:tasks.rados.rados.0.smithi122.stdout:2320: expect (ObjNum 798 snap 263 seq_num 798) 2024-09-15T03:56:08.258 INFO:tasks.rados.rados.0.smithi122.stdout:2319: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:56:08.258 INFO:tasks.rados.rados.0.smithi122.stdout:2319: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:56:08.258 INFO:tasks.rados.rados.0.smithi122.stdout:2317: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:56:08.259 INFO:tasks.rados.rados.0.smithi122.stdout:2319: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:56:08.259 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1286 (ObjNum 726 snap 221 seq_num 726) dirty exists 2024-09-15T03:56:08.259 INFO:tasks.rados.rados.0.smithi122.stdout:2317: finishing copy_from to smithi12223906-17 2024-09-15T03:56:08.259 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1036 (ObjNum 815 snap 267 seq_num 815) dirty exists 2024-09-15T03:56:09.147 INFO:tasks.rados.rados.0.smithi122.stdout:2317: done (4 left) 2024-09-15T03:56:09.148 INFO:tasks.rados.rados.0.smithi122.stdout:2318: done (3 left) 2024-09-15T03:56:09.148 INFO:tasks.rados.rados.0.smithi122.stdout:2319: done (2 left) 2024-09-15T03:56:09.148 INFO:tasks.rados.rados.0.smithi122.stdout:2320: done (1 left) 2024-09-15T03:56:09.148 INFO:tasks.rados.rados.0.smithi122.stdout:2321: done (0 left) 2024-09-15T03:56:09.148 INFO:tasks.rados.rados.0.smithi122.stdout:2322: snap_remove snap 265 2024-09-15T03:56:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:2322: done (0 left) 2024-09-15T03:56:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:2323: copy_from oid 44 from oid 41 current snap is 270 2024-09-15T03:56:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:2324: rollback oid 42 current snap is 270 2024-09-15T03:56:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 230 2024-09-15T03:56:10.051 INFO:tasks.rados.rados.0.smithi122.stdout:2325: rollback oid 14 current snap is 270 2024-09-15T03:56:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 262 2024-09-15T03:56:10.052 INFO:tasks.rados.rados.0.smithi122.stdout:2326: delete oid 17 current snap is 270 2024-09-15T03:56:10.056 INFO:tasks.rados.rados.0.smithi122.stdout:2323: finishing copy_from to smithi12223906-44 2024-09-15T03:56:10.056 INFO:tasks.rados.rados.0.smithi122.stdout:2323: got expected ENOENT (src dne) 2024-09-15T03:56:10.057 INFO:tasks.rados.rados.0.smithi122.stdout:2323: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:56:10.063 INFO:tasks.rados.rados.0.smithi122.stdout:2323: done (3 left) 2024-09-15T03:56:10.063 INFO:tasks.rados.rados.0.smithi122.stdout:2326: done (2 left) 2024-09-15T03:56:10.063 INFO:tasks.rados.rados.0.smithi122.stdout:2327: read oid 44 snap 247 2024-09-15T03:56:10.063 INFO:tasks.rados.rados.0.smithi122.stdout:2327: expect (ObjNum 766 snap 240 seq_num 766) 2024-09-15T03:56:10.063 INFO:tasks.rados.rados.0.smithi122.stdout:2328: snap_create 2024-09-15T03:56:10.070 INFO:tasks.rados.rados.0.smithi122.stdout:2327: expect (ObjNum 766 snap 240 seq_num 766) 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:2324: finishing rollback tid 0 to smithi12223906-42 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:2324: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:2324: finishing rollback tid 2 to smithi12223906-42 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1508 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:2325: finishing rollback tid 0 to smithi12223906-14 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:2325: finishing rollback tid 1 to smithi12223906-14 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:2325: finishing rollback tid 2 to smithi12223906-14 2024-09-15T03:56:10.162 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1512 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:56:10.967 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T03:56:10.967 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T03:56:10.967 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T03:56:10.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T03:56:10.971 INFO:tasks.ceph.mon.b:Started 2024-09-15T03:56:10.971 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T03:56:10.971 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T03:56:10.971 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T03:56:10.971 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T03:56:10.973 INFO:tasks.ceph.mon.f:Started 2024-09-15T03:56:10.974 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-15T03:56:10.974 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-15T03:56:10.974 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T03:56:10.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T03:56:10.977 INFO:tasks.ceph.mon.c:Started 2024-09-15T03:56:10.977 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:56:10.977 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T03:56:10.977 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T03:56:10.978 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T03:56:10.980 INFO:tasks.ceph.mon.g:Started 2024-09-15T03:56:10.981 INFO:tasks.mon_thrash:Sending STOP to mon h 2024-09-15T03:56:10.981 INFO:tasks.ceph.mon.h:Sent signal 19 2024-09-15T03:56:10.982 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T03:56:25.983 INFO:tasks.mon_thrash:Sending CONT to mon h 2024-09-15T03:56:25.984 INFO:tasks.ceph.mon.h:Sent signal 18 2024-09-15T03:56:25.984 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:56:25.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:56:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: get_monmap_and_config 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: monmap: 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:26.088189+0000 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:26.088189+0000 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.092 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 Processor -- start 2024-09-15T03:56:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- start start 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: init 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: _add_conns ranks=[7,1,5,6,2,8,3,0,4] 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient(hunting): picked mon.noname-i con 0x7f4da0099570 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient(hunting): picked mon.noname-f con 0x7f4da0068a90 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient(hunting): picked mon.noname-h con 0x7f4da013d500 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da004ca10 con 0x7f4da0068a90 2024-09-15T03:56:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00988a0 con 0x7f4da013d500 2024-09-15T03:56:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da0068e90 con 0x7f4da0099570 2024-09-15T03:56:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient(hunting): authenticate will time out at 1530.517731s 2024-09-15T03:56:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9e59c700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f4da0099570 0x7f4da0099940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:32848/0 (socket says 172.21.15.26:32848) 2024-09-15T03:56:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9e59c700 1 -- 172.21.15.26:0/4253647569 learned_addr learned my addr 172.21.15.26:0/4253647569 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d97fff700 1 --1- 172.21.15.26:0/4253647569 >> v1:172.21.15.122:6789/0 conn(0x7f4da0068a90 0x7f4da0098c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:52150/0 (socket says 172.21.15.26:52150) 2024-09-15T03:56:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148645241 0 0) 0x7f4da0068e90 con 0x7f4da0099570 2024-09-15T03:56:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient(hunting): my global_id is 134971 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8c003570 con 0x7f4da0099570 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3202751050 0 0) 0x7f4d8c003570 con 0x7f4da0099570 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 >> v1:172.21.15.122:6791/0 conn(0x7f4da013d500 legacy=0x7f4da013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 >> v1:172.21.15.122:6789/0 conn(0x7f4da0068a90 legacy=0x7f4da0098c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 monclient: found mon.noname-i 2024-09-15T03:56:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00a27a0 con 0x7f4da0099570 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 5 monclient: authenticate success, global_id 134971 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d900017b0 con 0x7f4da0099570 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: dump: 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d90001a60 con 0x7f4da0099570 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 1 -- 172.21.15.26:0/4253647569 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d90004c30 con 0x7f4da0099570 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 4 set_mon_vals no callback set 2024-09-15T03:56:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4d9dd9b700 10 monclient: dump: 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.108 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: shutdown 2024-09-15T03:56:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- 172.21.15.26:0/4253647569 >> v1:172.21.15.122:6792/0 conn(0x7f4da0099570 legacy=0x7f4da0099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- 172.21.15.26:0/4253647569 shutdown_connections 2024-09-15T03:56:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- 172.21.15.26:0/4253647569 >> 172.21.15.26:0/4253647569 conn(0x7f4da00a6c30 msgr2=0x7f4da006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- 172.21.15.26:0/4253647569 shutdown_connections 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 1 -- 172.21.15.26:0/4253647569 wait complete. 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.087+0000 7f4da4a03700 10 monclient: monmap: 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 1 Processor -- start 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 1 -- start start 2024-09-15T03:56:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient: init 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient: _add_conns ranks=[7,1,0,8,4,5,3,6,2] 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient(hunting): picked mon.noname-i con 0x7f4da0068a90 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient(hunting): picked mon.noname-f con 0x7f4da0099570 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient(hunting): picked mon.noname-a con 0x7f4da013d500 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da0127200 con 0x7f4da013d500 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da013c580 con 0x7f4da0099570 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da0126f20 con 0x7f4da0068a90 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4da4a03700 10 monclient(hunting): authenticate will time out at 1530.521379s 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4d9e59c700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f4da0068a90 0x7f4da006c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:32850/0 (socket says 172.21.15.26:32850) 2024-09-15T03:56:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.091+0000 7f4d9e59c700 1 -- 172.21.15.26:0/1693676709 learned_addr learned my addr 172.21.15.26:0/1693676709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109324334 0 0) 0x7f4da013c580 con 0x7f4da0099570 2024-09-15T03:56:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): my global_id is 204103 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d80003570 con 0x7f4da0099570 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801591144 0 0) 0x7f4da0127200 con 0x7f4da013d500 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): my global_id is 194310 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da013c580 con 0x7f4da013d500 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343707964 0 0) 0x7f4da0126f20 con 0x7f4da0068a90 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): my global_id is 134980 2024-09-15T03:56:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da0127200 con 0x7f4da0068a90 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 446418754 0 0) 0x7f4da013c580 con 0x7f4da013d500 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4da0126f20 con 0x7f4da013d500 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d98001b50 con 0x7f4da013d500 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): dump: 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 685060203 0 0) 0x7f4da0127200 con 0x7f4da0068a90 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4da013c580 con 0x7f4da0068a90 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 490114308 0 0) 0x7f4d80003570 con 0x7f4da0099570 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4da0127200 con 0x7f4da0099570 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2983395741 0 0) 0x7f4da0126f20 con 0x7f4da013d500 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 >> v1:172.21.15.122:6792/0 conn(0x7f4da0068a90 legacy=0x7f4da006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 >> v1:172.21.15.122:6789/0 conn(0x7f4da0099570 legacy=0x7f4da006c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 monclient: found mon.a 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da01274e0 con 0x7f4da013d500 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4d98002210 con 0x7f4da013d500 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4d980026c0 con 0x7f4da013d500 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d967fc700 10 monclient: dump: 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.099+0000 7f4d9d59a700 4 set_mon_vals no callback set 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 5 monclient: authenticate success, global_id 194310 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 10 monclient: _renew_subs 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da0127be0 con 0x7f4da013d500 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 10 monclient: _renew_subs 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da0125fc0 con 0x7f4da013d500 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4d980028f0 con 0x7f4da013d500 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(533..533 src has 224..533) v4 ==== 4015+0+0 (unknown 1683957262 0 0) 0x7f4d98041600 con 0x7f4da013d500 2024-09-15T03:56:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.103+0000 7f4d9e59c700 10 monclient: get_auth_request con 0x7f4d80037cd0 auth_method 0 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.107+0000 7f4da4a03700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.107+0000 7f4da4a03700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.107+0000 7f4da4a03700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.107+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da0126210 con 0x7f4da013d500 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.111+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f4d98012420 con 0x7f4da013d500 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.111+0000 7f4d967fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.111+0000 7f4d967fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4da4a03700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:56:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4da4a03700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4da4a03700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4da004bbf0 con 0x7f4da013d500 2024-09-15T03:56:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4d967fc700 1 -- 172.21.15.26:0/1693676709 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 108251505) 0x7f4d98011e20 con 0x7f4da013d500 2024-09-15T03:56:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4d967fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.331+0000 7f4d967fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:26.337 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"i","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:56:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 >> v1:172.21.15.122:6812/20895 conn(0x7f4d80037cd0 legacy=0x7f4d8003a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 10 monclient: shutdown 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 >> v1:172.21.15.26:6789/0 conn(0x7f4da013d500 legacy=0x7f4da006ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 shutdown_connections 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 >> 172.21.15.26:0/1693676709 conn(0x7f4da00a6c30 msgr2=0x7f4da00a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 shutdown_connections 2024-09-15T03:56:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.335+0000 7f4da4a03700 1 -- 172.21.15.26:0/1693676709 wait complete. 2024-09-15T03:56:26.350 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:56:26.351 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:56:26.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:56:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.451+0000 7fa7b0408700 10 monclient: get_monmap_and_config 2024-09-15T03:56:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.451+0000 7fa7b0408700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.451+0000 7fa7b0408700 10 monclient: monmap: 2024-09-15T03:56:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:26.454746+0000 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:26.454746+0000 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.459 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.451+0000 7fa7b0408700 1 Processor -- start 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.451+0000 7fa7b0408700 1 -- start start 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.451+0000 7fa7b0408700 10 monclient: init 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient: _add_conns ranks=[8,5,4,1,2,0,6,7,3] 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient(hunting): picked mon.noname-e con 0x7fa7a809bb40 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient(hunting): picked mon.noname-h con 0x7fa7a809ced0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient(hunting): picked mon.noname-c con 0x7fa7a8098a70 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a804ca10 con 0x7fa7a8098a70 2024-09-15T03:56:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a809cc70 con 0x7fa7a809ced0 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a809a760 con 0x7fa7a809bb40 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient(hunting): authenticate will time out at 1530.884228s 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ae9a5700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fa7a8098a70 0x7fa7a813f2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55322/0 (socket says 172.21.15.26:55322) 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ae1a4700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fa7a809bb40 0x7fa7a809bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:46764/0 (socket says 172.21.15.26:46764) 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ae9a5700 1 -- 172.21.15.26:0/2485423251 learned_addr learned my addr 172.21.15.26:0/2485423251 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286621531 0 0) 0x7fa7a809cc70 con 0x7fa7a809ced0 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): my global_id is 5549 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa794003570 con 0x7fa7a809ced0 2024-09-15T03:56:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179699999 0 0) 0x7fa7a804ca10 con 0x7fa7a8098a70 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): my global_id is 204106 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a809cc70 con 0x7fa7a8098a70 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18250685 0 0) 0x7fa7a809a760 con 0x7fa7a809bb40 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient(hunting): my global_id is 135008 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a804ca10 con 0x7fa7a809bb40 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965395526 0 0) 0x7fa7a809cc70 con 0x7fa7a8098a70 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 >> v1:172.21.15.26:6793/0 conn(0x7fa7a809bb40 legacy=0x7fa7a809bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 >> v1:172.21.15.122:6791/0 conn(0x7fa7a809ced0 legacy=0x7fa7a809a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 monclient: found mon.noname-c 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a809e290 con 0x7fa7a8098a70 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 5 monclient: authenticate success, global_id 204106 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa7a0001b10 con 0x7fa7a8098a70 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: dump: 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa7a0001db0 con 0x7fa7a8098a70 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 1 -- 172.21.15.26:0/2485423251 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa7a0004fb0 con 0x7fa7a8098a70 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 4 set_mon_vals no callback set 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7ad1a2700 10 monclient: dump: 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient: shutdown 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- 172.21.15.26:0/2485423251 >> v1:172.21.15.26:6791/0 conn(0x7fa7a8098a70 legacy=0x7fa7a813f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- 172.21.15.26:0/2485423251 shutdown_connections 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- 172.21.15.26:0/2485423251 >> 172.21.15.26:0/2485423251 conn(0x7fa7a8063f30 msgr2=0x7fa7a80a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- 172.21.15.26:0/2485423251 shutdown_connections 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 1 -- 172.21.15.26:0/2485423251 wait complete. 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.455+0000 7fa7b0408700 10 monclient: monmap: 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 Processor -- start 2024-09-15T03:56:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 -- start start 2024-09-15T03:56:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: init 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: _add_conns ranks=[8,3,7,4,5,1,0,2,6] 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient(hunting): picked mon.noname-e con 0x7fa7a8098a70 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient(hunting): picked mon.noname-g con 0x7fa7a809bb40 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient(hunting): picked mon.noname-i con 0x7fa7a809ced0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a809d710 con 0x7fa7a809bb40 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a813fae0 con 0x7fa7a809ced0 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a813f800 con 0x7fa7a8098a70 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient(hunting): authenticate will time out at 1530.889065s 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7ae1a4700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fa7a8098a70 0x7fa7a813bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:46780/0 (socket says 172.21.15.26:46780) 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7ae1a4700 1 -- 172.21.15.26:0/3928284478 learned_addr learned my addr 172.21.15.26:0/3928284478 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283588175 0 0) 0x7fa7a813f800 con 0x7fa7a8098a70 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): my global_id is 135017 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa78c003570 con 0x7fa7a8098a70 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818331216 0 0) 0x7fa7a813fae0 con 0x7fa7a809ced0 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): my global_id is 134989 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a813f800 con 0x7fa7a809ced0 2024-09-15T03:56:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431875133 0 0) 0x7fa7a809d710 con 0x7fa7a809bb40 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): my global_id is 204105 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a813fae0 con 0x7fa7a809bb40 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2484953718 0 0) 0x7fa78c003570 con 0x7fa7a8098a70 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa7a809d710 con 0x7fa7a8098a70 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa798001680 con 0x7fa7a8098a70 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): dump: 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.478 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2251487197 0 0) 0x7fa7a809d710 con 0x7fa7a8098a70 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.122:6792/0 conn(0x7fa7a809ced0 legacy=0x7fa7a814e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.122:6790/0 conn(0x7fa7a809bb40 legacy=0x7fa7a813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 monclient: found mon.e 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a813cd80 con 0x7fa7a8098a70 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 5 monclient: authenticate success, global_id 135017 2024-09-15T03:56:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: _renew_subs 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a806e410 con 0x7fa7a8098a70 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: _renew_subs 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a8139760 con 0x7fa7a8098a70 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa798001cd0 con 0x7fa7a8098a70 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa798004e20 con 0x7fa7a8098a70 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79ffff700 4 set_mon_vals no callback set 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa79e7fc700 10 monclient: dump: 2024-09-15T03:56:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.459+0000 7fa7b0408700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa790006590 con 0x7fa790002840 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa79800f860 con 0x7fa7a8098a70 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895335497 0 0) 0x7fa790006590 con 0x7fa790002840 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient(hunting): my global_id is 194319 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a809d710 con 0x7fa790002840 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(533..533 src has 224..533) v4 ==== 4015+0+0 (unknown 1683957262 0 0) 0x7fa798043ef0 con 0x7fa7a8098a70 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa7ae9a5700 10 monclient: get_auth_request con 0x7fa78c0377e0 auth_method 0 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2579677558 0 0) 0x7fa7a809d710 con 0x7fa790002840 2024-09-15T03:56:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa790006590 con 0x7fa790002840 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa7a40017c0 con 0x7fa790002840 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient: dump: 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4135614385 0 0) 0x7fa790006590 con 0x7fa790002840 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa79000a130 con 0x7fa790002840 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa7a4003b50 con 0x7fa790002840 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.26:6789/0 conn(0x7fa790002840 legacy=0x7fa790004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.463+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.26:6789/0 conn(0x7fa790002840 legacy=0x7fa790004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa7b0408700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa790009640 con 0x7fa7900090b0 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725257315 0 0) 0x7fa790009640 con 0x7fa7900090b0 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient(hunting): my global_id is 194328 2024-09-15T03:56:26.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa790006590 con 0x7fa7900090b0 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3381985156 0 0) 0x7fa790006590 con 0x7fa7900090b0 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa790009640 con 0x7fa7900090b0 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa7a40022f0 con 0x7fa7900090b0 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient: dump: 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 871617150 0 0) 0x7fa790009640 con 0x7fa7900090b0 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa7900098f0 con 0x7fa7900090b0 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 755633279) 0x7fa7a4003900 con 0x7fa7900090b0 2024-09-15T03:56:26.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.26:6789/0 conn(0x7fa7900090b0 legacy=0x7fa790008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.475+0000 7fa79e7fc700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.26:6789/0 conn(0x7fa7900090b0 legacy=0x7fa790008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:26.493 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:26.494 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.499 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:26.501 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.502 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.504 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.505 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.506 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.507 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:26.508 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:26.509 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.122:6812/20895 conn(0x7fa78c0377e0 legacy=0x7fa78c039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 10 monclient: shutdown 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 >> v1:172.21.15.26:6793/0 conn(0x7fa7a8098a70 legacy=0x7fa7a813bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7ae9a5700 1 -- 172.21.15.26:0/3928284478 reap_dead start 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7ae9a5700 1 -- 172.21.15.26:0/3928284478 reap_dead start 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 shutdown_connections 2024-09-15T03:56:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 >> 172.21.15.26:0/3928284478 conn(0x7fa7a8063f30 msgr2=0x7fa7a80a0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 shutdown_connections 2024-09-15T03:56:26.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.479+0000 7fa7b0408700 1 -- 172.21.15.26:0/3928284478 wait complete. 2024-09-15T03:56:26.512 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:26.512 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:56:26.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:56:26.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.619+0000 7fe33307f700 10 monclient: get_monmap_and_config 2024-09-15T03:56:26.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.619+0000 7fe33307f700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.619+0000 7fe33307f700 10 monclient: monmap: 2024-09-15T03:56:26.627 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.627 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.627 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:26.623707+0000 2024-09-15T03:56:26.628 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:26.623707+0000 2024-09-15T03:56:26.628 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.628 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.628 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.628 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.628 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.629 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.629 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.629 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.629 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.629 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.629 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.630 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 1 Processor -- start 2024-09-15T03:56:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 1 -- start start 2024-09-15T03:56:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient: init 2024-09-15T03:56:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient: _add_conns ranks=[7,0,2,4,8,1,5,3,6] 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient(hunting): picked mon.noname-i con 0x7fe32c0992d0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient(hunting): picked mon.noname-a con 0x7fe32c0996a0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient(hunting): picked mon.noname-b con 0x7fe32c099a70 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c04ca10 con 0x7fe32c0996a0 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0a2760 con 0x7fe32c099a70 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c133950 con 0x7fe32c0992d0 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient(hunting): authenticate will time out at 1531.053540s 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33161c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe32c099a70 0x7fe32c13b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37724/0 (socket says 172.21.15.26:37724) 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33161c700 1 -- 172.21.15.26:0/2555355425 learned_addr learned my addr 172.21.15.26:0/2555355425 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495706106 0 0) 0x7fe32c0a2760 con 0x7fe32c099a70 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): my global_id is 204104 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe314003570 con 0x7fe32c099a70 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781208712 0 0) 0x7fe32c04ca10 con 0x7fe32c0996a0 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): my global_id is 194337 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c0a2760 con 0x7fe32c0996a0 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330983430 0 0) 0x7fe32c133950 con 0x7fe32c0992d0 2024-09-15T03:56:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient(hunting): my global_id is 134998 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c04ca10 con 0x7fe32c0992d0 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 611809619 0 0) 0x7fe314003570 con 0x7fe32c099a70 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 >> v1:172.21.15.122:6792/0 conn(0x7fe32c0992d0 legacy=0x7fe32c133240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 >> v1:172.21.15.26:6789/0 conn(0x7fe32c0996a0 legacy=0x7fe32c137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 monclient: found mon.noname-b 2024-09-15T03:56:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c099ef0 con 0x7fe32c099a70 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe324001b50 con 0x7fe32c099a70 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: dump: 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 5 monclient: authenticate success, global_id 204104 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe324001e30 con 0x7fe32c099a70 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe33307f700 4 set_mon_vals no callback set 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 1 -- 172.21.15.26:0/2555355425 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe324002020 con 0x7fe32c099a70 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.623+0000 7fe32b7fe700 10 monclient: dump: 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.649 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.650 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient: shutdown 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- 172.21.15.26:0/2555355425 >> v1:172.21.15.26:6790/0 conn(0x7fe32c099a70 legacy=0x7fe32c13b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- 172.21.15.26:0/2555355425 shutdown_connections 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- 172.21.15.26:0/2555355425 >> 172.21.15.26:0/2555355425 conn(0x7fe32c06e470 msgr2=0x7fe32c0a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- 172.21.15.26:0/2555355425 shutdown_connections 2024-09-15T03:56:26.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- 172.21.15.26:0/2555355425 wait complete. 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient: monmap: 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.652 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 Processor -- start 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- start start 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient: init 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient: _add_conns ranks=[4,2,5,8,7,1,3,0,6] 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient(hunting): picked mon.noname-c con 0x7fe32c0992d0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient(hunting): picked mon.noname-b con 0x7fe32c0996a0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient(hunting): picked mon.noname-h con 0x7fe32c099a70 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c11f010 con 0x7fe32c0996a0 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c11ccf0 con 0x7fe32c0992d0 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c11ed30 con 0x7fe32c099a70 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.627+0000 7fe33307f700 10 monclient(hunting): authenticate will time out at 1531.059935s 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe32bfff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe32c0996a0 0x7fe32c146db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37734/0 (socket says 172.21.15.26:37734) 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe330e1b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fe32c0992d0 0x7fe32c131c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43864/0 (socket says 172.21.15.26:43864) 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe32bfff700 1 -- 172.21.15.26:0/2020167526 learned_addr learned my addr 172.21.15.26:0/2020167526 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350725246 0 0) 0x7fe32c11f010 con 0x7fe32c0996a0 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): my global_id is 204113 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe318003570 con 0x7fe32c0996a0 2024-09-15T03:56:26.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913515980 0 0) 0x7fe32c11ccf0 con 0x7fe32c0992d0 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): my global_id is 204115 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c11f010 con 0x7fe32c0992d0 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043943995 0 0) 0x7fe32c11ed30 con 0x7fe32c099a70 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): my global_id is 5558 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c11ccf0 con 0x7fe32c099a70 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4243695618 0 0) 0x7fe318003570 con 0x7fe32c0996a0 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe32c11ed30 con 0x7fe32c0996a0 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1509432763 0 0) 0x7fe32c11f010 con 0x7fe32c0992d0 2024-09-15T03:56:26.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe318003570 con 0x7fe32c0992d0 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe320001d10 con 0x7fe32c0996a0 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.656 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.657 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.657 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.657 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.657 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.657 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.658 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe31c0019e0 con 0x7fe32c0992d0 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:26.659 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.660 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.660 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.660 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.660 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.660 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.660 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.662 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3156492171 0 0) 0x7fe32c11ccf0 con 0x7fe32c099a70 2024-09-15T03:56:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe32c11f010 con 0x7fe32c099a70 2024-09-15T03:56:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe324001c00 con 0x7fe32c099a70 2024-09-15T03:56:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.664 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.664 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.664 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 729477033 0 0) 0x7fe318003570 con 0x7fe32c0992d0 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.122:6791/0 conn(0x7fe32c099a70 legacy=0x7fe32c14a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.26:6790/0 conn(0x7fe32c0996a0 legacy=0x7fe32c146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 monclient: found mon.c 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:26.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c11f2f0 con 0x7fe32c0992d0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 5 monclient: authenticate success, global_id 204115 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 10 monclient: _renew_subs 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c132320 con 0x7fe32c0992d0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 10 monclient: _renew_subs 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c0a1b90 con 0x7fe32c0992d0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe31c002d30 con 0x7fe32c0992d0 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe31c002270 con 0x7fe32c0992d0 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: dump: 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe32affd700 4 set_mon_vals no callback set 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe31c0023d0 con 0x7fe32c0992d0 2024-09-15T03:56:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe30c006590 con 0x7fe30c002840 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(533..533 src has 224..533) v4 ==== 4015+0+0 (unknown 1683957262 0 0) 0x7fe31c043d00 con 0x7fe32c0992d0 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe33161c700 10 monclient: get_auth_request con 0x7fe3180380e0 auth_method 0 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579117957 0 0) 0x7fe30c006590 con 0x7fe30c002840 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient(hunting): my global_id is 204122 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe318003570 con 0x7fe30c002840 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2607590202 0 0) 0x7fe318003570 con 0x7fe30c002840 2024-09-15T03:56:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe30c006590 con 0x7fe30c002840 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe320001740 con 0x7fe30c002840 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.631+0000 7fe328ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 10 monclient: dump: 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2316943734 0 0) 0x7fe30c006590 con 0x7fe30c002840 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe30c00a130 con 0x7fe30c002840 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe3200073c0 con 0x7fe30c002840 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.26:6790/0 conn(0x7fe30c002840 legacy=0x7fe30c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.635+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.26:6790/0 conn(0x7fe30c002840 legacy=0x7fe30c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe33307f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe30c007520 con 0x7fe30c0068d0 2024-09-15T03:56:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591412749 0 0) 0x7fe30c007520 con 0x7fe30c0068d0 2024-09-15T03:56:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 10 monclient(hunting): my global_id is 204131 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe30c006590 con 0x7fe30c0068d0 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4025062450 0 0) 0x7fe30c006590 con 0x7fe30c0068d0 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe30c007520 con 0x7fe30c0068d0 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe320005ae0 con 0x7fe30c0068d0 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 10 monclient: dump: 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.676 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2365744838 0 0) 0x7fe30c007520 con 0x7fe30c0068d0 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.643+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe30c008760 con 0x7fe30c0068d0 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 522011230) 0x7fe320007070 con 0x7fe30c0068d0 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe328ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe328ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.26:6790/0 conn(0x7fe30c0068d0 legacy=0x7fe30c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe328ff9700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.26:6790/0 conn(0x7fe30c0068d0 legacy=0x7fe30c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:26.678 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:26.679 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:26.679 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:26.679 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:26.679 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:26.679 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:26.679 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:26.680 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.681 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:26.682 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.683 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.684 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.684 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.684 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.684 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:26.684 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:26.685 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:26.685 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:26.685 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:26.685 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:26.685 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:26.685 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:26.686 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.687 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:26.688 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:26.689 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.690 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.691 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.692 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:26.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.694 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.695 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.697 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.700 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:26.700 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.700 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.700 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.700 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:26.701 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:26.701 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.701 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.701 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.701 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.702 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.702 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:26.702 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:26.702 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.703 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:26.703 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:26.703 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:26.703 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.703 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.705 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:26.705 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.122:6812/20895 conn(0x7fe3180380e0 legacy=0x7fe31803a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 10 monclient: shutdown 2024-09-15T03:56:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 >> v1:172.21.15.26:6791/0 conn(0x7fe32c0992d0 legacy=0x7fe32c131c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33161c700 1 -- 172.21.15.26:0/2020167526 reap_dead start 2024-09-15T03:56:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33161c700 1 -- 172.21.15.26:0/2020167526 reap_dead start 2024-09-15T03:56:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 shutdown_connections 2024-09-15T03:56:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 >> 172.21.15.26:0/2020167526 conn(0x7fe32c06e470 msgr2=0x7fe32c0988f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 shutdown_connections 2024-09-15T03:56:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.647+0000 7fe33307f700 1 -- 172.21.15.26:0/2020167526 wait complete. 2024-09-15T03:56:26.707 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:26.707 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:56:26.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:56:26.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.815+0000 7f4faffff700 10 monclient: get_monmap_and_config 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.815+0000 7f4faffff700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient: monmap: 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:26.819901+0000 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:26.819901+0000 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.824 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.824 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.824 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.824 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.824 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 1 Processor -- start 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 1 -- start start 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient: init 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient: _add_conns ranks=[4,6,7,0,5,2,1,3,8] 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient(hunting): picked mon.noname-c con 0x7f4fb00a2aa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient(hunting): picked mon.noname-d con 0x7f4fb006e160 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient(hunting): picked mon.noname-i con 0x7f4fb009b310 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb004ca10 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb0099950 con 0x7f4fb006e160 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb00995e0 con 0x7f4fb009b310 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient(hunting): authenticate will time out at 1531.249340s 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fae7fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f4fb006e160 0x7f4fb009cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48424/0 (socket says 172.21.15.26:48424) 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fae7fc700 1 -- 172.21.15.26:0/803029487 learned_addr learned my addr 172.21.15.26:0/803029487 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faeffd700 1 --1- 172.21.15.26:0/803029487 >> v1:172.21.15.26:6791/0 conn(0x7f4fb00a2aa0 0x7f4fb009c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43878/0 (socket says 172.21.15.26:43878) 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765635620 0 0) 0x7f4fb0099950 con 0x7f4fb006e160 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): my global_id is 194352 2024-09-15T03:56:26.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f9c003570 con 0x7f4fb006e160 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295795465 0 0) 0x7f4fb004ca10 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): my global_id is 204124 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb0099950 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110327746 0 0) 0x7f4fb00995e0 con 0x7f4fb009b310 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient(hunting): my global_id is 135007 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb004ca10 con 0x7f4fb009b310 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2783470532 0 0) 0x7f4f9c003570 con 0x7f4fb006e160 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 >> v1:172.21.15.122:6792/0 conn(0x7f4fb009b310 legacy=0x7f4fb0137640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 >> v1:172.21.15.26:6791/0 conn(0x7f4fb00a2aa0 legacy=0x7f4fb009c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 monclient: found mon.noname-d 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb009d630 con 0x7f4fb006e160 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4fa0001b30 con 0x7f4fb006e160 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 5 monclient: authenticate success, global_id 194352 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: dump: 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.832 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4fa0001e10 con 0x7f4fb006e160 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 1 -- 172.21.15.26:0/803029487 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4fa0004fb0 con 0x7f4fb006e160 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 4 set_mon_vals no callback set 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4fadffb700 10 monclient: dump: 2024-09-15T03:56:26.833 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.834 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 10 monclient: shutdown 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.819+0000 7f4faffff700 1 -- 172.21.15.26:0/803029487 >> v1:172.21.15.26:6792/0 conn(0x7f4fb006e160 legacy=0x7f4fb009cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- 172.21.15.26:0/803029487 shutdown_connections 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- 172.21.15.26:0/803029487 >> 172.21.15.26:0/803029487 conn(0x7f4fb0063cd0 msgr2=0x7f4fb00a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- 172.21.15.26:0/803029487 shutdown_connections 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- 172.21.15.26:0/803029487 wait complete. 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient: monmap: 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 Processor -- start 2024-09-15T03:56:26.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- start start 2024-09-15T03:56:26.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient: init 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient: _add_conns ranks=[1,7,6,0,8,4,2,5,3] 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient(hunting): picked mon.noname-f con 0x7f4fb006e160 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient(hunting): picked mon.noname-i con 0x7f4fb009b310 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient(hunting): picked mon.noname-d con 0x7f4fb0147e30 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb012a770 con 0x7f4fb006e160 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb012ba70 con 0x7f4fb0147e30 2024-09-15T03:56:26.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb009be80 con 0x7f4fb009b310 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faffff700 10 monclient(hunting): authenticate will time out at 1531.254721s 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faf7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f4fb0147e30 0x7f4fb014a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48438/0 (socket says 172.21.15.26:48438) 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4faf7fe700 1 -- 172.21.15.26:0/3127379035 learned_addr learned my addr 172.21.15.26:0/3127379035 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fae7fc700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f4fb009b310 0x7f4fb006d510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:34486/0 (socket says 172.21.15.26:34486) 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981371749 0 0) 0x7f4fb012ba70 con 0x7f4fb0147e30 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): my global_id is 194361 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f90003570 con 0x7f4fb0147e30 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880604127 0 0) 0x7f4fb009be80 con 0x7f4fb009b310 2024-09-15T03:56:26.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): my global_id is 135016 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb012ba70 con 0x7f4fb009b310 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892268888 0 0) 0x7f4fb012a770 con 0x7f4fb006e160 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): my global_id is 204112 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb009be80 con 0x7f4fb006e160 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4063251966 0 0) 0x7f4f90003570 con 0x7f4fb0147e30 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4fb012a770 con 0x7f4fb0147e30 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4fa8002af0 con 0x7f4fb0147e30 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.823+0000 7f4fa6ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.844 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4137055372 0 0) 0x7f4fb012a770 con 0x7f4fb0147e30 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.122:6792/0 conn(0x7f4fb009b310 legacy=0x7f4fb006d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.122:6789/0 conn(0x7f4fb006e160 legacy=0x7f4fb006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 monclient: found mon.d 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:26.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb009b9b0 con 0x7f4fb0147e30 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 5 monclient: authenticate success, global_id 194361 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 10 monclient: _renew_subs 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb006dc20 con 0x7f4fb0147e30 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 10 monclient: _renew_subs 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb0129fb0 con 0x7f4fb0147e30 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4fa80015a0 con 0x7f4fb0147e30 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4fa8001ad0 con 0x7f4fb0147e30 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fad7fa700 4 set_mon_vals no callback set 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: dump: 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.847 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4fb004bbf0 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4fa8001cb0 con 0x7f4fb0147e30 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(533..533 src has 224..533) v4 ==== 4015+0+0 (unknown 1683957262 0 0) 0x7f4fa8043fb0 con 0x7f4fb0147e30 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294894744 0 0) 0x7f4fb004bbf0 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient(hunting): my global_id is 204133 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb012a770 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fae7fc700 10 monclient: get_auth_request con 0x7f4f900376c0 auth_method 0 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3592155490 0 0) 0x7f4fb012a770 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4fb004bbf0 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f980019a0 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: dump: 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.850 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 191316809 0 0) 0x7f4fb004bbf0 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fb0066de0 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f4f98004830 con 0x7f4fb00a2aa0 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.26:6791/0 conn(0x7f4fb00a2aa0 legacy=0x7f4fb0068e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.827+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.26:6791/0 conn(0x7f4fb00a2aa0 legacy=0x7f4fb0068e20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4faffff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4fb0071610 con 0x7f4fb002ad10 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786633079 0 0) 0x7f4fb0071610 con 0x7f4fb002ad10 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient(hunting): my global_id is 204142 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb004bbf0 con 0x7f4fb002ad10 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3940093734 0 0) 0x7f4fb004bbf0 con 0x7f4fb002ad10 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4fb0071610 con 0x7f4fb002ad10 2024-09-15T03:56:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f98004ff0 con 0x7f4fb002ad10 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient: dump: 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1032543352 0 0) 0x7f4fb0071610 con 0x7f4fb002ad10 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4fb012ab80 con 0x7f4fb002ad10 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1641006917) 0x7f4f98007270 con 0x7f4fb002ad10 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.26:6791/0 conn(0x7f4fb002ad10 legacy=0x7f4fb0069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.843+0000 7f4fa6ffd700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.26:6791/0 conn(0x7f4fb002ad10 legacy=0x7f4fb0069530 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:26.856 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.857 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.858 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:26.859 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:26.860 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.861 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:26.862 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:26.863 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:26.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:26.865 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:26.866 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:26.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:26.868 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.870 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.873 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.874 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:26.875 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:26.876 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:26.877 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:26.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.122:6812/20895 conn(0x7f4f900376c0 legacy=0x7f4f90039b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 10 monclient: shutdown 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faf7fe700 1 -- 172.21.15.26:0/3127379035 reap_dead start 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 >> v1:172.21.15.26:6792/0 conn(0x7f4fb0147e30 legacy=0x7f4fb014a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 shutdown_connections 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 >> 172.21.15.26:0/3127379035 conn(0x7f4fb0063cd0 msgr2=0x7f4fb01286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 shutdown_connections 2024-09-15T03:56:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.847+0000 7f4faffff700 1 -- 172.21.15.26:0/3127379035 wait complete. 2024-09-15T03:56:26.879 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:26.880 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:56:26.880 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:56:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient: get_monmap_and_config 2024-09-15T03:56:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient: build_initial_monmap 2024-09-15T03:56:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient: monmap: 2024-09-15T03:56:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:26.993 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:26.990012+0000 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:26.990012+0000 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:26.994 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:26.995 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:26.995 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:26.995 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:26.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 1 Processor -- start 2024-09-15T03:56:26.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 1 -- start start 2024-09-15T03:56:26.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient: init 2024-09-15T03:56:26.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:26.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient: _add_conns ranks=[2,0,5,4,1,7,6,3,8] 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient(hunting): picked mon.noname-b con 0x7f304409e440 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient(hunting): picked mon.noname-a con 0x7f304412ffc0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient(hunting): picked mon.noname-h con 0x7f3044130390 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304404ca10 con 0x7f304412ffc0 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304409a0f0 con 0x7f304409e440 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304409e810 con 0x7f3044130390 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304bf7a700 10 monclient(hunting): authenticate will time out at 1531.419669s 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f3049515700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f304412ffc0 0x7f3044134be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52728/0 (socket says 172.21.15.26:52728) 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f3049d16700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f304409e440 0x7f304412f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37742/0 (socket says 172.21.15.26:37742) 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f3049515700 1 -- 172.21.15.26:0/508791456 learned_addr learned my addr 172.21.15.26:0/508791456 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.987+0000 7f304a517700 1 --1- 172.21.15.26:0/508791456 >> v1:172.21.15.122:6791/0 conn(0x7f3044130390 0x7f3044138310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:38132/0 (socket says 172.21.15.26:38132) 2024-09-15T03:56:26.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996087509 0 0) 0x7f304409e810 con 0x7f3044130390 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): my global_id is 5567 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3030003570 con 0x7f3044130390 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217332385 0 0) 0x7f304404ca10 con 0x7f304412ffc0 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): my global_id is 194346 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f304409e810 con 0x7f304412ffc0 2024-09-15T03:56:27.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713681420 0 0) 0x7f304409a0f0 con 0x7f304409e440 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient(hunting): my global_id is 204140 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f304404ca10 con 0x7f304409e440 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1707781636 0 0) 0x7f304409e810 con 0x7f304412ffc0 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 >> v1:172.21.15.122:6791/0 conn(0x7f3044130390 legacy=0x7f3044138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 >> v1:172.21.15.26:6790/0 conn(0x7f304409e440 legacy=0x7f304412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 monclient: found mon.noname-a 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304409c2a0 con 0x7f304412ffc0 2024-09-15T03:56:27.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3040001790 con 0x7f304412ffc0 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 5 monclient: authenticate success, global_id 194346 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: dump: 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.002 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.003 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3040001a30 con 0x7f304412ffc0 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 1 -- 172.21.15.26:0/508791456 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3040004c30 con 0x7f304412ffc0 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 4 set_mon_vals no callback set 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f3048d14700 10 monclient: dump: 2024-09-15T03:56:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.005 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 10 monclient: shutdown 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 -- 172.21.15.26:0/508791456 >> v1:172.21.15.26:6789/0 conn(0x7f304412ffc0 legacy=0x7f3044134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 -- 172.21.15.26:0/508791456 shutdown_connections 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 -- 172.21.15.26:0/508791456 >> 172.21.15.26:0/508791456 conn(0x7f3044063f30 msgr2=0x7f30440a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 -- 172.21.15.26:0/508791456 shutdown_connections 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 -- 172.21.15.26:0/508791456 wait complete. 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 10 monclient: monmap: 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.008 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 Processor -- start 2024-09-15T03:56:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 1 -- start start 2024-09-15T03:56:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.991+0000 7f304bf7a700 10 monclient: init 2024-09-15T03:56:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient: _add_conns ranks=[4,1,8,7,3,0,6,5,2] 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient(hunting): picked mon.noname-c con 0x7f304409e440 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient(hunting): picked mon.noname-f con 0x7f304412ffc0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient(hunting): picked mon.noname-e con 0x7f3044130390 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3044099720 con 0x7f304412ffc0 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30440994f0 con 0x7f304409e440 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304411cc50 con 0x7f3044130390 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient(hunting): authenticate will time out at 1531.424699s 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3049d16700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f304409e440 0x7f304412e840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43894/0 (socket says 172.21.15.26:43894) 2024-09-15T03:56:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3049d16700 1 -- 172.21.15.26:0/1327330160 learned_addr learned my addr 172.21.15.26:0/1327330160 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515576226 0 0) 0x7f30440994f0 con 0x7f304409e440 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): my global_id is 204151 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3028003570 con 0x7f304409e440 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540488669 0 0) 0x7f304411cc50 con 0x7f3044130390 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): my global_id is 135026 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30440994f0 con 0x7f3044130390 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017354917 0 0) 0x7f3044099720 con 0x7f304412ffc0 2024-09-15T03:56:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): my global_id is 204121 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f304411cc50 con 0x7f304412ffc0 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 713030642 0 0) 0x7f3028003570 con 0x7f304409e440 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3044099720 con 0x7f304409e440 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3034001510 con 0x7f304409e440 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): dump: 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2269100743 0 0) 0x7f30440994f0 con 0x7f3044130390 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3028003570 con 0x7f3044130390 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f303c001d80 con 0x7f3044130390 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient(hunting): dump: 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1610928200 0 0) 0x7f3044099720 con 0x7f304409e440 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.26:6793/0 conn(0x7f3044130390 legacy=0x7f3044148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.122:6789/0 conn(0x7f304412ffc0 legacy=0x7f3044144e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 monclient: found mon.c 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304409d070 con 0x7f304409e440 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 5 monclient: authenticate success, global_id 204151 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient: _renew_subs 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304411c970 con 0x7f304409e440 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient: _renew_subs 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f304412ef50 con 0x7f304409e440 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3034004ae0 con 0x7f304409e440 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3034004f90 con 0x7f304409e440 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f303bfff700 4 set_mon_vals no callback set 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 10 monclient: dump: 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3024006590 con 0x7f3024002840 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.995+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f303400f950 con 0x7f304409e440 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(533..533 src has 224..533) v4 ==== 4015+0+0 (unknown 1683957262 0 0) 0x7f3034043f00 con 0x7f304409e440 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306234902 0 0) 0x7f3024006590 con 0x7f3024002840 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient(hunting): my global_id is 194370 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3044099720 con 0x7f3024002840 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f304a517700 10 monclient: get_auth_request con 0x7f3028037cc0 auth_method 0 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2322501143 0 0) 0x7f3044099720 con 0x7f3024002840 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3024006590 con 0x7f3024002840 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3040001750 con 0x7f3024002840 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient: dump: 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1630559197 0 0) 0x7f3024006590 con 0x7f3024002840 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f302400a130 con 0x7f3024002840 2024-09-15T03:56:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3040004a90 con 0x7f3024002840 2024-09-15T03:56:27.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:27.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.26:6792/0 conn(0x7f3024002840 legacy=0x7f3024004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:26.999+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.26:6792/0 conn(0x7f3024002840 legacy=0x7f3024004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.026 INFO:tasks.rados.rados.0.smithi122.stdout:2324: done (3 left) 2024-09-15T03:56:27.026 INFO:tasks.rados.rados.0.smithi122.stdout:2325: done (2 left) 2024-09-15T03:56:27.026 INFO:tasks.rados.rados.0.smithi122.stdout:2327: done (1 left) 2024-09-15T03:56:27.026 INFO:tasks.rados.rados.0.smithi122.stdout:2328: done (0 left) 2024-09-15T03:56:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:2329: write (excl) oid 18 current snap is 271 2024-09-15T03:56:27.027 INFO:tasks.rados.rados.0.smithi122.stdout:2329: seq_num 829 ranges {733003=663898,1944396=506622,2865661=604381} 2024-09-15T03:56:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.011+0000 7f304bf7a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.011+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3024007520 con 0x7f30240068d0 2024-09-15T03:56:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894534545 0 0) 0x7f3024007520 con 0x7f30240068d0 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient(hunting): my global_id is 194379 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3024006590 con 0x7f30240068d0 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1964763098 0 0) 0x7f3024006590 con 0x7f30240068d0 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3024007520 con 0x7f30240068d0 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3040005530 con 0x7f30240068d0 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient: dump: 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1280551175 0 0) 0x7f3024007520 con 0x7f30240068d0 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3024008760 con 0x7f30240068d0 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2390271330) 0x7f3040007870 con 0x7f30240068d0 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.26:6792/0 conn(0x7f30240068d0 legacy=0x7f3024006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f3039ffb700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.26:6792/0 conn(0x7f30240068d0 legacy=0x7f3024006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:27.031 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.032 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.033 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:27.034 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:27.035 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.036 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.037 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.039 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.041 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.042 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.044 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.048 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.050 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.051 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.052 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.053 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.053 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:27.053 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.122:6812/20895 conn(0x7f3028037cc0 legacy=0x7f302803a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 10 monclient: shutdown 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 >> v1:172.21.15.26:6791/0 conn(0x7f304409e440 legacy=0x7f304412e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304a517700 1 -- 172.21.15.26:0/1327330160 reap_dead start 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304a517700 1 -- 172.21.15.26:0/1327330160 reap_dead start 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 shutdown_connections 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 >> 172.21.15.26:0/1327330160 conn(0x7f3044063f30 msgr2=0x7f304409ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 shutdown_connections 2024-09-15T03:56:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.019+0000 7f304bf7a700 1 -- 172.21.15.26:0/1327330160 wait complete. 2024-09-15T03:56:27.056 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:27.056 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:56:27.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:56:27.058 INFO:tasks.rados.rados.0.smithi122.stdout:2329: writing smithi12223906-18 from 733003 to 1396901 tid 1 2024-09-15T03:56:27.058 INFO:tasks.rados.rados.0.smithi122.stdout:2329: writing smithi12223906-18 from 1944396 to 2451018 tid 2 2024-09-15T03:56:27.058 INFO:tasks.rados.rados.0.smithi122.stdout:2329: writing smithi12223906-18 from 2865661 to 3470042 tid 3 2024-09-15T03:56:27.058 INFO:tasks.rados.rados.0.smithi122.stdout:2330: write oid 3 current snap is 271 2024-09-15T03:56:27.058 INFO:tasks.rados.rados.0.smithi122.stdout:2330: seq_num 830 ranges {749810=739147,2002878=650280,3293084=415653} 2024-09-15T03:56:27.058 INFO:tasks.rados.rados.0.smithi122.stdout:2330: writing smithi12223906-3 from 749810 to 1488957 tid 1 2024-09-15T03:56:27.062 INFO:tasks.rados.rados.0.smithi122.stdout:2330: writing smithi12223906-3 from 2002878 to 2653158 tid 2 2024-09-15T03:56:27.068 INFO:tasks.rados.rados.0.smithi122.stdout:2330: writing smithi12223906-3 from 3293084 to 3708737 tid 3 2024-09-15T03:56:27.068 INFO:tasks.rados.rados.0.smithi122.stdout:2331: write (excl) oid 44 current snap is 271 2024-09-15T03:56:27.068 INFO:tasks.rados.rados.0.smithi122.stdout:2331: seq_num 831 ranges {766617=414396,1945007=510291,3204154=626925} 2024-09-15T03:56:27.075 INFO:tasks.rados.rados.0.smithi122.stdout:2331: writing smithi12223906-44 from 766617 to 1181013 tid 1 2024-09-15T03:56:27.082 INFO:tasks.rados.rados.0.smithi122.stdout:2331: writing smithi12223906-44 from 1945007 to 2455298 tid 2 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2331: writing smithi12223906-44 from 3204154 to 3831079 tid 3 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2332: delete oid 12 current snap is 271 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2329: finishing write tid 1 to smithi12223906-18 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2329: finishing write tid 2 to smithi12223906-18 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2329: finishing write tid 3 to smithi12223906-18 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2329: finishing write tid 4 to smithi12223906-18 2024-09-15T03:56:27.090 INFO:tasks.rados.rados.0.smithi122.stdout:2329: finishing write tid 5 to smithi12223906-18 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1532 (ObjNum 829 snap 271 seq_num 829) dirty exists 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2329: left oid 18 (ObjNum 829 snap 271 seq_num 829) 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2330: finishing write tid 1 to smithi12223906-3 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2330: finishing write tid 2 to smithi12223906-3 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2330: finishing write tid 3 to smithi12223906-3 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2330: finishing write tid 4 to smithi12223906-3 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2330: finishing write tid 5 to smithi12223906-3 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 1521 (ObjNum 830 snap 271 seq_num 830) dirty exists 2024-09-15T03:56:27.091 INFO:tasks.rados.rados.0.smithi122.stdout:2330: left oid 3 (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:56:27.092 INFO:tasks.rados.rados.0.smithi122.stdout:2331: finishing write tid 1 to smithi12223906-44 2024-09-15T03:56:27.092 INFO:tasks.rados.rados.0.smithi122.stdout:2331: finishing write tid 2 to smithi12223906-44 2024-09-15T03:56:27.093 INFO:tasks.rados.rados.0.smithi122.stdout:2329: done (3 left) 2024-09-15T03:56:27.093 INFO:tasks.rados.rados.0.smithi122.stdout:2330: done (2 left) 2024-09-15T03:56:27.093 INFO:tasks.rados.rados.0.smithi122.stdout:2332: done (1 left) 2024-09-15T03:56:27.093 INFO:tasks.rados.rados.0.smithi122.stdout:2333: read oid 30 snap 217 2024-09-15T03:56:27.094 INFO:tasks.rados.rados.0.smithi122.stdout:2333: expect deleted 2024-09-15T03:56:27.094 INFO:tasks.rados.rados.0.smithi122.stdout:2334: snap_remove snap 208 2024-09-15T03:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.163+0000 7f90e90c0700 10 monclient: get_monmap_and_config 2024-09-15T03:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.163+0000 7f90e90c0700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.163+0000 7f90e90c0700 10 monclient: monmap: 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:27.166890+0000 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:27.166890+0000 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 1 Processor -- start 2024-09-15T03:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 1 -- start start 2024-09-15T03:56:27.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient: init 2024-09-15T03:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient: _add_conns ranks=[5,7,4,2,3,1,0,8,6] 2024-09-15T03:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient(hunting): picked mon.noname-h con 0x7f90e40a3890 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:27.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient(hunting): picked mon.noname-i con 0x7f90e4099360 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient(hunting): picked mon.noname-c con 0x7f90e4138ee0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e404ca10 con 0x7f90e4138ee0 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e406e6a0 con 0x7f90e40a3890 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e409e190 con 0x7f90e4099360 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient(hunting): authenticate will time out at 1531.597922s 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e359e700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f90e4138ee0 0x7f90e413b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43910/0 (socket says 172.21.15.26:43910) 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e359e700 1 -- 172.21.15.26:0/1820900271 learned_addr learned my addr 172.21.15.26:0/1820900271 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346294806 0 0) 0x7f90e404ca10 con 0x7f90e4138ee0 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): my global_id is 204160 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90cc003570 con 0x7f90e4138ee0 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646606857 0 0) 0x7f90e406e6a0 con 0x7f90e40a3890 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): my global_id is 5576 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e404ca10 con 0x7f90e40a3890 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1430997741 0 0) 0x7f90e409e190 con 0x7f90e4099360 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient(hunting): my global_id is 135025 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e406e6a0 con 0x7f90e4099360 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730128764 0 0) 0x7f90cc003570 con 0x7f90e4138ee0 2024-09-15T03:56:27.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 >> v1:172.21.15.122:6792/0 conn(0x7f90e4099360 legacy=0x7f90e4099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 >> v1:172.21.15.122:6791/0 conn(0x7f90e40a3890 legacy=0x7f90e41331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 monclient: found mon.noname-c 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e41338c0 con 0x7f90e4138ee0 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 5 monclient: authenticate success, global_id 204160 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90dc001b30 con 0x7f90e4138ee0 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: dump: 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f90dc001da0 con 0x7f90e4138ee0 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e90c0700 4 set_mon_vals no callback set 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 1 -- 172.21.15.26:0/1820900271 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90dc004fb0 con 0x7f90e4138ee0 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.167+0000 7f90e1d9b700 10 monclient: dump: 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.180 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.181 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 10 monclient: shutdown 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 -- 172.21.15.26:0/1820900271 >> v1:172.21.15.26:6791/0 conn(0x7f90e4138ee0 legacy=0x7f90e413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 -- 172.21.15.26:0/1820900271 shutdown_connections 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 -- 172.21.15.26:0/1820900271 >> 172.21.15.26:0/1820900271 conn(0x7f90e4071350 msgr2=0x7f90e40a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 -- 172.21.15.26:0/1820900271 shutdown_connections 2024-09-15T03:56:27.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 -- 172.21.15.26:0/1820900271 wait complete. 2024-09-15T03:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 10 monclient: monmap: 2024-09-15T03:56:27.187 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 Processor -- start 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 1 -- start start 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.171+0000 7f90e90c0700 10 monclient: init 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient: _add_conns ranks=[7,5,4,8,3,6,0,1,2] 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient(hunting): picked mon.noname-i con 0x7f90e4099360 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient(hunting): picked mon.noname-h con 0x7f90e4138ee0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient(hunting): picked mon.noname-c con 0x7f90e412f540 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e409ec80 con 0x7f90e412f540 2024-09-15T03:56:27.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e409e8c0 con 0x7f90e4138ee0 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e4131c70 con 0x7f90e4099360 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient(hunting): authenticate will time out at 1531.604712s 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e359e700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f90e412f540 0x7f90e414a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43918/0 (socket says 172.21.15.26:43918) 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e359e700 1 -- 172.21.15.26:0/827723011 learned_addr learned my addr 172.21.15.26:0/827723011 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925609429 0 0) 0x7f90e409ec80 con 0x7f90e412f540 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): my global_id is 204169 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d0003570 con 0x7f90e412f540 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853252876 0 0) 0x7f90e409e8c0 con 0x7f90e4138ee0 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): my global_id is 5585 2024-09-15T03:56:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e409ec80 con 0x7f90e4138ee0 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144524752 0 0) 0x7f90e4131c70 con 0x7f90e4099360 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): my global_id is 135034 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e409e8c0 con 0x7f90e4099360 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2336351219 0 0) 0x7f90d0003570 con 0x7f90e412f540 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90e4131c70 con 0x7f90e412f540 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90dc001b30 con 0x7f90e412f540 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): dump: 2024-09-15T03:56:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 175257672 0 0) 0x7f90e409ec80 con 0x7f90e4138ee0 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90d0003570 con 0x7f90e4138ee0 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90d8001a60 con 0x7f90e4138ee0 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient(hunting): dump: 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.193 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 759091826 0 0) 0x7f90e4131c70 con 0x7f90e412f540 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.122:6792/0 conn(0x7f90e4099360 legacy=0x7f90e412e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.122:6791/0 conn(0x7f90e4138ee0 legacy=0x7f90e412ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 monclient: found mon.c 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e4131990 con 0x7f90e412f540 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 5 monclient: authenticate success, global_id 204169 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient: _renew_subs 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e412fe60 con 0x7f90e412f540 2024-09-15T03:56:27.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient: _renew_subs 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e4132650 con 0x7f90e412f540 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f90dc004fb0 con 0x7f90e412f540 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90dc0055a0 con 0x7f90e412f540 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e159a700 4 set_mon_vals no callback set 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 10 monclient: dump: 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f90dc00ff60 con 0x7f90e412f540 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f90c4006590 con 0x7f90c4002840 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.175+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(534..534 src has 224..534) v4 ==== 3927+0+0 (unknown 2174197727 0 0) 0x7f90dc0443e0 con 0x7f90e412f540 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90e259c700 10 monclient: get_auth_request con 0x7f90d0037c00 auth_method 0 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306684075 0 0) 0x7f90c4006590 con 0x7f90c4002840 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient(hunting): my global_id is 135035 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e4131c70 con 0x7f90c4002840 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3601062533 0 0) 0x7f90e4131c70 con 0x7f90c4002840 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90c4006590 con 0x7f90c4002840 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90d40021b0 con 0x7f90c4002840 2024-09-15T03:56:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient: dump: 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2142601184 0 0) 0x7f90c4006590 con 0x7f90c4002840 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90c400a130 con 0x7f90c4002840 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f90d4004570 con 0x7f90c4002840 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.26:6793/0 conn(0x7f90c4002840 legacy=0x7f90c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.179+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.26:6793/0 conn(0x7f90c4002840 legacy=0x7f90c4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.183+0000 7f90e90c0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.183+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f90c4009640 con 0x7f90c40090b0 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248239212 0 0) 0x7f90c4009640 con 0x7f90c40090b0 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient(hunting): my global_id is 135044 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c4006590 con 0x7f90c40090b0 2024-09-15T03:56:27.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1675856249 0 0) 0x7f90c4006590 con 0x7f90c40090b0 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f90c4009640 con 0x7f90c40090b0 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f90d4002c60 con 0x7f90c40090b0 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient: dump: 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 396303223 0 0) 0x7f90c4009640 con 0x7f90c40090b0 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f90c40098f0 con 0x7f90c40090b0 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1336744327) 0x7f90d4004080 con 0x7f90c40090b0 2024-09-15T03:56:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.26:6793/0 conn(0x7f90c40090b0 legacy=0x7f90c4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90caffd700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.26:6793/0 conn(0x7f90c40090b0 legacy=0x7f90c4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:27.205 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:27.206 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:27.207 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:27.208 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:27.209 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.210 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.212 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.213 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:27.214 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.217 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.221 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.222 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.225 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:27.226 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.122:6812/20895 conn(0x7f90d0037c00 legacy=0x7f90d003a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 10 monclient: shutdown 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e359e700 1 -- 172.21.15.26:0/827723011 reap_dead start 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 >> v1:172.21.15.26:6791/0 conn(0x7f90e412f540 legacy=0x7f90e414a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 shutdown_connections 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 >> 172.21.15.26:0/827723011 conn(0x7f90e4071350 msgr2=0x7f90e40a1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 shutdown_connections 2024-09-15T03:56:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.187+0000 7f90e90c0700 1 -- 172.21.15.26:0/827723011 wait complete. 2024-09-15T03:56:27.229 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:27.229 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:56:27.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:56:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.339+0000 7f7da3307700 10 monclient: get_monmap_and_config 2024-09-15T03:56:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.339+0000 7f7da3307700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.339+0000 7f7da3307700 10 monclient: monmap: 2024-09-15T03:56:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:27.343585+0000 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:27.343585+0000 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.347 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 1 Processor -- start 2024-09-15T03:56:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 1 -- start start 2024-09-15T03:56:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient: init 2024-09-15T03:56:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient: _add_conns ranks=[3,6,0,4,5,8,1,7,2] 2024-09-15T03:56:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient(hunting): picked mon.noname-g con 0x7f7d9c0990c0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient(hunting): picked mon.noname-d con 0x7f7d9c0a1310 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient(hunting): picked mon.noname-a con 0x7f7d9c1335a0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c04ca10 con 0x7f7d9c1335a0 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0a1b20 con 0x7f7d9c0990c0 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0a10b0 con 0x7f7d9c0a1310 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient(hunting): authenticate will time out at 1531.773690s 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da08a2700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f7d9c0a1310 0x7f7d9c099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48446/0 (socket says 172.21.15.26:48446) 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da08a2700 1 -- 172.21.15.26:0/1773385020 learned_addr learned my addr 172.21.15.26:0/1773385020 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da18a4700 1 --1- 172.21.15.26:0/1773385020 >> v1:172.21.15.26:6789/0 conn(0x7f7d9c1335a0 0x7f7d9c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52732/0 (socket says 172.21.15.26:52732) 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742467524 0 0) 0x7f7d9c04ca10 con 0x7f7d9c1335a0 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): my global_id is 194355 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d88003570 con 0x7f7d9c1335a0 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584381047 0 0) 0x7f7d9c0a1b20 con 0x7f7d9c0990c0 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): my global_id is 204114 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d9c04ca10 con 0x7f7d9c0990c0 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177876650 0 0) 0x7f7d9c0a10b0 con 0x7f7d9c0a1310 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient(hunting): my global_id is 194388 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d9c0a1b20 con 0x7f7d9c0a1310 2024-09-15T03:56:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680748350 0 0) 0x7f7d88003570 con 0x7f7d9c1335a0 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 >> v1:172.21.15.26:6792/0 conn(0x7f7d9c0a1310 legacy=0x7f7d9c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 >> v1:172.21.15.122:6790/0 conn(0x7f7d9c0990c0 legacy=0x7f7d9c132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 monclient: found mon.noname-a 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c099ba0 con 0x7f7d9c1335a0 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d940017b0 con 0x7f7d9c1335a0 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 5 monclient: authenticate success, global_id 194355 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: dump: 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.356 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7d94001a90 con 0x7f7d9c1335a0 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7da3307700 4 set_mon_vals no callback set 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 1 -- 172.21.15.26:0/1773385020 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d94004b70 con 0x7f7d9c1335a0 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.343+0000 7f7d93fff700 10 monclient: dump: 2024-09-15T03:56:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.358 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient: shutdown 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- 172.21.15.26:0/1773385020 >> v1:172.21.15.26:6789/0 conn(0x7f7d9c1335a0 legacy=0x7f7d9c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- 172.21.15.26:0/1773385020 shutdown_connections 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- 172.21.15.26:0/1773385020 >> 172.21.15.26:0/1773385020 conn(0x7f7d9c06e370 msgr2=0x7f7d9c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- 172.21.15.26:0/1773385020 shutdown_connections 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- 172.21.15.26:0/1773385020 wait complete. 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient: monmap: 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 Processor -- start 2024-09-15T03:56:27.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- start start 2024-09-15T03:56:27.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient: init 2024-09-15T03:56:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient: _add_conns ranks=[4,7,5,3,8,1,2,6,0] 2024-09-15T03:56:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient(hunting): picked mon.noname-c con 0x7f7d9c0990c0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient(hunting): picked mon.noname-i con 0x7f7d9c0a1310 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient(hunting): picked mon.noname-h con 0x7f7d9c1335a0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0a1ce0 con 0x7f7d9c0990c0 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c09a010 con 0x7f7d9c1335a0 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c132980 con 0x7f7d9c0a1310 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da3307700 10 monclient(hunting): authenticate will time out at 1531.778851s 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da10a3700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f7d9c0990c0 0x7f7d9c131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43920/0 (socket says 172.21.15.26:43920) 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7da10a3700 1 -- 172.21.15.26:0/3939496335 learned_addr learned my addr 172.21.15.26:0/3939496335 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656024784 0 0) 0x7f7d9c0a1ce0 con 0x7f7d9c0990c0 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): my global_id is 204178 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d80003570 con 0x7f7d9c0990c0 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42764477 0 0) 0x7f7d9c132980 con 0x7f7d9c0a1310 2024-09-15T03:56:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): my global_id is 135043 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d9c0a1ce0 con 0x7f7d9c0a1310 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993704523 0 0) 0x7f7d9c09a010 con 0x7f7d9c1335a0 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 10 monclient(hunting): my global_id is 5594 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.347+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d9c132980 con 0x7f7d9c1335a0 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2293258339 0 0) 0x7f7d80003570 con 0x7f7d9c0990c0 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d9c09a010 con 0x7f7d9c0990c0 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 468379907 0 0) 0x7f7d9c0a1ce0 con 0x7f7d9c0a1310 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d80003570 con 0x7f7d9c0a1310 2024-09-15T03:56:27.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d98002ed0 con 0x7f7d9c0990c0 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): dump: 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d8c0019e0 con 0x7f7d9c0a1310 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): dump: 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 515524142 0 0) 0x7f7d9c09a010 con 0x7f7d9c0990c0 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6792/0 conn(0x7f7d9c0a1310 legacy=0x7f7d9c1470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6791/0 conn(0x7f7d9c1335a0 legacy=0x7f7d9c14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 monclient: found mon.c 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c120e40 con 0x7f7d9c0990c0 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 5 monclient: authenticate success, global_id 204178 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 10 monclient: _renew_subs 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c121d30 con 0x7f7d9c0990c0 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 10 monclient: _renew_subs 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c120270 con 0x7f7d9c0990c0 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7d980030c0 con 0x7f7d9c0990c0 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d98001de0 con 0x7f7d9c0990c0 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d937fe700 4 set_mon_vals no callback set 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: dump: 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.373 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7d7c006590 con 0x7f7d7c002840 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f7d98001fc0 con 0x7f7d9c0990c0 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(534..534 src has 224..534) v4 ==== 3927+0+0 (unknown 2174197727 0 0) 0x7f7d98044350 con 0x7f7d9c0990c0 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438965765 0 0) 0x7f7d7c006590 con 0x7f7d7c002840 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient(hunting): my global_id is 204130 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d9c09a010 con 0x7f7d7c002840 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7da18a4700 10 monclient: get_auth_request con 0x7f7d80037cc0 auth_method 0 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3032969446 0 0) 0x7f7d9c09a010 con 0x7f7d7c002840 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d7c006590 con 0x7f7d7c002840 2024-09-15T03:56:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d8c004b70 con 0x7f7d7c002840 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 10 monclient: dump: 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 794521067 0 0) 0x7f7d7c006590 con 0x7f7d7c002840 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.351+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d7c00a130 con 0x7f7d7c002840 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.355+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7d8c005000 con 0x7f7d7c002840 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.355+0000 7f7d917fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.355+0000 7f7d917fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.355+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6789/0 conn(0x7f7d7c002840 legacy=0x7f7d7c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.355+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6789/0 conn(0x7f7d7c002840 legacy=0x7f7d7c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.363+0000 7f7da3307700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.363+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7d7c009640 con 0x7f7d7c0090b0 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123095650 0 0) 0x7f7d7c009640 con 0x7f7d7c0090b0 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient(hunting): my global_id is 204139 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d7c006590 con 0x7f7d7c0090b0 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3323619961 0 0) 0x7f7d7c006590 con 0x7f7d7c0090b0 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7d7c009640 con 0x7f7d7c0090b0 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7d8c005860 con 0x7f7d7c0090b0 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient: dump: 2024-09-15T03:56:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1945937235 0 0) 0x7f7d7c009640 con 0x7f7d7c0090b0 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7d7c0098f0 con 0x7f7d7c0090b0 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1756949634) 0x7f7d8c005c80 con 0x7f7d7c0090b0 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6789/0 conn(0x7f7d7c0090b0 legacy=0x7f7d7c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7d917fa700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6789/0 conn(0x7f7d7c0090b0 legacy=0x7f7d7c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:27.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.382 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.383 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:27.384 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.385 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.386 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.386 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.386 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.386 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:27.388 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.390 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.391 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.392 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.393 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.394 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.396 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.397 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.398 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.399 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.400 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.401 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:27.402 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.122:6812/20895 conn(0x7f7d80037cc0 legacy=0x7f7d8003a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 10 monclient: shutdown 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 >> v1:172.21.15.26:6791/0 conn(0x7f7d9c0990c0 legacy=0x7f7d9c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da18a4700 1 -- 172.21.15.26:0/3939496335 reap_dead start 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da18a4700 1 -- 172.21.15.26:0/3939496335 reap_dead start 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 shutdown_connections 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 >> 172.21.15.26:0/3939496335 conn(0x7f7d9c06e370 msgr2=0x7f7d9c09f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 shutdown_connections 2024-09-15T03:56:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.367+0000 7f7da3307700 1 -- 172.21.15.26:0/3939496335 wait complete. 2024-09-15T03:56:27.405 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:27.405 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:56:27.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:56:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.515+0000 7f2f4b59e700 10 monclient: get_monmap_and_config 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.515+0000 7f2f4b59e700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.515+0000 7f2f4b59e700 10 monclient: monmap: 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:27.518061+0000 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:27.518061+0000 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.522 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.515+0000 7f2f4b59e700 1 Processor -- start 2024-09-15T03:56:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.515+0000 7f2f4b59e700 1 -- start start 2024-09-15T03:56:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.515+0000 7f2f4b59e700 10 monclient: init 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient: _add_conns ranks=[1,0,7,2,5,4,6,3,8] 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient(hunting): picked mon.noname-f con 0x7f2f4c063c60 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient(hunting): picked mon.noname-a con 0x7f2f4c09aca0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient(hunting): picked mon.noname-i con 0x7f2f4c136de0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c04ca10 con 0x7f2f4c09aca0 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c09c600 con 0x7f2f4c063c60 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c0a0230 con 0x7f2f4c136de0 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient(hunting): authenticate will time out at 1531.948345s 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f41d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2f4c09aca0 0x7f2f4c1356c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52736/0 (socket says 172.21.15.26:52736) 2024-09-15T03:56:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f41d9b700 1 -- 172.21.15.26:0/56738269 learned_addr learned my addr 172.21.15.26:0/56738269 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253731182 0 0) 0x7f2f4c04ca10 con 0x7f2f4c09aca0 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): my global_id is 194364 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f38003570 con 0x7f2f4c09aca0 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158273623 0 0) 0x7f2f4c09c600 con 0x7f2f4c063c60 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): my global_id is 204148 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c04ca10 con 0x7f2f4c063c60 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911476689 0 0) 0x7f2f4c0a0230 con 0x7f2f4c136de0 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient(hunting): my global_id is 135052 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c09c600 con 0x7f2f4c136de0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 396760046 0 0) 0x7f2f38003570 con 0x7f2f4c09aca0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 >> v1:172.21.15.122:6792/0 conn(0x7f2f4c136de0 legacy=0x7f2f4c1391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 >> v1:172.21.15.122:6789/0 conn(0x7f2f4c063c60 legacy=0x7f2f4c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 monclient: found mon.noname-a 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c09c320 con 0x7f2f4c09aca0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f340017b0 con 0x7f2f4c09aca0 2024-09-15T03:56:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 5 monclient: authenticate success, global_id 194364 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: dump: 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f34001a20 con 0x7f2f4c09aca0 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 1 -- 172.21.15.26:0/56738269 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f34004c30 con 0x7f2f4c09aca0 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f49d9b700 10 monclient: dump: 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 4 set_mon_vals no callback set 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 10 monclient: shutdown 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- 172.21.15.26:0/56738269 >> v1:172.21.15.26:6789/0 conn(0x7f2f4c09aca0 legacy=0x7f2f4c1356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- 172.21.15.26:0/56738269 shutdown_connections 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- 172.21.15.26:0/56738269 >> 172.21.15.26:0/56738269 conn(0x7f2f4c0a4410 msgr2=0x7f2f4c0a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- 172.21.15.26:0/56738269 shutdown_connections 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.519+0000 7f2f4b59e700 1 -- 172.21.15.26:0/56738269 wait complete. 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient: monmap: 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.535 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 1 Processor -- start 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 1 -- start start 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient: init 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient: _add_conns ranks=[0,6,1,8,3,5,4,2,7] 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient(hunting): picked mon.noname-a con 0x7f2f4c063c60 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient(hunting): picked mon.noname-d con 0x7f2f4c09aca0 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient(hunting): picked mon.noname-f con 0x7f2f4c136de0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c06e5f0 con 0x7f2f4c063c60 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4a59c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2f4c063c60 0x7f2f4c098eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52742/0 (socket says 172.21.15.26:52742) 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4a59c700 1 -- 172.21.15.26:0/2224306416 learned_addr learned my addr 172.21.15.26:0/2224306416 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c06e190 con 0x7f2f4c136de0 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c134870 con 0x7f2f4c09aca0 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f4b59e700 10 monclient(hunting): authenticate will time out at 1531.954818s 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147154223 0 0) 0x7f2f4c06e5f0 con 0x7f2f4c063c60 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): my global_id is 194373 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c003570 con 0x7f2f4c063c60 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736194773 0 0) 0x7f2f4c134870 con 0x7f2f4c09aca0 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): my global_id is 194397 2024-09-15T03:56:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c06e5f0 con 0x7f2f4c09aca0 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692507750 0 0) 0x7f2f4c06e190 con 0x7f2f4c136de0 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): my global_id is 204157 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c134870 con 0x7f2f4c136de0 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3278577618 0 0) 0x7f2f2c003570 con 0x7f2f4c063c60 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f4c06e190 con 0x7f2f4c063c60 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f3c002b30 con 0x7f2f4c063c60 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.523+0000 7f2f42ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3627946437 0 0) 0x7f2f4c06e5f0 con 0x7f2f4c09aca0 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f2c003570 con 0x7f2f4c09aca0 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f340017b0 con 0x7f2f4c09aca0 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2939567365 0 0) 0x7f2f4c134870 con 0x7f2f4c136de0 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f4c06e5f0 con 0x7f2f4c136de0 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f44001c80 con 0x7f2f4c136de0 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4024265869 0 0) 0x7f2f4c06e190 con 0x7f2f4c063c60 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.26:6792/0 conn(0x7f2f4c09aca0 legacy=0x7f2f4c0995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.122:6789/0 conn(0x7f2f4c136de0 legacy=0x7f2f4c148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 monclient: found mon.a 2024-09-15T03:56:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c099cd0 con 0x7f2f4c063c60 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 5 monclient: authenticate success, global_id 194373 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 10 monclient: _renew_subs 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c134550 con 0x7f2f4c063c60 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 10 monclient: _renew_subs 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2f3c0016a0 con 0x7f2f4c063c60 2024-09-15T03:56:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c132bf0 con 0x7f2f4c063c60 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f3c001bd0 con 0x7f2f4c063c60 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4959a700 4 set_mon_vals no callback set 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient: dump: 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2f3c001d30 con 0x7f2f4c063c60 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2f4c04bbf0 con 0x7f2f4c1362c0 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(534..534 src has 224..534) v4 ==== 3927+0+0 (unknown 2174197727 0 0) 0x7f2f3c043e20 con 0x7f2f4c063c60 2024-09-15T03:56:27.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f4ad9d700 10 monclient: get_auth_request con 0x7f2f2c038160 auth_method 0 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747172401 0 0) 0x7f2f4c04bbf0 con 0x7f2f4c1362c0 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 10 monclient(hunting): my global_id is 204123 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.527+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c06e190 con 0x7f2f4c1362c0 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4035410847 0 0) 0x7f2f4c06e190 con 0x7f2f4c1362c0 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f4c04bbf0 con 0x7f2f4c1362c0 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f34005240 con 0x7f2f4c1362c0 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 10 monclient: dump: 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.552 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1078203534 0 0) 0x7f2f4c04bbf0 con 0x7f2f4c1362c0 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f4c06fac0 con 0x7f2f4c1362c0 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2f34007570 con 0x7f2f4c1362c0 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.122:6790/0 conn(0x7f2f4c1362c0 legacy=0x7f2f4c12fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.531+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.122:6790/0 conn(0x7f2f4c1362c0 legacy=0x7f2f4c12fac0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f4b59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2f4c1356d0 con 0x7f2f4c135300 2024-09-15T03:56:27.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477064927 0 0) 0x7f2f4c1356d0 con 0x7f2f4c135300 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 10 monclient(hunting): my global_id is 204132 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c04bbf0 con 0x7f2f4c135300 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1370805216 0 0) 0x7f2f4c04bbf0 con 0x7f2f4c135300 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2f4c1356d0 con 0x7f2f4c135300 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2f34005c10 con 0x7f2f4c135300 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 10 monclient: dump: 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.556 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1625245798 0 0) 0x7f2f4c1356d0 con 0x7f2f4c135300 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.539+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f4c0714f0 con 0x7f2f4c135300 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1322912031) 0x7f2f34007090 con 0x7f2f4c135300 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f42ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f42ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.122:6790/0 conn(0x7f2f4c135300 legacy=0x7f2f4c02ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f42ffd700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.122:6790/0 conn(0x7f2f4c135300 legacy=0x7f2f4c02ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:27.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.568 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.569 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.571 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.572 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.574 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.575 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.576 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.577 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.577 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:27.577 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.122:6812/20895 conn(0x7f2f2c038160 legacy=0x7f2f2c03a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 10 monclient: shutdown 2024-09-15T03:56:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 >> v1:172.21.15.26:6789/0 conn(0x7f2f4c063c60 legacy=0x7f2f4c098eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4ad9d700 1 -- 172.21.15.26:0/2224306416 reap_dead start 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4ad9d700 1 -- 172.21.15.26:0/2224306416 reap_dead start 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 shutdown_connections 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 >> 172.21.15.26:0/2224306416 conn(0x7f2f4c0a4410 msgr2=0x7f2f4c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 shutdown_connections 2024-09-15T03:56:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.543+0000 7f2f4b59e700 1 -- 172.21.15.26:0/2224306416 wait complete. 2024-09-15T03:56:27.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:27.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:56:27.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:56:27.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient: get_monmap_and_config 2024-09-15T03:56:27.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient: monmap: 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:27.693673+0000 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:27.693673+0000 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 1 Processor -- start 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 1 -- start start 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient: init 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient: _add_conns ranks=[3,8,6,4,5,0,7,2,1] 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient(hunting): picked mon.noname-g con 0x7f8d20132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient(hunting): picked mon.noname-e con 0x7f8d2009faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient(hunting): picked mon.noname-d con 0x7f8d20138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2004ca10 con 0x7f8d20132b40 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2009d5d0 con 0x7f8d20138d40 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d20133620 con 0x7f8d2009faa0 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d24fb8700 10 monclient(hunting): authenticate will time out at 1532.123458s 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d1ffff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8d20138d40 0x7f8d2013b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48460/0 (socket says 172.21.15.26:48460) 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d1ffff700 1 -- 172.21.15.26:0/3980319622 learned_addr learned my addr 172.21.15.26:0/3980319622 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.691+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403740030 0 0) 0x7f8d2009d5d0 con 0x7f8d20138d40 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): my global_id is 194406 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d08003570 con 0x7f8d20138d40 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200196585 0 0) 0x7f8d20133620 con 0x7f8d2009faa0 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): my global_id is 135053 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2009d5d0 con 0x7f8d2009faa0 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022161048 0 0) 0x7f8d2004ca10 con 0x7f8d20132b40 2024-09-15T03:56:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient(hunting): my global_id is 204141 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d20133620 con 0x7f8d20132b40 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862108101 0 0) 0x7f8d2009d5d0 con 0x7f8d2009faa0 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 >> v1:172.21.15.26:6792/0 conn(0x7f8d20138d40 legacy=0x7f8d2013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 >> v1:172.21.15.122:6790/0 conn(0x7f8d20132b40 legacy=0x7f8d20132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 monclient: found mon.noname-e 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200a0cc0 con 0x7f8d2009faa0 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d14001790 con 0x7f8d2009faa0 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 5 monclient: authenticate success, global_id 135053 2024-09-15T03:56:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: dump: 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8d140019c0 con 0x7f8d2009faa0 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 1 -- 172.21.15.26:0/3980319622 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d14001ef0 con 0x7f8d2009faa0 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 4 set_mon_vals no callback set 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d1e7fc700 10 monclient: dump: 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: shutdown 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 -- 172.21.15.26:0/3980319622 >> v1:172.21.15.26:6793/0 conn(0x7f8d2009faa0 legacy=0x7f8d2009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 -- 172.21.15.26:0/3980319622 shutdown_connections 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 -- 172.21.15.26:0/3980319622 >> 172.21.15.26:0/3980319622 conn(0x7f8d2006e370 msgr2=0x7f8d200a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 -- 172.21.15.26:0/3980319622 shutdown_connections 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 -- 172.21.15.26:0/3980319622 wait complete. 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: monmap: 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 Processor -- start 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 1 -- start start 2024-09-15T03:56:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: init 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient: _add_conns ranks=[2,8,4,5,3,7,6,1,0] 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.695+0000 7f8d24fb8700 10 monclient(hunting): picked mon.noname-b con 0x7f8d2009faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient(hunting): picked mon.noname-e con 0x7f8d20132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient(hunting): picked mon.noname-c con 0x7f8d20138d40 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d201385f0 con 0x7f8d2009faa0 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2009cfa0 con 0x7f8d20138d40 2024-09-15T03:56:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2009ccc0 con 0x7f8d20132b40 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient(hunting): authenticate will time out at 1532.128136s 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d1effd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8d20132b40 0x7f8d201367d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50072/0 (socket says 172.21.15.26:50072) 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d1effd700 1 -- 172.21.15.26:0/710468573 learned_addr learned my addr 172.21.15.26:0/710468573 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725165659 0 0) 0x7f8d2009ccc0 con 0x7f8d20132b40 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): my global_id is 135062 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d0c003570 con 0x7f8d20132b40 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686143037 0 0) 0x7f8d201385f0 con 0x7f8d2009faa0 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): my global_id is 204149 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2009ccc0 con 0x7f8d2009faa0 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467480522 0 0) 0x7f8d2009cfa0 con 0x7f8d20138d40 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): my global_id is 204187 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d201385f0 con 0x7f8d20138d40 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1875293566 0 0) 0x7f8d0c003570 con 0x7f8d20132b40 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d2009cfa0 con 0x7f8d20132b40 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d14001e70 con 0x7f8d20132b40 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): dump: 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 334862045 0 0) 0x7f8d201385f0 con 0x7f8d20138d40 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d0c003570 con 0x7f8d20138d40 2024-09-15T03:56:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3188509496 0 0) 0x7f8d2009ccc0 con 0x7f8d2009faa0 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d201385f0 con 0x7f8d2009faa0 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d18001c60 con 0x7f8d20138d40 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): dump: 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.727 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d100019e0 con 0x7f8d2009faa0 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient(hunting): dump: 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3477806721 0 0) 0x7f8d2009cfa0 con 0x7f8d20132b40 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.26:6791/0 conn(0x7f8d20138d40 legacy=0x7f8d2014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.26:6790/0 conn(0x7f8d2009faa0 legacy=0x7f8d201360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 monclient: found mon.e 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200987e0 con 0x7f8d20132b40 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 5 monclient: authenticate success, global_id 135062 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient: _renew_subs 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d20098d10 con 0x7f8d20132b40 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient: _renew_subs 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d20137a20 con 0x7f8d20132b40 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8d14002520 con 0x7f8d20132b40 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d14002a50 con 0x7f8d20132b40 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 10 monclient: dump: 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d1d7fa700 4 set_mon_vals no callback set 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8d14002bb0 con 0x7f8d20132b40 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8d00006590 con 0x7f8d00002840 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.699+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(534..534 src has 224..534) v4 ==== 3927+0+0 (unknown 2174197727 0 0) 0x7f8d140440b0 con 0x7f8d20132b40 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d1ffff700 10 monclient: get_auth_request con 0x7f8d0c037fd0 auth_method 0 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841997198 0 0) 0x7f8d00006590 con 0x7f8d00002840 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient(hunting): my global_id is 5603 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2009cfa0 con 0x7f8d00002840 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3824225138 0 0) 0x7f8d2009cfa0 con 0x7f8d00002840 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d00006590 con 0x7f8d00002840 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d100024e0 con 0x7f8d00002840 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient: dump: 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3708034145 0 0) 0x7f8d00006590 con 0x7f8d00002840 2024-09-15T03:56:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d0000a130 con 0x7f8d00002840 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8d10002a90 con 0x7f8d00002840 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.122:6791/0 conn(0x7f8d00002840 legacy=0x7f8d00004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.703+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.122:6791/0 conn(0x7f8d00002840 legacy=0x7f8d00004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d24fb8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8d00007520 con 0x7f8d000068d0 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923649503 0 0) 0x7f8d00007520 con 0x7f8d000068d0 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient(hunting): my global_id is 5612 2024-09-15T03:56:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d00006590 con 0x7f8d000068d0 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2840984782 0 0) 0x7f8d00006590 con 0x7f8d000068d0 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d00007520 con 0x7f8d000068d0 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8d10002d40 con 0x7f8d000068d0 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient: dump: 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.738 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3180787486 0 0) 0x7f8d00007520 con 0x7f8d000068d0 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8d00008760 con 0x7f8d000068d0 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7738 (unknown 0 0 2354650225) 0x7f8d10005670 con 0x7f8d000068d0 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.715+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.122:6791/0 conn(0x7f8d000068d0 legacy=0x7f8d00006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d077fe700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.122:6791/0 conn(0x7f8d000068d0 legacy=0x7f8d00006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:27.740 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.742 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:27.743 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:27.744 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.745 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.749 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:27.750 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:27.751 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.752 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.754 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.755 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.758 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.759 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.759 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.759 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.759 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.759 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:27.760 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:27.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.760 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.760 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.760 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.761 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:56:27.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.763 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.764 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.764 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:27.764 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:27.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.122:6812/20895 conn(0x7f8d0c037fd0 legacy=0x7f8d0c03a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 10 monclient: shutdown 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 >> v1:172.21.15.26:6793/0 conn(0x7f8d20132b40 legacy=0x7f8d201367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d1ffff700 1 -- 172.21.15.26:0/710468573 reap_dead start 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d1ffff700 1 -- 172.21.15.26:0/710468573 reap_dead start 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 shutdown_connections 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 >> 172.21.15.26:0/710468573 conn(0x7f8d2006e370 msgr2=0x7f8d201321b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 shutdown_connections 2024-09-15T03:56:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.719+0000 7f8d24fb8700 1 -- 172.21.15.26:0/710468573 wait complete. 2024-09-15T03:56:27.767 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:27.768 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:56:27.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:56:27.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.879+0000 7f0cdc4d4700 10 monclient: get_monmap_and_config 2024-09-15T03:56:27.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.879+0000 7f0cdc4d4700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.879+0000 7f0cdc4d4700 10 monclient: monmap: 2024-09-15T03:56:27.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:27.882675+0000 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:27.882675+0000 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.879+0000 7f0cdc4d4700 1 Processor -- start 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.879+0000 7f0cdc4d4700 1 -- start start 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.879+0000 7f0cdc4d4700 10 monclient: init 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient: _add_conns ranks=[8,7,5,4,1,6,2,0,3] 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient(hunting): picked mon.noname-e con 0x7f0cd4132c90 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient(hunting): picked mon.noname-i con 0x7f0cd4133060 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient(hunting): picked mon.noname-h con 0x7f0cd4133470 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd404ca10 con 0x7f0cd4133470 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4068f90 con 0x7f0cd4133060 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40a1430 con 0x7f0cd4132c90 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient(hunting): authenticate will time out at 1532.312236s 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cda270700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f0cd4132c90 0x7f0cd409ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50084/0 (socket says 172.21.15.26:50084) 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cda270700 1 -- 172.21.15.26:0/2107800344 learned_addr learned my addr 172.21.15.26:0/2107800344 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292004247 0 0) 0x7f0cd40a1430 con 0x7f0cd4132c90 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): my global_id is 135071 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cc0003570 con 0x7f0cd4132c90 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258307037 0 0) 0x7f0cd4068f90 con 0x7f0cd4133060 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): my global_id is 135061 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd40a1430 con 0x7f0cd4133060 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139169898 0 0) 0x7f0cd404ca10 con 0x7f0cd4133470 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient(hunting): my global_id is 5621 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd4068f90 con 0x7f0cd4133470 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3552389252 0 0) 0x7f0cc0003570 con 0x7f0cd4132c90 2024-09-15T03:56:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 >> v1:172.21.15.122:6792/0 conn(0x7f0cd4133060 legacy=0x7f0cd4137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 >> v1:172.21.15.122:6791/0 conn(0x7f0cd4133470 legacy=0x7f0cd413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 monclient: found mon.noname-e 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd409da90 con 0x7f0cd4132c90 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 5 monclient: authenticate success, global_id 135071 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0cc40017b0 con 0x7f0cd4132c90 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: dump: 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0cc4001a10 con 0x7f0cd4132c90 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 4 set_mon_vals no callback set 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 1 -- 172.21.15.26:0/2107800344 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0cc4004c30 con 0x7f0cd4132c90 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cd926e700 10 monclient: dump: 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient: shutdown 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/2107800344 >> v1:172.21.15.26:6793/0 conn(0x7f0cd4132c90 legacy=0x7f0cd409ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/2107800344 shutdown_connections 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/2107800344 >> 172.21.15.26:0/2107800344 conn(0x7f0cd406e510 msgr2=0x7f0cd40a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/2107800344 shutdown_connections 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/2107800344 wait complete. 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient: build_initial_monmap 2024-09-15T03:56:27.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.883+0000 7f0cdc4d4700 10 monclient: monmap: 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:27.899 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:27.899 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 Processor -- start 2024-09-15T03:56:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 -- start start 2024-09-15T03:56:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: init 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: _add_conns ranks=[1,5,4,2,6,3,7,0,8] 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient(hunting): picked mon.noname-f con 0x7f0cd4132c90 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient(hunting): picked mon.noname-h con 0x7f0cd4133060 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient(hunting): picked mon.noname-c con 0x7f0cd4133470 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40a1e90 con 0x7f0cd4132c90 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4120060 con 0x7f0cd4133470 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd411ee20 con 0x7f0cd4133060 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient(hunting): authenticate will time out at 1532.317435s 2024-09-15T03:56:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdaa71700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f0cd4133470 0x7f0cd414a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43946/0 (socket says 172.21.15.26:43946) 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdaa71700 1 -- 172.21.15.26:0/1683854614 learned_addr learned my addr 172.21.15.26:0/1683854614 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572117862 0 0) 0x7f0cd4120060 con 0x7f0cd4133470 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): my global_id is 204196 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb8003570 con 0x7f0cd4133470 2024-09-15T03:56:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752863157 0 0) 0x7f0cd411ee20 con 0x7f0cd4133060 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): my global_id is 5630 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd4120060 con 0x7f0cd4133060 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283778486 0 0) 0x7f0cd40a1e90 con 0x7f0cd4132c90 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): my global_id is 204166 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd411ee20 con 0x7f0cd4132c90 2024-09-15T03:56:27.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2750555734 0 0) 0x7f0cb8003570 con 0x7f0cd4133470 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cd40a1e90 con 0x7f0cd4133470 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0ccc002dd0 con 0x7f0cd4133470 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): dump: 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3717201847 0 0) 0x7f0cd4120060 con 0x7f0cd4133060 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cb8003570 con 0x7f0cd4133060 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0cd0001a50 con 0x7f0cd4133060 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient(hunting): dump: 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3794718794 0 0) 0x7f0cd40a1e90 con 0x7f0cd4133470 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6791/0 conn(0x7f0cd4133060 legacy=0x7f0cd4147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6789/0 conn(0x7f0cd4132c90 legacy=0x7f0cd412fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 monclient: found mon.c 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd411fd80 con 0x7f0cd4133470 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 5 monclient: authenticate success, global_id 204196 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: _renew_subs 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd409d120 con 0x7f0cd4133470 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: _renew_subs 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd4142af0 con 0x7f0cd4133470 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0ccc001880 con 0x7f0cd4133470 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0ccc001d30 con 0x7f0cd4133470 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0cca7fc700 10 monclient: dump: 2024-09-15T03:56:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.887+0000 7f0ccbfff700 4 set_mon_vals no callback set 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0ccc001f10 con 0x7f0cd4133470 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cdc4d4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0cbc006590 con 0x7f0cbc002840 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(534..534 src has 224..534) v4 ==== 3927+0+0 (unknown 2174197727 0 0) 0x7f0ccc0442b0 con 0x7f0cd4133470 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cd9a6f700 10 monclient: get_auth_request con 0x7f0cb8037bf0 auth_method 0 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461708430 0 0) 0x7f0cbc006590 con 0x7f0cbc002840 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient(hunting): my global_id is 135070 2024-09-15T03:56:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cd40a1e90 con 0x7f0cbc002840 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 452020504 0 0) 0x7f0cd40a1e90 con 0x7f0cbc002840 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cbc006590 con 0x7f0cbc002840 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0cc4001210 con 0x7f0cbc002840 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient: dump: 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.915 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 978849524 0 0) 0x7f0cbc006590 con 0x7f0cbc002840 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cbc00a130 con 0x7f0cbc002840 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0cc40052c0 con 0x7f0cbc002840 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6792/0 conn(0x7f0cbc002840 legacy=0x7f0cbc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.891+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6792/0 conn(0x7f0cbc002840 legacy=0x7f0cbc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cdc4d4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0cbc007500 con 0x7f0cbc0068b0 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865621675 0 0) 0x7f0cbc007500 con 0x7f0cbc0068b0 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 10 monclient(hunting): my global_id is 135079 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cbc006590 con 0x7f0cbc0068b0 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 704044306 0 0) 0x7f0cbc006590 con 0x7f0cbc0068b0 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0cbc007500 con 0x7f0cbc0068b0 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0cc4005ad0 con 0x7f0cbc0068b0 2024-09-15T03:56:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.903+0000 7f0cca7fc700 10 monclient: dump: 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1960100953 0 0) 0x7f0cbc007500 con 0x7f0cbc0068b0 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0cbc008760 con 0x7f0cbc0068b0 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 1797102085) 0x7f0cc40082b0 con 0x7f0cbc0068b0 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6792/0 conn(0x7f0cbc0068b0 legacy=0x7f0cbc006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cca7fc700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6792/0 conn(0x7f0cbc0068b0 legacy=0x7f0cbc006d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:27.921 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.922 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.923 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:27.924 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:27.925 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:27.927 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:27.929 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.930 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:27.932 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:27.933 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.934 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.937 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:27.938 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.939 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:27.940 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:27.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:27.941 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.122:6812/20895 conn(0x7f0cb8037bf0 legacy=0x7f0cb803a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 10 monclient: shutdown 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 >> v1:172.21.15.26:6791/0 conn(0x7f0cd4133470 legacy=0x7f0cd414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdaa71700 1 -- 172.21.15.26:0/1683854614 reap_dead start 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdaa71700 1 -- 172.21.15.26:0/1683854614 reap_dead start 2024-09-15T03:56:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 shutdown_connections 2024-09-15T03:56:27.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 >> 172.21.15.26:0/1683854614 conn(0x7f0cd406e510 msgr2=0x7f0cd40a4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:27.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 shutdown_connections 2024-09-15T03:56:27.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:27.907+0000 7f0cdc4d4700 1 -- 172.21.15.26:0/1683854614 wait complete. 2024-09-15T03:56:27.943 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:27.943 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T03:56:27.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T03:56:28.043 INFO:tasks.rados.rados.0.smithi122.stdout:2334: done (2 left) 2024-09-15T03:56:28.043 INFO:tasks.rados.rados.0.smithi122.stdout:2335: copy_from oid 20 from oid 48 current snap is 271 2024-09-15T03:56:28.043 INFO:tasks.rados.rados.0.smithi122.stdout:2331: finishing write tid 3 to smithi12223906-44 2024-09-15T03:56:28.043 INFO:tasks.rados.rados.0.smithi122.stdout:2331: finishing write tid 4 to smithi12223906-44 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2331: finishing write tid 5 to smithi12223906-44 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1537 (ObjNum 831 snap 271 seq_num 831) dirty exists 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2331: left oid 44 (ObjNum 831 snap 271 seq_num 831) 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2331: done (2 left) 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2333: done (1 left) 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2336: read oid 15 snap -1 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2336: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2337: rollback oid 2 current snap is 271 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 240 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2338: write oid 45 current snap is 271 2024-09-15T03:56:28.044 INFO:tasks.rados.rados.0.smithi122.stdout:2338: seq_num 833 ranges {400231=564894,1661971=338260} 2024-09-15T03:56:28.049 INFO:tasks.rados.rados.0.smithi122.stdout:2338: writing smithi12223906-45 from 400231 to 965125 tid 1 2024-09-15T03:56:28.056 INFO:tasks.rados.rados.0.smithi122.stdout:2338: writing smithi12223906-45 from 1661971 to 2000231 tid 2 2024-09-15T03:56:28.056 INFO:tasks.rados.rados.0.smithi122.stdout:2339: write (excl) oid 5 current snap is 271 2024-09-15T03:56:28.056 INFO:tasks.rados.rados.0.smithi122.stdout:2339: seq_num 834 ranges {417038=640143,1720453=296585} 2024-09-15T03:56:28.062 INFO:tasks.rados.rados.0.smithi122.stdout:2339: writing smithi12223906-5 from 417038 to 1057181 tid 1 2024-09-15T03:56:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.055+0000 7f21fdb2a700 10 monclient: get_monmap_and_config 2024-09-15T03:56:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.055+0000 7f21fdb2a700 10 monclient: build_initial_monmap 2024-09-15T03:56:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.055+0000 7f21fdb2a700 10 monclient: monmap: 2024-09-15T03:56:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:28.059695+0000 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:28.059695+0000 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:28.064 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 Processor -- start 2024-09-15T03:56:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- start start 2024-09-15T03:56:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: init 2024-09-15T03:56:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: _add_conns ranks=[3,1,0,5,4,2,7,8,6] 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient(hunting): picked mon.noname-g con 0x7f21f80a6a80 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient(hunting): picked mon.noname-f con 0x7f21f809fb80 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient(hunting): picked mon.noname-a con 0x7f21f8138d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f804ca10 con 0x7f21f8138d30 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80a0f50 con 0x7f21f809fb80 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f8133680 con 0x7f21f80a6a80 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient(hunting): authenticate will time out at 1532.489184s 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fcb28700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f21f80a6a80 0x7f21f8132f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:39722/0 (socket says 172.21.15.26:39722) 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fcb28700 1 -- 172.21.15.26:0/4209423318 learned_addr learned my addr 172.21.15.26:0/4209423318 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914611736 0 0) 0x7f21f8133680 con 0x7f21f80a6a80 2024-09-15T03:56:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): my global_id is 204150 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e8003570 con 0x7f21f80a6a80 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339020594 0 0) 0x7f21f80a0f50 con 0x7f21f809fb80 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): my global_id is 204175 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f8133680 con 0x7f21f809fb80 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750265039 0 0) 0x7f21f804ca10 con 0x7f21f8138d30 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient(hunting): my global_id is 194382 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f80a0f50 con 0x7f21f8138d30 2024-09-15T03:56:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353448773 0 0) 0x7f21e8003570 con 0x7f21f80a6a80 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 >> v1:172.21.15.122:6789/0 conn(0x7f21f809fb80 legacy=0x7f21f809ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 >> v1:172.21.15.26:6789/0 conn(0x7f21f8138d30 legacy=0x7f21f813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 monclient: found mon.noname-g 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80a6e50 con 0x7f21f80a6a80 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21ec0017b0 con 0x7f21f80a6a80 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:56:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: dump: 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 5 monclient: authenticate success, global_id 204150 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21ec001a90 con 0x7f21f80a6a80 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 1 -- 172.21.15.26:0/4209423318 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21ec004c30 con 0x7f21f80a6a80 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 4 set_mon_vals no callback set 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:28.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21f77fe700 10 monclient: dump: 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: shutdown 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- 172.21.15.26:0/4209423318 >> v1:172.21.15.122:6790/0 conn(0x7f21f80a6a80 legacy=0x7f21f8132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- 172.21.15.26:0/4209423318 shutdown_connections 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- 172.21.15.26:0/4209423318 >> 172.21.15.26:0/4209423318 conn(0x7f21f806e380 msgr2=0x7f21f80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- 172.21.15.26:0/4209423318 shutdown_connections 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 1 -- 172.21.15.26:0/4209423318 wait complete. 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: build_initial_monmap 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.059+0000 7f21fdb2a700 10 monclient: monmap: 2024-09-15T03:56:28.078 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 Processor -- start 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- start start 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: init 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _add_conns ranks=[1,5,7,4,2,0,3,8,6] 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient(hunting): picked mon.noname-f con 0x7f21f809fb80 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient(hunting): picked mon.noname-h con 0x7f21f80a6a80 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:28.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient(hunting): picked mon.noname-i con 0x7f21f8138d30 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f809b940 con 0x7f21f809fb80 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80a1920 con 0x7f21f80a6a80 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f811cf10 con 0x7f21f8138d30 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient(hunting): authenticate will time out at 1532.492781s 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fcb28700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f21f809fb80 0x7f21f8099b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:43216/0 (socket says 172.21.15.26:43216) 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fcb28700 1 -- 172.21.15.26:0/3023298848 learned_addr learned my addr 172.21.15.26:0/3023298848 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256087920 0 0) 0x7f21f809b940 con 0x7f21f809fb80 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): my global_id is 204184 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21dc003570 con 0x7f21f809fb80 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203023548 0 0) 0x7f21f80a1920 con 0x7f21f80a6a80 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): my global_id is 5639 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f809b940 con 0x7f21f80a6a80 2024-09-15T03:56:28.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720091366 0 0) 0x7f21f811cf10 con 0x7f21f8138d30 2024-09-15T03:56:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): my global_id is 135088 2024-09-15T03:56:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f80a1920 con 0x7f21f8138d30 2024-09-15T03:56:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2818539222 0 0) 0x7f21dc003570 con 0x7f21f809fb80 2024-09-15T03:56:28.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21f811cf10 con 0x7f21f809fb80 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21ec004b70 con 0x7f21f809fb80 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1347105122 0 0) 0x7f21f809b940 con 0x7f21f80a6a80 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21dc003570 con 0x7f21f80a6a80 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21e4001a60 con 0x7f21f80a6a80 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:28.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1972670265 0 0) 0x7f21f80a1920 con 0x7f21f8138d30 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21f809b940 con 0x7f21f8138d30 2024-09-15T03:56:28.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21f0001cc0 con 0x7f21f8138d30 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:28.088 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 409093909 0 0) 0x7f21f811cf10 con 0x7f21f809fb80 2024-09-15T03:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 >> v1:172.21.15.122:6792/0 conn(0x7f21f8138d30 legacy=0x7f21f814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 >> v1:172.21.15.122:6791/0 conn(0x7f21f80a6a80 legacy=0x7f21f809a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 monclient: found mon.f 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f809bb80 con 0x7f21f809fb80 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: _finish_auth 0 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 5 monclient: authenticate success, global_id 204184 2024-09-15T03:56:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _renew_subs 2024-09-15T03:56:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:56:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f809b430 con 0x7f21f809fb80 2024-09-15T03:56:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _renew_subs 2024-09-15T03:56:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f809c1a0 con 0x7f21f809fb80 2024-09-15T03:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21ec0011d0 con 0x7f21f809fb80 2024-09-15T03:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21ec0052c0 con 0x7f21f809fb80 2024-09-15T03:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f6ffd700 4 set_mon_vals no callback set 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 10 monclient: dump: 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:28.093 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f21ec00fc80 con 0x7f21f809fb80 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d80050b0 con 0x7f21f809fb80 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.063+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(535..535 src has 224..535) v4 ==== 3971+0+0 (unknown 44455111 0 0) 0x7f21ec0440c0 con 0x7f21f809fb80 2024-09-15T03:56:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.067+0000 7f21f7fff700 10 monclient: get_auth_request con 0x7f21dc038130 auth_method 0 2024-09-15T03:56:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.071+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f21ec014fe0 con 0x7f21f809fb80 2024-09-15T03:56:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.071+0000 7f21f4ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:28.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.071+0000 7f21f4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:28.097 INFO:tasks.rados.rados.0.smithi122.stdout:2339: writing smithi12223906-5 from 1720453 to 2017038 tid 2 2024-09-15T03:56:28.097 INFO:tasks.rados.rados.0.smithi122.stdout:2340: snap_remove snap 247 2024-09-15T03:56:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.291+0000 7f21fdb2a700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T03:56:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.291+0000 7f21fdb2a700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T03:56:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.291+0000 7f21fdb2a700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T03:56:28.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.291+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 --> v1:172.21.15.122:6789/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f21d8005b70 con 0x7f21f809fb80 2024-09-15T03:56:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.295+0000 7f21f4ff9700 1 -- 172.21.15.26:0/3023298848 <== mon.1 v1:172.21.15.122:6789/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f21ec0149e0 con 0x7f21f809fb80 2024-09-15T03:56:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.295+0000 7f21f4ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T03:56:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.295+0000 7f21f4ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=31,config=1,health=12,kv=8,logm=48} crc {auth=3340815188,config=1228700967,health=2822512867,kv=2519098299,logm=2509426435}) 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=31,config=2,health=10,kv=8,logm=49} crc {auth=725527577,config=3140514009,health=2765110223,kv=2519098299,logm=2405916775}) 2024-09-15T03:56:28.303 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=31,config=2,health=12,kv=8,logm=47} crc {auth=94947094,config=3140514009,health=2822512867,kv=2519098299,logm=651181963}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=30,config=1,health=12,kv=7,logm=50} crc {auth=1488276033,config=1228700967,health=2822512867,kv=777372614,logm=938164566}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=30,config=2,health=11,kv=8,logm=49} crc {auth=3265506383,config=3140514009,health=547981409,kv=2519098299,logm=313590507}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=3894964140,config=3140514009,health=2822512867,kv=2519098299,logm=2251550817}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.304 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=1780117078,config=3140514009,health=2822512867,kv=2519098299,logm=3992815172}) 2024-09-15T03:56:28.305 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.305 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=2925689814,config=3140514009,health=487332443,kv=2519098299,logm=111250963}) 2024-09-15T03:56:28.305 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.297+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=1369934425,config=3140514009,health=1294055001,kv=2519098299,logm=4128494422}) 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 >> v1:172.21.15.122:6812/20895 conn(0x7f21dc038130 legacy=0x7f21dc03a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 10 monclient: shutdown 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 >> v1:172.21.15.122:6789/0 conn(0x7f21f809fb80 legacy=0x7f21f8099b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 shutdown_connections 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 >> 172.21.15.26:0/3023298848 conn(0x7f21f806e380 msgr2=0x7f21f809db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 shutdown_connections 2024-09-15T03:56:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:28.299+0000 7f21fdb2a700 1 -- 172.21.15.26:0/3023298848 wait complete. 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1397498433}) 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2216579789}) 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1000790910}) 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.309 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.301+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4193948961}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=901316601}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1630866249}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3109600697}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=466111944}) 2024-09-15T03:56:28.310 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.305+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1350901803}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3111794417}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=488267730}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1842226428}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.312 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=96114094}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=372398540}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2419159220}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=275170458}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2102873195}) 2024-09-15T03:56:28.313 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2623185649}) 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=800080210}) 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1794518944}) 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.315 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3751472962}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4143294153}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=232371067}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=92588448}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.309+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3772810901}) 2024-09-15T03:56:28.316 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.317 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=412664090}) 2024-09-15T03:56:28.317 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=74346946}) 2024-09-15T03:56:28.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3087524342}) 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1321148383}) 2024-09-15T03:56:28.329 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4134110151}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3646471933}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1631023087}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3969965962}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.330 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2872233223}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1839396312}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.313+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=581744045}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=675113384}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3988772254}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1465750955}) 2024-09-15T03:56:28.331 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1532488698}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2367119408}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3887814523}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3976552939}) 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.332 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848115050}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=132976021}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1612346789}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=467813616}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1010791379}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.333 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3845470847}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1017874163}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3635319070}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2355779286}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1715965637}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.317+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2614203284}) 2024-09-15T03:56:28.334 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1189190953}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=850654513}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2448224291}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4293709399}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.335 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2825099403}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4054268664}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1590589375}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4201260096}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=79667675}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1651563429}) 2024-09-15T03:56:28.336 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=347401042}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3625026600}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1195306167}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4047124577}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=123769847}) 2024-09-15T03:56:28.337 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1950507157}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3645080174}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.321+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2524360043}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3142006391}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4045741834}) 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.338 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2348370226}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2472085425}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=47988685}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3579977884}) 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.339 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.340 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3934931900}) 2024-09-15T03:56:28.340 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.340 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2471362228}) 2024-09-15T03:56:28.340 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2790284895}) 2024-09-15T03:56:28.357 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.357 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2593748498}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3667584680}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2617351382}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=106416481}) 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.358 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=909162406}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2999550969}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=727217671}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4203685989}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.325+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1986355102}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.359 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526896927}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1741608012}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2211293053}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1653909570}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1514478306}) 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.360 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3752087479}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=859141675}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=659119998}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1874621802}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2915514680}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.361 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2417810396}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2277337050}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.329+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=949046686}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2395546731}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1004627461}) 2024-09-15T03:56:28.362 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3613692998}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4190266129}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2374931140}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2121518935}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=596514668}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=497302334}) 2024-09-15T03:56:28.363 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3151614035}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1597039570}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3646708096}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3838587603}) 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.364 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3132144120}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.333+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=661857477}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=554713942}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=611567832}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=639297753}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.365 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2573279163}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2228992286}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=469662137}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3157949404}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4175134680}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4274305922}) 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.366 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2128385340}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2593733055}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=984408178}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2031180375}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3415865854}) 2024-09-15T03:56:28.367 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2376427252}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3711027495}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.337+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3978971452}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1506423851}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2673267075}) 2024-09-15T03:56:28.368 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=622157371}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=135327577}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=378293142}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3744248160}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.369 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2363109915}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=737797114}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1760275553}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2545309089}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608339022}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837981837}) 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.370 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3069237180}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1454454589}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2046906186}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3817271770}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.341+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.371 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=964566369}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1047424399}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3720732555}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2674800449}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1016480244}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2506408737}) 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.372 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2611526441}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2909354856}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1183321321}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3508314929}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4204007926}) 2024-09-15T03:56:28.373 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.345+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3369126816}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1414227566}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3444420216}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=388062741}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2469828624}) 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.374 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3240210081}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4248402883}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2055812893}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=717974559}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248970950}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1481976308}) 2024-09-15T03:56:28.375 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3989393402}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1814709298}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=510418779}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=953681049}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.376 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1464847971}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1472465721}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801202413}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.349+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1318073737}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1158024621}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3081638136}) 2024-09-15T03:56:28.377 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2542951891}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3526435679}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2599929439}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=419503324}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.378 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4024084526}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3191785445}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1556848526}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1304090546}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1221246614}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3356660214}) 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.379 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=943577769}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1717533887}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1537158926}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2295733758}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060877251}) 2024-09-15T03:56:28.380 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.353+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1015080029}) 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2404110985}) 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3372729054}) 2024-09-15T03:56:28.412 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=712330202}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1525169201}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2944788894}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=692212}) 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.413 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2008989136}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=383947760}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1814144471}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=715785400}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3586136460}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.414 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3485364771}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=495751428}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2387321589}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1077560863}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2515917212}) 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.415 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1097299493}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.357+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2612987368}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3185259117}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3992820853}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4051482519}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.416 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=8526966}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2052047387}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=407321359}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3629332027}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1984238698}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3752890375}) 2024-09-15T03:56:28.417 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.361+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=84373445}) 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2235088188}) 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2424474682}) 2024-09-15T03:56:28.418 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2031237823}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3472998107}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870416297}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2276338217}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2494255258}) 2024-09-15T03:56:28.419 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=884415165}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=184448309}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2414153320}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=793517646}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3828412523}) 2024-09-15T03:56:28.420 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=269352945}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773482559}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3840679332}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228231320}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.365+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1448871422}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.421 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2073032185}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=717507960}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2821930391}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2536616403}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.422 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2788474349}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=217639512}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3064097893}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1265284010}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1999092529}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2807651546}) 2024-09-15T03:56:28.423 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=755928985}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=238564465}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3680696491}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3561561467}) 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.424 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4105541905}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4084951667}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=178612560}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.369+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3484519902}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3066254017}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1968887998}) 2024-09-15T03:56:28.425 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4181464034}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1556477903}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=106395179}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4051726521}) 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.426 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2729299760}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=145690705}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1744848186}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=239878892}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1001515363}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4044800092}) 2024-09-15T03:56:28.427 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1916749796}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=735745633}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1385248774}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3255522319}) 2024-09-15T03:56:28.428 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119302671}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.373+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1296589268}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1264675242}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1196542588}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2278949764}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.429 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2449492042}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=488513108}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1678627574}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1707515076}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592172760}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=428122153}) 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.430 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=580161000}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2480589830}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1789942397}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2159811597}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.431 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4045592281}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4000841517}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3772113972}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2119551477}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.377+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1413336584}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=900158474}) 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.432 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=741523840}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1192413394}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3779929297}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3689176419}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.433 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3511391299}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3649524560}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1370330123}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.381+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=145742816}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3965217488}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2934312583}) 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.434 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=987018965}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2081265923}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3425924819}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=513041589}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2839504911}) 2024-09-15T03:56:28.435 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1319262980}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2694414853}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1246472743}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4288389537}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=510304110}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.436 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2362185835}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3218029039}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4205997473}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2747284181}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=554312685}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.385+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2985301809}) 2024-09-15T03:56:28.437 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2675949819}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304045478}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2533299940}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1303954589}) 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.438 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=460871540}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2521085776}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=902958716}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1003160189}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2722046378}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=158416919}) 2024-09-15T03:56:28.439 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=5875472}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=734049622}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1137851926}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=943151604}) 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.440 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=304225796}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4236240781}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=500429102}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.389+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2339107192}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=52,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=9} crc {logm=3744928874,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=3458249459}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=53,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=4061366413,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=1772136737,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=270945163}) 2024-09-15T03:56:28.441 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=54,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5,monmap=1,osd_metadata=6,osd_pg_creating=1,osd_snap=10} crc {logm=1305741020,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1772136737,mon_config_key=733955277,monmap=1228700967,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=270945163}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=54,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=9} crc {logm=1642712916,mdsmap=1531151339,mgr=273786614,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1478363750,mon_config_key=931815924,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=1594841280}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=56,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=4,monmap=2,osd_metadata=6,osd_snap=9} crc {logm=1368161410,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2000714203,mon_config_key=1595646671,monmap=2173287178,osd_metadata=2994937890,osd_snap=4002971978}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=56,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=4,osd_pg_creating=1,osd_snap=8} crc {logm=4268715049,mdsmap=3585761405,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=3865575228,osd_pg_creating=781936494,osd_snap=2021555217}) 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.442 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=56,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=11,mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=7} crc {logm=3557405185,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1044888168,mon_config_key=64957642,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=793879525}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=57,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=4} crc {logm=2103288723,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1044888168,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=781936494,osd_snap=157967165}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=52,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=13} crc {logm=1661743816,mdsmap=1531151339,mgr=73913752,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1133356339,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_snap=1511246189}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=3961832322}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=2283018100}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=2863834032}) 2024-09-15T03:56:28.443 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=100} crc {osd_snap=3034994745}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=100} crc {osd_snap=1765391617}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.393+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=100} crc {osd_snap=2220208133}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2137256224}) 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.444 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=4026790422}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1861411388}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=4251338329}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3777939189}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3722199861}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.445 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2146822277}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1865237629}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=890572371}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2445397109}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=833514169}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.397+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1930480260}) 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.446 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1493545691}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1017648636}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4106727177}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1033631705}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=260065442}) 2024-09-15T03:56:28.447 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=156076912}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2060595373}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1878207159}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1161568400}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3300814632}) 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.448 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2128798395}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2881827948}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1762415517}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1595369598}) 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.449 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3133233280}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1093862214}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3559177675}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.401+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1680413661}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1118183937}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.450 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1916692570}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=111489982}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1561374491}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3035231201}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.451 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3163060377}) 2024-09-15T03:56:28.452 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.452 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.452 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2056911277}) 2024-09-15T03:56:28.452 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.452 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615137151}) 2024-09-15T03:56:28.452 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.405+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1767792229}) 2024-09-15T03:56:28.454 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.454 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.454 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=56} crc {osdmap=3647899250}) 2024-09-15T03:56:28.454 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=55} crc {osdmap=79266501}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=54} crc {osdmap=1571953049}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=53} crc {osdmap=2569663586}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.455 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=55} crc {osdmap=731746696}) 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=53} crc {osdmap=4027200841}) 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=55} crc {osdmap=3499700887}) 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=57} crc {osdmap=2323926429}) 2024-09-15T03:56:28.456 INFO:tasks.ceph.mon.i.smithi122.stderr:2024-09-15T03:56:28.409+0000 7fa1eb042700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=57} crc {osdmap=3502337508}) 2024-09-15T03:56:29.046 INFO:tasks.rados.rados.0.smithi122.stdout:2340: done (5 left) 2024-09-15T03:56:29.046 INFO:tasks.rados.rados.0.smithi122.stdout:2341: read oid 42 snap -1 2024-09-15T03:56:29.046 INFO:tasks.rados.rados.0.smithi122.stdout:2341: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:56:29.050 INFO:tasks.rados.rados.0.smithi122.stdout:2342: read oid 28 snap -1 2024-09-15T03:56:29.050 INFO:tasks.rados.rados.0.smithi122.stdout:2342: expect (ObjNum 770 snap 244 seq_num 770) 2024-09-15T03:56:29.050 INFO:tasks.rados.rados.0.smithi122.stdout:2343: write (excl) oid 21 current snap is 271 2024-09-15T03:56:29.050 INFO:tasks.rados.rados.0.smithi122.stdout:2343: seq_num 835 ranges {433845=715392,1662582=371263} 2024-09-15T03:56:29.054 INFO:tasks.rados.rados.0.smithi122.stdout:2343: writing smithi12223906-21 from 433845 to 1149237 tid 1 2024-09-15T03:56:29.062 INFO:tasks.rados.rados.0.smithi122.stdout:2343: writing smithi12223906-21 from 1662582 to 2033845 tid 2 2024-09-15T03:56:29.065 INFO:tasks.rados.rados.0.smithi122.stdout:2336: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:56:29.199 INFO:tasks.rados.rados.0.smithi122.stdout:2335: finishing copy_from to smithi12223906-20 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:2335: got expected ENOENT (src dne) 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:2335: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:2337: finishing rollback tid 0 to smithi12223906-2 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:2337: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1039 (ObjNum 289 snap 0 seq_num 1843072368) dirty dne 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:2338: finishing write tid 1 to smithi12223906-45 2024-09-15T03:56:29.200 INFO:tasks.rados.rados.0.smithi122.stdout:2338: finishing write tid 2 to smithi12223906-45 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2338: finishing write tid 3 to smithi12223906-45 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2338: finishing write tid 4 to smithi12223906-45 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 827 (ObjNum 833 snap 271 seq_num 833) dirty exists 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2338: left oid 45 (ObjNum 833 snap 271 seq_num 833) 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2339: finishing write tid 1 to smithi12223906-5 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2339: finishing write tid 2 to smithi12223906-5 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2339: finishing write tid 3 to smithi12223906-5 2024-09-15T03:56:29.201 INFO:tasks.rados.rados.0.smithi122.stdout:2339: finishing write tid 4 to smithi12223906-5 2024-09-15T03:56:29.202 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 874 (ObjNum 834 snap 271 seq_num 834) dirty exists 2024-09-15T03:56:29.202 INFO:tasks.rados.rados.0.smithi122.stdout:2339: left oid 5 (ObjNum 834 snap 271 seq_num 834) 2024-09-15T03:56:29.202 INFO:tasks.rados.rados.0.smithi122.stdout:2342: expect (ObjNum 770 snap 244 seq_num 770) 2024-09-15T03:56:29.298 INFO:tasks.rados.rados.0.smithi122.stdout:2341: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:56:29.326 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:56:29.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:56:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient: get_monmap_and_config 2024-09-15T03:56:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient: build_initial_monmap 2024-09-15T03:56:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient: monmap: 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:29.437786+0000 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:29.437786+0000 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 1 Processor -- start 2024-09-15T03:56:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 1 -- start start 2024-09-15T03:56:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient: init 2024-09-15T03:56:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient: _add_conns ranks=[7,3,2,8,5,6,0,1,4] 2024-09-15T03:56:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient(hunting): picked mon.noname-i con 0x7fcea00a48d0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient(hunting): picked mon.noname-g con 0x7fcea006b6a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient(hunting): picked mon.noname-b con 0x7fcea0069380 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea004ca10 con 0x7fcea0069380 2024-09-15T03:56:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea0068d10 con 0x7fcea006b6a0 2024-09-15T03:56:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea00689a0 con 0x7fcea00a48d0 2024-09-15T03:56:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea86b2700 10 monclient(hunting): authenticate will time out at 1533.867458s 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea6c4f700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fcea0069380 0x7fcea013b2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37752/0 (socket says 172.21.15.26:37752) 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.435+0000 7fcea6c4f700 1 -- 172.21.15.26:0/2198446148 learned_addr learned my addr 172.21.15.26:0/2198446148 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180345068 0 0) 0x7fcea004ca10 con 0x7fcea0069380 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): my global_id is 204158 2024-09-15T03:56:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8c003570 con 0x7fcea0069380 2024-09-15T03:56:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758582264 0 0) 0x7fcea00689a0 con 0x7fcea00a48d0 2024-09-15T03:56:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): my global_id is 135097 2024-09-15T03:56:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea004ca10 con 0x7fcea00a48d0 2024-09-15T03:56:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888866589 0 0) 0x7fcea0068d10 con 0x7fcea006b6a0 2024-09-15T03:56:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient(hunting): my global_id is 204159 2024-09-15T03:56:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea00689a0 con 0x7fcea006b6a0 2024-09-15T03:56:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413224315 0 0) 0x7fcea004ca10 con 0x7fcea00a48d0 2024-09-15T03:56:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 >> v1:172.21.15.122:6790/0 conn(0x7fcea006b6a0 legacy=0x7fcea0137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 >> v1:172.21.15.26:6790/0 conn(0x7fcea0069380 legacy=0x7fcea013b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: _finish_hunting 0 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 monclient: found mon.noname-i 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea00a4ca0 con 0x7fcea00a48d0 2024-09-15T03:56:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: _finish_auth 0 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce90001790 con 0x7fcea00a48d0 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 5 monclient: authenticate success, global_id 135097 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T03:56:29.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: dump: 2024-09-15T03:56:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce90001a40 con 0x7fcea00a48d0 2024-09-15T03:56:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 1 -- 172.21.15.26:0/2198446148 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce90002240 con 0x7fcea00a48d0 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 4 set_mon_vals no callback set 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea544c700 10 monclient: dump: 2024-09-15T03:56:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.466 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 10 monclient: shutdown 2024-09-15T03:56:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 -- 172.21.15.26:0/2198446148 >> v1:172.21.15.122:6792/0 conn(0x7fcea00a48d0 legacy=0x7fcea006db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 -- 172.21.15.26:0/2198446148 shutdown_connections 2024-09-15T03:56:29.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 -- 172.21.15.26:0/2198446148 >> 172.21.15.26:0/2198446148 conn(0x7fcea0063a50 msgr2=0x7fcea00a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:29.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 -- 172.21.15.26:0/2198446148 shutdown_connections 2024-09-15T03:56:29.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 -- 172.21.15.26:0/2198446148 wait complete. 2024-09-15T03:56:29.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 10 monclient: build_initial_monmap 2024-09-15T03:56:29.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 10 monclient: monmap: 2024-09-15T03:56:29.467 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:29.468 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:29.468 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:29.468 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:29.468 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:29.468 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:29.468 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:29.469 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:29.469 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:29.469 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:29.469 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:29.469 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 Processor -- start 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 1 -- start start 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.439+0000 7fcea86b2700 10 monclient: init 2024-09-15T03:56:29.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient: _add_conns ranks=[6,0,5,7,3,2,1,4,8] 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient(hunting): picked mon.noname-d con 0x7fcea0069380 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient(hunting): picked mon.noname-a con 0x7fcea006b6a0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient(hunting): picked mon.noname-h con 0x7fcea00a48d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea006e320 con 0x7fcea006b6a0 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea0122660 con 0x7fcea00a48d0 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea0122c00 con 0x7fcea0069380 2024-09-15T03:56:29.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient(hunting): authenticate will time out at 1533.872419s 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea644e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fcea0069380 0x7fcea011f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48462/0 (socket says 172.21.15.26:48462) 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea644e700 1 -- 172.21.15.26:0/1404217818 learned_addr learned my addr 172.21.15.26:0/1404217818 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597976373 0 0) 0x7fcea0122c00 con 0x7fcea0069380 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): my global_id is 194415 2024-09-15T03:56:29.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce84003570 con 0x7fcea0069380 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970815878 0 0) 0x7fcea006e320 con 0x7fcea006b6a0 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): my global_id is 194391 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea0122c00 con 0x7fcea006b6a0 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582569314 0 0) 0x7fce8c003570 con 0x7fcea00a48d0 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): my global_id is 5648 2024-09-15T03:56:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea006e320 con 0x7fcea00a48d0 2024-09-15T03:56:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 791079301 0 0) 0x7fce84003570 con 0x7fcea0069380 2024-09-15T03:56:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce8c003570 con 0x7fcea0069380 2024-09-15T03:56:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce90001ef0 con 0x7fcea0069380 2024-09-15T03:56:29.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): dump: 2024-09-15T03:56:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.475 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.476 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3403896817 0 0) 0x7fcea006e320 con 0x7fcea00a48d0 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fce84003570 con 0x7fcea00a48d0 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce98001dc0 con 0x7fcea00a48d0 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): dump: 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.478 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2378375073 0 0) 0x7fcea0122c00 con 0x7fcea006b6a0 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcea006e320 con 0x7fcea006b6a0 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce9c0019e0 con 0x7fcea006b6a0 2024-09-15T03:56:29.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient(hunting): dump: 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.480 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 868234886 0 0) 0x7fce84003570 con 0x7fcea00a48d0 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 >> v1:172.21.15.26:6792/0 conn(0x7fcea0069380 legacy=0x7fcea011f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 >> v1:172.21.15.26:6789/0 conn(0x7fcea006b6a0 legacy=0x7fcea011fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 monclient: found mon.h 2024-09-15T03:56:29.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0121a30 con 0x7fcea00a48d0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 5 monclient: authenticate success, global_id 5648 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient: _renew_subs 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea0123b40 con 0x7fcea00a48d0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient: _renew_subs 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea0120e60 con 0x7fcea00a48d0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fce98004bb0 con 0x7fcea00a48d0 2024-09-15T03:56:29.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fce980050e0 con 0x7fcea00a48d0 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce97fff700 4 set_mon_vals no callback set 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.443+0000 7fce967fc700 10 monclient: dump: 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.483 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fce98005300 con 0x7fcea00a48d0 2024-09-15T03:56:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fcea86b2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fcea86b2700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fcea86b2700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:56:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce880050b0 con 0x7fcea00a48d0 2024-09-15T03:56:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(536..536 src has 224..536) v4 ==== 3987+0+0 (unknown 2744181624 0 0) 0x7fce98044040 con 0x7fcea00a48d0 2024-09-15T03:56:29.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.447+0000 7fcea644e700 10 monclient: get_auth_request con 0x7fce84038080 auth_method 0 2024-09-15T03:56:29.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.455+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fce98005980 con 0x7fcea00a48d0 2024-09-15T03:56:29.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.455+0000 7fce967fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:29.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.455+0000 7fce967fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2343: finishing write tid 1 to smithi12223906-21 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2343: finishing write tid 2 to smithi12223906-21 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2343: finishing write tid 3 to smithi12223906-21 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2343: finishing write tid 4 to smithi12223906-21 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1039 (ObjNum 835 snap 271 seq_num 835) dirty exists 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2343: left oid 21 (ObjNum 835 snap 271 seq_num 835) 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2335: done (7 left) 2024-09-15T03:56:29.489 INFO:tasks.rados.rados.0.smithi122.stdout:2336: done (6 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2337: done (5 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2338: done (4 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2339: done (3 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2341: done (2 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2342: done (1 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2343: done (0 left) 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2344: write (excl) oid 7 current snap is 271 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2344: seq_num 836 ranges {450652=790641,1721064=329588} 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2344: writing smithi12223906-7 from 450652 to 1241293 tid 1 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2344: writing smithi12223906-7 from 1721064 to 2050652 tid 2 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2345: write (excl) oid 44 current snap is 271 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2345: seq_num 837 ranges {467459=582243,1495899=404945} 2024-09-15T03:56:29.490 INFO:tasks.rados.rados.0.smithi122.stdout:2345: writing smithi12223906-44 from 467459 to 1049702 tid 1 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2345: writing smithi12223906-44 from 1495899 to 1900844 tid 2 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2346: read oid 23 snap 259 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2346: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2347: read oid 34 snap 220 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2347: expect deleted 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2348: delete oid 40 current snap is 271 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2344: finishing write tid 1 to smithi12223906-7 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2344: finishing write tid 2 to smithi12223906-7 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2344: finishing write tid 3 to smithi12223906-7 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2344: finishing write tid 4 to smithi12223906-7 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 947 (ObjNum 836 snap 271 seq_num 836) dirty exists 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2344: left oid 7 (ObjNum 836 snap 271 seq_num 836) 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2344: done (4 left) 2024-09-15T03:56:29.491 INFO:tasks.rados.rados.0.smithi122.stdout:2347: done (3 left) 2024-09-15T03:56:29.492 INFO:tasks.rados.rados.0.smithi122.stdout:2348: done (2 left) 2024-09-15T03:56:29.492 INFO:tasks.rados.rados.0.smithi122.stdout:2349: write oid 20 current snap is 271 2024-09-15T03:56:29.492 INFO:tasks.rados.rados.0.smithi122.stdout:2349: seq_num 839 ranges {501073=732741,1896510=204563} 2024-09-15T03:56:29.500 INFO:tasks.rados.rados.0.smithi122.stdout:2349: writing smithi12223906-20 from 501073 to 1233814 tid 1 2024-09-15T03:56:29.508 INFO:tasks.rados.rados.0.smithi122.stdout:2349: writing smithi12223906-20 from 1896510 to 2101073 tid 2 2024-09-15T03:56:29.508 INFO:tasks.rados.rados.0.smithi122.stdout:2350: read oid 14 snap -1 2024-09-15T03:56:29.508 INFO:tasks.rados.rados.0.smithi122.stdout:2350: expect (ObjNum 760 snap 235 seq_num 760) 2024-09-15T03:56:29.508 INFO:tasks.rados.rados.0.smithi122.stdout:2351: read oid 3 snap -1 2024-09-15T03:56:29.508 INFO:tasks.rados.rados.0.smithi122.stdout:2351: expect (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:56:29.508 INFO:tasks.rados.rados.0.smithi122.stdout:2352: copy_from oid 36 from oid 22 current snap is 271 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2345: finishing write tid 1 to smithi12223906-44 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2345: finishing write tid 2 to smithi12223906-44 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2353: write (excl) oid 24 current snap is 271 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2345: finishing write tid 3 to smithi12223906-44 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2345: finishing write tid 4 to smithi12223906-44 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1562 (ObjNum 837 snap 271 seq_num 837) dirty exists 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2345: left oid 44 (ObjNum 837 snap 271 seq_num 837) 2024-09-15T03:56:29.509 INFO:tasks.rados.rados.0.smithi122.stdout:2346: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:56:29.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:56:29.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:29.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T03:56:29.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fce880058e0 con 0x7fcea00a48d0 2024-09-15T03:56:29.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fce967fc700 1 -- 172.21.15.26:0/1404217818 <== mon.5 v1:172.21.15.122:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 2336305134) 0x7fce98014820 con 0x7fcea00a48d0 2024-09-15T03:56:29.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fce967fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:29.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fce967fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:29.696 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"i","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:56:29.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 >> v1:172.21.15.122:6812/20895 conn(0x7fce84038080 legacy=0x7fce8403a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 10 monclient: shutdown 2024-09-15T03:56:29.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:29.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 >> v1:172.21.15.122:6791/0 conn(0x7fcea00a48d0 legacy=0x7fcea0149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 shutdown_connections 2024-09-15T03:56:29.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.691+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 >> 172.21.15.26:0/1404217818 conn(0x7fcea0063a50 msgr2=0x7fcea0099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:29.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.695+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 shutdown_connections 2024-09-15T03:56:29.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.695+0000 7fcea86b2700 1 -- 172.21.15.26:0/1404217818 wait complete. 2024-09-15T03:56:29.710 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:56:29.710 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:56:29.710 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:56:29.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:56:29.781 INFO:tasks.rados.rados.0.smithi122.stdout:2349: finishing write tid 1 to smithi12223906-20 2024-09-15T03:56:29.781 INFO:tasks.rados.rados.0.smithi122.stdout:2349: finishing write tid 2 to smithi12223906-20 2024-09-15T03:56:29.782 INFO:tasks.rados.rados.0.smithi122.stdout:2349: finishing write tid 3 to smithi12223906-20 2024-09-15T03:56:29.782 INFO:tasks.rados.rados.0.smithi122.stdout:2349: finishing write tid 4 to smithi12223906-20 2024-09-15T03:56:29.782 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 670 (ObjNum 839 snap 271 seq_num 839) dirty exists 2024-09-15T03:56:29.782 INFO:tasks.rados.rados.0.smithi122.stdout:2349: left oid 20 (ObjNum 839 snap 271 seq_num 839) 2024-09-15T03:56:29.782 INFO:tasks.rados.rados.0.smithi122.stdout:2350: expect (ObjNum 760 snap 235 seq_num 760) 2024-09-15T03:56:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient: get_monmap_and_config 2024-09-15T03:56:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient: build_initial_monmap 2024-09-15T03:56:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient: monmap: 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:29.813854+0000 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:29.813854+0000 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:29.818 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 1 Processor -- start 2024-09-15T03:56:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 1 -- start start 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient: init 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient: _add_conns ranks=[5,1,6,2,0,7,4,3,8] 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient(hunting): picked mon.noname-h con 0x7fba5806e530 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient(hunting): picked mon.noname-f con 0x7fba58099240 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient(hunting): picked mon.noname-d con 0x7fba58138d70 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5804ca10 con 0x7fba58099240 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5809e310 con 0x7fba5806e530 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba58133730 con 0x7fba58138d70 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba5759e700 10 monclient(hunting): authenticate will time out at 1534.243341s 2024-09-15T03:56:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba56d9d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fba58138d70 0x7fba5813b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48472/0 (socket says 172.21.15.26:48472) 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.811+0000 7fba56d9d700 1 -- 172.21.15.26:0/2417228496 learned_addr learned my addr 172.21.15.26:0/2417228496 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243565258 0 0) 0x7fba58133730 con 0x7fba58138d70 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): my global_id is 194424 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba40003570 con 0x7fba58138d70 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785516369 0 0) 0x7fba5809e310 con 0x7fba5806e530 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): my global_id is 5657 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba58133730 con 0x7fba5806e530 2024-09-15T03:56:29.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906042454 0 0) 0x7fba5804ca10 con 0x7fba58099240 2024-09-15T03:56:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient(hunting): my global_id is 204193 2024-09-15T03:56:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba5809e310 con 0x7fba58099240 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043093207 0 0) 0x7fba40003570 con 0x7fba58138d70 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 >> v1:172.21.15.122:6791/0 conn(0x7fba5806e530 legacy=0x7fba58133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 >> v1:172.21.15.122:6789/0 conn(0x7fba58099240 legacy=0x7fba58099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: _finish_hunting 0 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 monclient: found mon.noname-d 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58097b00 con 0x7fba58138d70 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: _finish_auth 0 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba500017b0 con 0x7fba58138d70 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 5 monclient: authenticate success, global_id 194424 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: dump: 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba50001a10 con 0x7fba58138d70 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 1 -- 172.21.15.26:0/2417228496 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba50004c30 con 0x7fba58138d70 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 4 set_mon_vals no callback set 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5559a700 10 monclient: dump: 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 10 monclient: shutdown 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 -- 172.21.15.26:0/2417228496 >> v1:172.21.15.26:6792/0 conn(0x7fba58138d70 legacy=0x7fba5813b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 -- 172.21.15.26:0/2417228496 shutdown_connections 2024-09-15T03:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 -- 172.21.15.26:0/2417228496 >> 172.21.15.26:0/2417228496 conn(0x7fba58070de0 msgr2=0x7fba580711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 -- 172.21.15.26:0/2417228496 shutdown_connections 2024-09-15T03:56:29.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 -- 172.21.15.26:0/2417228496 wait complete. 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 10 monclient: build_initial_monmap 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 10 monclient: monmap: 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:29.834 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 Processor -- start 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 1 -- start start 2024-09-15T03:56:29.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.815+0000 7fba5759e700 10 monclient: init 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient: _add_conns ranks=[3,8,0,7,5,6,1,2,4] 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient(hunting): picked mon.noname-g con 0x7fba5806e530 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient(hunting): picked mon.noname-e con 0x7fba58099240 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient(hunting): picked mon.noname-a con 0x7fba58138d70 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5809ea60 con 0x7fba58138d70 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba581388c0 con 0x7fba5806e530 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba581300e0 con 0x7fba58099240 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient(hunting): authenticate will time out at 1534.248233s 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba56d9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba58138d70 0x7fba5814a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52774/0 (socket says 172.21.15.26:52774) 2024-09-15T03:56:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba55d9b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fba58099240 0x7fba58146ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50092/0 (socket says 172.21.15.26:50092) 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba56d9d700 1 -- 172.21.15.26:0/2732199686 learned_addr learned my addr 172.21.15.26:0/2732199686 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404096252 0 0) 0x7fba5809ea60 con 0x7fba58138d70 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): my global_id is 194400 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba38003570 con 0x7fba58138d70 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951583955 0 0) 0x7fba581300e0 con 0x7fba58099240 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): my global_id is 135080 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba5809ea60 con 0x7fba58099240 2024-09-15T03:56:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482085630 0 0) 0x7fba581388c0 con 0x7fba5806e530 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): my global_id is 204168 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba581300e0 con 0x7fba5806e530 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1245269237 0 0) 0x7fba38003570 con 0x7fba58138d70 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba581388c0 con 0x7fba58138d70 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba50001770 con 0x7fba58138d70 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): dump: 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.839 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2275243410 0 0) 0x7fba5809ea60 con 0x7fba58099240 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba38003570 con 0x7fba58099240 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba4c001a60 con 0x7fba58099240 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): dump: 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.840 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2375337013 0 0) 0x7fba581300e0 con 0x7fba5806e530 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba5809ea60 con 0x7fba5806e530 2024-09-15T03:56:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba48001610 con 0x7fba5806e530 2024-09-15T03:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): dump: 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1416446643 0 0) 0x7fba581388c0 con 0x7fba58138d70 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.26:6793/0 conn(0x7fba58099240 legacy=0x7fba58146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.122:6790/0 conn(0x7fba5806e530 legacy=0x7fba581381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 monclient: found mon.a 2024-09-15T03:56:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5809e790 con 0x7fba58138d70 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 5 monclient: authenticate success, global_id 194400 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient: _renew_subs 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba58132360 con 0x7fba58138d70 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient: _renew_subs 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fba58131610 con 0x7fba58138d70 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fba50004e60 con 0x7fba58138d70 2024-09-15T03:56:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba50004880 con 0x7fba58138d70 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient: dump: 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba54d99700 4 set_mon_vals no callback set 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fba5000fbf0 con 0x7fba58138d70 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fba3c006590 con 0x7fba3c002840 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(536..536 src has 224..536) v4 ==== 3987+0+0 (unknown 2744181624 0 0) 0x7fba500441e0 con 0x7fba58138d70 2024-09-15T03:56:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276701565 0 0) 0x7fba3c006590 con 0x7fba3c002840 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 10 monclient(hunting): my global_id is 194409 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.819+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba581388c0 con 0x7fba3c002840 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba55d9b700 10 monclient: get_auth_request con 0x7fba38038040 auth_method 0 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1148183698 0 0) 0x7fba581388c0 con 0x7fba3c002840 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba3c006590 con 0x7fba3c002840 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba48002590 con 0x7fba3c002840 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 10 monclient: dump: 2024-09-15T03:56:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 461999477 0 0) 0x7fba3c006590 con 0x7fba3c002840 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba3c00a130 con 0x7fba3c002840 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fba480048d0 con 0x7fba3c002840 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.26:6789/0 conn(0x7fba3c002840 legacy=0x7fba3c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.823+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.26:6789/0 conn(0x7fba3c002840 legacy=0x7fba3c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.831+0000 7fba5759e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.831+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fba3c007520 con 0x7fba3c0068d0 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292999633 0 0) 0x7fba3c007520 con 0x7fba3c0068d0 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient(hunting): my global_id is 194418 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3c006590 con 0x7fba3c0068d0 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2966618752 0 0) 0x7fba3c006590 con 0x7fba3c0068d0 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fba3c007520 con 0x7fba3c0068d0 2024-09-15T03:56:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fba48003090 con 0x7fba3c0068d0 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient: dump: 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2888378864 0 0) 0x7fba3c007520 con 0x7fba3c0068d0 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fba3c008760 con 0x7fba3c0068d0 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1547005289) 0x7fba48004530 con 0x7fba3c0068d0 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:29.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.26:6789/0 conn(0x7fba3c0068d0 legacy=0x7fba3c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba467fc700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.26:6789/0 conn(0x7fba3c0068d0 legacy=0x7fba3c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:29.856 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:29.857 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:29.858 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:29.859 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:29.860 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:29.861 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.862 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.864 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.865 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:29.866 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.867 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.868 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:29.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:29.871 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.872 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:29.873 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:29.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:29.876 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:29.876 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:29.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:29.876 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:29.876 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.122:6812/20895 conn(0x7fba38038040 legacy=0x7fba3803a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 10 monclient: shutdown 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 >> v1:172.21.15.26:6789/0 conn(0x7fba58138d70 legacy=0x7fba5814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba56d9d700 1 -- 172.21.15.26:0/2732199686 reap_dead start 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba56d9d700 1 -- 172.21.15.26:0/2732199686 reap_dead start 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 shutdown_connections 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 >> 172.21.15.26:0/2732199686 conn(0x7fba58070de0 msgr2=0x7fba580a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 shutdown_connections 2024-09-15T03:56:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.835+0000 7fba5759e700 1 -- 172.21.15.26:0/2732199686 wait complete. 2024-09-15T03:56:29.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:29.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:56:29.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:56:29.906 INFO:tasks.rados.rados.0.smithi122.stdout:2351: expect (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient: get_monmap_and_config 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient: build_initial_monmap 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient: monmap: 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:29.992364+0000 2024-09-15T03:56:29.995 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:29.992364+0000 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:29.996 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:29.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 1 Processor -- start 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 1 -- start start 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient: init 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient: _add_conns ranks=[2,5,4,6,1,7,8,0,3] 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient(hunting): picked mon.noname-b con 0x7f02380990c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient(hunting): picked mon.noname-h con 0x7f02380a1310 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient(hunting): picked mon.noname-c con 0x7f02381335a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023804ca10 con 0x7f02380990c0 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02380a1b20 con 0x7f02381335a0 2024-09-15T03:56:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02380a10b0 con 0x7f02380a1310 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f023e03f700 10 monclient(hunting): authenticate will time out at 1534.422767s 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02377fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f02380990c0 0x7f0238132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37760/0 (socket says 172.21.15.26:37760) 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02377fe700 1 -- 172.21.15.26:0/3657861883 learned_addr learned my addr 172.21.15.26:0/3657861883 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918780077 0 0) 0x7f023804ca10 con 0x7f02380990c0 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): my global_id is 204167 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0220003570 con 0x7f02380990c0 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602085099 0 0) 0x7f02380a1b20 con 0x7f02381335a0 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): my global_id is 204205 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f023804ca10 con 0x7f02381335a0 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184063778 0 0) 0x7f02380a10b0 con 0x7f02380a1310 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient(hunting): my global_id is 5666 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02380a1b20 con 0x7f02380a1310 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 597191374 0 0) 0x7f0220003570 con 0x7f02380990c0 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 >> v1:172.21.15.122:6791/0 conn(0x7f02380a1310 legacy=0x7f0238099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 >> v1:172.21.15.26:6791/0 conn(0x7f02381335a0 legacy=0x7f023813afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 monclient: found mon.noname-b 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238099ba0 con 0x7f02380990c0 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.991+0000 7f02367fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 5 monclient: authenticate success, global_id 204167 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0228001b30 con 0x7f02380990c0 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: dump: 2024-09-15T03:56:30.003 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0228001e10 con 0x7f02380990c0 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 1 -- 172.21.15.26:0/3657861883 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0228004fb0 con 0x7f02380990c0 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 4 set_mon_vals no callback set 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02367fc700 10 monclient: dump: 2024-09-15T03:56:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: shutdown 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- 172.21.15.26:0/3657861883 >> v1:172.21.15.26:6790/0 conn(0x7f02380990c0 legacy=0x7f0238132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- 172.21.15.26:0/3657861883 shutdown_connections 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- 172.21.15.26:0/3657861883 >> 172.21.15.26:0/3657861883 conn(0x7f023806e370 msgr2=0x7f02380a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- 172.21.15.26:0/3657861883 shutdown_connections 2024-09-15T03:56:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- 172.21.15.26:0/3657861883 wait complete. 2024-09-15T03:56:30.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: monmap: 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.013 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 Processor -- start 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- start start 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: init 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient: _add_conns ranks=[2,1,3,7,5,0,8,6,4] 2024-09-15T03:56:30.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient(hunting): picked mon.noname-b con 0x7f02380990c0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient(hunting): picked mon.noname-f con 0x7f02380a1310 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient(hunting): picked mon.noname-g con 0x7f02381335a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023809a010 con 0x7f02380a1310 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0238121f70 con 0x7f02380990c0 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0238142be0 con 0x7f02381335a0 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f023e03f700 10 monclient(hunting): authenticate will time out at 1534.427769s 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02377fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f02380990c0 0x7f0238131b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37764/0 (socket says 172.21.15.26:37764) 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.995+0000 7f02377fe700 1 -- 172.21.15.26:0/2968531006 learned_addr learned my addr 172.21.15.26:0/2968531006 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473942185 0 0) 0x7f0238121f70 con 0x7f02380990c0 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): my global_id is 204176 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0224003570 con 0x7f02380990c0 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374069834 0 0) 0x7f023809a010 con 0x7f02380a1310 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): my global_id is 204202 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0238121f70 con 0x7f02380a1310 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008577377 0 0) 0x7f0238142be0 con 0x7f02381335a0 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): my global_id is 204177 2024-09-15T03:56:30.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f023809a010 con 0x7f02381335a0 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1970153201 0 0) 0x7f0224003570 con 0x7f02380990c0 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0238142be0 con 0x7f02380990c0 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0228004ef0 con 0x7f02380990c0 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): dump: 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.017 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1574236048 0 0) 0x7f0238121f70 con 0x7f02380a1310 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0224003570 con 0x7f02380a1310 2024-09-15T03:56:30.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f022c001a60 con 0x7f02380a1310 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): dump: 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.019 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1668037440 0 0) 0x7f023809a010 con 0x7f02381335a0 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0238121f70 con 0x7f02381335a0 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0230001880 con 0x7f02381335a0 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): dump: 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1453426789 0 0) 0x7f0238142be0 con 0x7f02380990c0 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.122:6790/0 conn(0x7f02381335a0 legacy=0x7f023814a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.122:6789/0 conn(0x7f02380a1310 legacy=0x7f0238147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 monclient: found mon.b 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238120160 con 0x7f02380990c0 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 5 monclient: authenticate success, global_id 204176 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 10 monclient: _renew_subs 2024-09-15T03:56:30.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0238121c90 con 0x7f02380990c0 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 10 monclient: _renew_subs 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0238132250 con 0x7f02380990c0 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0228005280 con 0x7f02380990c0 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0228001550 con 0x7f02380990c0 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f0235ffb700 4 set_mon_vals no callback set 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient: dump: 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.025 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.026 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0228010010 con 0x7f02380990c0 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(536..536 src has 224..536) v4 ==== 3987+0+0 (unknown 2744181624 0 0) 0x7f0228044540 con 0x7f02380990c0 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f0236ffd700 10 monclient: get_auth_request con 0x7f0224038060 auth_method 0 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0218006590 con 0x7f0218002840 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066193749 0 0) 0x7f0218006590 con 0x7f0218002840 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 10 monclient(hunting): my global_id is 204185 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:29.999+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0238142be0 con 0x7f0218002840 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2082961648 0 0) 0x7f0238142be0 con 0x7f0218002840 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0218006590 con 0x7f0218002840 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0230002050 con 0x7f0218002840 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 10 monclient: dump: 2024-09-15T03:56:30.028 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.029 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.029 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.029 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.029 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.029 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3607752133 0 0) 0x7f0218006590 con 0x7f0218002840 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f021800a130 con 0x7f0218002840 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0230003b60 con 0x7f0218002840 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.26:6790/0 conn(0x7f0218002840 legacy=0x7f0218004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.003+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.26:6790/0 conn(0x7f0218002840 legacy=0x7f0218004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0218009640 con 0x7f02180090b0 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359042637 0 0) 0x7f0218009640 con 0x7f02180090b0 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient(hunting): my global_id is 204194 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0218006590 con 0x7f02180090b0 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2588783096 0 0) 0x7f0218006590 con 0x7f02180090b0 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0218009640 con 0x7f02180090b0 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f02300025b0 con 0x7f02180090b0 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:30.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient: dump: 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.034 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1990688744 0 0) 0x7f0218009640 con 0x7f02180090b0 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f02180098f0 con 0x7f02180090b0 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3533462416) 0x7f02300027a0 con 0x7f02180090b0 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.26:6790/0 conn(0x7f02180090b0 legacy=0x7f0218008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f021f7fe700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.26:6790/0 conn(0x7f02180090b0 legacy=0x7f0218008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:30.036 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:30.037 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:30.038 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:30.039 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:30.040 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:30.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.042 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.043 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.045 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:30.046 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.048 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.050 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.051 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.052 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.053 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.054 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.055 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:30.056 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.122:6812/20895 conn(0x7f0224038060 legacy=0x7f022403a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 10 monclient: shutdown 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 >> v1:172.21.15.26:6790/0 conn(0x7f02380990c0 legacy=0x7f0238131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f0237fff700 1 -- 172.21.15.26:0/2968531006 reap_dead start 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 shutdown_connections 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 >> 172.21.15.26:0/2968531006 conn(0x7f023806e370 msgr2=0x7f02380a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 shutdown_connections 2024-09-15T03:56:30.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.011+0000 7f023e03f700 1 -- 172.21.15.26:0/2968531006 wait complete. 2024-09-15T03:56:30.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:30.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:56:30.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.163+0000 7f815569f700 10 monclient: get_monmap_and_config 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.163+0000 7f815569f700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.163+0000 7f815569f700 10 monclient: monmap: 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:30.166571+0000 2024-09-15T03:56:30.169 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:30.166571+0000 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.170 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.163+0000 7f815569f700 1 Processor -- start 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.163+0000 7f815569f700 1 -- start start 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.163+0000 7f815569f700 10 monclient: init 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient: _add_conns ranks=[6,5,2,1,8,3,7,4,0] 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient(hunting): picked mon.noname-d con 0x7f81500a3890 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient(hunting): picked mon.noname-h con 0x7f8150099360 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient(hunting): picked mon.noname-b con 0x7f8150138ee0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815004ca10 con 0x7f8150138ee0 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815006e6a0 con 0x7f8150099360 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814effd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f81500a3890 0x7f81501331b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48476/0 (socket says 172.21.15.26:48476) 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815009e190 con 0x7f81500a3890 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814f7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f8150138ee0 0x7f815013b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37788/0 (socket says 172.21.15.26:37788) 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814effd700 1 -- 172.21.15.26:0/1454956721 learned_addr learned my addr 172.21.15.26:0/1454956721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient(hunting): authenticate will time out at 1534.596603s 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049457746 0 0) 0x7f815004ca10 con 0x7f8150138ee0 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): my global_id is 204203 2024-09-15T03:56:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f813c003570 con 0x7f8150138ee0 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322384273 0 0) 0x7f815009e190 con 0x7f81500a3890 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): my global_id is 194433 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815004ca10 con 0x7f81500a3890 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137462590 0 0) 0x7f815006e6a0 con 0x7f8150099360 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient(hunting): my global_id is 5675 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815009e190 con 0x7f8150099360 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454638548 0 0) 0x7f815004ca10 con 0x7f81500a3890 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 >> v1:172.21.15.122:6791/0 conn(0x7f8150099360 legacy=0x7f8150099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 >> v1:172.21.15.26:6790/0 conn(0x7f8150138ee0 legacy=0x7f815013b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 monclient: found mon.noname-d 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81501338c0 con 0x7f81500a3890 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8138001790 con 0x7f81500a3890 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: dump: 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.177 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 5 monclient: authenticate success, global_id 194433 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8138001a40 con 0x7f81500a3890 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 4 set_mon_vals no callback set 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 1 -- 172.21.15.26:0/1454956721 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8138001c30 con 0x7f81500a3890 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f814dffb700 10 monclient: dump: 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.179 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 10 monclient: shutdown 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- 172.21.15.26:0/1454956721 >> v1:172.21.15.26:6792/0 conn(0x7f81500a3890 legacy=0x7f81501331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- 172.21.15.26:0/1454956721 shutdown_connections 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- 172.21.15.26:0/1454956721 >> 172.21.15.26:0/1454956721 conn(0x7f8150071350 msgr2=0x7f81500a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- 172.21.15.26:0/1454956721 shutdown_connections 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.167+0000 7f815569f700 1 -- 172.21.15.26:0/1454956721 wait complete. 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient: monmap: 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.186 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 1 Processor -- start 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 1 -- start start 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient: init 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient: _add_conns ranks=[2,5,8,3,0,4,7,6,1] 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient(hunting): picked mon.noname-b con 0x7f8150099360 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.171+0000 7f815569f700 10 monclient(hunting): picked mon.noname-h con 0x7f8150138ee0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient(hunting): picked mon.noname-e con 0x7f8150131630 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815009ec80 con 0x7f8150099360 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f814effd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f8150099360 0x7f8150141850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37802/0 (socket says 172.21.15.26:37802) 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f814effd700 1 -- 172.21.15.26:0/2422715637 learned_addr learned my addr 172.21.15.26:0/2422715637 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8150141f60 con 0x7f8150138ee0 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8150138ca0 con 0x7f8150131630 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient(hunting): authenticate will time out at 1534.604236s 2024-09-15T03:56:30.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609444837 0 0) 0x7f815009ec80 con 0x7f8150099360 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): my global_id is 204212 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8130003570 con 0x7f8150099360 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53405029 0 0) 0x7f8150138ca0 con 0x7f8150131630 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): my global_id is 135089 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815009ec80 con 0x7f8150131630 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468668022 0 0) 0x7f8150141f60 con 0x7f8150138ee0 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): my global_id is 5684 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8150138ca0 con 0x7f8150138ee0 2024-09-15T03:56:30.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1559049214 0 0) 0x7f8130003570 con 0x7f8150099360 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8150141f60 con 0x7f8150099360 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8138001880 con 0x7f8150099360 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.190 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 466482816 0 0) 0x7f815009ec80 con 0x7f8150131630 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8130003570 con 0x7f8150131630 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81400019c0 con 0x7f8150131630 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:30.191 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.192 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1819979992 0 0) 0x7f8150141f60 con 0x7f8150099360 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.26:6793/0 conn(0x7f8150131630 legacy=0x7f8150149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.122:6791/0 conn(0x7f8150138ee0 legacy=0x7f8150138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 monclient: found mon.b 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815009e8c0 con 0x7f8150099360 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 5 monclient: authenticate success, global_id 204212 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient: _renew_subs 2024-09-15T03:56:30.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8150130f10 con 0x7f8150099360 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8138004e20 con 0x7f8150099360 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient: _renew_subs 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8138005350 con 0x7f8150099360 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8150142740 con 0x7f8150099360 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f814d7fa700 4 set_mon_vals no callback set 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: dump: 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.194 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.195 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f813800fd90 con 0x7f8150099360 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(537..537 src has 224..537) v4 ==== 3971+0+0 (unknown 468753986 0 0) 0x7f81380441f0 con 0x7f8150099360 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f815004bbf0 con 0x7f81500a3890 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f814e7fc700 10 monclient: get_auth_request con 0x7f8130037cf0 auth_method 0 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357474798 0 0) 0x7f815004bbf0 con 0x7f81500a3890 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient(hunting): my global_id is 204214 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8150141f60 con 0x7f81500a3890 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 672207318 0 0) 0x7f8150141f60 con 0x7f81500a3890 2024-09-15T03:56:30.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f815004bbf0 con 0x7f81500a3890 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8140004b70 con 0x7f81500a3890 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.175+0000 7f8146ffd700 10 monclient: dump: 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.197 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 789716562 0 0) 0x7f815004bbf0 con 0x7f81500a3890 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8150132590 con 0x7f81500a3890 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8140006e90 con 0x7f81500a3890 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.26:6791/0 conn(0x7f81500a3890 legacy=0x7f815009a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.179+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.26:6791/0 conn(0x7f81500a3890 legacy=0x7f815009a390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f815006fac0 con 0x7f815002ad10 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827338113 0 0) 0x7f815006fac0 con 0x7f815002ad10 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient(hunting): my global_id is 204223 2024-09-15T03:56:30.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815004bbf0 con 0x7f815002ad10 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3393732505 0 0) 0x7f815004bbf0 con 0x7f815002ad10 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f815006fac0 con 0x7f815002ad10 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8140005660 con 0x7f815002ad10 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient: dump: 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.201 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2149110554 0 0) 0x7f815006fac0 con 0x7f815002ad10 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8150133cb0 con 0x7f815002ad10 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 4039702190) 0x7f8140005ab0 con 0x7f815002ad10 2024-09-15T03:56:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.26:6791/0 conn(0x7f815002ad10 legacy=0x7f815009aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f8146ffd700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.26:6791/0 conn(0x7f815002ad10 legacy=0x7f815009aaa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:30.203 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.204 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.205 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.206 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:30.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.208 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:30.209 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.210 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.211 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.214 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.215 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:30.216 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.218 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:30.219 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.220 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.222 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.223 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.223 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:30.223 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.122:6812/20895 conn(0x7f8130037cf0 legacy=0x7f813003a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 10 monclient: shutdown 2024-09-15T03:56:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 >> v1:172.21.15.26:6790/0 conn(0x7f8150099360 legacy=0x7f8150141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f814f7fe700 1 -- 172.21.15.26:0/2422715637 reap_dead start 2024-09-15T03:56:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f814f7fe700 1 -- 172.21.15.26:0/2422715637 reap_dead start 2024-09-15T03:56:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 shutdown_connections 2024-09-15T03:56:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 >> 172.21.15.26:0/2422715637 conn(0x7f8150071350 msgr2=0x7f8150098970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 shutdown_connections 2024-09-15T03:56:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.187+0000 7f815569f700 1 -- 172.21.15.26:0/2422715637 wait complete. 2024-09-15T03:56:30.225 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:30.226 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:56:30.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2352: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2352: finishing copy_from to smithi12223906-36 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 1456 (ObjNum 819 snap 267 seq_num 819) dirty exists 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2353: seq_num 840 ranges {517880=407990,1554992=562888} 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2353: writing smithi12223906-24 from 517880 to 925870 tid 1 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2353: writing smithi12223906-24 from 1554992 to 2117880 tid 2 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2345: done (6 left) 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2346: done (5 left) 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2349: done (4 left) 2024-09-15T03:56:30.228 INFO:tasks.rados.rados.0.smithi122.stdout:2350: done (3 left) 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2351: done (2 left) 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2352: done (1 left) 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2354: rollback oid 41 current snap is 271 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 198 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2355: copy_from oid 42 from oid 25 current snap is 271 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2356: delete oid 8 current snap is 271 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2356: done (3 left) 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2357: copy_from oid 17 from oid 4 current snap is 271 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2358: snap_create 2024-09-15T03:56:30.229 INFO:tasks.rados.rados.0.smithi122.stdout:2354: finishing rollback tid 1 to smithi12223906-41 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 0 (ObjNum 145 snap 0 seq_num 1842552816) dirty dne 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2353: finishing write tid 1 to smithi12223906-24 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2353: finishing write tid 2 to smithi12223906-24 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2353: finishing write tid 3 to smithi12223906-24 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2353: finishing write tid 4 to smithi12223906-24 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 1550 (ObjNum 840 snap 271 seq_num 840) dirty exists 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2353: left oid 24 (ObjNum 840 snap 271 seq_num 840) 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2355: finishing copy_from to smithi12223906-42 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2355: got expected ENOENT (src dne) 2024-09-15T03:56:30.230 INFO:tasks.rados.rados.0.smithi122.stdout:2355: finishing copy_from racing read to smithi12223906-42 2024-09-15T03:56:30.239 INFO:tasks.rados.rados.0.smithi122.stdout:2357: finishing copy_from racing read to smithi12223906-17 2024-09-15T03:56:30.240 INFO:tasks.rados.rados.0.smithi122.stdout:2357: finishing copy_from to smithi12223906-17 2024-09-15T03:56:30.241 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 816 (ObjNum 758 snap 234 seq_num 758) dirty exists 2024-09-15T03:56:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient: get_monmap_and_config 2024-09-15T03:56:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient: monmap: 2024-09-15T03:56:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.336 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:30.333314+0000 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:30.333314+0000 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.337 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 1 Processor -- start 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 1 -- start start 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient: init 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient: _add_conns ranks=[5,8,3,0,4,7,6,1,2] 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient(hunting): picked mon.noname-h con 0x7efe68132b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient(hunting): picked mon.noname-e con 0x7efe6809faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient(hunting): picked mon.noname-g con 0x7efe68138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6804ca10 con 0x7efe68138d40 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6809d5d0 con 0x7efe68132b40 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe68133620 con 0x7efe6809faa0 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6eb8b700 10 monclient(hunting): authenticate will time out at 1534.763375s 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6d388700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7efe6809faa0 0x7efe6809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50100/0 (socket says 172.21.15.26:50100) 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.331+0000 7efe6d388700 1 -- 172.21.15.26:0/1517604720 learned_addr learned my addr 172.21.15.26:0/1517604720 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336683320 0 0) 0x7efe68133620 con 0x7efe6809faa0 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): my global_id is 135098 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe54003570 con 0x7efe6809faa0 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214856765 0 0) 0x7efe6804ca10 con 0x7efe68138d40 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): my global_id is 204186 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe68133620 con 0x7efe68138d40 2024-09-15T03:56:30.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23261813 0 0) 0x7efe6809d5d0 con 0x7efe68132b40 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient(hunting): my global_id is 5693 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6804ca10 con 0x7efe68132b40 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2862079519 0 0) 0x7efe54003570 con 0x7efe6809faa0 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 >> v1:172.21.15.122:6791/0 conn(0x7efe68132b40 legacy=0x7efe68132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 >> v1:172.21.15.122:6790/0 conn(0x7efe68138d40 legacy=0x7efe6813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 monclient: found mon.noname-e 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe680a0cc0 con 0x7efe6809faa0 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 5 monclient: authenticate success, global_id 135098 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe580017b0 con 0x7efe6809faa0 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: dump: 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.350 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe580019d0 con 0x7efe6809faa0 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:30.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 4 set_mon_vals no callback set 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 1 -- 172.21.15.26:0/1517604720 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe580048f0 con 0x7efe6809faa0 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6cb87700 10 monclient: dump: 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: shutdown 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1517604720 >> v1:172.21.15.26:6793/0 conn(0x7efe6809faa0 legacy=0x7efe6809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1517604720 shutdown_connections 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1517604720 >> 172.21.15.26:0/1517604720 conn(0x7efe6806e370 msgr2=0x7efe680a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1517604720 shutdown_connections 2024-09-15T03:56:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1517604720 wait complete. 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: monmap: 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.354 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 Processor -- start 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- start start 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: init 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient: _add_conns ranks=[3,0,6,1,2,8,7,4,5] 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient(hunting): picked mon.noname-g con 0x7efe6809faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient(hunting): picked mon.noname-a con 0x7efe68132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient(hunting): picked mon.noname-d con 0x7efe68138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6809cfa0 con 0x7efe68132b40 2024-09-15T03:56:30.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6809ccc0 con 0x7efe6809faa0 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6811cca0 con 0x7efe68138d40 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6eb8b700 10 monclient(hunting): authenticate will time out at 1534.767188s 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6d388700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efe68132b40 0x7efe68135e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52814/0 (socket says 172.21.15.26:52814) 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe6d388700 1 -- 172.21.15.26:0/1807813329 learned_addr learned my addr 172.21.15.26:0/1807813329 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81718356 0 0) 0x7efe6811cca0 con 0x7efe68138d40 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): my global_id is 194442 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe4c003570 con 0x7efe68138d40 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189952212 0 0) 0x7efe6809cfa0 con 0x7efe68132b40 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): my global_id is 194427 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6811cca0 con 0x7efe68132b40 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104148556 0 0) 0x7efe6809ccc0 con 0x7efe6809faa0 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 10 monclient(hunting): my global_id is 204195 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.335+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6809cfa0 con 0x7efe6809faa0 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2715557137 0 0) 0x7efe6811cca0 con 0x7efe68132b40 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe6809ccc0 con 0x7efe68132b40 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe580016b0 con 0x7efe68132b40 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): dump: 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 145144458 0 0) 0x7efe6809cfa0 con 0x7efe6809faa0 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe6811cca0 con 0x7efe6809faa0 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2773660417 0 0) 0x7efe4c003570 con 0x7efe68138d40 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe6809cfa0 con 0x7efe68138d40 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe64001aa0 con 0x7efe6809faa0 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): dump: 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.359 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe60001c80 con 0x7efe68138d40 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): dump: 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.361 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2329804808 0 0) 0x7efe6809ccc0 con 0x7efe68132b40 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.26:6792/0 conn(0x7efe68138d40 legacy=0x7efe6814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.122:6790/0 conn(0x7efe6809faa0 legacy=0x7efe68135740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 monclient: found mon.a 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe681375f0 con 0x7efe68132b40 2024-09-15T03:56:30.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 5 monclient: authenticate success, global_id 194427 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 10 monclient: _renew_subs 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efe58004950 con 0x7efe68132b40 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6811c6e0 con 0x7efe68132b40 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe580051b0 con 0x7efe68132b40 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5f7fe700 4 set_mon_vals no callback set 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: dump: 2024-09-15T03:56:30.363 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.364 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 10 monclient: _renew_subs 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68137cb0 con 0x7efe68132b40 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efe6804bbf0 con 0x7efe68136560 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7efe580053e0 con 0x7efe68132b40 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(537..537 src has 224..537) v4 ==== 3971+0+0 (unknown 468753986 0 0) 0x7efe58044140 con 0x7efe68132b40 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282693652 0 0) 0x7efe6804bbf0 con 0x7efe68136560 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient(hunting): my global_id is 194451 2024-09-15T03:56:30.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6809ccc0 con 0x7efe68136560 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe6e38a700 10 monclient: get_auth_request con 0x7efe4c0380b0 auth_method 0 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2398778467 0 0) 0x7efe6809ccc0 con 0x7efe68136560 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe6804bbf0 con 0x7efe68136560 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe640021b0 con 0x7efe68136560 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 10 monclient: dump: 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.366 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2383406314 0 0) 0x7efe6804bbf0 con 0x7efe68136560 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.339+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe6811c8d0 con 0x7efe68136560 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.343+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7efe64002670 con 0x7efe68136560 2024-09-15T03:56:30.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.343+0000 7efe5dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.343+0000 7efe5dffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.343+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.26:6792/0 conn(0x7efe68136560 legacy=0x7efe68099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.343+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.26:6792/0 conn(0x7efe68136560 legacy=0x7efe68099a70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe6eb8b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efe6806fac0 con 0x7efe6802ad10 2024-09-15T03:56:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140861301 0 0) 0x7efe6806fac0 con 0x7efe6802ad10 2024-09-15T03:56:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient(hunting): my global_id is 194460 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe6804bbf0 con 0x7efe6802ad10 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 409043756 0 0) 0x7efe6804bbf0 con 0x7efe6802ad10 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe6806fac0 con 0x7efe6802ad10 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efe64003000 con 0x7efe6802ad10 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient: dump: 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.370 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3917705715 0 0) 0x7efe6806fac0 con 0x7efe6802ad10 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efe680a1ee0 con 0x7efe6802ad10 2024-09-15T03:56:30.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 2432433596) 0x7efe640051e0 con 0x7efe6802ad10 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.26:6792/0 conn(0x7efe6802ad10 legacy=0x7efe6809a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.351+0000 7efe5dffb700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.26:6792/0 conn(0x7efe6802ad10 legacy=0x7efe6809a180 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:30.372 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:56:30.373 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:30.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:30.375 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:30.376 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.382 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.384 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:30.386 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.389 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.390 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.391 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.392 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.392 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:30.392 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.122:6812/20895 conn(0x7efe4c0380b0 legacy=0x7efe4c03a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 10 monclient: shutdown 2024-09-15T03:56:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 >> v1:172.21.15.26:6789/0 conn(0x7efe68132b40 legacy=0x7efe68135e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6e38a700 1 -- 172.21.15.26:0/1807813329 reap_dead start 2024-09-15T03:56:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 shutdown_connections 2024-09-15T03:56:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 >> 172.21.15.26:0/1807813329 conn(0x7efe6806e370 msgr2=0x7efe6809da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 shutdown_connections 2024-09-15T03:56:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.355+0000 7efe6eb8b700 1 -- 172.21.15.26:0/1807813329 wait complete. 2024-09-15T03:56:30.394 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:30.394 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:56:30.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:56:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.507+0000 7f952f59e700 10 monclient: get_monmap_and_config 2024-09-15T03:56:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.507+0000 7f952f59e700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.507+0000 7f952f59e700 10 monclient: monmap: 2024-09-15T03:56:30.514 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:30.511604+0000 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:30.511604+0000 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.516 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 1 Processor -- start 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 1 -- start start 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient: init 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient: _add_conns ranks=[5,7,3,8,0,1,6,2,4] 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient(hunting): picked mon.noname-h con 0x7f95300a6b20 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient(hunting): picked mon.noname-i con 0x7f9530099110 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient(hunting): picked mon.noname-g con 0x7f9530138da0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953004ca10 con 0x7f9530138da0 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300a0d30 con 0x7f95300a6b20 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300a22f0 con 0x7f9530099110 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient(hunting): authenticate will time out at 1534.941399s 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952ed9d700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f9530138da0 0x7f953013b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:39790/0 (socket says 172.21.15.26:39790) 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952ed9d700 1 -- 172.21.15.26:0/1314626182 learned_addr learned my addr 172.21.15.26:0/1314626182 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179807626 0 0) 0x7f9518001040 con 0x7f95300a6b20 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): my global_id is 5702 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f951c003570 con 0x7f95300a6b20 2024-09-15T03:56:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474064712 0 0) 0x7f95300a22f0 con 0x7f9530099110 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): my global_id is 135106 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9518001040 con 0x7f9530099110 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220892766 0 0) 0x7f953004ca10 con 0x7f9530138da0 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient(hunting): my global_id is 204204 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95300a22f0 con 0x7f9530138da0 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674622544 0 0) 0x7f951c003570 con 0x7f95300a6b20 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 >> v1:172.21.15.122:6792/0 conn(0x7f9530099110 legacy=0x7f95300994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 >> v1:172.21.15.122:6790/0 conn(0x7f9530138da0 legacy=0x7f953013b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 monclient: found mon.noname-h 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95301337f0 con 0x7f95300a6b20 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 5 monclient: authenticate success, global_id 5702 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9518001870 con 0x7f95300a6b20 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: dump: 2024-09-15T03:56:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9518001b50 con 0x7f95300a6b20 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952f59e700 4 set_mon_vals no callback set 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 1 -- 172.21.15.26:0/1314626182 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9518002240 con 0x7f95300a6b20 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.511+0000 7f952d59a700 10 monclient: dump: 2024-09-15T03:56:30.525 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.526 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient: shutdown 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- 172.21.15.26:0/1314626182 >> v1:172.21.15.122:6791/0 conn(0x7f95300a6b20 legacy=0x7f95301330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- 172.21.15.26:0/1314626182 shutdown_connections 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- 172.21.15.26:0/1314626182 >> 172.21.15.26:0/1314626182 conn(0x7f953006e2d0 msgr2=0x7f95300a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- 172.21.15.26:0/1314626182 shutdown_connections 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- 172.21.15.26:0/1314626182 wait complete. 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient: monmap: 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.528 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.529 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 Processor -- start 2024-09-15T03:56:30.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- start start 2024-09-15T03:56:30.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient: init 2024-09-15T03:56:30.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient: _add_conns ranks=[4,5,2,7,3,8,0,6,1] 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient(hunting): picked mon.noname-c con 0x7f9530099110 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient(hunting): picked mon.noname-h con 0x7f95300a6b20 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient(hunting): picked mon.noname-b con 0x7f9530138da0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300a1880 con 0x7f9530138da0 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9530099e30 con 0x7f9530099110 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953011d2a0 con 0x7f95300a6b20 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952f59e700 10 monclient(hunting): authenticate will time out at 1534.946468s 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952ed9d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f9530138da0 0x7f95301363b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37816/0 (socket says 172.21.15.26:37816) 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952ed9d700 1 -- 172.21.15.26:0/558524363 learned_addr learned my addr 172.21.15.26:0/558524363 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f952e59c700 1 --1- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6791/0 conn(0x7f9530099110 0x7f9530135590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43980/0 (socket says 172.21.15.26:43980) 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831447050 0 0) 0x7f95300a1880 con 0x7f9530138da0 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): my global_id is 204221 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9510003570 con 0x7f9530138da0 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658843331 0 0) 0x7f9530099e30 con 0x7f9530099110 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): my global_id is 204232 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95300a1880 con 0x7f9530099110 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421308021 0 0) 0x7f953011d2a0 con 0x7f95300a6b20 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): my global_id is 5711 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9530099e30 con 0x7f95300a6b20 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 989172881 0 0) 0x7f95300a1880 con 0x7f9530099110 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f953011d2a0 con 0x7f9530099110 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3070333211 0 0) 0x7f9510003570 con 0x7f9530138da0 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95300a1880 con 0x7f9530138da0 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9518002000 con 0x7f9530099110 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): dump: 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:30.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f95200018c0 con 0x7f9530138da0 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.515+0000 7f95267fc700 10 monclient(hunting): dump: 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.539 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3489025334 0 0) 0x7f9530099e30 con 0x7f95300a6b20 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9510003570 con 0x7f95300a6b20 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3091327121 0 0) 0x7f953011d2a0 con 0x7f9530099110 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.122:6791/0 conn(0x7f95300a6b20 legacy=0x7f9530135ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6790/0 conn(0x7f9530138da0 legacy=0x7f95301363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 monclient: found mon.c 2024-09-15T03:56:30.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953011ca00 con 0x7f9530099110 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 5 monclient: authenticate success, global_id 204232 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 10 monclient: _renew_subs 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953011cce0 con 0x7f9530099110 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 10 monclient: _renew_subs 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9530137da0 con 0x7f9530099110 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f953011ce70 con 0x7f953009e5b0 2024-09-15T03:56:30.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9518001290 con 0x7f9530099110 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9518002bc0 con 0x7f9530099110 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952cd99700 4 set_mon_vals no callback set 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: dump: 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.542 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9518002da0 con 0x7f9530099110 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(537..537 src has 224..537) v4 ==== 3971+0+0 (unknown 468753986 0 0) 0x7f9518044020 con 0x7f9530099110 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465460867 0 0) 0x7f953011ce70 con 0x7f953009e5b0 2024-09-15T03:56:30.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient(hunting): my global_id is 135107 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953011d2a0 con 0x7f953009e5b0 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f952ed9d700 10 monclient: get_auth_request con 0x7f9510037ef0 auth_method 0 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3331721312 0 0) 0x7f953011d2a0 con 0x7f953009e5b0 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f953011ce70 con 0x7f953009e5b0 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9528002540 con 0x7f953009e5b0 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 10 monclient: dump: 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.544 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3873102682 0 0) 0x7f953011ce70 con 0x7f953009e5b0 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.519+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f953004bbf0 con 0x7f953009e5b0 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.523+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9528004910 con 0x7f953009e5b0 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.523+0000 7f95267fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.523+0000 7f95267fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.523+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6793/0 conn(0x7f953009e5b0 legacy=0x7f953009e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.523+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6793/0 conn(0x7f953009e5b0 legacy=0x7f953009e980 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:30.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f95300a0d10 con 0x7f953009f090 2024-09-15T03:56:30.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523888702 0 0) 0x7f95300a0d10 con 0x7f953009f090 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient(hunting): my global_id is 135116 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953011ce70 con 0x7f953009f090 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 401504294 0 0) 0x7f953011ce70 con 0x7f953009f090 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95300a0d10 con 0x7f953009f090 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f95280030e0 con 0x7f953009f090 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient: dump: 2024-09-15T03:56:30.548 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.549 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1092901114 0 0) 0x7f95300a0d10 con 0x7f953009f090 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9530071370 con 0x7f953009f090 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1361985881) 0x7f9528004630 con 0x7f953009f090 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6793/0 conn(0x7f953009f090 legacy=0x7f953002ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f95267fc700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6793/0 conn(0x7f953009f090 legacy=0x7f953002ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.551 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:30.553 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:30.554 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:30.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.559 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:30.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:30.565 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.568 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:30.569 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.122:6812/20895 conn(0x7f9510037ef0 legacy=0x7f951003a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 10 monclient: shutdown 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 >> v1:172.21.15.26:6791/0 conn(0x7f9530099110 legacy=0x7f9530135590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952ed9d700 1 -- 172.21.15.26:0/558524363 reap_dead start 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952ed9d700 1 -- 172.21.15.26:0/558524363 reap_dead start 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 shutdown_connections 2024-09-15T03:56:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 >> 172.21.15.26:0/558524363 conn(0x7f953006e2d0 msgr2=0x7f95300a49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 shutdown_connections 2024-09-15T03:56:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.531+0000 7f952f59e700 1 -- 172.21.15.26:0/558524363 wait complete. 2024-09-15T03:56:30.572 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:30.572 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:56:30.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.679+0000 7f8041963700 10 monclient: get_monmap_and_config 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.679+0000 7f8041963700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.679+0000 7f8041963700 10 monclient: monmap: 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:30.683209+0000 2024-09-15T03:56:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:30.683209+0000 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.687 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 1 Processor -- start 2024-09-15T03:56:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 1 -- start start 2024-09-15T03:56:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient: init 2024-09-15T03:56:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient: _add_conns ranks=[4,1,8,2,5,7,6,3,0] 2024-09-15T03:56:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient(hunting): picked mon.noname-c con 0x7f803c12fd20 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient(hunting): picked mon.noname-f con 0x7f803c1300f0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient(hunting): picked mon.noname-e con 0x7f803c130520 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c04ca10 con 0x7f803c1300f0 2024-09-15T03:56:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0a2830 con 0x7f803c12fd20 2024-09-15T03:56:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c097b70 con 0x7f803c130520 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient(hunting): authenticate will time out at 1535.113166s 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803affd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f803c12fd20 0x7f803c133270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43984/0 (socket says 172.21.15.26:43984) 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803affd700 1 -- 172.21.15.26:0/3401809962 learned_addr learned my addr 172.21.15.26:0/3401809962 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803b7fe700 1 --1- 172.21.15.26:0/3401809962 >> v1:172.21.15.26:6793/0 conn(0x7f803c130520 0x7f803c13b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50130/0 (socket says 172.21.15.26:50130) 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087936211 0 0) 0x7f803c0a2830 con 0x7f803c12fd20 2024-09-15T03:56:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): my global_id is 204241 2024-09-15T03:56:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8028003570 con 0x7f803c12fd20 2024-09-15T03:56:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574880073 0 0) 0x7f803c097b70 con 0x7f803c130520 2024-09-15T03:56:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): my global_id is 135125 2024-09-15T03:56:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803c0a2830 con 0x7f803c130520 2024-09-15T03:56:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712965784 0 0) 0x7f803c04ca10 con 0x7f803c1300f0 2024-09-15T03:56:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient(hunting): my global_id is 204211 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803c097b70 con 0x7f803c1300f0 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3837900797 0 0) 0x7f8028003570 con 0x7f803c12fd20 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 >> v1:172.21.15.26:6793/0 conn(0x7f803c130520 legacy=0x7f803c13b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 >> v1:172.21.15.122:6789/0 conn(0x7f803c1300f0 legacy=0x7f803c137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 monclient: found mon.noname-c 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c133980 con 0x7f803c12fd20 2024-09-15T03:56:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f802c0017b0 con 0x7f803c12fd20 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 5 monclient: authenticate success, global_id 204241 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: dump: 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.698 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f802c001a20 con 0x7f803c12fd20 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 4 set_mon_vals no callback set 2024-09-15T03:56:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 1 -- 172.21.15.26:0/3401809962 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f802c004c30 con 0x7f803c12fd20 2024-09-15T03:56:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f803a7fc700 10 monclient: dump: 2024-09-15T03:56:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 10 monclient: shutdown 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.683+0000 7f8041963700 1 -- 172.21.15.26:0/3401809962 >> v1:172.21.15.26:6791/0 conn(0x7f803c12fd20 legacy=0x7f803c133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- 172.21.15.26:0/3401809962 shutdown_connections 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- 172.21.15.26:0/3401809962 >> 172.21.15.26:0/3401809962 conn(0x7f803c06e410 msgr2=0x7f803c0a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- 172.21.15.26:0/3401809962 shutdown_connections 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- 172.21.15.26:0/3401809962 wait complete. 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient: monmap: 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.703 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.704 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 Processor -- start 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- start start 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient: init 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient: _add_conns ranks=[6,7,8,3,1,2,0,4,5] 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient(hunting): picked mon.noname-d con 0x7f803c12fd20 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient(hunting): picked mon.noname-i con 0x7f803c1300f0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient(hunting): picked mon.noname-e con 0x7f803c130520 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c11d8a0 con 0x7f803c12fd20 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0a1b50 con 0x7f803c1300f0 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c11f070 con 0x7f803c130520 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f8041963700 10 monclient(hunting): authenticate will time out at 1535.119759s 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f803affd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f803c12fd20 0x7f803c12e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48506/0 (socket says 172.21.15.26:48506) 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f803affd700 1 -- 172.21.15.26:0/564207319 learned_addr learned my addr 172.21.15.26:0/564207319 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.687+0000 7f803b7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f803c130520 0x7f803c14a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50136/0 (socket says 172.21.15.26:50136) 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586256818 0 0) 0x7f803c11d8a0 con 0x7f803c12fd20 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): my global_id is 194469 2024-09-15T03:56:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f801c003570 con 0x7f803c12fd20 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447648662 0 0) 0x7f803c11f070 con 0x7f803c130520 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): my global_id is 135134 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803c11d8a0 con 0x7f803c130520 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693226197 0 0) 0x7f803c0a1b50 con 0x7f803c1300f0 2024-09-15T03:56:30.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): my global_id is 135115 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803c11f070 con 0x7f803c1300f0 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4250550095 0 0) 0x7f803c11d8a0 con 0x7f803c130520 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f803c0a1b50 con 0x7f803c130520 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8034001d00 con 0x7f803c130520 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 121035767 0 0) 0x7f801c003570 con 0x7f803c12fd20 2024-09-15T03:56:30.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f803c11d8a0 con 0x7f803c12fd20 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f802c0017b0 con 0x7f803c12fd20 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.715 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2813648877 0 0) 0x7f803c0a1b50 con 0x7f803c130520 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.122:6792/0 conn(0x7f803c1300f0 legacy=0x7f803c146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.26:6792/0 conn(0x7f803c12fd20 legacy=0x7f803c12e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 monclient: found mon.e 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c11f350 con 0x7f803c130520 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 5 monclient: authenticate success, global_id 135134 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 10 monclient: _renew_subs 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c09c7a0 con 0x7f803c130520 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 10 monclient: _renew_subs 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c1429c0 con 0x7f803c130520 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8034004c30 con 0x7f803c130520 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8034005160 con 0x7f803c130520 2024-09-15T03:56:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8039ffb700 4 set_mon_vals no callback set 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient: dump: 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8020006590 con 0x7f8020002840 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8034005340 con 0x7f803c130520 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(537..537 src has 224..537) v4 ==== 3971+0+0 (unknown 468753986 0 0) 0x7f80340441a0 con 0x7f803c130520 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539669288 0 0) 0x7f8020006590 con 0x7f8020002840 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8033fff700 10 monclient: get_auth_request con 0x7f801c037c90 auth_method 0 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.691+0000 7f8032ffd700 10 monclient(hunting): my global_id is 204220 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f803c0a1b50 con 0x7f8020002840 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2358563116 0 0) 0x7f803c0a1b50 con 0x7f8020002840 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8020006590 con 0x7f8020002840 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f802c0011d0 con 0x7f8020002840 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 10 monclient: dump: 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2249881893 0 0) 0x7f8020006590 con 0x7f8020002840 2024-09-15T03:56:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f802000a130 con 0x7f8020002840 2024-09-15T03:56:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f802c0070b0 con 0x7f8020002840 2024-09-15T03:56:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.122:6789/0 conn(0x7f8020002840 legacy=0x7f8020004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.695+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.122:6789/0 conn(0x7f8020002840 legacy=0x7f8020004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8041963700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8020009620 con 0x7f8020009090 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335247074 0 0) 0x7f8020009620 con 0x7f8020009090 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 10 monclient(hunting): my global_id is 204229 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8020006590 con 0x7f8020009090 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1202660253 0 0) 0x7f8020006590 con 0x7f8020009090 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8020009620 con 0x7f8020009090 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f802c005810 con 0x7f8020009090 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 10 monclient: dump: 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 378299702 0 0) 0x7f8020009620 con 0x7f8020009090 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.707+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f80200098d0 con 0x7f8020009090 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 510867415) 0x7f802c006ce0 con 0x7f8020009090 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8032ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8032ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.122:6789/0 conn(0x7f8020009090 legacy=0x7f8020008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8032ffd700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.122:6789/0 conn(0x7f8020009090 legacy=0x7f8020008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:30.727 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.728 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.729 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.730 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:30.731 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:30.732 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.733 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.734 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.738 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.739 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.742 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:30.743 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:30.744 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.748 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.749 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:30.749 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.122:6812/20895 conn(0x7f801c037c90 legacy=0x7f801c03a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 10 monclient: shutdown 2024-09-15T03:56:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 >> v1:172.21.15.26:6793/0 conn(0x7f803c130520 legacy=0x7f803c14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f803b7fe700 1 -- 172.21.15.26:0/564207319 reap_dead start 2024-09-15T03:56:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f803b7fe700 1 -- 172.21.15.26:0/564207319 reap_dead start 2024-09-15T03:56:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 shutdown_connections 2024-09-15T03:56:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 >> 172.21.15.26:0/564207319 conn(0x7f803c06e410 msgr2=0x7f803c0a4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 shutdown_connections 2024-09-15T03:56:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.711+0000 7f8041963700 1 -- 172.21.15.26:0/564207319 wait complete. 2024-09-15T03:56:30.751 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:30.751 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:56:30.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:56:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.863+0000 7f0b8e0f7700 10 monclient: get_monmap_and_config 2024-09-15T03:56:30.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.863+0000 7f0b8e0f7700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.863+0000 7f0b8e0f7700 10 monclient: monmap: 2024-09-15T03:56:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.872 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:30.866219+0000 2024-09-15T03:56:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:30.866219+0000 2024-09-15T03:56:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.873 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.874 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.875 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.875 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.863+0000 7f0b8e0f7700 1 Processor -- start 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.863+0000 7f0b8e0f7700 1 -- start start 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.863+0000 7f0b8e0f7700 10 monclient: init 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient: _add_conns ranks=[4,6,8,2,0,5,1,3,7] 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient(hunting): picked mon.noname-c con 0x7f0b88070fb0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient(hunting): picked mon.noname-d con 0x7f0b880950b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient(hunting): picked mon.noname-e con 0x7f0b88095b90 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8804ca10 con 0x7f0b88070fb0 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8809c7e0 con 0x7f0b880950b0 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8809b840 con 0x7f0b88095b90 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient(hunting): authenticate will time out at 1535.296967s 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8c8f4700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f0b880950b0 0x7f0b88095480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48520/0 (socket says 172.21.15.26:48520) 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8c8f4700 1 -- 172.21.15.26:0/3914325561 learned_addr learned my addr 172.21.15.26:0/3914325561 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8d8f6700 1 --1- 172.21.15.26:0/3914325561 >> v1:172.21.15.26:6793/0 conn(0x7f0b88095b90 0x7f0b88137c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50146/0 (socket says 172.21.15.26:50146) 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953348197 0 0) 0x7f0b8809b840 con 0x7f0b88095b90 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): my global_id is 135143 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b74003570 con 0x7f0b88095b90 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796030693 0 0) 0x7f0b8804ca10 con 0x7f0b88070fb0 2024-09-15T03:56:30.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): my global_id is 204250 2024-09-15T03:56:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8809b840 con 0x7f0b88070fb0 2024-09-15T03:56:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396002707 0 0) 0x7f0b8809c7e0 con 0x7f0b880950b0 2024-09-15T03:56:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient(hunting): my global_id is 194478 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8804ca10 con 0x7f0b880950b0 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141938916 0 0) 0x7f0b74003570 con 0x7f0b88095b90 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 >> v1:172.21.15.26:6792/0 conn(0x7f0b880950b0 legacy=0x7f0b88095480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 >> v1:172.21.15.26:6791/0 conn(0x7f0b88070fb0 legacy=0x7f0b8809bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 monclient: found mon.noname-e 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8809b210 con 0x7f0b88095b90 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 5 monclient: authenticate success, global_id 135143 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0b80001b30 con 0x7f0b88095b90 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: dump: 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0b80001da0 con 0x7f0b88095b90 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 1 -- 172.21.15.26:0/3914325561 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0b80004ef0 con 0x7f0b88095b90 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 4 set_mon_vals no callback set 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b7bfff700 10 monclient: dump: 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 10 monclient: shutdown 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/3914325561 >> v1:172.21.15.26:6793/0 conn(0x7f0b88095b90 legacy=0x7f0b88137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/3914325561 shutdown_connections 2024-09-15T03:56:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/3914325561 >> 172.21.15.26:0/3914325561 conn(0x7f0b880a2ea0 msgr2=0x7f0b8809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/3914325561 shutdown_connections 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.867+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/3914325561 wait complete. 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient: build_initial_monmap 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient: monmap: 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:30.898 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 1 Processor -- start 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 1 -- start start 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient: init 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient: _add_conns ranks=[7,3,4,1,5,0,6,2,8] 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient(hunting): picked mon.noname-i con 0x7f0b880950b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient(hunting): picked mon.noname-g con 0x7f0b88095b90 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient(hunting): picked mon.noname-c con 0x7f0b881344a0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:30.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b88130350 con 0x7f0b88095b90 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b880960c0 con 0x7f0b881344a0 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b881278a0 con 0x7f0b880950b0 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8e0f7700 10 monclient(hunting): authenticate will time out at 1535.302628s 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8d8f6700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f0b881344a0 0x7f0b8814a8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:43998/0 (socket says 172.21.15.26:43998) 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b8d8f6700 1 -- 172.21.15.26:0/1349468681 learned_addr learned my addr 172.21.15.26:0/1349468681 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811625953 0 0) 0x7f0b880960c0 con 0x7f0b881344a0 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): my global_id is 204259 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b6c003570 con 0x7f0b881344a0 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143971439 0 0) 0x7f0b881278a0 con 0x7f0b880950b0 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): my global_id is 135124 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b880960c0 con 0x7f0b880950b0 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705828732 0 0) 0x7f0b88130350 con 0x7f0b88095b90 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): my global_id is 204213 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b881278a0 con 0x7f0b88095b90 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2748290264 0 0) 0x7f0b6c003570 con 0x7f0b881344a0 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b88130350 con 0x7f0b881344a0 2024-09-15T03:56:30.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0b800017b0 con 0x7f0b881344a0 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): dump: 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.902 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.903 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.871+0000 7f0b797fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1624923615 0 0) 0x7f0b88130350 con 0x7f0b881344a0 2024-09-15T03:56:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6792/0 conn(0x7f0b880950b0 legacy=0x7f0b88141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6790/0 conn(0x7f0b88095b90 legacy=0x7f0b88147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: _finish_hunting 0 2024-09-15T03:56:30.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 monclient: found mon.c 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88128790 con 0x7f0b881344a0 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: _finish_auth 0 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 5 monclient: authenticate success, global_id 204259 2024-09-15T03:56:30.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 10 monclient: _renew_subs 2024-09-15T03:56:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b88135070 con 0x7f0b881344a0 2024-09-15T03:56:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 10 monclient: _renew_subs 2024-09-15T03:56:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b88129bc0 con 0x7f0b881344a0 2024-09-15T03:56:30.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0b80005120 con 0x7f0b881344a0 2024-09-15T03:56:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0b800055d0 con 0x7f0b881344a0 2024-09-15T03:56:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b7b7fe700 4 set_mon_vals no callback set 2024-09-15T03:56:30.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: dump: 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.908 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0b80005730 con 0x7f0b881344a0 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0b68006590 con 0x7f0b68002840 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(537..537 src has 224..537) v4 ==== 3971+0+0 (unknown 468753986 0 0) 0x7f0b80044400 con 0x7f0b881344a0 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b8c8f4700 10 monclient: get_auth_request con 0x7f0b6c037680 auth_method 0 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385001536 0 0) 0x7f0b68006590 con 0x7f0b68002840 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient(hunting): my global_id is 204222 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b88130350 con 0x7f0b68002840 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2587056301 0 0) 0x7f0b88130350 con 0x7f0b68002840 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b68006590 con 0x7f0b68002840 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0b840019f0 con 0x7f0b68002840 2024-09-15T03:56:30.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 10 monclient: dump: 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2076960588 0 0) 0x7f0b68006590 con 0x7f0b68002840 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.875+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b6800a130 con 0x7f0b68002840 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.879+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0b84001da0 con 0x7f0b68002840 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.879+0000 7f0b797fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.879+0000 7f0b797fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:30.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.879+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6790/0 conn(0x7f0b68002840 legacy=0x7f0b68004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.879+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6790/0 conn(0x7f0b68002840 legacy=0x7f0b68004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b8e0f7700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:30.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0b68007520 con 0x7f0b680068d0 2024-09-15T03:56:30.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5162392 0 0) 0x7f0b68007520 con 0x7f0b680068d0 2024-09-15T03:56:30.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 10 monclient(hunting): my global_id is 204231 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b68006590 con 0x7f0b680068d0 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1643900720 0 0) 0x7f0b68006590 con 0x7f0b680068d0 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b68007520 con 0x7f0b680068d0 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0b84002680 con 0x7f0b680068d0 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 10 monclient: dump: 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:30.915 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:30.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 916520716 0 0) 0x7f0b68007520 con 0x7f0b680068d0 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.891+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0b68008760 con 0x7f0b680068d0 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 940692042) 0x7f0b84002a30 con 0x7f0b680068d0 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b797fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b797fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6790/0 conn(0x7f0b680068d0 legacy=0x7f0b68006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b797fa700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6790/0 conn(0x7f0b680068d0 legacy=0x7f0b68006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.917 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.918 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.919 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:30.921 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:30.922 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.923 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:30.925 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:30.926 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.928 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:30.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.934 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.935 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.937 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:30.938 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:30.939 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:30.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:30.939 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:30.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:30.939 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:30.939 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.122:6812/20895 conn(0x7f0b6c037680 legacy=0x7f0b6c039b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 10 monclient: shutdown 2024-09-15T03:56:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:30.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 >> v1:172.21.15.26:6791/0 conn(0x7f0b881344a0 legacy=0x7f0b8814a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8d8f6700 1 -- 172.21.15.26:0/1349468681 reap_dead start 2024-09-15T03:56:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8d8f6700 1 -- 172.21.15.26:0/1349468681 reap_dead start 2024-09-15T03:56:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 shutdown_connections 2024-09-15T03:56:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 >> 172.21.15.26:0/1349468681 conn(0x7f0b880a2ea0 msgr2=0x7f0b880a0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 shutdown_connections 2024-09-15T03:56:30.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:30.895+0000 7f0b8e0f7700 1 -- 172.21.15.26:0/1349468681 wait complete. 2024-09-15T03:56:30.942 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:30.942 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:56:30.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient: get_monmap_and_config 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient: build_initial_monmap 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient: monmap: 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:31.053077+0000 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:31.053077+0000 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:31.057 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 1 Processor -- start 2024-09-15T03:56:31.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 1 -- start start 2024-09-15T03:56:31.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient: init 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient: _add_conns ranks=[8,3,1,4,7,0,2,6,5] 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient(hunting): picked mon.noname-e con 0x7f53fc132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient(hunting): picked mon.noname-g con 0x7f53fc09faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient(hunting): picked mon.noname-f con 0x7f53fc138d40 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc04ca10 con 0x7f53fc138d40 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc09d5d0 con 0x7f53fc09faa0 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc133620 con 0x7f53fc132b40 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f5400976700 10 monclient(hunting): authenticate will time out at 1535.483783s 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f53fad9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f53fc132b40 0x7f53fc132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50150/0 (socket says 172.21.15.26:50150) 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.051+0000 7f53fad9d700 1 -- 172.21.15.26:0/1178977890 learned_addr learned my addr 172.21.15.26:0/1178977890 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177784744 0 0) 0x7f53fc133620 con 0x7f53fc132b40 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): my global_id is 135152 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e8003570 con 0x7f53fc132b40 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072997673 0 0) 0x7f53fc09d5d0 con 0x7f53fc09faa0 2024-09-15T03:56:31.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): my global_id is 204240 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc133620 con 0x7f53fc09faa0 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804859284 0 0) 0x7f53fc04ca10 con 0x7f53fc138d40 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient(hunting): my global_id is 204238 2024-09-15T03:56:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc09d5d0 con 0x7f53fc138d40 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3171552994 0 0) 0x7f53e8003570 con 0x7f53fc132b40 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 >> v1:172.21.15.122:6790/0 conn(0x7f53fc09faa0 legacy=0x7f53fc09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 >> v1:172.21.15.122:6789/0 conn(0x7f53fc138d40 legacy=0x7f53fc13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: _finish_hunting 0 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 monclient: found mon.noname-e 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc0a0cc0 con 0x7f53fc132b40 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: _finish_auth 0 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 5 monclient: authenticate success, global_id 135152 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53ec0017b0 con 0x7f53fc132b40 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: dump: 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.065 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53ec001a20 con 0x7f53fc132b40 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 4 set_mon_vals no callback set 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 1 -- 172.21.15.26:0/1178977890 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53ec004c30 con 0x7f53fc132b40 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:31.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f53fa59c700 10 monclient: dump: 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 10 monclient: shutdown 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 -- 172.21.15.26:0/1178977890 >> v1:172.21.15.26:6793/0 conn(0x7f53fc132b40 legacy=0x7f53fc132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 -- 172.21.15.26:0/1178977890 shutdown_connections 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 -- 172.21.15.26:0/1178977890 >> 172.21.15.26:0/1178977890 conn(0x7f53fc06e370 msgr2=0x7f53fc0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 -- 172.21.15.26:0/1178977890 shutdown_connections 2024-09-15T03:56:31.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 -- 172.21.15.26:0/1178977890 wait complete. 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 10 monclient: build_initial_monmap 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 10 monclient: monmap: 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 Processor -- start 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 1 -- start start 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.055+0000 7f5400976700 10 monclient: init 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient: _add_conns ranks=[8,5,0,1,2,3,4,6,7] 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient(hunting): picked mon.noname-e con 0x7f53fc09faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient(hunting): picked mon.noname-h con 0x7f53fc132b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient(hunting): picked mon.noname-a con 0x7f53fc138d40 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc1378a0 con 0x7f53fc138d40 2024-09-15T03:56:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc09cfa0 con 0x7f53fc132b40 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc09ccc0 con 0x7f53fc09faa0 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient(hunting): authenticate will time out at 1535.488753s 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53fb59e700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53fc138d40 0x7f53fc14ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52824/0 (socket says 172.21.15.26:52824) 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53fb59e700 1 -- 172.21.15.26:0/2507089404 learned_addr learned my addr 172.21.15.26:0/2507089404 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031847730 0 0) 0x7f53fc1378a0 con 0x7f53fc138d40 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): my global_id is 194436 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53dc003570 con 0x7f53fc138d40 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416305001 0 0) 0x7f53fc09ccc0 con 0x7f53fc09faa0 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): my global_id is 135161 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc1378a0 con 0x7f53fc09faa0 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617104400 0 0) 0x7f53fc09cfa0 con 0x7f53fc132b40 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): my global_id is 5720 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc09ccc0 con 0x7f53fc132b40 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3337054145 0 0) 0x7f53dc003570 con 0x7f53fc138d40 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53fc09cfa0 con 0x7f53fc138d40 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53f4002dd0 con 0x7f53fc138d40 2024-09-15T03:56:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): dump: 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 85289248 0 0) 0x7f53fc1378a0 con 0x7f53fc09faa0 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53dc003570 con 0x7f53fc09faa0 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53ec001770 con 0x7f53fc09faa0 2024-09-15T03:56:31.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): dump: 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4190894827 0 0) 0x7f53fc09ccc0 con 0x7f53fc132b40 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53fc1378a0 con 0x7f53fc132b40 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53e40019a0 con 0x7f53fc132b40 2024-09-15T03:56:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient(hunting): dump: 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 432400668 0 0) 0x7f53fc09cfa0 con 0x7f53fc138d40 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.26:6793/0 conn(0x7f53fc09faa0 legacy=0x7f53fc142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.122:6791/0 conn(0x7f53fc132b40 legacy=0x7f53fc1473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 monclient: found mon.a 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc11c6e0 con 0x7f53fc138d40 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 5 monclient: authenticate success, global_id 194436 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient: _renew_subs 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc137320 con 0x7f53fc138d40 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient: _renew_subs 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc137ef0 con 0x7f53fc138d40 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53f4001880 con 0x7f53fc138d40 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53f4001d30 con 0x7f53fc138d40 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f959a700 4 set_mon_vals no callback set 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.059+0000 7f53f37fe700 10 monclient: dump: 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.087 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.088 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.088 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.088 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.088 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.088 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.088 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.089 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.089 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.089 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.089 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.089 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f53f4001f10 con 0x7f53fc138d40 2024-09-15T03:56:31.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f5400976700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53e0006590 con 0x7f53e0002840 2024-09-15T03:56:31.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(537..537 src has 224..537) v4 ==== 3971+0+0 (unknown 468753986 0 0) 0x7f53f4044290 con 0x7f53fc138d40 2024-09-15T03:56:31.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f259c700 10 monclient: get_auth_request con 0x7f53dc0380e0 auth_method 0 2024-09-15T03:56:31.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954824984 0 0) 0x7f53e0006590 con 0x7f53e0002840 2024-09-15T03:56:31.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient(hunting): my global_id is 5729 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53fc09cfa0 con 0x7f53e0002840 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 404554634 0 0) 0x7f53fc09cfa0 con 0x7f53e0002840 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53e0006590 con 0x7f53e0002840 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53ec004bb0 con 0x7f53e0002840 2024-09-15T03:56:31.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient: dump: 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.092 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.093 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.093 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.093 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.093 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.093 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.093 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.094 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3600716224 0 0) 0x7f53e0006590 con 0x7f53e0002840 2024-09-15T03:56:31.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53e000a130 con 0x7f53e0002840 2024-09-15T03:56:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f53ec005220 con 0x7f53e0002840 2024-09-15T03:56:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.122:6791/0 conn(0x7f53e0002840 legacy=0x7f53e0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.063+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.122:6791/0 conn(0x7f53e0002840 legacy=0x7f53e0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:31.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.071+0000 7f5400976700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.071+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53e0007520 con 0x7f53e00068d0 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233781819 0 0) 0x7f53e0007520 con 0x7f53e00068d0 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient(hunting): my global_id is 5738 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e0006590 con 0x7f53e00068d0 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 922585395 0 0) 0x7f53e0006590 con 0x7f53e00068d0 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53e0007520 con 0x7f53e00068d0 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53ec0059d0 con 0x7f53e00068d0 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient: dump: 2024-09-15T03:56:31.097 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.098 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2252771985 0 0) 0x7f53e0007520 con 0x7f53e00068d0 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f53e0008760 con 0x7f53e00068d0 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7738 (unknown 0 0 322262810) 0x7f53ec006da0 con 0x7f53e00068d0 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.122:6791/0 conn(0x7f53e00068d0 legacy=0x7f53e0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53f37fe700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.122:6791/0 conn(0x7f53e00068d0 legacy=0x7f53e0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:31.099 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:31.100 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:31.101 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:31.102 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:31.103 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:31.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:31.105 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.106 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:31.107 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.108 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.110 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:31.111 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.112 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:31.113 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:31.115 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:31.116 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.117 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:31.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.119 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:31.119 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:31.120 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.122 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.123 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.124 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:56:31.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:31.126 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:31.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.122:6812/20895 conn(0x7f53dc0380e0 legacy=0x7f53dc03a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 10 monclient: shutdown 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 >> v1:172.21.15.26:6789/0 conn(0x7f53fc138d40 legacy=0x7f53fc14ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53fb59e700 1 -- 172.21.15.26:0/2507089404 reap_dead start 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f53fb59e700 1 -- 172.21.15.26:0/2507089404 reap_dead start 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 shutdown_connections 2024-09-15T03:56:31.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 >> 172.21.15.26:0/2507089404 conn(0x7f53fc06e370 msgr2=0x7f53fc09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 shutdown_connections 2024-09-15T03:56:31.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.075+0000 7f5400976700 1 -- 172.21.15.26:0/2507089404 wait complete. 2024-09-15T03:56:31.130 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:31.130 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:56:31.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:56:31.133 INFO:tasks.rados.rados.0.smithi122.stdout:2353: done (4 left) 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2354: done (3 left) 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2355: done (2 left) 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2357: done (1 left) 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2358: done (0 left) 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2359: write oid 30 current snap is 272 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2359: seq_num 842 ranges {551494=558488,1671956=479538} 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2359: writing smithi12223906-30 from 551494 to 1109982 tid 1 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2359: writing smithi12223906-30 from 1671956 to 2151494 tid 2 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2360: rollback oid 11 current snap is 272 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 11 to 237 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2359: finishing write tid 1 to smithi12223906-30 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2361: read oid 29 snap -1 2024-09-15T03:56:31.134 INFO:tasks.rados.rados.0.smithi122.stdout:2361: expect deleted 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2362: delete oid 20 current snap is 272 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2359: finishing write tid 2 to smithi12223906-30 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2359: finishing write tid 3 to smithi12223906-30 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2359: finishing write tid 4 to smithi12223906-30 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 821 (ObjNum 842 snap 272 seq_num 842) dirty exists 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2359: left oid 30 (ObjNum 842 snap 272 seq_num 842) 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2359: done (3 left) 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2361: done (2 left) 2024-09-15T03:56:31.135 INFO:tasks.rados.rados.0.smithi122.stdout:2362: done (1 left) 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2363: rollback oid 25 current snap is 272 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 252 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2364: read oid 19 snap -1 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2364: expect deleted 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2365: snap_create 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2360: finishing rollback tid 0 to smithi12223906-11 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2360: finishing rollback tid 1 to smithi12223906-11 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2360: finishing rollback tid 2 to smithi12223906-11 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1301 (ObjNum 621 snap 185 seq_num 621) dirty exists 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2363: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:2363: finishing rollback tid 2 to smithi12223906-25 2024-09-15T03:56:31.136 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 1552 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:56:31.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.243+0000 7fdad9b80700 10 monclient: get_monmap_and_config 2024-09-15T03:56:31.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.243+0000 7fdad9b80700 10 monclient: build_initial_monmap 2024-09-15T03:56:31.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.243+0000 7fdad9b80700 10 monclient: monmap: 2024-09-15T03:56:31.249 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:31.246677+0000 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:31.246677+0000 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:31.250 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:31.251 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:31.251 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.243+0000 7fdad9b80700 1 Processor -- start 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.243+0000 7fdad9b80700 1 -- start start 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.243+0000 7fdad9b80700 10 monclient: init 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient: _add_conns ranks=[5,1,3,8,6,7,0,2,4] 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient(hunting): picked mon.noname-h con 0x7fdad40990c0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient(hunting): picked mon.noname-f con 0x7fdad40a1310 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient(hunting): picked mon.noname-g con 0x7fdad41335a0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad404ca10 con 0x7fdad40a1310 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40a1b20 con 0x7fdad41335a0 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40a10b0 con 0x7fdad40990c0 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient(hunting): authenticate will time out at 1535.676975s 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad3fff700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fdad41335a0 0x7fdad413afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:39850/0 (socket says 172.21.15.26:39850) 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad2ffd700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fdad40a1310 0x7fdad4099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:43312/0 (socket says 172.21.15.26:43312) 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad3fff700 1 -- 172.21.15.26:0/4255431170 learned_addr learned my addr 172.21.15.26:0/4255431170 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad37fe700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7fdad40990c0 0x7fdad4132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:38290/0 (socket says 172.21.15.26:38290) 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034949736 0 0) 0x7fdad40a10b0 con 0x7fdad40990c0 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): my global_id is 5747 2024-09-15T03:56:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac0003570 con 0x7fdad40990c0 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107414722 0 0) 0x7fdad40a1b20 con 0x7fdad41335a0 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): my global_id is 204249 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad40a10b0 con 0x7fdad41335a0 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027882895 0 0) 0x7fdad404ca10 con 0x7fdad40a1310 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient(hunting): my global_id is 204247 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad40a1b20 con 0x7fdad40a1310 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 371155048 0 0) 0x7fdac0003570 con 0x7fdad40990c0 2024-09-15T03:56:31.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 >> v1:172.21.15.122:6790/0 conn(0x7fdad41335a0 legacy=0x7fdad413afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 >> v1:172.21.15.122:6789/0 conn(0x7fdad40a1310 legacy=0x7fdad4099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 monclient: found mon.noname-h 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4099ba0 con 0x7fdad40990c0 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdac40016e0 con 0x7fdad40990c0 2024-09-15T03:56:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 5 monclient: authenticate success, global_id 5747 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: dump: 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdac40019c0 con 0x7fdad40990c0 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 1 -- 172.21.15.26:0/4255431170 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdac4004b70 con 0x7fdad40990c0 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad27fc700 10 monclient: dump: 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 4 set_mon_vals no callback set 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 10 monclient: shutdown 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.247+0000 7fdad9b80700 1 -- 172.21.15.26:0/4255431170 >> v1:172.21.15.122:6791/0 conn(0x7fdad40990c0 legacy=0x7fdad4132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- 172.21.15.26:0/4255431170 shutdown_connections 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- 172.21.15.26:0/4255431170 >> 172.21.15.26:0/4255431170 conn(0x7fdad406e370 msgr2=0x7fdad40a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- 172.21.15.26:0/4255431170 shutdown_connections 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- 172.21.15.26:0/4255431170 wait complete. 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient: build_initial_monmap 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient: monmap: 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 Processor -- start 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- start start 2024-09-15T03:56:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient: init 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient: _add_conns ranks=[0,6,7,4,8,5,1,3,2] 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient(hunting): picked mon.noname-a con 0x7fdad40990c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient(hunting): picked mon.noname-d con 0x7fdad40a1310 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient(hunting): picked mon.noname-i con 0x7fdad41335a0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad4122210 con 0x7fdad40990c0 2024-09-15T03:56:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad409a010 con 0x7fdad40a1310 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad41200b0 con 0x7fdad41335a0 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad9b80700 10 monclient(hunting): authenticate will time out at 1535.682830s 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad2ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fdad40a1310 0x7fdad4147030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48536/0 (socket says 172.21.15.26:48536) 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad37fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdad40990c0 0x7fdad4131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52834/0 (socket says 172.21.15.26:52834) 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdad2ffd700 1 -- 172.21.15.26:0/4193085050 learned_addr learned my addr 172.21.15.26:0/4193085050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671535415 0 0) 0x7fdad409a010 con 0x7fdad40a1310 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): my global_id is 194487 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab4003570 con 0x7fdad40a1310 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437949262 0 0) 0x7fdad4099ba0 con 0x7fdad40990c0 2024-09-15T03:56:31.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): my global_id is 194445 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad409a010 con 0x7fdad40990c0 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320614458 0 0) 0x7fdad41200b0 con 0x7fdad41335a0 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 10 monclient(hunting): my global_id is 135133 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.251+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad4099ba0 con 0x7fdad41335a0 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 263628262 0 0) 0x7fdad409a010 con 0x7fdad40990c0 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdad41200b0 con 0x7fdad40990c0 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdac4004c30 con 0x7fdad40990c0 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): dump: 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 324746523 0 0) 0x7fdab4003570 con 0x7fdad40a1310 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdad409a010 con 0x7fdad40a1310 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 168538296 0 0) 0x7fdad4099ba0 con 0x7fdad41335a0 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdab4003570 con 0x7fdad41335a0 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdabc0019e0 con 0x7fdad40a1310 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): dump: 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.276 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdacc001c80 con 0x7fdad41335a0 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): dump: 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.277 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.278 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2377830620 0 0) 0x7fdad41200b0 con 0x7fdad40990c0 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.122:6792/0 conn(0x7fdad41335a0 legacy=0x7fdad414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.26:6792/0 conn(0x7fdad40a1310 legacy=0x7fdad4147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 monclient: found mon.a 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4121be0 con 0x7fdad40990c0 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:31.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 5 monclient: authenticate success, global_id 194445 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 10 monclient: _renew_subs 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad4142bb0 con 0x7fdad40990c0 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 10 monclient: _renew_subs 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad4120c80 con 0x7fdad40990c0 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdac4004e30 con 0x7fdad40990c0 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdac40052e0 con 0x7fdad40990c0 2024-09-15T03:56:31.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad1ffb700 4 set_mon_vals no callback set 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient: dump: 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.282 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdac400fca0 con 0x7fdad40990c0 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdab8006590 con 0x7fdab8002840 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(538..538 src has 224..538) v4 ==== 3911+0+0 (unknown 2710781976 0 0) 0x7fdac4043f20 con 0x7fdad40990c0 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdad3fff700 10 monclient: get_auth_request con 0x7fdab40381d0 auth_method 0 2024-09-15T03:56:31.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534844884 0 0) 0x7fdab8006590 con 0x7fdab8002840 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 10 monclient(hunting): my global_id is 135142 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.255+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad41200b0 con 0x7fdab8002840 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4094369257 0 0) 0x7fdad41200b0 con 0x7fdab8002840 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdab8006590 con 0x7fdab8002840 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdabc004b70 con 0x7fdab8002840 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 10 monclient: dump: 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 459574452 0 0) 0x7fdab8006590 con 0x7fdab8002840 2024-09-15T03:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdab800a130 con 0x7fdab8002840 2024-09-15T03:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdabc005860 con 0x7fdab8002840 2024-09-15T03:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.122:6792/0 conn(0x7fdab8002840 legacy=0x7fdab8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.259+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.122:6792/0 conn(0x7fdab8002840 legacy=0x7fdab8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdad9b80700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdab8009640 con 0x7fdab80090b0 2024-09-15T03:56:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281588783 0 0) 0x7fdab8009640 con 0x7fdab80090b0 2024-09-15T03:56:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdacb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdacb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdacb7fe700 10 monclient(hunting): my global_id is 135151 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.267+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab8006590 con 0x7fdab80090b0 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2617126282 0 0) 0x7fdab8006590 con 0x7fdab80090b0 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdab8009640 con 0x7fdab80090b0 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdabc005630 con 0x7fdab80090b0 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 10 monclient: dump: 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.288 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2866783063 0 0) 0x7fdab8009640 con 0x7fdab80090b0 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdab80098f0 con 0x7fdab80090b0 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7570 (unknown 0 0 1941210397) 0x7fdabc005da0 con 0x7fdab80090b0 2024-09-15T03:56:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.122:6792/0 conn(0x7fdab80090b0 legacy=0x7fdab8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdacb7fe700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.122:6792/0 conn(0x7fdab80090b0 legacy=0x7fdab8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:31.290 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 86, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:56:31.291 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:31.292 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:31.293 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:31.294 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:31.295 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:31.296 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:31.297 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.298 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.299 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:31.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:31.301 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:31.302 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:31.303 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:31.304 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:31.305 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.306 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:31.308 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.309 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:31.317 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.318 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:31.320 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:31.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:31.323 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:31.323 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:31.323 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:31.323 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.122:6812/20895 conn(0x7fdab40381d0 legacy=0x7fdab403a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 10 monclient: shutdown 2024-09-15T03:56:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 >> v1:172.21.15.26:6789/0 conn(0x7fdad40990c0 legacy=0x7fdad4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad3fff700 1 -- 172.21.15.26:0/4193085050 reap_dead start 2024-09-15T03:56:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad3fff700 1 -- 172.21.15.26:0/4193085050 reap_dead start 2024-09-15T03:56:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 shutdown_connections 2024-09-15T03:56:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 >> 172.21.15.26:0/4193085050 conn(0x7fdad406e370 msgr2=0x7fdad4099490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 shutdown_connections 2024-09-15T03:56:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.271+0000 7fdad9b80700 1 -- 172.21.15.26:0/4193085050 wait complete. 2024-09-15T03:56:31.326 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:56:31.326 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i'] 2024-09-15T03:56:31.326 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T03:56:31.326 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-15T03:56:31.326 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T03:56:31.327 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T03:56:31.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:56:31.346 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T03:56:31.346 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-15T03:56:31.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:56:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient: get_monmap_and_config 2024-09-15T03:56:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient: build_initial_monmap 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient: monmap: 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:31.457946+0000 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:31.457946+0000 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:31.464 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 1 Processor -- start 2024-09-15T03:56:31.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 1 -- start start 2024-09-15T03:56:31.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient: init 2024-09-15T03:56:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient: _add_conns ranks=[7,3,0,2,4,1,5,6,8] 2024-09-15T03:56:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient(hunting): picked mon.noname-i con 0x7f626009b6e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:31.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient(hunting): picked mon.noname-g con 0x7f626006e160 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient(hunting): picked mon.noname-a con 0x7f6260136d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626004ca10 con 0x7f6260136d30 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6260098880 con 0x7f626006e160 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626006e550 con 0x7f626009b6e0 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f6264899700 10 monclient(hunting): authenticate will time out at 1535.887538s 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f625ed9d700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f626009b6e0 legacy=0x7f6260097a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:31.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f625f59e700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6260136d30 0x7f6260139110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52838/0 (socket says 172.21.15.26:52838) 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.455+0000 7f625f59e700 1 -- 172.21.15.26:0/2241856747 learned_addr learned my addr 172.21.15.26:0/2241856747 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231455106 0 0) 0x7f626004ca10 con 0x7f6260136d30 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient(hunting): my global_id is 194454 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6248003570 con 0x7f6260136d30 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979334858 0 0) 0x7f6260098880 con 0x7f626006e160 2024-09-15T03:56:31.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient(hunting): my global_id is 204258 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626004ca10 con 0x7f626006e160 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 382394344 0 0) 0x7f6248003570 con 0x7f6260136d30 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 >> v1:172.21.15.122:6792/0 conn(0x7f626009b6e0 legacy=0x7f6260097a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 >> v1:172.21.15.122:6790/0 conn(0x7f626006e160 legacy=0x7f6260098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: _finish_hunting 0 2024-09-15T03:56:31.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 monclient: found mon.noname-a 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626009d2e0 con 0x7f6260136d30 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: _finish_auth 0 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 5 monclient: authenticate success, global_id 194454 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:31.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6258001b30 con 0x7f6260136d30 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: dump: 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.479 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:31.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6258001e10 con 0x7f6260136d30 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 1 -- 172.21.15.26:0/2241856747 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6258004fb0 con 0x7f6260136d30 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 4 set_mon_vals no callback set 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f625dd9b700 10 monclient: dump: 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.481 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 10 monclient: shutdown 2024-09-15T03:56:31.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 -- 172.21.15.26:0/2241856747 >> v1:172.21.15.26:6789/0 conn(0x7f6260136d30 legacy=0x7f6260139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 -- 172.21.15.26:0/2241856747 shutdown_connections 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 -- 172.21.15.26:0/2241856747 >> 172.21.15.26:0/2241856747 conn(0x7f6260063d90 msgr2=0x7f62600a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 -- 172.21.15.26:0/2241856747 shutdown_connections 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 -- 172.21.15.26:0/2241856747 wait complete. 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 10 monclient: build_initial_monmap 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 10 monclient: monmap: 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:31.483 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 Processor -- start 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 1 -- start start 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.459+0000 7f6264899700 10 monclient: init 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _add_conns ranks=[5,8,2,6,4,3,7,1,0] 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient(hunting): picked mon.noname-h con 0x7f626006e160 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient(hunting): picked mon.noname-e con 0x7f6260136d30 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:31.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient(hunting): picked mon.noname-b con 0x7f626006bb30 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626006a890 con 0x7f626006bb30 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626009c810 con 0x7f626006e160 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626009cbe0 con 0x7f6260136d30 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient(hunting): authenticate will time out at 1535.892272s 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f625e59c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f6260136d30 0x7f6260069b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50166/0 (socket says 172.21.15.26:50166) 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f625e59c700 1 -- 172.21.15.26:0/259487858 learned_addr learned my addr 172.21.15.26:0/259487858 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801950744 0 0) 0x7f626006a890 con 0x7f626006bb30 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): my global_id is 204230 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f624c003570 con 0x7f626006bb30 2024-09-15T03:56:31.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44094068 0 0) 0x7f626009cbe0 con 0x7f6260136d30 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): my global_id is 135170 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626006a890 con 0x7f6260136d30 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015563269 0 0) 0x7f626009c810 con 0x7f626006e160 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): my global_id is 5756 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626009cbe0 con 0x7f626006e160 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2834132065 0 0) 0x7f624c003570 con 0x7f626006bb30 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f626009c810 con 0x7f626006bb30 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6258004ef0 con 0x7f626006bb30 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.488 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2442336371 0 0) 0x7f626006a890 con 0x7f6260136d30 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f624c003570 con 0x7f6260136d30 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3421012411 0 0) 0x7f626009c810 con 0x7f626006bb30 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 >> v1:172.21.15.26:6793/0 conn(0x7f6260136d30 legacy=0x7f6260069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 >> v1:172.21.15.122:6791/0 conn(0x7f626006e160 legacy=0x7f6260069420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 monclient: found mon.b 2024-09-15T03:56:31.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626006cd70 con 0x7f626006bb30 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 5 monclient: authenticate success, global_id 204230 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _renew_subs 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626006a2a0 con 0x7f626006bb30 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _renew_subs 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f626006ca90 con 0x7f626006bb30 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6258001be0 con 0x7f626006bb30 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6258005520 con 0x7f626006bb30 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 10 monclient: dump: 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f625cd99700 4 set_mon_vals no callback set 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f625800ff60 con 0x7f626006bb30 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626009e6d0 con 0x7f626006bb30 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(538..538 src has 224..538) v4 ==== 3911+0+0 (unknown 2710781976 0 0) 0x7f62580442a0 con 0x7f626006bb30 2024-09-15T03:56:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.463+0000 7f625ed9d700 10 monclient: get_auth_request con 0x7f624c037a10 auth_method 0 2024-09-15T03:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.471+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f626009e6d0 con 0x7f626006bb30 2024-09-15T03:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.471+0000 7f6246ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:31.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.471+0000 7f6246ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6264899700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6264899700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6264899700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f626009e6d0 con 0x7f626006bb30 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6246ffd700 1 -- 172.21.15.26:0/259487858 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 2496173089) 0x7f626009e6d0 con 0x7f626006bb30 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6246ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.695+0000 7f6246ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:31.701 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"i","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 >> v1:172.21.15.122:6812/20895 conn(0x7f624c037a10 legacy=0x7f624c039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 10 monclient: shutdown 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 >> v1:172.21.15.26:6790/0 conn(0x7f626006bb30 legacy=0x7f62601481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 shutdown_connections 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 >> 172.21.15.26:0/259487858 conn(0x7f6260063d90 msgr2=0x7f626012f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 shutdown_connections 2024-09-15T03:56:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:31.699+0000 7f6264899700 1 -- 172.21.15.26:0/259487858 wait complete. 2024-09-15T03:56:34.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:56:34.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.819+0000 7f627fcb3700 10 monclient: get_monmap_and_config 2024-09-15T03:56:34.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.819+0000 7f627fcb3700 10 monclient: build_initial_monmap 2024-09-15T03:56:34.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.819+0000 7f627fcb3700 10 monclient: monmap: 2024-09-15T03:56:34.827 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:34.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:34.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:34.823614+0000 2024-09-15T03:56:34.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:34.823614+0000 2024-09-15T03:56:34.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:34.828 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:34.828 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:34.828 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:34.828 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:34.829 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:34.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 1 Processor -- start 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 1 -- start start 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient: init 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient: _add_conns ranks=[2,1,4,3,5,6,0,7,8] 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient(hunting): picked mon.noname-b con 0x7f6278099310 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient(hunting): picked mon.noname-f con 0x7f6278132fa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient(hunting): picked mon.noname-c con 0x7f62781333f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627804ca10 con 0x7f6278132fa0 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6278097e50 con 0x7f6278099310 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62780a1410 con 0x7f62781333f0 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:34.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient(hunting): authenticate will time out at 1539.253343s 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627da4f700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6278099310 0x7f62780996e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37822/0 (socket says 172.21.15.26:37822) 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627da4f700 1 -- 172.21.15.26:0/3034512524 learned_addr learned my addr 172.21.15.26:0/3034512524 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134392425 0 0) 0x7f62780a1410 con 0x7f62781333f0 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): my global_id is 204268 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6264003570 con 0x7f62781333f0 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630767538 0 0) 0x7f6278097e50 con 0x7f6278099310 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): my global_id is 204239 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62780a1410 con 0x7f6278099310 2024-09-15T03:56:34.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287757835 0 0) 0x7f627804ca10 con 0x7f6278132fa0 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient(hunting): my global_id is 204256 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6278097e50 con 0x7f6278132fa0 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348818045 0 0) 0x7f6264003570 con 0x7f62781333f0 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 >> v1:172.21.15.26:6790/0 conn(0x7f6278099310 legacy=0x7f62780996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 >> v1:172.21.15.122:6789/0 conn(0x7f6278132fa0 legacy=0x7f62781380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: _finish_hunting 0 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 monclient: found mon.noname-c 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780a2710 con 0x7f62781333f0 2024-09-15T03:56:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: _finish_auth 0 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 5 monclient: authenticate success, global_id 204268 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6270001b70 con 0x7f62781333f0 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: dump: 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:34.844 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:34.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6270001d90 con 0x7f62781333f0 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 1 -- 172.21.15.26:0/3034512524 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f62700025c0 con 0x7f62781333f0 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627fcb3700 4 set_mon_vals no callback set 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.823+0000 7f627ca4d700 10 monclient: dump: 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:34.846 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:34.847 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient: shutdown 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- 172.21.15.26:0/3034512524 >> v1:172.21.15.26:6791/0 conn(0x7f62781333f0 legacy=0x7f627813b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- 172.21.15.26:0/3034512524 shutdown_connections 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- 172.21.15.26:0/3034512524 >> 172.21.15.26:0/3034512524 conn(0x7f62780a6c30 msgr2=0x7f627806dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- 172.21.15.26:0/3034512524 shutdown_connections 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- 172.21.15.26:0/3034512524 wait complete. 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient: build_initial_monmap 2024-09-15T03:56:34.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient: monmap: 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:34.849 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 Processor -- start 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- start start 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient: init 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient: _add_conns ranks=[7,2,5,4,3,0,1,6,8] 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient(hunting): picked mon.noname-i con 0x7f6278099310 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient(hunting): picked mon.noname-b con 0x7f6278132fa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient(hunting): picked mon.noname-h con 0x7f62781333f0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627811ff20 con 0x7f6278132fa0 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6278120e80 con 0x7f62781333f0 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6278121440 con 0x7f6278099310 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627fcb3700 10 monclient(hunting): authenticate will time out at 1539.259424s 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627d24e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6278132fa0 0x7f6278147110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:37830/0 (socket says 172.21.15.26:37830) 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627d24e700 1 -- 172.21.15.26:0/671304010 learned_addr learned my addr 172.21.15.26:0/671304010 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f627da4f700 1 -- 172.21.15.26:0/671304010 >> v1:172.21.15.122:6792/0 conn(0x7f6278099310 legacy=0x7f6278142130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:34.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032727441 0 0) 0x7f627811ff20 con 0x7f6278132fa0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f6269ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f6269ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.827+0000 7f6269ffb700 10 monclient(hunting): my global_id is 204248 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f625c003570 con 0x7f6278132fa0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195604560 0 0) 0x7f6278120e80 con 0x7f62781333f0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): my global_id is 5765 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f627811ff20 con 0x7f62781333f0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3921195788 0 0) 0x7f625c003570 con 0x7f6278132fa0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6278120e80 con 0x7f6278132fa0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f626c001b40 con 0x7f6278132fa0 2024-09-15T03:56:34.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): dump: 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2017700692 0 0) 0x7f6278120e80 con 0x7f6278132fa0 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 >> v1:172.21.15.122:6792/0 conn(0x7f6278099310 legacy=0x7f6278142130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 >> v1:172.21.15.122:6791/0 conn(0x7f62781333f0 legacy=0x7f627814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 monclient: found mon.b 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627811fa50 con 0x7f6278132fa0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 5 monclient: authenticate success, global_id 204248 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 monclient: _renew_subs 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6278122030 con 0x7f6278132fa0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 monclient: _renew_subs 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f62780a1f10 con 0x7f6278132fa0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f626c002270 con 0x7f6278132fa0 2024-09-15T03:56:34.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f626c0027a0 con 0x7f6278132fa0 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f626bfff700 4 set_mon_vals no callback set 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 10 monclient: dump: 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:34.856 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f626c002900 con 0x7f6278132fa0 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:34.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(538..538 src has 224..538) v4 ==== 3911+0+0 (unknown 2710781976 0 0) 0x7f626c043eb0 con 0x7f6278132fa0 2024-09-15T03:56:34.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62580050b0 con 0x7f6278132fa0 2024-09-15T03:56:34.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.831+0000 7f627da4f700 10 monclient: get_auth_request con 0x7f625c037600 auth_method 0 2024-09-15T03:56:34.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.839+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f626c014d40 con 0x7f6278132fa0 2024-09-15T03:56:34.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.839+0000 7f6269ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:34.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:34.839+0000 7f6269ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f627fcb3700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:56:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f627fcb3700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f627fcb3700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f62580058e0 con 0x7f6278132fa0 2024-09-15T03:56:35.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f6269ffb700 1 -- 172.21.15.26:0/671304010 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 2883314623) 0x7f626c014740 con 0x7f6278132fa0 2024-09-15T03:56:35.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f6269ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:35.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.063+0000 7f6269ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:35.072 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":86,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"i","quorum_age":9,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 >> v1:172.21.15.122:6812/20895 conn(0x7f625c037600 legacy=0x7f625c039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 10 monclient: shutdown 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 >> v1:172.21.15.26:6790/0 conn(0x7f6278132fa0 legacy=0x7f6278147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 shutdown_connections 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 >> 172.21.15.26:0/671304010 conn(0x7f62780a6c30 msgr2=0x7f6278067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 shutdown_connections 2024-09-15T03:56:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:35.067+0000 7f627fcb3700 1 -- 172.21.15.26:0/671304010 wait complete. 2024-09-15T03:56:38.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:56:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient: get_monmap_and_config 2024-09-15T03:56:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient: build_initial_monmap 2024-09-15T03:56:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient: monmap: 2024-09-15T03:56:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:38.193201+0000 2024-09-15T03:56:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:38.193201+0000 2024-09-15T03:56:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:38.201 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:38.201 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:38.201 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:38.201 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:38.201 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:38.201 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:38.202 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:38.202 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:38.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 1 Processor -- start 2024-09-15T03:56:38.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 1 -- start start 2024-09-15T03:56:38.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient: init 2024-09-15T03:56:38.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:38.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient: _add_conns ranks=[1,6,2,3,7,8,5,0,4] 2024-09-15T03:56:38.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient(hunting): picked mon.noname-f con 0x7f198c099310 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:38.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient(hunting): picked mon.noname-d con 0x7f198c132fa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:38.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient(hunting): picked mon.noname-b con 0x7f198c1333f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:38.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c04ca10 con 0x7f198c099310 2024-09-15T03:56:38.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c097e50 con 0x7f198c1333f0 2024-09-15T03:56:38.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0a1410 con 0x7f198c132fa0 2024-09-15T03:56:38.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:38.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f19921ce700 10 monclient(hunting): authenticate will time out at 1542.622703s 2024-09-15T03:56:38.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f198bfff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f198c1333f0 0x7f198c13b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:36156/0 (socket says 172.21.15.26:36156) 2024-09-15T03:56:38.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:38.190+0000 7f198bfff700 1 -- 172.21.15.26:0/1172178096 learned_addr learned my addr 172.21.15.26:0/1172178096 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient(hunting): tick 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6792/0 conn(0x7f198c132fa0 legacy=0x7f198c1380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6790/0 conn(0x7f198c1333f0 legacy=0x7f198c13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6789/0 conn(0x7f198c099310 legacy=0x7f198c0996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient: _add_conns ranks=[8,0,4,5,3,7] 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-e con 0x7f196c003830 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-a con 0x7f196c0074b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-c con 0x7f196c00b210 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c0a2710 con 0x7f196c0074b0 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c00fff0 con 0x7f196c00b210 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c011240 con 0x7f196c003830 2024-09-15T03:56:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:41.190+0000 7f1989ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:44.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient(hunting): tick 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6793/0 conn(0x7f196c003830 legacy=0x7f196c005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6791/0 conn(0x7f196c00b210 legacy=0x7f196c00d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6789/0 conn(0x7f196c0074b0 legacy=0x7f196c009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient: _add_conns ranks=[3,7,5] 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f198bfff700 1 -- 172.21.15.26:0/1172178096 reap_dead start 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-g con 0x7f196c010400 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f198bfff700 1 -- 172.21.15.26:0/1172178096 reap_dead start 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-i con 0x7f196c00a110 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-h con 0x7f196c00a4e0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0124f0 con 0x7f196c010400 2024-09-15T03:56:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c00a990 con 0x7f196c00a4e0 2024-09-15T03:56:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c00abb0 con 0x7f196c00a110 2024-09-15T03:56:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1989ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.190+0000 7f1983fff700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6792/0 conn(0x7f196c00a110 legacy=0x7f196c005c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:44.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.398+0000 7f1983fff700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6792/0 conn(0x7f196c00a110 legacy=0x7f196c005c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:44.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:44.802+0000 7f1983fff700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6792/0 conn(0x7f196c00a110 legacy=0x7f196c005c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:45.606+0000 7f1983fff700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6792/0 conn(0x7f196c00a110 legacy=0x7f196c005c70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.190+0000 7f1989ffb700 10 monclient(hunting): tick 2024-09-15T03:56:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.190+0000 7f1989ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:56:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.190+0000 7f1989ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:56:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6792/0 conn(0x7f196c00a110 legacy=0x7f196c005c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:56:47.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6791/0 conn(0x7f196c00a4e0 legacy=0x7f196c00d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.190+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6790/0 conn(0x7f196c010400 legacy=0x7f196c0107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 10 monclient: _add_conns ranks=[1,4,6,8,5,3,0,2,7] 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-f con 0x7f196c00deb0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-c con 0x7f196c00e340 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-d con 0x7f196c00e8d0 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c00adf0 con 0x7f196c00deb0 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c00abb0 con 0x7f196c00e340 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c01e6c0 con 0x7f196c00e8d0 2024-09-15T03:56:47.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:47.194+0000 7f1989ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient(hunting): tick 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 monclient(hunting): continuing hunt 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6792/0 conn(0x7f196c00e8d0 legacy=0x7f196c01af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6791/0 conn(0x7f196c00e340 legacy=0x7f196c017810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.122:6789/0 conn(0x7f196c00deb0 legacy=0x7f196c009960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient: _add_conns ranks=[0,2,8,3,5,7] 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-a con 0x7f196c01ea40 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f198bfff700 1 -- 172.21.15.26:0/1172178096 reap_dead start 2024-09-15T03:56:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-b con 0x7f196c020b30 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient(hunting): picked mon.noname-e con 0x7f196c00deb0 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c01f8f0 con 0x7f196c01ea40 2024-09-15T03:56:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0107d0 con 0x7f196c020b30 2024-09-15T03:56:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c010960 con 0x7f196c00deb0 2024-09-15T03:56:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:50.194+0000 7f1989ffb700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551992452 0 0) 0x7f196c010960 con 0x7f196c00deb0 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient(hunting): my global_id is 135179 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1978003570 con 0x7f196c00deb0 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759425031 0 0) 0x7f196c0107d0 con 0x7f196c020b30 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient(hunting): my global_id is 204257 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f196c010960 con 0x7f196c020b30 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2958791313 0 0) 0x7f1978003570 con 0x7f196c00deb0 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6790/0 conn(0x7f196c020b30 legacy=0x7f196c022f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6789/0 conn(0x7f196c01ea40 legacy=0x7f196c01ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 monclient: found mon.noname-e 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c010af0 con 0x7f196c00deb0 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 5 monclient: authenticate success, global_id 135179 2024-09-15T03:56:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f19840019b0 con 0x7f196c00deb0 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: dump: 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1984001c50 con 0x7f196c00deb0 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 1 -- 172.21.15.26:0/1172178096 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1984004c70 con 0x7f196c00deb0 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 4 set_mon_vals no callback set 2024-09-15T03:56:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f198affd700 10 monclient: dump: 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.125 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 10 monclient: shutdown 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 1 -- 172.21.15.26:0/1172178096 >> v1:172.21.15.26:6793/0 conn(0x7f196c00deb0 legacy=0x7f196c00d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 1 -- 172.21.15.26:0/1172178096 shutdown_connections 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 1 -- 172.21.15.26:0/1172178096 >> 172.21.15.26:0/1172178096 conn(0x7f198c0a6c30 msgr2=0x7f198c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 1 -- 172.21.15.26:0/1172178096 shutdown_connections 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.114+0000 7f19921ce700 1 -- 172.21.15.26:0/1172178096 wait complete. 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient: monmap: 2024-09-15T03:56:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 1 Processor -- start 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 1 -- start start 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient: init 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient: _add_conns ranks=[7,1,3,8,6,5,0,2,4] 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient(hunting): picked mon.noname-i con 0x7f196c00deb0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient(hunting): picked mon.noname-f con 0x7f196c01ea40 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient(hunting): picked mon.noname-g con 0x7f196c020b30 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c120340 con 0x7f196c01ea40 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c11ee90 con 0x7f196c020b30 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c120060 con 0x7f196c00deb0 2024-09-15T03:56:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f19921ce700 10 monclient(hunting): authenticate will time out at 1555.549982s 2024-09-15T03:56:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f198b7fe700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f196c00deb0 legacy=0x7f198c12fc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f1983fff700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f196c01ea40 0x7f198c1470f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:56528/0 (socket says 172.21.15.26:56528) 2024-09-15T03:56:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f1983fff700 1 -- 172.21.15.26:0/3220228388 learned_addr learned my addr 172.21.15.26:0/3220228388 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.118+0000 7f198bfff700 1 --1- 172.21.15.26:0/3220228388 >> v1:172.21.15.122:6790/0 conn(0x7f196c020b30 0x7f198c14a820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:41634/0 (socket says 172.21.15.26:41634) 2024-09-15T03:56:51.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526087803 0 0) 0x7f198c120340 con 0x7f196c01ea40 2024-09-15T03:56:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 10 monclient(hunting): my global_id is 204265 2024-09-15T03:56:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f196c0107d0 con 0x7f196c01ea40 2024-09-15T03:56:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001487594 0 0) 0x7f198c11ee90 con 0x7f196c020b30 2024-09-15T03:56:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 10 monclient(hunting): my global_id is 204267 2024-09-15T03:56:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.134+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198c120340 con 0x7f196c020b30 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 666801205 0 0) 0x7f198c120340 con 0x7f196c020b30 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f198c11ee90 con 0x7f196c020b30 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f19840019b0 con 0x7f196c020b30 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient(hunting): dump: 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2826008923 0 0) 0x7f198c11ee90 con 0x7f196c020b30 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 >> v1:172.21.15.122:6792/0 conn(0x7f196c00deb0 legacy=0x7f198c12fc30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 >> v1:172.21.15.122:6789/0 conn(0x7f196c01ea40 legacy=0x7f198c1470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 monclient: found mon.g 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c11fa70 con 0x7f196c020b30 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 5 monclient: authenticate success, global_id 204267 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 10 monclient: _renew_subs 2024-09-15T03:56:51.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c1342a0 con 0x7f196c020b30 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 10 monclient: _renew_subs 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c130340 con 0x7f196c020b30 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1984001290 con 0x7f196c020b30 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1984005220 con 0x7f196c020b30 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f19837fe700 10 monclient: dump: 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.153 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.142+0000 7f198a7fc700 4 set_mon_vals no callback set 2024-09-15T03:56:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f198400fbe0 con 0x7f196c020b30 2024-09-15T03:56:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f19921ce700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f19921ce700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f19921ce700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:56:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c130340 con 0x7f196c020b30 2024-09-15T03:56:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 8 ==== osd_map(538..538 src has 224..538) v4 ==== 3911+0+0 (unknown 2710781976 0 0) 0x7f1984043dc0 con 0x7f196c020b30 2024-09-15T03:56:51.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.146+0000 7f198b7fe700 10 monclient: get_auth_request con 0x7f196c03ad80 auth_method 0 2024-09-15T03:56:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.150+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f19840148f0 con 0x7f196c020b30 2024-09-15T03:56:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.150+0000 7f19837fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.150+0000 7f19837fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:51.192 INFO:tasks.rados.rados.0.smithi122.stdout:2360: done (3 left) 2024-09-15T03:56:51.192 INFO:tasks.rados.rados.0.smithi122.stdout:2363: done (2 left) 2024-09-15T03:56:51.192 INFO:tasks.rados.rados.0.smithi122.stdout:2364: done (1 left) 2024-09-15T03:56:51.192 INFO:tasks.rados.rados.0.smithi122.stdout:2365: done (0 left) 2024-09-15T03:56:51.193 INFO:tasks.rados.rados.0.smithi122.stdout:2366: write oid 30 current snap is 273 2024-09-15T03:56:51.193 INFO:tasks.rados.rados.0.smithi122.stdout:2366: seq_num 844 ranges {585108=425339,1788920=396188} 2024-09-15T03:56:51.199 INFO:tasks.rados.rados.0.smithi122.stdout:2366: writing smithi12223906-30 from 585108 to 1010447 tid 1 2024-09-15T03:56:51.206 INFO:tasks.rados.rados.0.smithi122.stdout:2366: writing smithi12223906-30 from 1788920 to 2185108 tid 2 2024-09-15T03:56:51.206 INFO:tasks.rados.rados.0.smithi122.stdout:2367: write (excl) oid 1 current snap is 273 2024-09-15T03:56:51.206 INFO:tasks.rados.rados.0.smithi122.stdout:2367: seq_num 845 ranges {601915=500588,1847402=354513} 2024-09-15T03:56:51.212 INFO:tasks.rados.rados.0.smithi122.stdout:2367: writing smithi12223906-1 from 601915 to 1102503 tid 1 2024-09-15T03:56:51.219 INFO:tasks.rados.rados.0.smithi122.stdout:2367: writing smithi12223906-1 from 1847402 to 2201915 tid 2 2024-09-15T03:56:51.219 INFO:tasks.rados.rados.0.smithi122.stdout:2368: write oid 13 current snap is 273 2024-09-15T03:56:51.219 INFO:tasks.rados.rados.0.smithi122.stdout:2366: finishing write tid 1 to smithi12223906-30 2024-09-15T03:56:51.219 INFO:tasks.rados.rados.0.smithi122.stdout:2368: seq_num 846 ranges {618722=575837,1905884=312838} 2024-09-15T03:56:51.225 INFO:tasks.rados.rados.0.smithi122.stdout:2368: writing smithi12223906-13 from 618722 to 1194559 tid 1 2024-09-15T03:56:51.231 INFO:tasks.rados.rados.0.smithi122.stdout:2368: writing smithi12223906-13 from 1905884 to 2218722 tid 2 2024-09-15T03:56:51.231 INFO:tasks.rados.rados.0.smithi122.stdout:2369: read oid 27 snap 267 2024-09-15T03:56:51.231 INFO:tasks.rados.rados.0.smithi122.stdout:2369: expect deleted 2024-09-15T03:56:51.232 INFO:tasks.rados.rados.0.smithi122.stdout:2370: write oid 6 current snap is 273 2024-09-15T03:56:51.232 INFO:tasks.rados.rados.0.smithi122.stdout:2366: finishing write tid 2 to smithi12223906-30 2024-09-15T03:56:51.232 INFO:tasks.rados.rados.0.smithi122.stdout:2370: seq_num 847 ranges {635529=651086,1848013=387516} 2024-09-15T03:56:51.238 INFO:tasks.rados.rados.0.smithi122.stdout:2370: writing smithi12223906-6 from 635529 to 1286615 tid 1 2024-09-15T03:56:51.244 INFO:tasks.rados.rados.0.smithi122.stdout:2370: writing smithi12223906-6 from 1848013 to 2235529 tid 2 2024-09-15T03:56:51.244 INFO:tasks.rados.rados.0.smithi122.stdout:2371: snap_create 2024-09-15T03:56:51.244 INFO:tasks.rados.rados.0.smithi122.stdout:2366: finishing write tid 3 to smithi12223906-30 2024-09-15T03:56:51.244 INFO:tasks.rados.rados.0.smithi122.stdout:2366: finishing write tid 4 to smithi12223906-30 2024-09-15T03:56:51.244 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 824 (ObjNum 844 snap 273 seq_num 844) dirty exists 2024-09-15T03:56:51.244 INFO:tasks.rados.rados.0.smithi122.stdout:2366: left oid 30 (ObjNum 844 snap 273 seq_num 844) 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2367: finishing write tid 1 to smithi12223906-1 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2367: finishing write tid 2 to smithi12223906-1 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2367: finishing write tid 3 to smithi12223906-1 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2367: finishing write tid 4 to smithi12223906-1 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1304 (ObjNum 845 snap 273 seq_num 845) dirty exists 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2367: left oid 1 (ObjNum 845 snap 273 seq_num 845) 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2368: finishing write tid 1 to smithi12223906-13 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2368: finishing write tid 2 to smithi12223906-13 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2368: finishing write tid 3 to smithi12223906-13 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2368: finishing write tid 4 to smithi12223906-13 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 830 (ObjNum 846 snap 273 seq_num 846) dirty exists 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2368: left oid 13 (ObjNum 846 snap 273 seq_num 846) 2024-09-15T03:56:51.245 INFO:tasks.rados.rados.0.smithi122.stdout:2370: finishing write tid 1 to smithi12223906-6 2024-09-15T03:56:51.247 INFO:tasks.rados.rados.0.smithi122.stdout:2370: finishing write tid 2 to smithi12223906-6 2024-09-15T03:56:51.248 INFO:tasks.rados.rados.0.smithi122.stdout:2370: finishing write tid 3 to smithi12223906-6 2024-09-15T03:56:51.248 INFO:tasks.rados.rados.0.smithi122.stdout:2370: finishing write tid 4 to smithi12223906-6 2024-09-15T03:56:51.248 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 839 (ObjNum 847 snap 273 seq_num 847) dirty exists 2024-09-15T03:56:51.248 INFO:tasks.rados.rados.0.smithi122.stdout:2370: left oid 6 (ObjNum 847 snap 273 seq_num 847) 2024-09-15T03:56:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19921ce700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:56:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19921ce700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19921ce700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:56:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 --> v1:172.21.15.122:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f198c130340 con 0x7f196c020b30 2024-09-15T03:56:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19837fe700 1 -- 172.21.15.26:0/3220228388 <== mon.3 v1:172.21.15.122:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2694 (unknown 254746657 0 1043502839) 0x7f1984014710 con 0x7f196c020b30 2024-09-15T03:56:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19837fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:56:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.390+0000 7f19837fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:51.398 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":92,"quorum":[0,1,2,3,4,5,6,8],"quorum_names":["a","f","b","g","c","h","d","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:56:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 >> v1:172.21.15.122:6812/20895 conn(0x7f196c03ad80 legacy=0x7f196c03d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 10 monclient: shutdown 2024-09-15T03:56:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 >> v1:172.21.15.122:6790/0 conn(0x7f196c020b30 legacy=0x7f198c14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 shutdown_connections 2024-09-15T03:56:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 >> 172.21.15.26:0/3220228388 conn(0x7f198c0a6c30 msgr2=0x7f198c067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 shutdown_connections 2024-09-15T03:56:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.394+0000 7f19921ce700 1 -- 172.21.15.26:0/3220228388 wait complete. 2024-09-15T03:56:51.413 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-15T03:56:51.413 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-15T03:56:51.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: get_monmap_and_config 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: monmap: 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:51.516207+0000 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:51.516207+0000 2024-09-15T03:56:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.520 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 Processor -- start 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 -- start start 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: init 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: _add_conns ranks=[4,2,5,8,7,1,0,3,6] 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient(hunting): picked mon.noname-c con 0x7fac8c132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient(hunting): picked mon.noname-b con 0x7fac8c09faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient(hunting): picked mon.noname-h con 0x7fac8c138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c04ca10 con 0x7fac8c09faa0 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c09d5d0 con 0x7fac8c132b40 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c133620 con 0x7fac8c138d40 2024-09-15T03:56:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient(hunting): authenticate will time out at 1555.945889s 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac91cda700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fac8c09faa0 0x7fac8c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:32876/0 (socket says 172.21.15.26:32876) 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac91cda700 1 -- 172.21.15.26:0/3880046314 learned_addr learned my addr 172.21.15.26:0/3880046314 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485886197 0 0) 0x7fac8c09d5d0 con 0x7fac8c132b40 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): my global_id is 204277 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac78003570 con 0x7fac8c132b40 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718552424 0 0) 0x7fac8c04ca10 con 0x7fac8c09faa0 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): my global_id is 204266 2024-09-15T03:56:51.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac8c09d5d0 con 0x7fac8c09faa0 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451600907 0 0) 0x7fac8c133620 con 0x7fac8c138d40 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient(hunting): my global_id is 5774 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac8c04ca10 con 0x7fac8c138d40 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1334175136 0 0) 0x7fac8c09d5d0 con 0x7fac8c09faa0 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 >> v1:172.21.15.122:6791/0 conn(0x7fac8c138d40 legacy=0x7fac8c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 >> v1:172.21.15.26:6791/0 conn(0x7fac8c132b40 legacy=0x7fac8c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 monclient: found mon.noname-b 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:56:51.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c0a0cc0 con 0x7fac8c09faa0 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac88001790 con 0x7fac8c09faa0 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: dump: 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.526 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 5 monclient: authenticate success, global_id 204266 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fac88001a40 con 0x7fac8c09faa0 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 1 -- 172.21.15.26:0/3880046314 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac88004c30 con 0x7fac8c09faa0 2024-09-15T03:56:51.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac914d9700 10 monclient: dump: 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.528 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 4 set_mon_vals no callback set 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 10 monclient: shutdown 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 -- 172.21.15.26:0/3880046314 >> v1:172.21.15.26:6790/0 conn(0x7fac8c09faa0 legacy=0x7fac8c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.514+0000 7fac934dd700 1 -- 172.21.15.26:0/3880046314 shutdown_connections 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- 172.21.15.26:0/3880046314 >> 172.21.15.26:0/3880046314 conn(0x7fac8c06e370 msgr2=0x7fac8c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- 172.21.15.26:0/3880046314 shutdown_connections 2024-09-15T03:56:51.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- 172.21.15.26:0/3880046314 wait complete. 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: monmap: 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.535 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 Processor -- start 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- start start 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: init 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: _add_conns ranks=[5,8,0,2,7,3,6,1,4] 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient(hunting): picked mon.noname-h con 0x7fac8c09faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:51.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient(hunting): picked mon.noname-e con 0x7fac8c132b40 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient(hunting): picked mon.noname-a con 0x7fac8c138d40 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c099f90 con 0x7fac8c138d40 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c09bb70 con 0x7fac8c09faa0 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c09cfa0 con 0x7fac8c132b40 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac91cda700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fac8c132b40 0x7fac8c098620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:42766/0 (socket says 172.21.15.26:42766) 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac91cda700 1 -- 172.21.15.26:0/3528630357 learned_addr learned my addr 172.21.15.26:0/3528630357 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient(hunting): authenticate will time out at 1555.950419s 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570678911 0 0) 0x7fac8c099f90 con 0x7fac8c138d40 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): my global_id is 194463 2024-09-15T03:56:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac70003570 con 0x7fac8c138d40 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013503248 0 0) 0x7fac78003570 con 0x7fac8c09faa0 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): my global_id is 5783 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac8c099f90 con 0x7fac8c09faa0 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224752064 0 0) 0x7fac8c09cfa0 con 0x7fac8c132b40 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): my global_id is 135188 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac78003570 con 0x7fac8c132b40 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 555378736 0 0) 0x7fac70003570 con 0x7fac8c138d40 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac8c09cfa0 con 0x7fac8c138d40 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac84001d00 con 0x7fac8c138d40 2024-09-15T03:56:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): dump: 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1363828296 0 0) 0x7fac8c099f90 con 0x7fac8c09faa0 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac70003570 con 0x7fac8c09faa0 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac7c001a80 con 0x7fac8c09faa0 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient(hunting): dump: 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3354490414 0 0) 0x7fac8c09cfa0 con 0x7fac8c138d40 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.26:6793/0 conn(0x7fac8c132b40 legacy=0x7fac8c098620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.122:6791/0 conn(0x7fac8c09faa0 legacy=0x7fac8c097f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 monclient: found mon.a 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c09ccc0 con 0x7fac8c138d40 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac827fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 5 monclient: authenticate success, global_id 194463 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: _renew_subs 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c099a10 con 0x7fac8c138d40 2024-09-15T03:56:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: _renew_subs 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.518+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c09c200 con 0x7fac8c138d40 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fac840030d0 con 0x7fac8c138d40 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac840025e0 con 0x7fac8c138d40 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac90cd8700 4 set_mon_vals no callback set 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: dump: 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fac840027c0 con 0x7fac8c138d40 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac934dd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fac74006590 con 0x7fac74002840 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(539..539 src has 224..539) v4 ==== 3911+0+0 (unknown 2512508809 0 0) 0x7fac840418a0 con 0x7fac8c138d40 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771395321 0 0) 0x7fac74006590 con 0x7fac74002840 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient(hunting): my global_id is 194472 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac8c09cfa0 con 0x7fac74002840 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac91cda700 10 monclient: get_auth_request con 0x7fac70037cb0 auth_method 0 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1509315020 0 0) 0x7fac8c09cfa0 con 0x7fac74002840 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac74006590 con 0x7fac74002840 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac7c002290 con 0x7fac74002840 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: dump: 2024-09-15T03:56:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 745393433 0 0) 0x7fac74006590 con 0x7fac74002840 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac7400a130 con 0x7fac74002840 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fac7c002800 con 0x7fac74002840 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.26:6789/0 conn(0x7fac74002840 legacy=0x7fac74004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.522+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.26:6789/0 conn(0x7fac74002840 legacy=0x7fac74004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:51.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fac74007520 con 0x7fac740068d0 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940661089 0 0) 0x7fac74007520 con 0x7fac740068d0 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient(hunting): my global_id is 194481 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac74006590 con 0x7fac740068d0 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1452075990 0 0) 0x7fac74006590 con 0x7fac740068d0 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac74007520 con 0x7fac740068d0 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fac7c002ca0 con 0x7fac740068d0 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient: dump: 2024-09-15T03:56:51.549 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.550 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 161758156 0 0) 0x7fac74007520 con 0x7fac740068d0 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fac74008760 con 0x7fac740068d0 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 3968833466) 0x7fac7c0053a0 con 0x7fac740068d0 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.26:6789/0 conn(0x7fac740068d0 legacy=0x7fac74006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac827fc700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.26:6789/0 conn(0x7fac740068d0 legacy=0x7fac74006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:51.571 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.122:6812/20895 conn(0x7fac70037cb0 legacy=0x7fac7003a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 10 monclient: shutdown 2024-09-15T03:56:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 >> v1:172.21.15.26:6789/0 conn(0x7fac8c138d40 legacy=0x7fac8c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac92cdc700 1 -- 172.21.15.26:0/3528630357 reap_dead start 2024-09-15T03:56:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac92cdc700 1 -- 172.21.15.26:0/3528630357 reap_dead start 2024-09-15T03:56:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 shutdown_connections 2024-09-15T03:56:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 >> 172.21.15.26:0/3528630357 conn(0x7fac8c06e370 msgr2=0x7fac8c0669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 shutdown_connections 2024-09-15T03:56:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.530+0000 7fac934dd700 1 -- 172.21.15.26:0/3528630357 wait complete. 2024-09-15T03:56:51.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:51.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-15T03:56:51.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: get_monmap_and_config 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: monmap: 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:51.688072+0000 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:51.688072+0000 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.692 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 1 Processor -- start 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 1 -- start start 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: init 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: _add_conns ranks=[2,4,6,3,5,0,8,1,7] 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient(hunting): picked mon.noname-b con 0x7f915c0a6880 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f915c09faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient(hunting): picked mon.noname-d con 0x7f915c138d30 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:51.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c04ca10 con 0x7f915c0a6880 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c0a0580 con 0x7f915c09faa0 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c133780 con 0x7f915c138d30 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient(hunting): authenticate will time out at 1556.117864s 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915affd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f915c0a6880 0x7f915c133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:32878/0 (socket says 172.21.15.26:32878) 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f915c138d30 0x7f915c13b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:47294/0 (socket says 172.21.15.26:47294) 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915affd700 1 -- 172.21.15.26:0/48261497 learned_addr learned my addr 172.21.15.26:0/48261497 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915a7fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f915c09faa0 0x7f915c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:59762/0 (socket says 172.21.15.26:59762) 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793648476 0 0) 0x7f915c04ca10 con 0x7f915c0a6880 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): my global_id is 204275 2024-09-15T03:56:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9144003570 con 0x7f915c0a6880 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288891527 0 0) 0x7f914c001110 con 0x7f915c09faa0 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): my global_id is 204286 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c04ca10 con 0x7f915c09faa0 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162158887 0 0) 0x7f915c133780 con 0x7f915c138d30 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient(hunting): my global_id is 194496 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f914c001110 con 0x7f915c138d30 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275105120 0 0) 0x7f9144003570 con 0x7f915c0a6880 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 >> v1:172.21.15.26:6792/0 conn(0x7f915c138d30 legacy=0x7f915c13b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 >> v1:172.21.15.26:6791/0 conn(0x7f915c09faa0 legacy=0x7f915c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 monclient: found mon.noname-b 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0a6c50 con 0x7f915c0a6880 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 5 monclient: authenticate success, global_id 204275 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91500016e0 con 0x7f915c0a6880 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: dump: 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f91500019c0 con 0x7f915c0a6880 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 1 -- 172.21.15.26:0/48261497 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f91500048f0 con 0x7f915c0a6880 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f915bfff700 4 set_mon_vals no callback set 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.686+0000 7f9159ffb700 10 monclient: dump: 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.700 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: shutdown 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- 172.21.15.26:0/48261497 >> v1:172.21.15.26:6790/0 conn(0x7f915c0a6880 legacy=0x7f915c133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- 172.21.15.26:0/48261497 shutdown_connections 2024-09-15T03:56:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- 172.21.15.26:0/48261497 >> 172.21.15.26:0/48261497 conn(0x7f915c06e2a0 msgr2=0x7f915c0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- 172.21.15.26:0/48261497 shutdown_connections 2024-09-15T03:56:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- 172.21.15.26:0/48261497 wait complete. 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: monmap: 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:51.707 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 Processor -- start 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- start start 2024-09-15T03:56:51.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: init 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: _add_conns ranks=[3,4,2,8,6,1,7,0,5] 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient(hunting): picked mon.noname-g con 0x7f915c09faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f915c0a6880 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient(hunting): picked mon.noname-b con 0x7f915c138d30 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c09b9d0 con 0x7f915c138d30 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c11d2a0 con 0x7f915c09faa0 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915c09b4e0 con 0x7f915c0a6880 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient(hunting): authenticate will time out at 1556.122319s 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915a7fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f915c0a6880 0x7f915c099be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:59766/0 (socket says 172.21.15.26:59766) 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915b7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f915c138d30 0x7f915c14a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:32882/0 (socket says 172.21.15.26:32882) 2024-09-15T03:56:51.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915a7fc700 1 -- 172.21.15.26:0/3230012143 learned_addr learned my addr 172.21.15.26:0/3230012143 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515080043 0 0) 0x7f915c09b9d0 con 0x7f915c138d30 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): my global_id is 204284 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9148003570 con 0x7f915c138d30 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916322175 0 0) 0x7f915c09b4e0 con 0x7f915c0a6880 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): my global_id is 204295 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c09b9d0 con 0x7f915c0a6880 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200724230 0 0) 0x7f915c11d2a0 con 0x7f915c09faa0 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): my global_id is 204276 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c09b4e0 con 0x7f915c09faa0 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2276145775 0 0) 0x7f9148003570 con 0x7f915c138d30 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f915c11d2a0 con 0x7f915c138d30 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9154001c00 con 0x7f915c138d30 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.711 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1713460561 0 0) 0x7f915c09b9d0 con 0x7f915c0a6880 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9148003570 con 0x7f915c0a6880 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f914c0016a0 con 0x7f915c0a6880 2024-09-15T03:56:51.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient(hunting): dump: 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.713 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 101354867 0 0) 0x7f915c11d2a0 con 0x7f915c138d30 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.26:6791/0 conn(0x7f915c0a6880 legacy=0x7f915c099be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.122:6790/0 conn(0x7f915c09faa0 legacy=0x7f915c0994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 monclient: found mon.b 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0a18c0 con 0x7f915c138d30 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f9142ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 5 monclient: authenticate success, global_id 204284 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: _renew_subs 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.690+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c09cf10 con 0x7f915c138d30 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f915bfff700 10 monclient: _renew_subs 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f915bfff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c09afd0 con 0x7f915c138d30 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9154004bb0 con 0x7f915c138d30 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9154005060 con 0x7f915c138d30 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f91597fa700 4 set_mon_vals no callback set 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: dump: 2024-09-15T03:56:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f915400fa20 con 0x7f915c138d30 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f915bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f913c006590 con 0x7f913c002840 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(539..539 src has 224..539) v4 ==== 3911+0+0 (unknown 2512508809 0 0) 0x7f9154044050 con 0x7f915c138d30 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f915a7fc700 10 monclient: get_auth_request con 0x7f9148037c90 auth_method 0 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254134292 0 0) 0x7f913c006590 con 0x7f913c002840 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient(hunting): my global_id is 204293 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915c11d2a0 con 0x7f913c002840 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3357986379 0 0) 0x7f915c11d2a0 con 0x7f913c002840 2024-09-15T03:56:51.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f913c006590 con 0x7f913c002840 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9150004940 con 0x7f913c002840 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: dump: 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.718 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2576389961 0 0) 0x7f913c006590 con 0x7f913c002840 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f913c00a130 con 0x7f913c002840 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9150007210 con 0x7f913c002840 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.26:6790/0 conn(0x7f913c002840 legacy=0x7f913c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.694+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.26:6790/0 conn(0x7f913c002840 legacy=0x7f913c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f915bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f913c007520 con 0x7f913c0068d0 2024-09-15T03:56:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348839168 0 0) 0x7f913c007520 con 0x7f913c0068d0 2024-09-15T03:56:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient(hunting): my global_id is 204302 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f913c006590 con 0x7f913c0068d0 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2045556554 0 0) 0x7f913c006590 con 0x7f913c0068d0 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f913c007520 con 0x7f913c0068d0 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9150005a10 con 0x7f913c0068d0 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient: dump: 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.722 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3102692879 0 0) 0x7f913c007520 con 0x7f913c0068d0 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f913c008760 con 0x7f913c0068d0 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 664634466) 0x7f915000a200 con 0x7f913c0068d0 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:51.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.26:6790/0 conn(0x7f913c0068d0 legacy=0x7f913c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.702+0000 7f9142ffd700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.26:6790/0 conn(0x7f913c0068d0 legacy=0x7f913c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:51.724 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:51.743 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:51.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.122:6812/20895 conn(0x7f9148037c90 legacy=0x7f914803a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 10 monclient: shutdown 2024-09-15T03:56:51.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 >> v1:172.21.15.26:6790/0 conn(0x7f915c138d30 legacy=0x7f915c14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915b7fe700 1 -- 172.21.15.26:0/3230012143 reap_dead start 2024-09-15T03:56:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 shutdown_connections 2024-09-15T03:56:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 >> 172.21.15.26:0/3230012143 conn(0x7f915c06e2a0 msgr2=0x7f915c09d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 shutdown_connections 2024-09-15T03:56:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.706+0000 7f915bfff700 1 -- 172.21.15.26:0/3230012143 wait complete. 2024-09-15T03:56:51.745 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:51.745 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-15T03:56:51.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:56:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.858+0000 7f1e1a6d4700 10 monclient: get_monmap_and_config 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.858+0000 7f1e1a6d4700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.858+0000 7f1e1a6d4700 10 monclient: monmap: 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:51.863664+0000 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:51.863664+0000 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.868 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 1 Processor -- start 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 1 -- start start 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient: init 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient: _add_conns ranks=[8,5,4,7,0,6,1,3,2] 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient(hunting): picked mon.noname-e con 0x7f1e140a6880 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient(hunting): picked mon.noname-h con 0x7f1e1409faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient(hunting): picked mon.noname-c con 0x7f1e14138d30 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1404ca10 con 0x7f1e14138d30 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140a0580 con 0x7f1e1409faa0 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e14133780 con 0x7f1e140a6880 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient(hunting): authenticate will time out at 1556.293137s 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e19ed3700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f1e14138d30 0x7f1e1413b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:59772/0 (socket says 172.21.15.26:59772) 2024-09-15T03:56:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e19ed3700 1 -- 172.21.15.26:0/3984599718 learned_addr learned my addr 172.21.15.26:0/3984599718 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259844758 0 0) 0x7f1e14133780 con 0x7f1e140a6880 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): my global_id is 135197 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc003570 con 0x7f1e140a6880 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309062875 0 0) 0x7f1e1404ca10 con 0x7f1e14138d30 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): my global_id is 204304 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e14133780 con 0x7f1e14138d30 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30889778 0 0) 0x7f1e04001110 con 0x7f1e1409faa0 2024-09-15T03:56:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient(hunting): my global_id is 5792 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1404ca10 con 0x7f1e1409faa0 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 475077070 0 0) 0x7f1dfc003570 con 0x7f1e140a6880 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 >> v1:172.21.15.122:6791/0 conn(0x7f1e1409faa0 legacy=0x7f1e1409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 >> v1:172.21.15.26:6791/0 conn(0x7f1e14138d30 legacy=0x7f1e1413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 monclient: found mon.noname-e 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140a6c50 con 0x7f1e140a6880 2024-09-15T03:56:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e0c001ab0 con 0x7f1e140a6880 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 5 monclient: authenticate success, global_id 135197 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: dump: 2024-09-15T03:56:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e0c001cd0 con 0x7f1e140a6880 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 1 -- 172.21.15.26:0/3984599718 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e0c002200 con 0x7f1e140a6880 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 4 set_mon_vals no callback set 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e0bfff700 10 monclient: dump: 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.877 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 10 monclient: shutdown 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.862+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/3984599718 >> v1:172.21.15.26:6793/0 conn(0x7f1e140a6880 legacy=0x7f1e14133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/3984599718 shutdown_connections 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/3984599718 >> 172.21.15.26:0/3984599718 conn(0x7f1e1406e2a0 msgr2=0x7f1e140a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/3984599718 shutdown_connections 2024-09-15T03:56:51.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/3984599718 wait complete. 2024-09-15T03:56:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient: build_initial_monmap 2024-09-15T03:56:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient: monmap: 2024-09-15T03:56:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:51.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 Processor -- start 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- start start 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient: init 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient: _add_conns ranks=[7,6,3,8,0,1,4,5,2] 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient(hunting): picked mon.noname-i con 0x7f1e1409faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient(hunting): picked mon.noname-d con 0x7f1e140a6880 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient(hunting): picked mon.noname-g con 0x7f1e14138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140a18c0 con 0x7f1e14138d30 2024-09-15T03:56:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1409b800 con 0x7f1e140a6880 2024-09-15T03:56:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1409b550 con 0x7f1e1409faa0 2024-09-15T03:56:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e1a6d4700 10 monclient(hunting): authenticate will time out at 1556.299186s 2024-09-15T03:56:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e18ed1700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1e140a6880 0x7f1e14099c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:47302/0 (socket says 172.21.15.26:47302) 2024-09-15T03:56:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e18ed1700 1 -- 172.21.15.26:0/1439597179 learned_addr learned my addr 172.21.15.26:0/1439597179 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e196d2700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.122:6792/0 conn(0x7f1e1409faa0 legacy=0x7f1e14099540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281392324 0 0) 0x7f1e1409b800 con 0x7f1e140a6880 2024-09-15T03:56:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e097fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e097fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e097fa700 10 monclient(hunting): my global_id is 194505 2024-09-15T03:56:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e00003570 con 0x7f1e140a6880 2024-09-15T03:56:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.866+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3426933062 0 0) 0x7f1e00003570 con 0x7f1e140a6880 2024-09-15T03:56:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e1409b800 con 0x7f1e140a6880 2024-09-15T03:56:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e04001bf0 con 0x7f1e140a6880 2024-09-15T03:56:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): dump: 2024-09-15T03:56:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771687510 0 0) 0x7f1e140a18c0 con 0x7f1e14138d30 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): my global_id is 204285 2024-09-15T03:56:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e00003570 con 0x7f1e14138d30 2024-09-15T03:56:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1303411765 0 0) 0x7f1e1409b800 con 0x7f1e140a6880 2024-09-15T03:56:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.122:6792/0 conn(0x7f1e1409faa0 legacy=0x7f1e14099540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.122:6790/0 conn(0x7f1e14138d30 legacy=0x7f1e1409a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: _finish_hunting 0 2024-09-15T03:56:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 monclient: found mon.d 2024-09-15T03:56:51.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1409b040 con 0x7f1e140a6880 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: _finish_auth 0 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 5 monclient: authenticate success, global_id 194505 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 10 monclient: _renew_subs 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1409bc10 con 0x7f1e140a6880 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 10 monclient: _renew_subs 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1414b390 con 0x7f1e140a6880 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e04002250 con 0x7f1e140a6880 2024-09-15T03:56:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e04002780 con 0x7f1e140a6880 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: dump: 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.897 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e0affd700 4 set_mon_vals no callback set 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1e04002960 con 0x7f1e140a6880 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1df4006590 con 0x7f1df4002840 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(539..539 src has 224..539) v4 ==== 3911+0+0 (unknown 2512508809 0 0) 0x7f1e04043f40 con 0x7f1e140a6880 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423508806 0 0) 0x7f1df4006590 con 0x7f1df4002840 2024-09-15T03:56:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient(hunting): my global_id is 204313 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1409b800 con 0x7f1df4002840 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e19ed3700 10 monclient: get_auth_request con 0x7f1e00037480 auth_method 0 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4128866153 0 0) 0x7f1e1409b800 con 0x7f1df4002840 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1df4006590 con 0x7f1df4002840 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e0c0018d0 con 0x7f1df4002840 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: dump: 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.899 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2652817381 0 0) 0x7f1df4006590 con 0x7f1df4002840 2024-09-15T03:56:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1df400a130 con 0x7f1df4002840 2024-09-15T03:56:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1e0c006fc0 con 0x7f1df4002840 2024-09-15T03:56:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.26:6791/0 conn(0x7f1df4002840 legacy=0x7f1df4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.870+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.26:6791/0 conn(0x7f1df4002840 legacy=0x7f1df4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e1a6d4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1df4009640 con 0x7f1df40090b0 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692174618 0 0) 0x7f1df4009640 con 0x7f1df40090b0 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 10 monclient(hunting): my global_id is 204322 2024-09-15T03:56:51.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df4006590 con 0x7f1df40090b0 2024-09-15T03:56:51.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 907732040 0 0) 0x7f1df4006590 con 0x7f1df40090b0 2024-09-15T03:56:51.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1df4009640 con 0x7f1df40090b0 2024-09-15T03:56:51.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e0c005530 con 0x7f1df40090b0 2024-09-15T03:56:51.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:51.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 10 monclient: dump: 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 478401343 0 0) 0x7f1df4009640 con 0x7f1df40090b0 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.878+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1df40098f0 con 0x7f1df40090b0 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 3074714298) 0x7f1e0c006a00 con 0x7f1df40090b0 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e097fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e097fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.26:6791/0 conn(0x7f1df40090b0 legacy=0x7f1df4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e097fa700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.26:6791/0 conn(0x7f1df40090b0 legacy=0x7f1df4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:51.935 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.122:6812/20895 conn(0x7f1e00037480 legacy=0x7f1e00039930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 10 monclient: shutdown 2024-09-15T03:56:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 >> v1:172.21.15.26:6792/0 conn(0x7f1e140a6880 legacy=0x7f1e14099c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e19ed3700 1 -- 172.21.15.26:0/1439597179 reap_dead start 2024-09-15T03:56:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 shutdown_connections 2024-09-15T03:56:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 >> 172.21.15.26:0/1439597179 conn(0x7f1e1406e2a0 msgr2=0x7f1e14067880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 shutdown_connections 2024-09-15T03:56:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:51.882+0000 7f1e1a6d4700 1 -- 172.21.15.26:0/1439597179 wait complete. 2024-09-15T03:56:51.938 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:51.939 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-15T03:56:51.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:56:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.046+0000 7fca84a83700 10 monclient: get_monmap_and_config 2024-09-15T03:56:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.046+0000 7fca84a83700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.046+0000 7fca84a83700 10 monclient: monmap: 2024-09-15T03:56:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:52.051226+0000 2024-09-15T03:56:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:52.051226+0000 2024-09-15T03:56:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.055 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.057 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 Processor -- start 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- start start 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient: init 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient: _add_conns ranks=[2,0,8,3,7,6,4,5,1] 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient(hunting): picked mon.noname-b con 0x7fca80132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient(hunting): picked mon.noname-a con 0x7fca8009faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient(hunting): picked mon.noname-e con 0x7fca80138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:52.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8004ca10 con 0x7fca8009faa0 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8009d5d0 con 0x7fca80132b40 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca80133620 con 0x7fca80138d40 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient(hunting): authenticate will time out at 1556.481406s 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca76ffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fca8009faa0 0x7fca8009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33506/0 (socket says 172.21.15.26:33506) 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7ffff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fca80138d40 0x7fca8013b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:42780/0 (socket says 172.21.15.26:42780) 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca76ffd700 1 -- 172.21.15.26:0/2365242245 learned_addr learned my addr 172.21.15.26:0/2365242245 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492961955 0 0) 0x7fca8004ca10 con 0x7fca8009faa0 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): my global_id is 194490 2024-09-15T03:56:52.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca6c003570 con 0x7fca8009faa0 2024-09-15T03:56:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462293418 0 0) 0x7fca80133620 con 0x7fca80138d40 2024-09-15T03:56:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): my global_id is 135206 2024-09-15T03:56:52.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8004ca10 con 0x7fca80138d40 2024-09-15T03:56:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003684823 0 0) 0x7fca8009d5d0 con 0x7fca80132b40 2024-09-15T03:56:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient(hunting): my global_id is 204311 2024-09-15T03:56:52.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca80133620 con 0x7fca80132b40 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394066398 0 0) 0x7fca6c003570 con 0x7fca8009faa0 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 >> v1:172.21.15.26:6793/0 conn(0x7fca80138d40 legacy=0x7fca8013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 >> v1:172.21.15.26:6790/0 conn(0x7fca80132b40 legacy=0x7fca80132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 monclient: found mon.noname-a 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800a0cc0 con 0x7fca8009faa0 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca680017b0 con 0x7fca8009faa0 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: dump: 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 5 monclient: authenticate success, global_id 194490 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fca68001a20 con 0x7fca8009faa0 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 4 set_mon_vals no callback set 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 1 -- 172.21.15.26:0/2365242245 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca68004c30 con 0x7fca8009faa0 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca7effd700 10 monclient: dump: 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.087 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 10 monclient: shutdown 2024-09-15T03:56:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- 172.21.15.26:0/2365242245 >> v1:172.21.15.26:6789/0 conn(0x7fca8009faa0 legacy=0x7fca8009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- 172.21.15.26:0/2365242245 shutdown_connections 2024-09-15T03:56:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- 172.21.15.26:0/2365242245 >> 172.21.15.26:0/2365242245 conn(0x7fca8006e370 msgr2=0x7fca800a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- 172.21.15.26:0/2365242245 shutdown_connections 2024-09-15T03:56:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.050+0000 7fca84a83700 1 -- 172.21.15.26:0/2365242245 wait complete. 2024-09-15T03:56:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: monmap: 2024-09-15T03:56:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 Processor -- start 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 -- start start 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: init 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: _add_conns ranks=[1,7,8,4,2,5,6,3,0] 2024-09-15T03:56:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient(hunting): picked mon.noname-f con 0x7fca8009faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient(hunting): picked mon.noname-i con 0x7fca80132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient(hunting): picked mon.noname-e con 0x7fca80138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8009cfa0 con 0x7fca8009faa0 2024-09-15T03:56:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8009ccc0 con 0x7fca80132b40 2024-09-15T03:56:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8011cca0 con 0x7fca80138d40 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient(hunting): authenticate will time out at 1556.486160s 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca76ffd700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fca80132b40 legacy=0x7fca80136040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca7ffff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fca80138d40 0x7fca8014a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:42782/0 (socket says 172.21.15.26:42782) 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca7ffff700 1 -- 172.21.15.26:0/2723973248 learned_addr learned my addr 172.21.15.26:0/2723973248 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605717603 0 0) 0x7fca8011cca0 con 0x7fca80138d40 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): my global_id is 135215 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca60003570 con 0x7fca80138d40 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678568000 0 0) 0x7fca8009cfa0 con 0x7fca8009faa0 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): my global_id is 204274 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8011cca0 con 0x7fca8009faa0 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3552936 0 0) 0x7fca60003570 con 0x7fca80138d40 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca8009cfa0 con 0x7fca80138d40 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca78001d20 con 0x7fca80138d40 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:56:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): dump: 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3773106784 0 0) 0x7fca8009cfa0 con 0x7fca80138d40 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.122:6792/0 conn(0x7fca80132b40 legacy=0x7fca80136040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.122:6789/0 conn(0x7fca8009faa0 legacy=0x7fca80135930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 monclient: found mon.e 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80137990 con 0x7fca80138d40 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca77fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 5 monclient: authenticate success, global_id 135215 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: _renew_subs 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca80137240 con 0x7fca80138d40 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: _renew_subs 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.054+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fca80137fb0 con 0x7fca80138d40 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fca780020f0 con 0x7fca80138d40 2024-09-15T03:56:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca78005260 con 0x7fca80138d40 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca7e7fc700 4 set_mon_vals no callback set 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: dump: 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca84a83700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fca64006590 con 0x7fca64002840 2024-09-15T03:56:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fca78005440 con 0x7fca80138d40 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(539..539 src has 224..539) v4 ==== 3911+0+0 (unknown 2512508809 0 0) 0x7fca78044240 con 0x7fca80138d40 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272933541 0 0) 0x7fca64006590 con 0x7fca64002840 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient(hunting): my global_id is 194514 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8009cfa0 con 0x7fca64002840 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca76ffd700 10 monclient: get_auth_request con 0x7fca60037510 auth_method 0 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2719636535 0 0) 0x7fca8009cfa0 con 0x7fca64002840 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca64006590 con 0x7fca64002840 2024-09-15T03:56:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca70002240 con 0x7fca64002840 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: dump: 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1275017254 0 0) 0x7fca64006590 con 0x7fca64002840 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca6400a130 con 0x7fca64002840 2024-09-15T03:56:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fca700027e0 con 0x7fca64002840 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.26:6792/0 conn(0x7fca64002840 legacy=0x7fca64004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.058+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.26:6792/0 conn(0x7fca64002840 legacy=0x7fca64004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fca64009640 con 0x7fca640090b0 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473141129 0 0) 0x7fca64009640 con 0x7fca640090b0 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient(hunting): my global_id is 194523 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca64006590 con 0x7fca640090b0 2024-09-15T03:56:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2746665633 0 0) 0x7fca64006590 con 0x7fca640090b0 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fca64009640 con 0x7fca640090b0 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fca70002d00 con 0x7fca640090b0 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient: dump: 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 990785115 0 0) 0x7fca64009640 con 0x7fca640090b0 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fca640098f0 con 0x7fca640090b0 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 1319051600) 0x7fca70005380 con 0x7fca640090b0 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.26:6792/0 conn(0x7fca640090b0 legacy=0x7fca64008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca77fff700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.26:6792/0 conn(0x7fca640090b0 legacy=0x7fca64008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.107 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:52.133 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.122:6812/20895 conn(0x7fca60037510 legacy=0x7fca600399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 10 monclient: shutdown 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca7ffff700 1 -- 172.21.15.26:0/2723973248 reap_dead start 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 >> v1:172.21.15.26:6793/0 conn(0x7fca80138d40 legacy=0x7fca8014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 shutdown_connections 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 >> 172.21.15.26:0/2723973248 conn(0x7fca8006e370 msgr2=0x7fca8009da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 shutdown_connections 2024-09-15T03:56:52.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.070+0000 7fca84a83700 1 -- 172.21.15.26:0/2723973248 wait complete. 2024-09-15T03:56:52.135 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:52.136 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-15T03:56:52.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:56:52.202 INFO:tasks.rados.rados.0.smithi122.stdout:2366: done (5 left) 2024-09-15T03:56:52.202 INFO:tasks.rados.rados.0.smithi122.stdout:2367: done (4 left) 2024-09-15T03:56:52.202 INFO:tasks.rados.rados.0.smithi122.stdout:2368: done (3 left) 2024-09-15T03:56:52.202 INFO:tasks.rados.rados.0.smithi122.stdout:2369: done (2 left) 2024-09-15T03:56:52.202 INFO:tasks.rados.rados.0.smithi122.stdout:2370: done (1 left) 2024-09-15T03:56:52.202 INFO:tasks.rados.rados.0.smithi122.stdout:2371: done (0 left) 2024-09-15T03:56:52.203 INFO:tasks.rados.rados.0.smithi122.stdout:2372: copy_from oid 19 from oid 4 current snap is 274 2024-09-15T03:56:52.203 INFO:tasks.rados.rados.0.smithi122.stdout:2373: snap_remove snap 268 2024-09-15T03:56:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 10 monclient: get_monmap_and_config 2024-09-15T03:56:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 10 monclient: monmap: 2024-09-15T03:56:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:52.254053+0000 2024-09-15T03:56:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:52.254053+0000 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.260 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.260 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 1 Processor -- start 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 1 -- start start 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 10 monclient: init 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.250+0000 7fabade09700 10 monclient: _add_conns ranks=[7,2,6,8,4,0,5,3,1] 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient(hunting): picked mon.noname-i con 0x7faba8132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient(hunting): picked mon.noname-b con 0x7faba809faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient(hunting): picked mon.noname-d con 0x7faba8138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba804ca10 con 0x7faba809faa0 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba809d5d0 con 0x7faba8138d40 2024-09-15T03:56:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba8133620 con 0x7faba8132b40 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient(hunting): authenticate will time out at 1556.684162s 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabace07700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7faba8132b40 legacy=0x7faba8132f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba7fff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7faba809faa0 0x7faba809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:32920/0 (socket says 172.21.15.26:32920) 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabad608700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7faba8138d40 0x7faba813b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:47314/0 (socket says 172.21.15.26:47314) 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba7fff700 1 -- 172.21.15.26:0/566528028 learned_addr learned my addr 172.21.15.26:0/566528028 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826624351 0 0) 0x7faba804ca10 con 0x7faba809faa0 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient(hunting): my global_id is 204320 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab94003570 con 0x7faba809faa0 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744572551 0 0) 0x7faba809d5d0 con 0x7faba8138d40 2024-09-15T03:56:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient(hunting): my global_id is 194532 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba804ca10 con 0x7faba8138d40 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 337119793 0 0) 0x7faba804ca10 con 0x7faba8138d40 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 >> v1:172.21.15.122:6792/0 conn(0x7faba8132b40 legacy=0x7faba8132f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 >> v1:172.21.15.26:6790/0 conn(0x7faba809faa0 legacy=0x7faba809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 monclient: found mon.noname-d 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80a0cc0 con 0x7faba8138d40 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faba0001b10 con 0x7faba8138d40 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: dump: 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 5 monclient: authenticate success, global_id 194532 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faba0001d40 con 0x7faba8138d40 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 1 -- 172.21.15.26:0/566528028 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faba0004bb0 con 0x7faba8138d40 2024-09-15T03:56:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 4 set_mon_vals no callback set 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7faba77fe700 10 monclient: dump: 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient: shutdown 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- 172.21.15.26:0/566528028 >> v1:172.21.15.26:6792/0 conn(0x7faba8138d40 legacy=0x7faba813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- 172.21.15.26:0/566528028 shutdown_connections 2024-09-15T03:56:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- 172.21.15.26:0/566528028 >> 172.21.15.26:0/566528028 conn(0x7faba806e370 msgr2=0x7faba80a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- 172.21.15.26:0/566528028 shutdown_connections 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- 172.21.15.26:0/566528028 wait complete. 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient: monmap: 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 Processor -- start 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 1 -- start start 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.254+0000 7fabade09700 10 monclient: init 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient: _add_conns ranks=[7,3,2,1,0,4,6,8,5] 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient(hunting): picked mon.noname-i con 0x7faba809faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient(hunting): picked mon.noname-g con 0x7faba8132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient(hunting): picked mon.noname-b con 0x7faba8138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba809cfa0 con 0x7faba8138d40 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba809ccc0 con 0x7faba8132b40 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba809b620 con 0x7faba809faa0 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient(hunting): authenticate will time out at 1556.688632s 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabad608700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7faba8138d40 0x7faba814a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:32928/0 (socket says 172.21.15.26:32928) 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabad608700 1 -- 172.21.15.26:0/3427007821 learned_addr learned my addr 172.21.15.26:0/3427007821 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabace07700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.122:6792/0 conn(0x7faba809faa0 legacy=0x7faba8098060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788098815 0 0) 0x7faba809cfa0 con 0x7faba8138d40 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): my global_id is 204329 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab880035a0 con 0x7faba8138d40 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181195991 0 0) 0x7faba809ccc0 con 0x7faba8132b40 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): my global_id is 204294 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba809cfa0 con 0x7faba8132b40 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3950802286 0 0) 0x7fab880035a0 con 0x7faba8138d40 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faba809ccc0 con 0x7faba8138d40 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faba0001b90 con 0x7faba8138d40 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): dump: 2024-09-15T03:56:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2259286626 0 0) 0x7faba809cfa0 con 0x7faba8132b40 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab880035a0 con 0x7faba8132b40 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2170430331 0 0) 0x7faba809ccc0 con 0x7faba8138d40 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.122:6792/0 conn(0x7faba809faa0 legacy=0x7faba8098060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.122:6790/0 conn(0x7faba8132b40 legacy=0x7faba8098770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 monclient: found mon.b 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba809a730 con 0x7faba8138d40 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 5 monclient: authenticate success, global_id 204329 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient: _renew_subs 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba809a920 con 0x7faba8138d40 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient: _renew_subs 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faba8099020 con 0x7faba8138d40 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7faba0005030 con 0x7faba8138d40 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7faba0005560 con 0x7faba8138d40 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba67fc700 4 set_mon_vals no callback set 2024-09-15T03:56:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7faba4ff9700 10 monclient: dump: 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.258+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fab8c006590 con 0x7fab8c002840 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7faba0005740 con 0x7faba8138d40 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(540..540 src has 224..540) v4 ==== 3911+0+0 (unknown 410792318 0 0) 0x7faba00444a0 con 0x7faba8138d40 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300342269 0 0) 0x7fab8c006590 con 0x7fab8c002840 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient(hunting): my global_id is 135224 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba809ccc0 con 0x7fab8c002840 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba7fff700 10 monclient: get_auth_request con 0x7fab88037840 auth_method 0 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1838791203 0 0) 0x7faba809ccc0 con 0x7fab8c002840 2024-09-15T03:56:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab8c006590 con 0x7fab8c002840 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab90004080 con 0x7fab8c002840 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient: dump: 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2154840944 0 0) 0x7fab8c006590 con 0x7fab8c002840 2024-09-15T03:56:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab8c00a130 con 0x7fab8c002840 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fab90006470 con 0x7fab8c002840 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.26:6793/0 conn(0x7fab8c002840 legacy=0x7fab8c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.262+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.26:6793/0 conn(0x7fab8c002840 legacy=0x7fab8c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.270+0000 7fabade09700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.270+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fab8c009640 con 0x7fab8c0090b0 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889225726 0 0) 0x7fab8c009640 con 0x7fab8c0090b0 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient(hunting): my global_id is 135233 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab8c006590 con 0x7fab8c0090b0 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 668005102 0 0) 0x7fab8c006590 con 0x7fab8c0090b0 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab8c009640 con 0x7fab8c0090b0 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab90004be0 con 0x7fab8c0090b0 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient: dump: 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2531141891 0 0) 0x7fab8c009640 con 0x7fab8c0090b0 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fab8c0098f0 con 0x7fab8c0090b0 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7557 (unknown 0 0 475613461) 0x7fab90006160 con 0x7fab8c0090b0 2024-09-15T03:56:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.26:6793/0 conn(0x7fab8c0090b0 legacy=0x7fab8c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7faba4ff9700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.26:6793/0 conn(0x7fab8c0090b0 legacy=0x7fab8c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:52.309 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.122:6812/20895 conn(0x7fab88037840 legacy=0x7fab88039cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 10 monclient: shutdown 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 >> v1:172.21.15.26:6790/0 conn(0x7faba8138d40 legacy=0x7faba814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabad608700 1 -- 172.21.15.26:0/3427007821 reap_dead start 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabad608700 1 -- 172.21.15.26:0/3427007821 reap_dead start 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 shutdown_connections 2024-09-15T03:56:52.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 >> 172.21.15.26:0/3427007821 conn(0x7faba806e370 msgr2=0x7faba80a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 shutdown_connections 2024-09-15T03:56:52.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.274+0000 7fabade09700 1 -- 172.21.15.26:0/3427007821 wait complete. 2024-09-15T03:56:52.312 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:52.312 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-15T03:56:52.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:56:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient: get_monmap_and_config 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient: monmap: 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:52.424866+0000 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:52.424866+0000 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 1 Processor -- start 2024-09-15T03:56:52.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 1 -- start start 2024-09-15T03:56:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient: init 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient: _add_conns ranks=[4,5,8,0,6,1,7,2,3] 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient(hunting): picked mon.noname-c con 0x7fbf040a3890 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient(hunting): picked mon.noname-h con 0x7fbf04099360 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient(hunting): picked mon.noname-e con 0x7fbf04138ee0 addr v1:172.21.15.26:6793/0 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0404ca10 con 0x7fbf040a3890 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0406e6a0 con 0x7fbf04099360 2024-09-15T03:56:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0409e190 con 0x7fbf04138ee0 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf0b860700 10 monclient(hunting): authenticate will time out at 1556.855712s 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf095fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fbf040a3890 0x7fbf041331b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:59812/0 (socket says 172.21.15.26:59812) 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.422+0000 7fbf095fc700 1 -- 172.21.15.26:0/520994974 learned_addr learned my addr 172.21.15.26:0/520994974 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708651466 0 0) 0x7fbf0404ca10 con 0x7fbf040a3890 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): my global_id is 204331 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbef0003570 con 0x7fbf040a3890 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905975063 0 0) 0x7fbf0409e190 con 0x7fbf04138ee0 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): my global_id is 135242 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0404ca10 con 0x7fbf04138ee0 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922255267 0 0) 0x7fbf0406e6a0 con 0x7fbf04099360 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient(hunting): my global_id is 5801 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0409e190 con 0x7fbf04099360 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760962149 0 0) 0x7fbef0003570 con 0x7fbf040a3890 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 >> v1:172.21.15.26:6793/0 conn(0x7fbf04138ee0 legacy=0x7fbf0413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 >> v1:172.21.15.122:6791/0 conn(0x7fbf04099360 legacy=0x7fbf04099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 monclient: found mon.noname-c 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:56:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf041338c0 con 0x7fbf040a3890 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbf000017b0 con 0x7fbf040a3890 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 5 monclient: authenticate success, global_id 204331 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: dump: 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf00001a20 con 0x7fbf040a3890 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 1 -- 172.21.15.26:0/520994974 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbf00004c30 con 0x7fbf040a3890 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 4 set_mon_vals no callback set 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbefbfff700 10 monclient: dump: 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.439 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 10 monclient: shutdown 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 1 -- 172.21.15.26:0/520994974 >> v1:172.21.15.26:6791/0 conn(0x7fbf040a3890 legacy=0x7fbf041331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 1 -- 172.21.15.26:0/520994974 shutdown_connections 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 1 -- 172.21.15.26:0/520994974 >> 172.21.15.26:0/520994974 conn(0x7fbf04071350 msgr2=0x7fbf040a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 1 -- 172.21.15.26:0/520994974 shutdown_connections 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.426+0000 7fbf0b860700 1 -- 172.21.15.26:0/520994974 wait complete. 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient: monmap: 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 1 Processor -- start 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 1 -- start start 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient: init 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient: _add_conns ranks=[4,5,0,2,8,3,1,7,6] 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient(hunting): picked mon.noname-c con 0x7fbf04099360 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient(hunting): picked mon.noname-h con 0x7fbf04138ee0 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient(hunting): picked mon.noname-a con 0x7fbf0411a5c0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0409ec80 con 0x7fbf0411a5c0 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf0409e8c0 con 0x7fbf04099360 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf095fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fbf04099360 0x7fbf04137790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:59826/0 (socket says 172.21.15.26:59826) 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf095fc700 1 -- 172.21.15.26:0/1412251884 learned_addr learned my addr 172.21.15.26:0/1412251884 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf09dfd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbf0411a5c0 0x7fbf041385b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33514/0 (socket says 172.21.15.26:33514) 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf041315a0 con 0x7fbf04138ee0 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 10 monclient(hunting): authenticate will time out at 1556.862734s 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707682145 0 0) 0x7fbf0409e8c0 con 0x7fbf04099360 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): my global_id is 204340 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee8003570 con 0x7fbf04099360 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157453365 0 0) 0x7fbf0409ec80 con 0x7fbf0411a5c0 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): my global_id is 194499 2024-09-15T03:56:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0409e8c0 con 0x7fbf0411a5c0 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528665217 0 0) 0x7fbf041315a0 con 0x7fbf04138ee0 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): my global_id is 5810 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0409ec80 con 0x7fbf04138ee0 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 244248440 0 0) 0x7fbee8003570 con 0x7fbf04099360 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf041315a0 con 0x7fbf04099360 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbf00001830 con 0x7fbf04099360 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): dump: 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2524008460 0 0) 0x7fbf0409e8c0 con 0x7fbf0411a5c0 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbee8003570 con 0x7fbf0411a5c0 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbefc001c80 con 0x7fbf0411a5c0 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient(hunting): dump: 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 51374775 0 0) 0x7fbf041315a0 con 0x7fbf04099360 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.122:6791/0 conn(0x7fbf04138ee0 legacy=0x7fbf04137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.26:6789/0 conn(0x7fbf0411a5c0 legacy=0x7fbf041385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 monclient: found mon.c 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf041199f0 con 0x7fbf04099360 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbef97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.430+0000 7fbf0b860700 5 monclient: authenticate success, global_id 204340 2024-09-15T03:56:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 10 monclient: _renew_subs 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0411b520 con 0x7fbf04099360 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 10 monclient: _renew_subs 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0414b540 con 0x7fbf04099360 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbf000011d0 con 0x7fbf04099360 2024-09-15T03:56:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbf00005240 con 0x7fbf04099360 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbefb7fe700 4 set_mon_vals no callback set 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: dump: 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbf0000fc00 con 0x7fbf04099360 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbee4006590 con 0x7fbee4002840 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(540..540 src has 224..540) v4 ==== 3911+0+0 (unknown 410792318 0 0) 0x7fbf000441e0 con 0x7fbf04099360 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbf09dfd700 10 monclient: get_auth_request con 0x7fbee8037c60 auth_method 0 2024-09-15T03:56:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375073638 0 0) 0x7fbee4006590 con 0x7fbee4002840 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient(hunting): my global_id is 204283 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf041315a0 con 0x7fbee4002840 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1471127266 0 0) 0x7fbf041315a0 con 0x7fbee4002840 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbee4006590 con 0x7fbee4002840 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbef4001920 con 0x7fbee4002840 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: dump: 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3418350219 0 0) 0x7fbee4006590 con 0x7fbee4002840 2024-09-15T03:56:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbee400a130 con 0x7fbee4002840 2024-09-15T03:56:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbef4001f20 con 0x7fbee4002840 2024-09-15T03:56:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.122:6789/0 conn(0x7fbee4002840 legacy=0x7fbee4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.434+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.122:6789/0 conn(0x7fbee4002840 legacy=0x7fbee4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbf0b860700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:52.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbee4007520 con 0x7fbee40068d0 2024-09-15T03:56:52.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469004015 0 0) 0x7fbee4007520 con 0x7fbee40068d0 2024-09-15T03:56:52.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 10 monclient(hunting): my global_id is 204292 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee4006590 con 0x7fbee40068d0 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 241391431 0 0) 0x7fbee4006590 con 0x7fbee40068d0 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbee4007520 con 0x7fbee40068d0 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbef4002730 con 0x7fbee40068d0 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 10 monclient: dump: 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.460 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3633743308 0 0) 0x7fbee4007520 con 0x7fbee40068d0 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.446+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbee4008760 con 0x7fbee40068d0 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 2783402880) 0x7fbef4002b10 con 0x7fbee40068d0 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbef97fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbef97fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.122:6789/0 conn(0x7fbee40068d0 legacy=0x7fbee4006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbef97fa700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.122:6789/0 conn(0x7fbee40068d0 legacy=0x7fbee4006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:52.482 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.122:6812/20895 conn(0x7fbee8037c60 legacy=0x7fbee803a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 10 monclient: shutdown 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 >> v1:172.21.15.26:6791/0 conn(0x7fbf04099360 legacy=0x7fbf04137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf09dfd700 1 -- 172.21.15.26:0/1412251884 reap_dead start 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 shutdown_connections 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 >> 172.21.15.26:0/1412251884 conn(0x7fbf04071350 msgr2=0x7fbf04098970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 shutdown_connections 2024-09-15T03:56:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.450+0000 7fbf0b860700 1 -- 172.21.15.26:0/1412251884 wait complete. 2024-09-15T03:56:52.484 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:52.484 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-09-15T03:56:52.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:56:52.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient: get_monmap_and_config 2024-09-15T03:56:52.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient: monmap: 2024-09-15T03:56:52.599 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.599 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.599 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:52.596325+0000 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:52.596325+0000 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.600 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 1 Processor -- start 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 1 -- start start 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient: init 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient: _add_conns ranks=[1,0,4,5,8,6,7,3,2] 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient(hunting): picked mon.noname-f con 0x7f44700a3890 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient(hunting): picked mon.noname-a con 0x7f4470099360 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient(hunting): picked mon.noname-c con 0x7f4470138ee0 addr v1:172.21.15.26:6791/0 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447004ca10 con 0x7f4470099360 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447006e6a0 con 0x7f44700a3890 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447009e190 con 0x7f4470138ee0 2024-09-15T03:56:52.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4477fc8700 10 monclient(hunting): authenticate will time out at 1557.026554s 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4475563700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4470099360 0x7f4470099730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33530/0 (socket says 172.21.15.26:33530) 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4475563700 1 -- 172.21.15.26:0/1203648205 learned_addr learned my addr 172.21.15.26:0/1203648205 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573402979 0 0) 0x7f447004ca10 con 0x7f4470099360 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): my global_id is 194508 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f445c003570 con 0x7f4470099360 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754512339 0 0) 0x7f447006e6a0 con 0x7f44700a3890 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): my global_id is 204301 2024-09-15T03:56:52.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447004ca10 con 0x7f44700a3890 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481981382 0 0) 0x7f447009e190 con 0x7f4470138ee0 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 10 monclient(hunting): my global_id is 204349 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447006e6a0 con 0x7f4470138ee0 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.594+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 231890402 0 0) 0x7f445c003570 con 0x7f4470099360 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 >> v1:172.21.15.26:6791/0 conn(0x7f4470138ee0 legacy=0x7f447013b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 >> v1:172.21.15.122:6789/0 conn(0x7f44700a3890 legacy=0x7f44701331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 monclient: found mon.noname-a 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44701338c0 con 0x7f4470099360 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 5 monclient: authenticate success, global_id 194508 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f44640017b0 con 0x7f4470099360 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: dump: 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.608 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f44640019a0 con 0x7f4470099360 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 4 set_mon_vals no callback set 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 1 -- 172.21.15.26:0/1203648205 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f44640048f0 con 0x7f4470099360 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4474d62700 10 monclient: dump: 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.610 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: shutdown 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- 172.21.15.26:0/1203648205 >> v1:172.21.15.26:6789/0 conn(0x7f4470099360 legacy=0x7f4470099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- 172.21.15.26:0/1203648205 shutdown_connections 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- 172.21.15.26:0/1203648205 >> 172.21.15.26:0/1203648205 conn(0x7f4470071350 msgr2=0x7f44700a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- 172.21.15.26:0/1203648205 shutdown_connections 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- 172.21.15.26:0/1203648205 wait complete. 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: monmap: 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:52.612 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 Processor -- start 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- start start 2024-09-15T03:56:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: init 2024-09-15T03:56:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient: _add_conns ranks=[3,7,6,8,5,0,4,1,2] 2024-09-15T03:56:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient(hunting): picked mon.noname-g con 0x7f4470099360 addr v1:172.21.15.122:6790/0 2024-09-15T03:56:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient(hunting): picked mon.noname-i con 0x7f4470138ee0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient(hunting): picked mon.noname-d con 0x7f4470138890 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447009ec80 con 0x7f4470099360 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44701326e0 con 0x7f4470138890 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447009e8c0 con 0x7f4470138ee0 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4477fc8700 10 monclient(hunting): authenticate will time out at 1557.031073s 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4475563700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f4470138ee0 legacy=0x7f447012f740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4476565700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f4470138890 0x7f447012fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:47322/0 (socket says 172.21.15.26:47322) 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4476565700 1 -- 172.21.15.26:0/2035922957 learned_addr learned my addr 172.21.15.26:0/2035922957 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321091227 0 0) 0x7f44701326e0 con 0x7f4470138890 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 10 monclient(hunting): my global_id is 194541 2024-09-15T03:56:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4454003570 con 0x7f4470138890 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045279047 0 0) 0x7f447009ec80 con 0x7f4470099360 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 10 monclient(hunting): my global_id is 204303 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.598+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44701326e0 con 0x7f4470099360 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 836758424 0 0) 0x7f4454003570 con 0x7f4470138890 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f447009ec80 con 0x7f4470138890 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4468001c20 con 0x7f4470138890 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): dump: 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 357462037 0 0) 0x7f44701326e0 con 0x7f4470099360 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4454003570 con 0x7f4470099360 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f446c001b70 con 0x7f4470099360 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): dump: 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3008629864 0 0) 0x7f447009ec80 con 0x7f4470138890 2024-09-15T03:56:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6792/0 conn(0x7f4470138ee0 legacy=0x7f447012f740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6790/0 conn(0x7f4470099360 legacy=0x7f447012f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 monclient: found mon.d 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44701310c0 con 0x7f4470138890 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 5 monclient: authenticate success, global_id 194541 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 10 monclient: _renew_subs 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4470130bb0 con 0x7f4470138890 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 10 monclient: _renew_subs 2024-09-15T03:56:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f447014b540 con 0x7f4470138890 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4468004c00 con 0x7f4470138890 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f44680050b0 con 0x7f4470138890 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4463fff700 4 set_mon_vals no callback set 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient: dump: 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f446800fa70 con 0x7f4470138890 2024-09-15T03:56:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(540..540 src has 224..540) v4 ==== 3911+0+0 (unknown 410792318 0 0) 0x7f44680440d0 con 0x7f4470138890 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4450006590 con 0x7f4450002840 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4475d64700 10 monclient: get_auth_request con 0x7f4454037bf0 auth_method 0 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592666783 0 0) 0x7f4450006590 con 0x7f4450002840 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 10 monclient(hunting): my global_id is 204312 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.602+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f447009ec80 con 0x7f4450002840 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3081678207 0 0) 0x7f447009ec80 con 0x7f4450002840 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4450006590 con 0x7f4450002840 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f44640015d0 con 0x7f4450002840 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 10 monclient: dump: 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 186150179 0 0) 0x7f4450006590 con 0x7f4450002840 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f445000a130 con 0x7f4450002840 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f44640048f0 con 0x7f4450002840 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6790/0 conn(0x7f4450002840 legacy=0x7f4450004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.606+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6790/0 conn(0x7f4450002840 legacy=0x7f4450004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4477fc8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4450007520 con 0x7f44500068d0 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357459325 0 0) 0x7f4450007520 con 0x7f44500068d0 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 10 monclient(hunting): my global_id is 204321 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4450006590 con 0x7f44500068d0 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2883247266 0 0) 0x7f4450006590 con 0x7f44500068d0 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4450007520 con 0x7f44500068d0 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4464005210 con 0x7f44500068d0 2024-09-15T03:56:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 10 monclient: dump: 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.635 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3948804031 0 0) 0x7f4450007520 con 0x7f44500068d0 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.618+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4450008760 con 0x7f44500068d0 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7387 (unknown 0 0 1021943664) 0x7f44640074f0 con 0x7f44500068d0 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4461ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4461ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6790/0 conn(0x7f44500068d0 legacy=0x7f4450006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4461ffb700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6790/0 conn(0x7f44500068d0 legacy=0x7f4450006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:52.656 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.122:6812/20895 conn(0x7f4454037bf0 legacy=0x7f445403a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 10 monclient: shutdown 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 >> v1:172.21.15.26:6792/0 conn(0x7f4470138890 legacy=0x7f447012fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4476565700 1 -- 172.21.15.26:0/2035922957 reap_dead start 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4476565700 1 -- 172.21.15.26:0/2035922957 reap_dead start 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 shutdown_connections 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 >> 172.21.15.26:0/2035922957 conn(0x7f4470071350 msgr2=0x7f44700a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 shutdown_connections 2024-09-15T03:56:52.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.622+0000 7f4477fc8700 1 -- 172.21.15.26:0/2035922957 wait complete. 2024-09-15T03:56:52.659 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:52.659 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-15T03:56:52.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient: get_monmap_and_config 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient: monmap: 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:56:52.768946+0000 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:56:52.768946+0000 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.772 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.773 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 1 Processor -- start 2024-09-15T03:56:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 1 -- start start 2024-09-15T03:56:52.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient: init 2024-09-15T03:56:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient: _add_conns ranks=[5,6,7,8,1,4,2,3,0] 2024-09-15T03:56:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient(hunting): picked mon.noname-h con 0x7f830c12f270 addr v1:172.21.15.122:6791/0 2024-09-15T03:56:52.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.766+0000 7f8312165700 10 monclient(hunting): picked mon.noname-d con 0x7f830c06d5b0 addr v1:172.21.15.26:6792/0 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): picked mon.noname-i con 0x7f830c1353b0 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c04ca10 con 0x7f830c12f270 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c06aab0 con 0x7f830c06d5b0 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c09f050 con 0x7f830c1353b0 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830bfff700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f830c1353b0 legacy=0x7f830c137790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): authenticate will time out at 1557.200219s 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8303fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f830c06d5b0 0x7f830c06d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:47328/0 (socket says 172.21.15.26:47328) 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8303fff700 1 -- 172.21.15.26:0/3610694015 learned_addr learned my addr 172.21.15.26:0/3610694015 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702014977 0 0) 0x7f830c06aab0 con 0x7f830c06d5b0 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient(hunting): my global_id is 194550 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f8003570 con 0x7f830c06d5b0 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671332882 0 0) 0x7f830c04ca10 con 0x7f830c12f270 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient(hunting): my global_id is 5819 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c06aab0 con 0x7f830c12f270 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607246573 0 0) 0x7f82f8003570 con 0x7f830c06d5b0 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 >> v1:172.21.15.122:6792/0 conn(0x7f830c1353b0 legacy=0x7f830c137790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 >> v1:172.21.15.122:6791/0 conn(0x7f830c12f270 legacy=0x7f830c12f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 monclient: found mon.noname-d 2024-09-15T03:56:52.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c06a7d0 con 0x7f830c06d5b0 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82f40017b0 con 0x7f830c06d5b0 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 5 monclient: authenticate success, global_id 194550 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: dump: 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.780 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.781 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f82f4001a90 con 0x7f830c06d5b0 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 1 -- 172.21.15.26:0/3610694015 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82f4004c30 con 0x7f830c06d5b0 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: get_monmap_and_config success 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 4 set_mon_vals no callback set 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f830affd700 10 monclient: dump: 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.782 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.783 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.784 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: shutdown 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- 172.21.15.26:0/3610694015 >> v1:172.21.15.26:6792/0 conn(0x7f830c06d5b0 legacy=0x7f830c06d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- 172.21.15.26:0/3610694015 shutdown_connections 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- 172.21.15.26:0/3610694015 >> 172.21.15.26:0/3610694015 conn(0x7f830c063e10 msgr2=0x7f830c0a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- 172.21.15.26:0/3610694015 shutdown_connections 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- 172.21.15.26:0/3610694015 wait complete. 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: build_initial_monmap 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: monmap: 2024-09-15T03:56:52.785 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:56:52.786 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:56:52.787 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:56:52.787 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 Processor -- start 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- start start 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: init 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: _reopen_session rank -1 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient: _add_conns ranks=[1,7,0,5,2,6,8,3,4] 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): picked mon.noname-f con 0x7f830c06d5b0 addr v1:172.21.15.122:6789/0 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): picked mon.noname-i con 0x7f830c12f270 addr v1:172.21.15.122:6792/0 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): picked mon.noname-a con 0x7f830c1353b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c134560 con 0x7f830c1353b0 2024-09-15T03:56:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c09f560 con 0x7f830c06d5b0 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830c134280 con 0x7f830c12f270 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): _renew_subs 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.770+0000 7f8312165700 10 monclient(hunting): authenticate will time out at 1557.203971s 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8303fff700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f830c12f270 legacy=0x7f830c146be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f830bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f830c1353b0 0x7f830c14a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33546/0 (socket says 172.21.15.26:33546) 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f830bfff700 1 -- 172.21.15.26:0/2759381574 learned_addr learned my addr 172.21.15.26:0/2759381574 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f830b7fe700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f830c06d5b0 0x7f830c1326e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:56576/0 (socket says 172.21.15.26:56576) 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463159882 0 0) 0x7f830c134560 con 0x7f830c1353b0 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): my global_id is 194517 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ec003570 con 0x7f830c1353b0 2024-09-15T03:56:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347565856 0 0) 0x7f830c09f560 con 0x7f830c06d5b0 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): my global_id is 204310 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c134560 con 0x7f830c06d5b0 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1020409672 0 0) 0x7f82ec003570 con 0x7f830c1353b0 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f830c09f560 con 0x7f830c1353b0 2024-09-15T03:56:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f83040045b0 con 0x7f830c1353b0 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): dump: 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.795 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2706836639 0 0) 0x7f830c134560 con 0x7f830c06d5b0 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82ec003570 con 0x7f830c06d5b0 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82fc001b70 con 0x7f830c06d5b0 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient(hunting): dump: 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 972093269 0 0) 0x7f830c09f560 con 0x7f830c1353b0 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6792/0 conn(0x7f830c12f270 legacy=0x7f830c146be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6789/0 conn(0x7f830c06d5b0 legacy=0x7f830c1326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: _finish_hunting 0 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 monclient: found mon.a 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c134840 con 0x7f830c1353b0 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: _finish_auth 0 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 5 monclient: authenticate success, global_id 194517 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 10 monclient: _renew_subs 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c132400 con 0x7f830c1353b0 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 10 monclient: _renew_subs 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c132df0 con 0x7f830c1353b0 2024-09-15T03:56:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8304004cb0 con 0x7f830c1353b0 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8304005160 con 0x7f830c1353b0 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f830a7fc700 4 set_mon_vals no callback set 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 10 monclient: dump: 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f82f0006590 con 0x7f82f0002840 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.774+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f83040052c0 con 0x7f830c1353b0 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(540..540 src has 224..540) v4 ==== 3911+0+0 (unknown 410792318 0 0) 0x7f8304044100 con 0x7f830c1353b0 2024-09-15T03:56:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774874544 0 0) 0x7f82f0006590 con 0x7f82f0002840 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient(hunting): my global_id is 5828 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f830c09f560 con 0x7f82f0002840 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f8303fff700 10 monclient: get_auth_request con 0x7f82ec037b10 auth_method 0 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3867542602 0 0) 0x7f830c09f560 con 0x7f82f0002840 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82f0006590 con 0x7f82f0002840 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82fc0022c0 con 0x7f82f0002840 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient: dump: 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 377327226 0 0) 0x7f82f0006590 con 0x7f82f0002840 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82f000a130 con 0x7f82f0002840 2024-09-15T03:56:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f82fc005810 con 0x7f82f0002840 2024-09-15T03:56:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:56:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:56:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6791/0 conn(0x7f82f0002840 legacy=0x7f82f0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.778+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6791/0 conn(0x7f82f0002840 legacy=0x7f82f0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.786+0000 7f8312165700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:56:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.786+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f82f0007520 con 0x7f82f00068d0 2024-09-15T03:56:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546683026 0 0) 0x7f82f0007520 con 0x7f82f00068d0 2024-09-15T03:56:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient(hunting): my global_id is 5837 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f0006590 con 0x7f82f00068d0 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1668576482 0 0) 0x7f82f0006590 con 0x7f82f00068d0 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82f0007520 con 0x7f82f00068d0 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82fc002c50 con 0x7f82f00068d0 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient: dump: 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:56:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1820716770 0 0) 0x7f82f0007520 con 0x7f82f00068d0 2024-09-15T03:56:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f82f0008760 con 0x7f82f00068d0 2024-09-15T03:56:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7729 (unknown 0 0 578950657) 0x7f82fc007e40 con 0x7f82f00068d0 2024-09-15T03:56:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:56:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:56:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6791/0 conn(0x7f82f00068d0 legacy=0x7f82f0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f83037fe700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6791/0 conn(0x7f82f00068d0 legacy=0x7f82f0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:56:52.808 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 92, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:56:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:56:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:56:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:56:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:56:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:56:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:56:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:56:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:56:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:56:52.831 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:56:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.122:6812/20895 conn(0x7f82ec037b10 legacy=0x7f82ec039fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 10 monclient: shutdown 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 >> v1:172.21.15.26:6789/0 conn(0x7f830c1353b0 legacy=0x7f830c14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f830bfff700 1 -- 172.21.15.26:0/2759381574 reap_dead start 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f830bfff700 1 -- 172.21.15.26:0/2759381574 reap_dead start 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 shutdown_connections 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 >> 172.21.15.26:0/2759381574 conn(0x7f830c063e10 msgr2=0x7f830c0a06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 shutdown_connections 2024-09-15T03:56:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:56:52.790+0000 7f8312165700 1 -- 172.21.15.26:0/2759381574 wait complete. 2024-09-15T03:56:52.833 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:56:52.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:2372: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:2372: finishing copy_from to smithi12223906-19 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 816 (ObjNum 758 snap 234 seq_num 758) dirty exists 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:2372: done (1 left) 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:2373: done (0 left) 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:2374: rollback oid 40 current snap is 274 2024-09-15T03:56:53.209 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 269 2024-09-15T03:56:53.210 INFO:tasks.rados.rados.0.smithi122.stdout:2375: snap_create 2024-09-15T03:56:53.221 INFO:tasks.rados.rados.0.smithi122.stdout:2374: finishing rollback tid 1 to smithi12223906-40 2024-09-15T03:56:53.222 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 814 (ObjNum 33 snap 0 seq_num 1817663720) dirty dne 2024-09-15T03:56:54.212 INFO:tasks.rados.rados.0.smithi122.stdout:2374: done (1 left) 2024-09-15T03:56:54.212 INFO:tasks.rados.rados.0.smithi122.stdout:2375: done (0 left) 2024-09-15T03:56:54.212 INFO:tasks.rados.rados.0.smithi122.stdout:2376: snap_create 2024-09-15T03:56:55.214 INFO:tasks.rados.rados.0.smithi122.stdout:2376: done (0 left) 2024-09-15T03:56:55.214 INFO:tasks.rados.rados.0.smithi122.stdout:2377: write oid 19 current snap is 276 2024-09-15T03:56:55.215 INFO:tasks.rados.rados.0.smithi122.stdout:2377: seq_num 848 ranges {652336=726335,1906495=345841} 2024-09-15T03:56:55.225 INFO:tasks.rados.rados.0.smithi122.stdout:2377: writing smithi12223906-19 from 652336 to 1378671 tid 1 2024-09-15T03:56:55.230 INFO:tasks.rados.rados.0.smithi122.stdout:2377: writing smithi12223906-19 from 1906495 to 2252336 tid 2 2024-09-15T03:56:55.231 INFO:tasks.rados.rados.0.smithi122.stdout:2378: rollback oid 7 current snap is 276 2024-09-15T03:56:55.231 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 267 2024-09-15T03:56:55.231 INFO:tasks.rados.rados.0.smithi122.stdout:2379: write oid 12 current snap is 276 2024-09-15T03:56:55.231 INFO:tasks.rados.rados.0.smithi122.stdout:2379: seq_num 849 ranges {669143=401584,1564977=426959} 2024-09-15T03:56:55.236 INFO:tasks.rados.rados.0.smithi122.stdout:2379: writing smithi12223906-12 from 669143 to 1070727 tid 1 2024-09-15T03:56:55.242 INFO:tasks.rados.rados.0.smithi122.stdout:2379: writing smithi12223906-12 from 1564977 to 1991936 tid 2 2024-09-15T03:56:55.242 INFO:tasks.rados.rados.0.smithi122.stdout:2380: delete oid 9 current snap is 276 2024-09-15T03:56:55.242 INFO:tasks.rados.rados.0.smithi122.stdout:2377: finishing write tid 1 to smithi12223906-19 2024-09-15T03:56:55.242 INFO:tasks.rados.rados.0.smithi122.stdout:2377: finishing write tid 2 to smithi12223906-19 2024-09-15T03:56:55.242 INFO:tasks.rados.rados.0.smithi122.stdout:2377: finishing write tid 3 to smithi12223906-19 2024-09-15T03:56:55.242 INFO:tasks.rados.rados.0.smithi122.stdout:2377: finishing write tid 4 to smithi12223906-19 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 833 (ObjNum 848 snap 276 seq_num 848) dirty exists 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:2377: left oid 19 (ObjNum 848 snap 276 seq_num 848) 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:2378: finishing rollback tid 0 to smithi12223906-7 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:2378: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:2378: finishing rollback tid 2 to smithi12223906-7 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 950 (ObjNum 687 snap 211 seq_num 687) dirty exists 2024-09-15T03:56:55.243 INFO:tasks.rados.rados.0.smithi122.stdout:2379: finishing write tid 1 to smithi12223906-12 2024-09-15T03:56:55.246 INFO:tasks.rados.rados.0.smithi122.stdout:2379: finishing write tid 2 to smithi12223906-12 2024-09-15T03:56:55.247 INFO:tasks.rados.rados.0.smithi122.stdout:2379: finishing write tid 3 to smithi12223906-12 2024-09-15T03:56:55.248 INFO:tasks.rados.rados.0.smithi122.stdout:2379: finishing write tid 4 to smithi12223906-12 2024-09-15T03:56:55.248 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 752 (ObjNum 849 snap 276 seq_num 849) dirty exists 2024-09-15T03:56:55.248 INFO:tasks.rados.rados.0.smithi122.stdout:2379: left oid 12 (ObjNum 849 snap 276 seq_num 849) 2024-09-15T03:56:55.248 INFO:tasks.rados.rados.0.smithi122.stdout:2377: done (3 left) 2024-09-15T03:56:55.248 INFO:tasks.rados.rados.0.smithi122.stdout:2378: done (2 left) 2024-09-15T03:56:55.249 INFO:tasks.rados.rados.0.smithi122.stdout:2379: done (1 left) 2024-09-15T03:56:55.249 INFO:tasks.rados.rados.0.smithi122.stdout:2380: done (0 left) 2024-09-15T03:56:55.249 INFO:tasks.rados.rados.0.smithi122.stdout:2381: delete oid 20 current snap is 276 2024-09-15T03:56:55.249 INFO:tasks.rados.rados.0.smithi122.stdout:2381: done (0 left) 2024-09-15T03:56:55.249 INFO:tasks.rados.rados.0.smithi122.stdout:2382: snap_create 2024-09-15T03:56:56.004 INFO:tasks.rados.rados.0.smithi122.stdout:2382: done (0 left) 2024-09-15T03:56:56.004 INFO:tasks.rados.rados.0.smithi122.stdout:2383: delete oid 42 current snap is 277 2024-09-15T03:56:56.011 INFO:tasks.rados.rados.0.smithi122.stdout:2383: done (0 left) 2024-09-15T03:56:56.012 INFO:tasks.rados.rados.0.smithi122.stdout:2384: rollback oid 45 current snap is 277 2024-09-15T03:56:56.012 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 215 2024-09-15T03:56:56.012 INFO:tasks.rados.rados.0.smithi122.stdout:2385: write (excl) oid 42 current snap is 277 2024-09-15T03:56:56.012 INFO:tasks.rados.rados.0.smithi122.stdout:2385: seq_num 853 ranges {736371=418933,1798905=434297} 2024-09-15T03:56:56.019 INFO:tasks.rados.rados.0.smithi122.stdout:2385: writing smithi12223906-42 from 736371 to 1155304 tid 1 2024-09-15T03:56:56.027 INFO:tasks.rados.rados.0.smithi122.stdout:2385: writing smithi12223906-42 from 1798905 to 2233202 tid 2 2024-09-15T03:56:56.027 INFO:tasks.rados.rados.0.smithi122.stdout:2386: rollback oid 12 current snap is 277 2024-09-15T03:56:56.027 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 227 2024-09-15T03:56:56.027 INFO:tasks.rados.rados.0.smithi122.stdout:2384: finishing rollback tid 0 to smithi12223906-45 2024-09-15T03:56:56.027 INFO:tasks.rados.rados.0.smithi122.stdout:2387: snap_remove snap 269 2024-09-15T03:56:57.023 INFO:tasks.rados.rados.0.smithi122.stdout:2387: done (3 left) 2024-09-15T03:56:57.024 INFO:tasks.rados.rados.0.smithi122.stdout:2388: write (excl) oid 1 current snap is 277 2024-09-15T03:56:57.024 INFO:tasks.rados.rados.0.smithi122.stdout:2388: seq_num 854 ranges {753178=494182,1857387=495791} 2024-09-15T03:56:57.033 INFO:tasks.rados.rados.0.smithi122.stdout:2388: writing smithi12223906-1 from 753178 to 1247360 tid 1 2024-09-15T03:56:57.040 INFO:tasks.rados.rados.0.smithi122.stdout:2388: writing smithi12223906-1 from 1857387 to 2353178 tid 2 2024-09-15T03:56:57.040 INFO:tasks.rados.rados.0.smithi122.stdout:2389: snap_remove snap 260 2024-09-15T03:56:58.031 INFO:tasks.rados.rados.0.smithi122.stdout:2389: done (4 left) 2024-09-15T03:56:58.032 INFO:tasks.rados.rados.0.smithi122.stdout:2390: read oid 25 snap -1 2024-09-15T03:56:58.032 INFO:tasks.rados.rados.0.smithi122.stdout:2390: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:56:58.032 INFO:tasks.rados.rados.0.smithi122.stdout:2391: snap_remove snap 261 2024-09-15T03:56:59.038 INFO:tasks.rados.rados.0.smithi122.stdout:2391: done (5 left) 2024-09-15T03:56:59.039 INFO:tasks.rados.rados.0.smithi122.stdout:2392: copy_from oid 7 from oid 44 current snap is 277 2024-09-15T03:56:59.042 INFO:tasks.rados.rados.0.smithi122.stdout:2384: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:56:59.042 INFO:tasks.rados.rados.0.smithi122.stdout:2384: finishing rollback tid 2 to smithi12223906-45 2024-09-15T03:56:59.042 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 836 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2385: finishing write tid 1 to smithi12223906-42 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2385: finishing write tid 2 to smithi12223906-42 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2385: finishing write tid 3 to smithi12223906-42 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2386: finishing rollback tid 0 to smithi12223906-12 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2385: finishing write tid 4 to smithi12223906-42 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1565 (ObjNum 853 snap 277 seq_num 853) dirty exists 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2385: left oid 42 (ObjNum 853 snap 277 seq_num 853) 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2386: finishing rollback tid 1 to smithi12223906-12 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2386: finishing rollback tid 2 to smithi12223906-12 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 755 (ObjNum 719 snap 218 seq_num 719) dirty exists 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2388: finishing write tid 1 to smithi12223906-1 2024-09-15T03:56:59.043 INFO:tasks.rados.rados.0.smithi122.stdout:2388: finishing write tid 2 to smithi12223906-1 2024-09-15T03:56:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2388: finishing write tid 3 to smithi12223906-1 2024-09-15T03:56:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2388: finishing write tid 4 to smithi12223906-1 2024-09-15T03:56:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1307 (ObjNum 854 snap 277 seq_num 854) dirty exists 2024-09-15T03:56:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2388: left oid 1 (ObjNum 854 snap 277 seq_num 854) 2024-09-15T03:56:59.044 INFO:tasks.rados.rados.0.smithi122.stdout:2390: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:56:59.204 INFO:tasks.rados.rados.0.smithi122.stdout:2392: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:56:59.204 INFO:tasks.rados.rados.0.smithi122.stdout:2392: finishing copy_from to smithi12223906-7 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1562 (ObjNum 837 snap 271 seq_num 837) dirty exists 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2384: done (5 left) 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2385: done (4 left) 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2386: done (3 left) 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2388: done (2 left) 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2390: done (1 left) 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2392: done (0 left) 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2393: write oid 50 current snap is 277 2024-09-15T03:56:59.205 INFO:tasks.rados.rados.0.smithi122.stdout:2393: seq_num 855 ranges {769985=569431,1915869=437966} 2024-09-15T03:56:59.211 INFO:tasks.rados.rados.0.smithi122.stdout:2393: writing smithi12223906-50 from 769985 to 1339416 tid 1 2024-09-15T03:56:59.218 INFO:tasks.rados.rados.0.smithi122.stdout:2393: writing smithi12223906-50 from 1915869 to 2353835 tid 2 2024-09-15T03:56:59.219 INFO:tasks.rados.rados.0.smithi122.stdout:2394: delete oid 24 current snap is 277 2024-09-15T03:56:59.226 INFO:tasks.rados.rados.0.smithi122.stdout:2393: finishing write tid 1 to smithi12223906-50 2024-09-15T03:56:59.227 INFO:tasks.rados.rados.0.smithi122.stdout:2393: finishing write tid 2 to smithi12223906-50 2024-09-15T03:56:59.227 INFO:tasks.rados.rados.0.smithi122.stdout:2393: finishing write tid 3 to smithi12223906-50 2024-09-15T03:56:59.227 INFO:tasks.rados.rados.0.smithi122.stdout:2393: finishing write tid 4 to smithi12223906-50 2024-09-15T03:56:59.227 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1593 (ObjNum 855 snap 277 seq_num 855) dirty exists 2024-09-15T03:56:59.227 INFO:tasks.rados.rados.0.smithi122.stdout:2393: left oid 50 (ObjNum 855 snap 277 seq_num 855) 2024-09-15T03:56:59.228 INFO:tasks.rados.rados.0.smithi122.stdout:2393: done (1 left) 2024-09-15T03:56:59.228 INFO:tasks.rados.rados.0.smithi122.stdout:2394: done (0 left) 2024-09-15T03:56:59.228 INFO:tasks.rados.rados.0.smithi122.stdout:2395: write oid 16 current snap is 277 2024-09-15T03:56:59.228 INFO:tasks.rados.rados.0.smithi122.stdout:2395: seq_num 857 ranges {403599=719929,1916480=441635} 2024-09-15T03:56:59.237 INFO:tasks.rados.rados.0.smithi122.stdout:2395: writing smithi12223906-16 from 403599 to 1123528 tid 1 2024-09-15T03:56:59.246 INFO:tasks.rados.rados.0.smithi122.stdout:2395: writing smithi12223906-16 from 1916480 to 2358115 tid 2 2024-09-15T03:56:59.246 INFO:tasks.rados.rados.0.smithi122.stdout:2396: write (excl) oid 27 current snap is 277 2024-09-15T03:56:59.246 INFO:tasks.rados.rados.0.smithi122.stdout:2396: seq_num 858 ranges {420406=795178,1974962=445444} 2024-09-15T03:56:59.253 INFO:tasks.rados.rados.0.smithi122.stdout:2396: writing smithi12223906-27 from 420406 to 1215584 tid 1 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2396: writing smithi12223906-27 from 1974962 to 2420406 tid 2 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2397: rollback oid 2 current snap is 277 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 250 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2395: finishing write tid 1 to smithi12223906-16 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2398: delete oid 18 current snap is 277 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2395: finishing write tid 2 to smithi12223906-16 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2395: finishing write tid 3 to smithi12223906-16 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2395: finishing write tid 4 to smithi12223906-16 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1310 (ObjNum 857 snap 277 seq_num 857) dirty exists 2024-09-15T03:56:59.260 INFO:tasks.rados.rados.0.smithi122.stdout:2395: left oid 16 (ObjNum 857 snap 277 seq_num 857) 2024-09-15T03:56:59.262 INFO:tasks.rados.rados.0.smithi122.stdout:2397: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:56:59.262 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1039 (ObjNum 129 snap 0 seq_num 1843199792) dirty dne 2024-09-15T03:56:59.267 INFO:tasks.rados.rados.0.smithi122.stdout:2396: finishing write tid 1 to smithi12223906-27 2024-09-15T03:56:59.275 INFO:tasks.rados.rados.0.smithi122.stdout:2396: finishing write tid 2 to smithi12223906-27 2024-09-15T03:56:59.275 INFO:tasks.rados.rados.0.smithi122.stdout:2396: finishing write tid 3 to smithi12223906-27 2024-09-15T03:56:59.275 INFO:tasks.rados.rados.0.smithi122.stdout:2396: finishing write tid 4 to smithi12223906-27 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1596 (ObjNum 858 snap 277 seq_num 858) dirty exists 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:2396: left oid 27 (ObjNum 858 snap 277 seq_num 858) 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:2395: done (3 left) 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:2396: done (2 left) 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:2397: done (1 left) 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:2398: done (0 left) 2024-09-15T03:56:59.276 INFO:tasks.rados.rados.0.smithi122.stdout:2399: copy_from oid 5 from oid 33 current snap is 277 2024-09-15T03:56:59.277 INFO:tasks.rados.rados.0.smithi122.stdout:2400: snap_remove snap 227 2024-09-15T03:57:00.052 INFO:tasks.rados.rados.0.smithi122.stdout:2400: done (1 left) 2024-09-15T03:57:00.052 INFO:tasks.rados.rados.0.smithi122.stdout:2401: read oid 35 snap 230 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2401: expect deleted 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2402: read oid 45 snap -1 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2402: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2403: read oid 28 snap 259 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2403: expect (ObjNum 770 snap 244 seq_num 770) 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2404: write (excl) oid 25 current snap is 277 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2399: finishing copy_from to smithi12223906-5 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2399: got expected ENOENT (src dne) 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2399: finishing copy_from racing read to smithi12223906-5 2024-09-15T03:57:00.053 INFO:tasks.rados.rados.0.smithi122.stdout:2404: seq_num 860 ranges {454020=662029,1691926=588962} 2024-09-15T03:57:00.062 INFO:tasks.rados.rados.0.smithi122.stdout:2404: writing smithi12223906-25 from 454020 to 1116049 tid 1 2024-09-15T03:57:00.070 INFO:tasks.rados.rados.0.smithi122.stdout:2404: writing smithi12223906-25 from 1691926 to 2280888 tid 2 2024-09-15T03:57:00.070 INFO:tasks.rados.rados.0.smithi122.stdout:2399: done (4 left) 2024-09-15T03:57:00.070 INFO:tasks.rados.rados.0.smithi122.stdout:2405: copy_from oid 21 from oid 33 current snap is 277 2024-09-15T03:57:00.070 INFO:tasks.rados.rados.0.smithi122.stdout:2402: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:57:00.181 INFO:tasks.rados.rados.0.smithi122.stdout:2403: expect (ObjNum 770 snap 244 seq_num 770) 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2405: finishing copy_from to smithi12223906-21 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2405: got expected ENOENT (src dne) 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2405: finishing copy_from racing read to smithi12223906-21 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2404: finishing write tid 1 to smithi12223906-25 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2404: finishing write tid 2 to smithi12223906-25 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2404: finishing write tid 3 to smithi12223906-25 2024-09-15T03:57:00.280 INFO:tasks.rados.rados.0.smithi122.stdout:2404: finishing write tid 4 to smithi12223906-25 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 1615 (ObjNum 860 snap 277 seq_num 860) dirty exists 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2404: left oid 25 (ObjNum 860 snap 277 seq_num 860) 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2401: done (4 left) 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2402: done (3 left) 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2403: done (2 left) 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2404: done (1 left) 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2405: done (0 left) 2024-09-15T03:57:00.281 INFO:tasks.rados.rados.0.smithi122.stdout:2406: snap_create 2024-09-15T03:57:01.006 INFO:tasks.rados.rados.0.smithi122.stdout:2406: done (0 left) 2024-09-15T03:57:01.007 INFO:tasks.rados.rados.0.smithi122.stdout:2407: rollback oid 6 current snap is 278 2024-09-15T03:57:01.007 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 6 to 231 2024-09-15T03:57:01.007 INFO:tasks.rados.rados.0.smithi122.stdout:2408: write (excl) oid 39 current snap is 278 2024-09-15T03:57:01.007 INFO:tasks.rados.rados.0.smithi122.stdout:2408: seq_num 861 ranges {470827=737278,1750408=720419} 2024-09-15T03:57:01.013 INFO:tasks.rados.rados.0.smithi122.stdout:2408: writing smithi12223906-39 from 470827 to 1208105 tid 1 2024-09-15T03:57:01.021 INFO:tasks.rados.rados.0.smithi122.stdout:2408: writing smithi12223906-39 from 1750408 to 2470827 tid 2 2024-09-15T03:57:01.021 INFO:tasks.rados.rados.0.smithi122.stdout:2409: write oid 32 current snap is 278 2024-09-15T03:57:01.021 INFO:tasks.rados.rados.0.smithi122.stdout:2407: finishing rollback tid 0 to smithi12223906-6 2024-09-15T03:57:01.021 INFO:tasks.rados.rados.0.smithi122.stdout:2409: seq_num 862 ranges {487634=412527,1408890=592631} 2024-09-15T03:57:01.025 INFO:tasks.rados.rados.0.smithi122.stdout:2409: writing smithi12223906-32 from 487634 to 900161 tid 1 2024-09-15T03:57:01.032 INFO:tasks.rados.rados.0.smithi122.stdout:2409: writing smithi12223906-32 from 1408890 to 2001521 tid 2 2024-09-15T03:57:01.032 INFO:tasks.rados.rados.0.smithi122.stdout:2410: read oid 40 snap -1 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2410: expect deleted 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2411: write (excl) oid 42 current snap is 278 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2407: finishing rollback tid 1 to smithi12223906-6 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2407: finishing rollback tid 2 to smithi12223906-6 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 843 (ObjNum 744 snap 226 seq_num 744) dirty exists 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2408: finishing write tid 1 to smithi12223906-39 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2408: finishing write tid 2 to smithi12223906-39 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2408: finishing write tid 3 to smithi12223906-39 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2408: finishing write tid 4 to smithi12223906-39 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 1621 (ObjNum 861 snap 278 seq_num 861) dirty exists 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2408: left oid 39 (ObjNum 861 snap 278 seq_num 861) 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2409: finishing write tid 1 to smithi12223906-32 2024-09-15T03:57:01.033 INFO:tasks.rados.rados.0.smithi122.stdout:2411: seq_num 863 ranges {504441=487776,1467372=736289} 2024-09-15T03:57:01.038 INFO:tasks.rados.rados.0.smithi122.stdout:2411: writing smithi12223906-42 from 504441 to 992217 tid 1 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2411: writing smithi12223906-42 from 1467372 to 2203661 tid 2 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2407: done (4 left) 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2408: done (3 left) 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2412: snap_create 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2409: finishing write tid 2 to smithi12223906-32 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2409: finishing write tid 3 to smithi12223906-32 2024-09-15T03:57:01.045 INFO:tasks.rados.rados.0.smithi122.stdout:2409: finishing write tid 4 to smithi12223906-32 2024-09-15T03:57:01.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 1613 (ObjNum 862 snap 278 seq_num 862) dirty exists 2024-09-15T03:57:01.046 INFO:tasks.rados.rados.0.smithi122.stdout:2409: left oid 32 (ObjNum 862 snap 278 seq_num 862) 2024-09-15T03:57:01.046 INFO:tasks.rados.rados.0.smithi122.stdout:2411: finishing write tid 1 to smithi12223906-42 2024-09-15T03:57:01.052 INFO:tasks.rados.rados.0.smithi122.stdout:2411: finishing write tid 2 to smithi12223906-42 2024-09-15T03:57:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:2411: finishing write tid 3 to smithi12223906-42 2024-09-15T03:57:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:2411: finishing write tid 4 to smithi12223906-42 2024-09-15T03:57:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1625 (ObjNum 863 snap 278 seq_num 863) dirty exists 2024-09-15T03:57:01.053 INFO:tasks.rados.rados.0.smithi122.stdout:2411: left oid 42 (ObjNum 863 snap 278 seq_num 863) 2024-09-15T03:57:02.063 INFO:tasks.rados.rados.0.smithi122.stdout:2409: done (3 left) 2024-09-15T03:57:02.064 INFO:tasks.rados.rados.0.smithi122.stdout:2410: done (2 left) 2024-09-15T03:57:02.064 INFO:tasks.rados.rados.0.smithi122.stdout:2411: done (1 left) 2024-09-15T03:57:02.064 INFO:tasks.rados.rados.0.smithi122.stdout:2412: done (0 left) 2024-09-15T03:57:02.064 INFO:tasks.rados.rados.0.smithi122.stdout:2413: delete oid 48 current snap is 279 2024-09-15T03:57:02.069 INFO:tasks.rados.rados.0.smithi122.stdout:2413: done (0 left) 2024-09-15T03:57:02.069 INFO:tasks.rados.rados.0.smithi122.stdout:2414: write (excl) oid 7 current snap is 279 2024-09-15T03:57:02.069 INFO:tasks.rados.rados.0.smithi122.stdout:2414: seq_num 865 ranges {538055=638274,1867983=670072} 2024-09-15T03:57:02.079 INFO:tasks.rados.rados.0.smithi122.stdout:2414: writing smithi12223906-7 from 538055 to 1176329 tid 1 2024-09-15T03:57:02.088 INFO:tasks.rados.rados.0.smithi122.stdout:2414: writing smithi12223906-7 from 1867983 to 2538055 tid 2 2024-09-15T03:57:02.088 INFO:tasks.rados.rados.0.smithi122.stdout:2415: snap_create 2024-09-15T03:57:02.088 INFO:tasks.rados.rados.0.smithi122.stdout:2414: finishing write tid 1 to smithi12223906-7 2024-09-15T03:57:02.094 INFO:tasks.rados.rados.0.smithi122.stdout:2414: finishing write tid 2 to smithi12223906-7 2024-09-15T03:57:02.094 INFO:tasks.rados.rados.0.smithi122.stdout:2414: finishing write tid 3 to smithi12223906-7 2024-09-15T03:57:02.094 INFO:tasks.rados.rados.0.smithi122.stdout:2414: finishing write tid 4 to smithi12223906-7 2024-09-15T03:57:02.095 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1565 (ObjNum 865 snap 279 seq_num 865) dirty exists 2024-09-15T03:57:02.095 INFO:tasks.rados.rados.0.smithi122.stdout:2414: left oid 7 (ObjNum 865 snap 279 seq_num 865) 2024-09-15T03:57:03.068 INFO:tasks.rados.rados.0.smithi122.stdout:2414: done (1 left) 2024-09-15T03:57:03.068 INFO:tasks.rados.rados.0.smithi122.stdout:2415: done (0 left) 2024-09-15T03:57:03.068 INFO:tasks.rados.rados.0.smithi122.stdout:2416: delete oid 21 current snap is 280 2024-09-15T03:57:03.074 INFO:tasks.rados.rados.0.smithi122.stdout:2416: done (0 left) 2024-09-15T03:57:03.074 INFO:tasks.rados.rados.0.smithi122.stdout:2417: snap_create 2024-09-15T03:57:04.081 INFO:tasks.rados.rados.0.smithi122.stdout:2417: done (0 left) 2024-09-15T03:57:04.081 INFO:tasks.rados.rados.0.smithi122.stdout:2418: snap_remove snap 240 2024-09-15T03:57:05.086 INFO:tasks.rados.rados.0.smithi122.stdout:2418: done (0 left) 2024-09-15T03:57:05.086 INFO:tasks.rados.rados.0.smithi122.stdout:2419: read oid 23 snap -1 2024-09-15T03:57:05.086 INFO:tasks.rados.rados.0.smithi122.stdout:2419: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:57:05.086 INFO:tasks.rados.rados.0.smithi122.stdout:2420: snap_remove snap 259 2024-09-15T03:57:06.012 INFO:tasks.rados.rados.0.smithi122.stdout:2420: done (1 left) 2024-09-15T03:57:06.013 INFO:tasks.rados.rados.0.smithi122.stdout:2421: delete oid 10 current snap is 281 2024-09-15T03:57:06.013 INFO:tasks.rados.rados.0.smithi122.stdout:2419: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:57:06.282 INFO:tasks.rados.rados.0.smithi122.stdout:2419: done (1 left) 2024-09-15T03:57:06.282 INFO:tasks.rados.rados.0.smithi122.stdout:2421: done (0 left) 2024-09-15T03:57:06.282 INFO:tasks.rados.rados.0.smithi122.stdout:2422: copy_from oid 44 from oid 33 current snap is 281 2024-09-15T03:57:06.282 INFO:tasks.rados.rados.0.smithi122.stdout:2423: write oid 41 current snap is 281 2024-09-15T03:57:06.282 INFO:tasks.rados.rados.0.smithi122.stdout:2423: seq_num 868 ranges {588476=580374,1643429=603638} 2024-09-15T03:57:06.297 INFO:tasks.rados.rados.0.smithi122.stdout:2423: writing smithi12223906-41 from 588476 to 1168850 tid 1 2024-09-15T03:57:06.303 INFO:tasks.rados.rados.0.smithi122.stdout:2423: writing smithi12223906-41 from 1643429 to 2247067 tid 2 2024-09-15T03:57:06.303 INFO:tasks.rados.rados.0.smithi122.stdout:2424: read oid 15 snap -1 2024-09-15T03:57:06.303 INFO:tasks.rados.rados.0.smithi122.stdout:2424: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:57:06.303 INFO:tasks.rados.rados.0.smithi122.stdout:2425: snap_remove snap 263 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2425: done (3 left) 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2426: rollback oid 35 current snap is 281 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 279 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2422: finishing copy_from to smithi12223906-44 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2422: got expected ENOENT (src dne) 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2422: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2423: finishing write tid 1 to smithi12223906-41 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2423: finishing write tid 2 to smithi12223906-41 2024-09-15T03:57:07.102 INFO:tasks.rados.rados.0.smithi122.stdout:2424: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:57:07.234 INFO:tasks.rados.rados.0.smithi122.stdout:2423: finishing write tid 3 to smithi12223906-41 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2423: finishing write tid 4 to smithi12223906-41 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 827 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2423: left oid 41 (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2422: done (3 left) 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2423: done (2 left) 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2424: done (1 left) 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2427: delete oid 10 current snap is 281 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2426: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2426: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:2426: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:57:07.235 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 603 (ObjNum 818 snap 267 seq_num 818) dirty exists 2024-09-15T03:57:07.238 INFO:tasks.rados.rados.0.smithi122.stdout:2426: done (1 left) 2024-09-15T03:57:07.239 INFO:tasks.rados.rados.0.smithi122.stdout:2427: done (0 left) 2024-09-15T03:57:07.239 INFO:tasks.rados.rados.0.smithi122.stdout:2428: write (excl) oid 27 current snap is 281 2024-09-15T03:57:07.239 INFO:tasks.rados.rados.0.smithi122.stdout:2428: seq_num 870 ranges {622090=730872,1760393=607307} 2024-09-15T03:57:07.246 INFO:tasks.rados.rados.0.smithi122.stdout:2428: writing smithi12223906-27 from 622090 to 1352962 tid 1 2024-09-15T03:57:07.252 INFO:tasks.rados.rados.0.smithi122.stdout:2428: writing smithi12223906-27 from 1760393 to 2367700 tid 2 2024-09-15T03:57:07.253 INFO:tasks.rados.rados.0.smithi122.stdout:2429: write (excl) oid 37 current snap is 281 2024-09-15T03:57:07.253 INFO:tasks.rados.rados.0.smithi122.stdout:2429: seq_num 871 ranges {638897=406121,1818875=750965} 2024-09-15T03:57:07.258 INFO:tasks.rados.rados.0.smithi122.stdout:2429: writing smithi12223906-37 from 638897 to 1045018 tid 1 2024-09-15T03:57:07.266 INFO:tasks.rados.rados.0.smithi122.stdout:2429: writing smithi12223906-37 from 1818875 to 2569840 tid 2 2024-09-15T03:57:07.267 INFO:tasks.rados.rados.0.smithi122.stdout:2430: write (excl) oid 21 current snap is 281 2024-09-15T03:57:07.267 INFO:tasks.rados.rados.0.smithi122.stdout:2430: seq_num 872 ranges {655704=481370,1761004=494623} 2024-09-15T03:57:07.274 INFO:tasks.rados.rados.0.smithi122.stdout:2430: writing smithi12223906-21 from 655704 to 1137074 tid 1 2024-09-15T03:57:07.280 INFO:tasks.rados.rados.0.smithi122.stdout:2430: writing smithi12223906-21 from 1761004 to 2255627 tid 2 2024-09-15T03:57:07.280 INFO:tasks.rados.rados.0.smithi122.stdout:2431: read oid 45 snap 273 2024-09-15T03:57:07.280 INFO:tasks.rados.rados.0.smithi122.stdout:2431: expect (ObjNum 833 snap 271 seq_num 833) 2024-09-15T03:57:07.280 INFO:tasks.rados.rados.0.smithi122.stdout:2432: rollback oid 9 current snap is 281 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 278 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2428: finishing write tid 1 to smithi12223906-27 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2433: read oid 50 snap -1 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2433: expect (ObjNum 855 snap 277 seq_num 855) 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2434: read oid 38 snap -1 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2434: expect (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2435: copy_from oid 13 from oid 35 current snap is 281 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2428: finishing write tid 2 to smithi12223906-27 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2436: delete oid 7 current snap is 281 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2429: finishing write tid 1 to smithi12223906-37 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2428: finishing write tid 3 to smithi12223906-27 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2428: finishing write tid 4 to smithi12223906-27 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1647 (ObjNum 870 snap 281 seq_num 870) dirty exists 2024-09-15T03:57:07.281 INFO:tasks.rados.rados.0.smithi122.stdout:2428: left oid 27 (ObjNum 870 snap 281 seq_num 870) 2024-09-15T03:57:07.282 INFO:tasks.rados.rados.0.smithi122.stdout:2429: finishing write tid 2 to smithi12223906-37 2024-09-15T03:57:07.282 INFO:tasks.rados.rados.0.smithi122.stdout:2429: finishing write tid 3 to smithi12223906-37 2024-09-15T03:57:07.282 INFO:tasks.rados.rados.0.smithi122.stdout:2429: finishing write tid 4 to smithi12223906-37 2024-09-15T03:57:07.282 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1044 (ObjNum 871 snap 281 seq_num 871) dirty exists 2024-09-15T03:57:07.282 INFO:tasks.rados.rados.0.smithi122.stdout:2429: left oid 37 (ObjNum 871 snap 281 seq_num 871) 2024-09-15T03:57:07.282 INFO:tasks.rados.rados.0.smithi122.stdout:2430: finishing write tid 1 to smithi12223906-21 2024-09-15T03:57:07.283 INFO:tasks.rados.rados.0.smithi122.stdout:2428: done (8 left) 2024-09-15T03:57:07.283 INFO:tasks.rados.rados.0.smithi122.stdout:2429: done (7 left) 2024-09-15T03:57:07.284 INFO:tasks.rados.rados.0.smithi122.stdout:2436: done (6 left) 2024-09-15T03:57:07.284 INFO:tasks.rados.rados.0.smithi122.stdout:2437: snap_remove snap 270 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:2437: done (6 left) 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:2438: rollback oid 10 current snap is 281 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 271 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:2432: finishing rollback tid 1 to smithi12223906-9 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 840 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:2430: finishing write tid 2 to smithi12223906-21 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:2430: finishing write tid 3 to smithi12223906-21 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:2430: finishing write tid 4 to smithi12223906-21 2024-09-15T03:57:08.125 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1047 (ObjNum 872 snap 281 seq_num 872) dirty exists 2024-09-15T03:57:08.126 INFO:tasks.rados.rados.0.smithi122.stdout:2430: left oid 21 (ObjNum 872 snap 281 seq_num 872) 2024-09-15T03:57:08.126 INFO:tasks.rados.rados.0.smithi122.stdout:2434: expect (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:57:08.237 INFO:tasks.rados.rados.0.smithi122.stdout:2431: expect (ObjNum 833 snap 271 seq_num 833) 2024-09-15T03:57:08.362 INFO:tasks.rados.rados.0.smithi122.stdout:2433: expect (ObjNum 855 snap 277 seq_num 855) 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:2435: finishing copy_from racing read to smithi12223906-13 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:2435: finishing copy_from to smithi12223906-13 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 603 (ObjNum 818 snap 267 seq_num 818) dirty exists 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:2438: finishing rollback tid 1 to smithi12223906-10 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:2438: finishing rollback tid 2 to smithi12223906-10 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1049 (ObjNum 815 snap 267 seq_num 815) dirty exists 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:2430: done (6 left) 2024-09-15T03:57:08.439 INFO:tasks.rados.rados.0.smithi122.stdout:2431: done (5 left) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2432: done (4 left) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2433: done (3 left) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2434: done (2 left) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2435: done (1 left) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2438: done (0 left) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2439: read oid 3 snap -1 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2439: expect (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2440: read oid 31 snap 239 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2440: expect (ObjNum 741 snap 224 seq_num 741) 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2441: copy_from oid 6 from oid 30 current snap is 281 2024-09-15T03:57:08.440 INFO:tasks.rados.rados.0.smithi122.stdout:2442: snap_remove snap 198 2024-09-15T03:57:09.139 INFO:tasks.rados.rados.0.smithi122.stdout:2442: done (3 left) 2024-09-15T03:57:09.139 INFO:tasks.rados.rados.0.smithi122.stdout:2443: read oid 37 snap 266 2024-09-15T03:57:09.139 INFO:tasks.rados.rados.0.smithi122.stdout:2443: expect deleted 2024-09-15T03:57:09.139 INFO:tasks.rados.rados.0.smithi122.stdout:2444: copy_from oid 46 from oid 33 current snap is 281 2024-09-15T03:57:09.139 INFO:tasks.rados.rados.0.smithi122.stdout:2445: delete oid 7 current snap is 281 2024-09-15T03:57:09.139 INFO:tasks.rados.rados.0.smithi122.stdout:2441: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:57:09.140 INFO:tasks.rados.rados.0.smithi122.stdout:2440: expect (ObjNum 741 snap 224 seq_num 741) 2024-09-15T03:57:09.222 INFO:tasks.rados.rados.0.smithi122.stdout:2441: finishing copy_from to smithi12223906-6 2024-09-15T03:57:09.223 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 824 (ObjNum 844 snap 273 seq_num 844) dirty exists 2024-09-15T03:57:09.223 INFO:tasks.rados.rados.0.smithi122.stdout:2439: expect (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:57:09.525 INFO:tasks.rados.rados.0.smithi122.stdout:2444: finishing copy_from to smithi12223906-46 2024-09-15T03:57:09.525 INFO:tasks.rados.rados.0.smithi122.stdout:2444: got expected ENOENT (src dne) 2024-09-15T03:57:09.525 INFO:tasks.rados.rados.0.smithi122.stdout:2444: finishing copy_from racing read to smithi12223906-46 2024-09-15T03:57:09.525 INFO:tasks.rados.rados.0.smithi122.stdout:2439: done (5 left) 2024-09-15T03:57:09.525 INFO:tasks.rados.rados.0.smithi122.stdout:2440: done (4 left) 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2441: done (3 left) 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2443: done (2 left) 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2444: done (1 left) 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2445: done (0 left) 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2446: copy_from oid 34 from oid 33 current snap is 281 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2447: delete oid 39 current snap is 281 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2446: finishing copy_from to smithi12223906-34 2024-09-15T03:57:09.526 INFO:tasks.rados.rados.0.smithi122.stdout:2446: got expected ENOENT (src dne) 2024-09-15T03:57:09.527 INFO:tasks.rados.rados.0.smithi122.stdout:2446: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:57:09.533 INFO:tasks.rados.rados.0.smithi122.stdout:2446: done (1 left) 2024-09-15T03:57:09.533 INFO:tasks.rados.rados.0.smithi122.stdout:2447: done (0 left) 2024-09-15T03:57:09.533 INFO:tasks.rados.rados.0.smithi122.stdout:2448: write oid 29 current snap is 281 2024-09-15T03:57:09.534 INFO:tasks.rados.rados.0.smithi122.stdout:2448: seq_num 876 ranges {722932=498719,1994932=501961} 2024-09-15T03:57:09.548 INFO:tasks.rados.rados.0.smithi122.stdout:2448: writing smithi12223906-29 from 722932 to 1221651 tid 1 2024-09-15T03:57:09.556 INFO:tasks.rados.rados.0.smithi122.stdout:2448: writing smithi12223906-29 from 1994932 to 2496893 tid 2 2024-09-15T03:57:09.556 INFO:tasks.rados.rados.0.smithi122.stdout:2449: delete oid 49 current snap is 281 2024-09-15T03:57:09.556 INFO:tasks.rados.rados.0.smithi122.stdout:2448: finishing write tid 1 to smithi12223906-29 2024-09-15T03:57:09.561 INFO:tasks.rados.rados.0.smithi122.stdout:2448: finishing write tid 2 to smithi12223906-29 2024-09-15T03:57:09.561 INFO:tasks.rados.rados.0.smithi122.stdout:2448: finishing write tid 3 to smithi12223906-29 2024-09-15T03:57:09.561 INFO:tasks.rados.rados.0.smithi122.stdout:2448: finishing write tid 4 to smithi12223906-29 2024-09-15T03:57:09.562 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 1569 (ObjNum 876 snap 281 seq_num 876) dirty exists 2024-09-15T03:57:09.562 INFO:tasks.rados.rados.0.smithi122.stdout:2448: left oid 29 (ObjNum 876 snap 281 seq_num 876) 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2448: done (1 left) 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2449: done (0 left) 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2450: read oid 35 snap -1 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2450: expect (ObjNum 818 snap 267 seq_num 818) 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2451: copy_from oid 45 from oid 46 current snap is 281 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2452: read oid 21 snap -1 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2452: expect (ObjNum 872 snap 281 seq_num 872) 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2453: write oid 15 current snap is 281 2024-09-15T03:57:09.563 INFO:tasks.rados.rados.0.smithi122.stdout:2453: seq_num 878 ranges {756546=649217,2111896=505630} 2024-09-15T03:57:09.570 INFO:tasks.rados.rados.0.smithi122.stdout:2453: writing smithi12223906-15 from 756546 to 1405763 tid 1 2024-09-15T03:57:09.577 INFO:tasks.rados.rados.0.smithi122.stdout:2453: writing smithi12223906-15 from 2111896 to 2617526 tid 2 2024-09-15T03:57:09.577 INFO:tasks.rados.rados.0.smithi122.stdout:2454: snap_create 2024-09-15T03:57:09.577 INFO:tasks.rados.rados.0.smithi122.stdout:2452: expect (ObjNum 872 snap 281 seq_num 872) 2024-09-15T03:57:09.670 INFO:tasks.rados.rados.0.smithi122.stdout:2450: expect (ObjNum 818 snap 267 seq_num 818) 2024-09-15T03:57:09.791 INFO:tasks.rados.rados.0.smithi122.stdout:2453: finishing write tid 1 to smithi12223906-15 2024-09-15T03:57:09.791 INFO:tasks.rados.rados.0.smithi122.stdout:2453: finishing write tid 2 to smithi12223906-15 2024-09-15T03:57:09.791 INFO:tasks.rados.rados.0.smithi122.stdout:2453: finishing write tid 3 to smithi12223906-15 2024-09-15T03:57:09.791 INFO:tasks.rados.rados.0.smithi122.stdout:2453: finishing write tid 4 to smithi12223906-15 2024-09-15T03:57:09.791 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1313 (ObjNum 878 snap 281 seq_num 878) dirty exists 2024-09-15T03:57:09.791 INFO:tasks.rados.rados.0.smithi122.stdout:2453: left oid 15 (ObjNum 878 snap 281 seq_num 878) 2024-09-15T03:57:09.792 INFO:tasks.rados.rados.0.smithi122.stdout:2451: finishing copy_from racing read to smithi12223906-45 2024-09-15T03:57:09.792 INFO:tasks.rados.rados.0.smithi122.stdout:2451: finishing copy_from to smithi12223906-45 2024-09-15T03:57:09.792 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 811 (ObjNum 773 snap 250 seq_num 773) dirty exists 2024-09-15T03:57:10.158 INFO:tasks.rados.rados.0.smithi122.stdout:2450: done (4 left) 2024-09-15T03:57:10.159 INFO:tasks.rados.rados.0.smithi122.stdout:2451: done (3 left) 2024-09-15T03:57:10.159 INFO:tasks.rados.rados.0.smithi122.stdout:2452: done (2 left) 2024-09-15T03:57:10.159 INFO:tasks.rados.rados.0.smithi122.stdout:2453: done (1 left) 2024-09-15T03:57:10.159 INFO:tasks.rados.rados.0.smithi122.stdout:2454: done (0 left) 2024-09-15T03:57:10.159 INFO:tasks.rados.rados.0.smithi122.stdout:2455: snap_create 2024-09-15T03:57:11.011 INFO:tasks.rados.rados.0.smithi122.stdout:2455: done (0 left) 2024-09-15T03:57:11.011 INFO:tasks.rados.rados.0.smithi122.stdout:2456: read oid 22 snap 230 2024-09-15T03:57:11.011 INFO:tasks.rados.rados.0.smithi122.stdout:2456: expect (ObjNum 731 snap 222 seq_num 731) 2024-09-15T03:57:11.011 INFO:tasks.rados.rados.0.smithi122.stdout:2457: snap_remove snap 220 2024-09-15T03:57:12.016 INFO:tasks.rados.rados.0.smithi122.stdout:2457: done (1 left) 2024-09-15T03:57:12.016 INFO:tasks.rados.rados.0.smithi122.stdout:2458: delete oid 11 current snap is 283 2024-09-15T03:57:12.016 INFO:tasks.rados.rados.0.smithi122.stdout:2456: expect (ObjNum 731 snap 222 seq_num 731) 2024-09-15T03:57:12.161 INFO:tasks.rados.rados.0.smithi122.stdout:2456: done (1 left) 2024-09-15T03:57:12.161 INFO:tasks.rados.rados.0.smithi122.stdout:2458: done (0 left) 2024-09-15T03:57:12.162 INFO:tasks.rados.rados.0.smithi122.stdout:2459: write (excl) oid 12 current snap is 283 2024-09-15T03:57:12.162 INFO:tasks.rados.rados.0.smithi122.stdout:2459: seq_num 880 ranges {790160=799715,2112507=509299} 2024-09-15T03:57:12.176 INFO:tasks.rados.rados.0.smithi122.stdout:2459: writing smithi12223906-12 from 790160 to 1589875 tid 1 2024-09-15T03:57:12.182 INFO:tasks.rados.rados.0.smithi122.stdout:2459: writing smithi12223906-12 from 2112507 to 2621806 tid 2 2024-09-15T03:57:12.182 INFO:tasks.rados.rados.0.smithi122.stdout:2460: delete oid 25 current snap is 283 2024-09-15T03:57:12.186 INFO:tasks.rados.rados.0.smithi122.stdout:2459: finishing write tid 1 to smithi12223906-12 2024-09-15T03:57:12.187 INFO:tasks.rados.rados.0.smithi122.stdout:2459: finishing write tid 2 to smithi12223906-12 2024-09-15T03:57:12.188 INFO:tasks.rados.rados.0.smithi122.stdout:2459: finishing write tid 3 to smithi12223906-12 2024-09-15T03:57:12.188 INFO:tasks.rados.rados.0.smithi122.stdout:2459: finishing write tid 4 to smithi12223906-12 2024-09-15T03:57:12.188 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 758 (ObjNum 880 snap 283 seq_num 880) dirty exists 2024-09-15T03:57:12.188 INFO:tasks.rados.rados.0.smithi122.stdout:2459: left oid 12 (ObjNum 880 snap 283 seq_num 880) 2024-09-15T03:57:12.189 INFO:tasks.rados.rados.0.smithi122.stdout:2459: done (1 left) 2024-09-15T03:57:12.189 INFO:tasks.rados.rados.0.smithi122.stdout:2460: done (0 left) 2024-09-15T03:57:12.189 INFO:tasks.rados.rados.0.smithi122.stdout:2461: read oid 23 snap -1 2024-09-15T03:57:12.189 INFO:tasks.rados.rados.0.smithi122.stdout:2461: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:57:12.190 INFO:tasks.rados.rados.0.smithi122.stdout:2462: write oid 42 current snap is 283 2024-09-15T03:57:12.190 INFO:tasks.rados.rados.0.smithi122.stdout:2462: seq_num 882 ranges {423774=666566,1545824=512968,2704609=119165} 2024-09-15T03:57:12.195 INFO:tasks.rados.rados.0.smithi122.stdout:2462: writing smithi12223906-42 from 423774 to 1090340 tid 1 2024-09-15T03:57:12.203 INFO:tasks.rados.rados.0.smithi122.stdout:2462: writing smithi12223906-42 from 1545824 to 2058792 tid 2 2024-09-15T03:57:12.208 INFO:tasks.rados.rados.0.smithi122.stdout:2462: writing smithi12223906-42 from 2704609 to 2823774 tid 3 2024-09-15T03:57:12.209 INFO:tasks.rados.rados.0.smithi122.stdout:2463: snap_create 2024-09-15T03:57:12.209 INFO:tasks.rados.rados.0.smithi122.stdout:2461: expect (ObjNum 711 snap 217 seq_num 711) 2024-09-15T03:57:12.485 INFO:tasks.rados.rados.0.smithi122.stdout:2462: finishing write tid 1 to smithi12223906-42 2024-09-15T03:57:12.485 INFO:tasks.rados.rados.0.smithi122.stdout:2462: finishing write tid 2 to smithi12223906-42 2024-09-15T03:57:12.485 INFO:tasks.rados.rados.0.smithi122.stdout:2462: finishing write tid 3 to smithi12223906-42 2024-09-15T03:57:12.486 INFO:tasks.rados.rados.0.smithi122.stdout:2462: finishing write tid 4 to smithi12223906-42 2024-09-15T03:57:12.486 INFO:tasks.rados.rados.0.smithi122.stdout:2462: finishing write tid 5 to smithi12223906-42 2024-09-15T03:57:12.486 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1704 (ObjNum 882 snap 283 seq_num 882) dirty exists 2024-09-15T03:57:12.486 INFO:tasks.rados.rados.0.smithi122.stdout:2462: left oid 42 (ObjNum 882 snap 283 seq_num 882) 2024-09-15T03:57:13.168 INFO:tasks.rados.rados.0.smithi122.stdout:2461: done (2 left) 2024-09-15T03:57:13.168 INFO:tasks.rados.rados.0.smithi122.stdout:2462: done (1 left) 2024-09-15T03:57:13.168 INFO:tasks.rados.rados.0.smithi122.stdout:2463: done (0 left) 2024-09-15T03:57:13.168 INFO:tasks.rados.rados.0.smithi122.stdout:2464: read oid 49 snap 226 2024-09-15T03:57:13.168 INFO:tasks.rados.rados.0.smithi122.stdout:2464: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:57:13.168 INFO:tasks.rados.rados.0.smithi122.stdout:2465: delete oid 43 current snap is 284 2024-09-15T03:57:13.173 INFO:tasks.rados.rados.0.smithi122.stdout:2464: expect (ObjNum 596 snap 180 seq_num 596) 2024-09-15T03:57:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:2464: done (1 left) 2024-09-15T03:57:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:2465: done (0 left) 2024-09-15T03:57:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:2466: read oid 11 snap -1 2024-09-15T03:57:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:2466: expect deleted 2024-09-15T03:57:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:2467: read oid 34 snap -1 2024-09-15T03:57:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:2467: expect (ObjNum 809 snap 265 seq_num 809) 2024-09-15T03:57:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:2468: read oid 1 snap 264 2024-09-15T03:57:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:2468: expect deleted 2024-09-15T03:57:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:2469: delete oid 15 current snap is 284 2024-09-15T03:57:13.291 INFO:tasks.rados.rados.0.smithi122.stdout:2467: expect (ObjNum 809 snap 265 seq_num 809) 2024-09-15T03:57:13.457 INFO:tasks.rados.rados.0.smithi122.stdout:2466: done (3 left) 2024-09-15T03:57:13.457 INFO:tasks.rados.rados.0.smithi122.stdout:2467: done (2 left) 2024-09-15T03:57:13.457 INFO:tasks.rados.rados.0.smithi122.stdout:2468: done (1 left) 2024-09-15T03:57:13.457 INFO:tasks.rados.rados.0.smithi122.stdout:2469: done (0 left) 2024-09-15T03:57:13.457 INFO:tasks.rados.rados.0.smithi122.stdout:2470: write oid 21 current snap is 284 2024-09-15T03:57:13.457 INFO:tasks.rados.rados.0.smithi122.stdout:2470: seq_num 885 ranges {474195=492313,1604917=660295,2670525=203670} 2024-09-15T03:57:13.472 INFO:tasks.rados.rados.0.smithi122.stdout:2470: writing smithi12223906-21 from 474195 to 966508 tid 1 2024-09-15T03:57:13.479 INFO:tasks.rados.rados.0.smithi122.stdout:2470: writing smithi12223906-21 from 1604917 to 2265212 tid 2 2024-09-15T03:57:13.483 INFO:tasks.rados.rados.0.smithi122.stdout:2470: writing smithi12223906-21 from 2670525 to 2874195 tid 3 2024-09-15T03:57:13.483 INFO:tasks.rados.rados.0.smithi122.stdout:2471: snap_create 2024-09-15T03:57:13.483 INFO:tasks.rados.rados.0.smithi122.stdout:2470: finishing write tid 1 to smithi12223906-21 2024-09-15T03:57:13.488 INFO:tasks.rados.rados.0.smithi122.stdout:2470: finishing write tid 2 to smithi12223906-21 2024-09-15T03:57:13.488 INFO:tasks.rados.rados.0.smithi122.stdout:2470: finishing write tid 3 to smithi12223906-21 2024-09-15T03:57:13.489 INFO:tasks.rados.rados.0.smithi122.stdout:2470: finishing write tid 4 to smithi12223906-21 2024-09-15T03:57:13.489 INFO:tasks.rados.rados.0.smithi122.stdout:2470: finishing write tid 5 to smithi12223906-21 2024-09-15T03:57:13.489 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1053 (ObjNum 885 snap 284 seq_num 885) dirty exists 2024-09-15T03:57:13.489 INFO:tasks.rados.rados.0.smithi122.stdout:2470: left oid 21 (ObjNum 885 snap 284 seq_num 885) 2024-09-15T03:57:14.178 INFO:tasks.rados.rados.0.smithi122.stdout:2470: done (1 left) 2024-09-15T03:57:14.178 INFO:tasks.rados.rados.0.smithi122.stdout:2471: done (0 left) 2024-09-15T03:57:14.178 INFO:tasks.rados.rados.0.smithi122.stdout:2472: snap_create 2024-09-15T03:57:15.185 INFO:tasks.rados.rados.0.smithi122.stdout:2472: done (0 left) 2024-09-15T03:57:15.185 INFO:tasks.rados.rados.0.smithi122.stdout:2473: write oid 4 current snap is 286 2024-09-15T03:57:15.185 INFO:tasks.rados.rados.0.smithi122.stdout:2473: seq_num 886 ranges {491002=567562,1663399=520306,2814301=76701} 2024-09-15T03:57:15.191 INFO:tasks.rados.rados.0.smithi122.stdout:2473: writing smithi12223906-4 from 491002 to 1058564 tid 1 2024-09-15T03:57:15.197 INFO:tasks.rados.rados.0.smithi122.stdout:2473: writing smithi12223906-4 from 1663399 to 2183705 tid 2 2024-09-15T03:57:15.202 INFO:tasks.rados.rados.0.smithi122.stdout:2473: writing smithi12223906-4 from 2814301 to 2891002 tid 3 2024-09-15T03:57:15.202 INFO:tasks.rados.rados.0.smithi122.stdout:2474: delete oid 21 current snap is 286 2024-09-15T03:57:15.202 INFO:tasks.rados.rados.0.smithi122.stdout:2473: finishing write tid 1 to smithi12223906-4 2024-09-15T03:57:15.205 INFO:tasks.rados.rados.0.smithi122.stdout:2473: finishing write tid 2 to smithi12223906-4 2024-09-15T03:57:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2473: finishing write tid 3 to smithi12223906-4 2024-09-15T03:57:15.208 INFO:tasks.rados.rados.0.smithi122.stdout:2474: done (1 left) 2024-09-15T03:57:15.208 INFO:tasks.rados.rados.0.smithi122.stdout:2475: delete oid 1 current snap is 286 2024-09-15T03:57:15.208 INFO:tasks.rados.rados.0.smithi122.stdout:2473: finishing write tid 4 to smithi12223906-4 2024-09-15T03:57:15.208 INFO:tasks.rados.rados.0.smithi122.stdout:2473: finishing write tid 5 to smithi12223906-4 2024-09-15T03:57:15.208 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 820 (ObjNum 886 snap 286 seq_num 886) dirty exists 2024-09-15T03:57:15.209 INFO:tasks.rados.rados.0.smithi122.stdout:2473: left oid 4 (ObjNum 886 snap 286 seq_num 886) 2024-09-15T03:57:15.212 INFO:tasks.rados.rados.0.smithi122.stdout:2473: done (1 left) 2024-09-15T03:57:15.212 INFO:tasks.rados.rados.0.smithi122.stdout:2475: done (0 left) 2024-09-15T03:57:15.212 INFO:tasks.rados.rados.0.smithi122.stdout:2476: copy_from oid 37 from oid 10 current snap is 286 2024-09-15T03:57:15.212 INFO:tasks.rados.rados.0.smithi122.stdout:2477: snap_remove snap 278 2024-09-15T03:57:16.014 INFO:tasks.rados.rados.0.smithi122.stdout:2477: done (1 left) 2024-09-15T03:57:16.014 INFO:tasks.rados.rados.0.smithi122.stdout:2478: snap_remove snap 226 2024-09-15T03:57:17.017 INFO:tasks.rados.rados.0.smithi122.stdout:2478: done (1 left) 2024-09-15T03:57:17.017 INFO:tasks.rados.rados.0.smithi122.stdout:2479: delete oid 21 current snap is 286 2024-09-15T03:57:17.017 INFO:tasks.rados.rados.0.smithi122.stdout:2476: finishing copy_from racing read to smithi12223906-37 2024-09-15T03:57:17.017 INFO:tasks.rados.rados.0.smithi122.stdout:2476: finishing copy_from to smithi12223906-37 2024-09-15T03:57:17.017 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1049 (ObjNum 815 snap 267 seq_num 815) dirty exists 2024-09-15T03:57:17.024 INFO:tasks.rados.rados.0.smithi122.stdout:2476: done (1 left) 2024-09-15T03:57:17.024 INFO:tasks.rados.rados.0.smithi122.stdout:2479: done (0 left) 2024-09-15T03:57:17.024 INFO:tasks.rados.rados.0.smithi122.stdout:2480: write (excl) oid 25 current snap is 286 2024-09-15T03:57:17.025 INFO:tasks.rados.rados.0.smithi122.stdout:2480: seq_num 890 ranges {558230=584911,1897327=527644,2923993=34237} 2024-09-15T03:57:17.031 INFO:tasks.rados.rados.0.smithi122.stdout:2480: writing smithi12223906-25 from 558230 to 1143141 tid 1 2024-09-15T03:57:17.038 INFO:tasks.rados.rados.0.smithi122.stdout:2480: writing smithi12223906-25 from 1897327 to 2424971 tid 2 2024-09-15T03:57:17.041 INFO:tasks.rados.rados.0.smithi122.stdout:2480: writing smithi12223906-25 from 2923993 to 2958230 tid 3 2024-09-15T03:57:17.041 INFO:tasks.rados.rados.0.smithi122.stdout:2481: write oid 23 current snap is 286 2024-09-15T03:57:17.041 INFO:tasks.rados.rados.0.smithi122.stdout:2481: seq_num 891 ranges {575037=660160,1955809=671302} 2024-09-15T03:57:17.048 INFO:tasks.rados.rados.0.smithi122.stdout:2481: writing smithi12223906-23 from 575037 to 1235197 tid 1 2024-09-15T03:57:17.056 INFO:tasks.rados.rados.0.smithi122.stdout:2481: writing smithi12223906-23 from 1955809 to 2627111 tid 2 2024-09-15T03:57:17.056 INFO:tasks.rados.rados.0.smithi122.stdout:2482: write oid 14 current snap is 286 2024-09-15T03:57:17.056 INFO:tasks.rados.rados.0.smithi122.stdout:2480: finishing write tid 1 to smithi12223906-25 2024-09-15T03:57:17.056 INFO:tasks.rados.rados.0.smithi122.stdout:2482: seq_num 892 ranges {591844=735409,1897938=531313,2862486=129358} 2024-09-15T03:57:17.063 INFO:tasks.rados.rados.0.smithi122.stdout:2482: writing smithi12223906-14 from 591844 to 1327253 tid 1 2024-09-15T03:57:17.069 INFO:tasks.rados.rados.0.smithi122.stdout:2482: writing smithi12223906-14 from 1897938 to 2429251 tid 2 2024-09-15T03:57:17.073 INFO:tasks.rados.rados.0.smithi122.stdout:2482: writing smithi12223906-14 from 2862486 to 2991844 tid 3 2024-09-15T03:57:17.073 INFO:tasks.rados.rados.0.smithi122.stdout:2483: snap_remove snap 250 2024-09-15T03:57:17.073 INFO:tasks.rados.rados.0.smithi122.stdout:2480: finishing write tid 2 to smithi12223906-25 2024-09-15T03:57:18.027 INFO:tasks.rados.rados.0.smithi122.stdout:2483: done (3 left) 2024-09-15T03:57:18.027 INFO:tasks.rados.rados.0.smithi122.stdout:2484: copy_from oid 36 from oid 29 current snap is 286 2024-09-15T03:57:18.027 INFO:tasks.rados.rados.0.smithi122.stdout:2480: finishing write tid 3 to smithi12223906-25 2024-09-15T03:57:18.027 INFO:tasks.rados.rados.0.smithi122.stdout:2480: finishing write tid 4 to smithi12223906-25 2024-09-15T03:57:18.027 INFO:tasks.rados.rados.0.smithi122.stdout:2480: finishing write tid 5 to smithi12223906-25 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 1732 (ObjNum 890 snap 286 seq_num 890) dirty exists 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2480: left oid 25 (ObjNum 890 snap 286 seq_num 890) 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2481: finishing write tid 1 to smithi12223906-23 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2481: finishing write tid 2 to smithi12223906-23 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2481: finishing write tid 3 to smithi12223906-23 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2481: finishing write tid 4 to smithi12223906-23 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 1695 (ObjNum 891 snap 286 seq_num 891) dirty exists 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2481: left oid 23 (ObjNum 891 snap 286 seq_num 891) 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2482: finishing write tid 1 to smithi12223906-14 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2482: finishing write tid 2 to smithi12223906-14 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2482: finishing write tid 3 to smithi12223906-14 2024-09-15T03:57:18.028 INFO:tasks.rados.rados.0.smithi122.stdout:2482: finishing write tid 4 to smithi12223906-14 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2482: finishing write tid 5 to smithi12223906-14 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1737 (ObjNum 892 snap 286 seq_num 892) dirty exists 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2482: left oid 14 (ObjNum 892 snap 286 seq_num 892) 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2480: done (3 left) 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2481: done (2 left) 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2482: done (1 left) 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2485: rollback oid 22 current snap is 286 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 22 to 274 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2486: copy_from oid 40 from oid 49 current snap is 286 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2487: read oid 18 snap 235 2024-09-15T03:57:18.029 INFO:tasks.rados.rados.0.smithi122.stdout:2487: expect (ObjNum 698 snap 214 seq_num 698) 2024-09-15T03:57:18.030 INFO:tasks.rados.rados.0.smithi122.stdout:2488: rollback oid 12 current snap is 286 2024-09-15T03:57:18.030 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 274 2024-09-15T03:57:18.030 INFO:tasks.rados.rados.0.smithi122.stdout:2489: delete oid 1 current snap is 286 2024-09-15T03:57:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2488: finishing rollback tid 0 to smithi12223906-12 2024-09-15T03:57:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2489: done (5 left) 2024-09-15T03:57:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2490: read oid 5 snap -1 2024-09-15T03:57:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2490: expect (ObjNum 834 snap 271 seq_num 834) 2024-09-15T03:57:18.036 INFO:tasks.rados.rados.0.smithi122.stdout:2491: write (excl) oid 20 current snap is 286 2024-09-15T03:57:18.036 INFO:tasks.rados.rados.0.smithi122.stdout:2491: seq_num 894 ranges {625458=485907,1614902=534982,2917332=108126} 2024-09-15T03:57:18.042 INFO:tasks.rados.rados.0.smithi122.stdout:2491: writing smithi12223906-20 from 625458 to 1111365 tid 1 2024-09-15T03:57:18.049 INFO:tasks.rados.rados.0.smithi122.stdout:2491: writing smithi12223906-20 from 1614902 to 2149884 tid 2 2024-09-15T03:57:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:2491: writing smithi12223906-20 from 2917332 to 3025458 tid 3 2024-09-15T03:57:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:2492: delete oid 27 current snap is 286 2024-09-15T03:57:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:2486: finishing copy_from to smithi12223906-40 2024-09-15T03:57:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:2486: got expected ENOENT (src dne) 2024-09-15T03:57:18.054 INFO:tasks.rados.rados.0.smithi122.stdout:2486: finishing copy_from racing read to smithi12223906-40 2024-09-15T03:57:18.055 INFO:tasks.rados.rados.0.smithi122.stdout:2488: finishing rollback tid 1 to smithi12223906-12 2024-09-15T03:57:18.055 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 760 (ObjNum 145 snap 0 seq_num 1842338144) dirty dne 2024-09-15T03:57:18.055 INFO:tasks.rados.rados.0.smithi122.stdout:2490: expect (ObjNum 834 snap 271 seq_num 834) 2024-09-15T03:57:18.120 INFO:tasks.rados.rados.0.smithi122.stdout:2491: finishing write tid 1 to smithi12223906-20 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2486: done (7 left) 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2488: done (6 left) 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2490: done (5 left) 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2492: done (4 left) 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2493: read oid 39 snap 282 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2493: expect deleted 2024-09-15T03:57:18.121 INFO:tasks.rados.rados.0.smithi122.stdout:2494: snap_remove snap 221 2024-09-15T03:57:19.033 INFO:tasks.rados.rados.0.smithi122.stdout:2494: done (5 left) 2024-09-15T03:57:19.033 INFO:tasks.rados.rados.0.smithi122.stdout:2495: snap_remove snap 279 2024-09-15T03:57:20.031 INFO:tasks.rados.rados.0.smithi122.stdout:2495: done (5 left) 2024-09-15T03:57:20.031 INFO:tasks.rados.rados.0.smithi122.stdout:2496: snap_remove snap 262 2024-09-15T03:57:21.026 INFO:tasks.rados.rados.0.smithi122.stdout:2496: done (5 left) 2024-09-15T03:57:21.027 INFO:tasks.rados.rados.0.smithi122.stdout:2497: write oid 23 current snap is 286 2024-09-15T03:57:21.027 INFO:tasks.rados.rados.0.smithi122.stdout:2497: seq_num 896 ranges {659072=636405,1731866=422298,2855825=203247} 2024-09-15T03:57:21.028 INFO:tasks.rados.rados.0.smithi122.stdout:2497: writing smithi12223906-23 from 659072 to 1295477 tid 1 2024-09-15T03:57:21.033 INFO:tasks.rados.rados.0.smithi122.stdout:2497: writing smithi12223906-23 from 1731866 to 2154164 tid 2 2024-09-15T03:57:21.039 INFO:tasks.rados.rados.0.smithi122.stdout:2497: writing smithi12223906-23 from 2855825 to 3059072 tid 3 2024-09-15T03:57:21.039 INFO:tasks.rados.rados.0.smithi122.stdout:2498: write oid 24 current snap is 286 2024-09-15T03:57:21.039 INFO:tasks.rados.rados.0.smithi122.stdout:2498: seq_num 897 ranges {675879=711654,2073995=682309} 2024-09-15T03:57:21.047 INFO:tasks.rados.rados.0.smithi122.stdout:2498: writing smithi12223906-24 from 675879 to 1387533 tid 1 2024-09-15T03:57:21.055 INFO:tasks.rados.rados.0.smithi122.stdout:2498: writing smithi12223906-24 from 2073995 to 2756304 tid 2 2024-09-15T03:57:21.055 INFO:tasks.rados.rados.0.smithi122.stdout:2499: snap_remove snap 276 2024-09-15T03:57:22.063 INFO:tasks.rados.rados.0.smithi122.stdout:2499: done (7 left) 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2500: read oid 3 snap 251 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2500: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2501: rollback oid 35 current snap is 286 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 215 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2485: finishing rollback tid 0 to smithi12223906-22 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2485: finishing rollback tid 1 to smithi12223906-22 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2485: finishing rollback tid 2 to smithi12223906-22 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1703 (ObjNum 819 snap 267 seq_num 819) dirty exists 2024-09-15T03:57:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2487: expect (ObjNum 698 snap 214 seq_num 698) 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2491: finishing write tid 2 to smithi12223906-20 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2491: finishing write tid 3 to smithi12223906-20 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2491: finishing write tid 4 to smithi12223906-20 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2491: finishing write tid 5 to smithi12223906-20 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 675 (ObjNum 894 snap 286 seq_num 894) dirty exists 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2491: left oid 20 (ObjNum 894 snap 286 seq_num 894) 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2484: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:57:22.288 INFO:tasks.rados.rados.0.smithi122.stdout:2484: finishing copy_from to smithi12223906-36 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 1569 (ObjNum 876 snap 281 seq_num 876) dirty exists 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2497: finishing write tid 1 to smithi12223906-23 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2497: finishing write tid 2 to smithi12223906-23 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2497: finishing write tid 3 to smithi12223906-23 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2497: finishing write tid 4 to smithi12223906-23 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2497: finishing write tid 5 to smithi12223906-23 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 1749 (ObjNum 896 snap 286 seq_num 896) dirty exists 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2497: left oid 23 (ObjNum 896 snap 286 seq_num 896) 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2498: finishing write tid 1 to smithi12223906-24 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2498: finishing write tid 2 to smithi12223906-24 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2498: finishing write tid 3 to smithi12223906-24 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2498: finishing write tid 4 to smithi12223906-24 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 1792 (ObjNum 897 snap 286 seq_num 897) dirty exists 2024-09-15T03:57:22.289 INFO:tasks.rados.rados.0.smithi122.stdout:2498: left oid 24 (ObjNum 897 snap 286 seq_num 897) 2024-09-15T03:57:22.290 INFO:tasks.rados.rados.0.smithi122.stdout:2501: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:57:22.290 INFO:tasks.rados.rados.0.smithi122.stdout:2501: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:57:22.290 INFO:tasks.rados.rados.0.smithi122.stdout:2501: finishing rollback tid 2 to smithi12223906-35 2024-09-15T03:57:22.290 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 606 (ObjNum 616 snap 183 seq_num 616) dirty exists 2024-09-15T03:57:22.290 INFO:tasks.rados.rados.0.smithi122.stdout:2500: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:57:22.505 INFO:tasks.rados.rados.0.smithi122.stdout:2484: done (8 left) 2024-09-15T03:57:22.505 INFO:tasks.rados.rados.0.smithi122.stdout:2485: done (7 left) 2024-09-15T03:57:22.505 INFO:tasks.rados.rados.0.smithi122.stdout:2487: done (6 left) 2024-09-15T03:57:22.505 INFO:tasks.rados.rados.0.smithi122.stdout:2491: done (5 left) 2024-09-15T03:57:22.506 INFO:tasks.rados.rados.0.smithi122.stdout:2493: done (4 left) 2024-09-15T03:57:22.506 INFO:tasks.rados.rados.0.smithi122.stdout:2497: done (3 left) 2024-09-15T03:57:22.506 INFO:tasks.rados.rados.0.smithi122.stdout:2498: done (2 left) 2024-09-15T03:57:22.506 INFO:tasks.rados.rados.0.smithi122.stdout:2500: done (1 left) 2024-09-15T03:57:22.506 INFO:tasks.rados.rados.0.smithi122.stdout:2501: done (0 left) 2024-09-15T03:57:22.506 INFO:tasks.rados.rados.0.smithi122.stdout:2502: snap_remove snap 215 2024-09-15T03:57:23.081 INFO:tasks.rados.rados.0.smithi122.stdout:2502: done (0 left) 2024-09-15T03:57:23.081 INFO:tasks.rados.rados.0.smithi122.stdout:2503: read oid 7 snap -1 2024-09-15T03:57:23.081 INFO:tasks.rados.rados.0.smithi122.stdout:2503: expect deleted 2024-09-15T03:57:23.081 INFO:tasks.rados.rados.0.smithi122.stdout:2504: snap_remove snap 217 2024-09-15T03:57:24.107 INFO:tasks.rados.rados.0.smithi122.stdout:2504: done (1 left) 2024-09-15T03:57:24.107 INFO:tasks.rados.rados.0.smithi122.stdout:2505: read oid 29 snap -1 2024-09-15T03:57:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2505: expect (ObjNum 876 snap 281 seq_num 876) 2024-09-15T03:57:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2506: copy_from oid 12 from oid 49 current snap is 286 2024-09-15T03:57:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2507: write (excl) oid 5 current snap is 286 2024-09-15T03:57:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2507: seq_num 898 ranges {692686=503256,1848830=425967,2910671=182015} 2024-09-15T03:57:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2507: writing smithi12223906-5 from 692686 to 1195942 tid 1 2024-09-15T03:57:24.109 INFO:tasks.rados.rados.0.smithi122.stdout:2507: writing smithi12223906-5 from 1848830 to 2274797 tid 2 2024-09-15T03:57:24.114 INFO:tasks.rados.rados.0.smithi122.stdout:2507: writing smithi12223906-5 from 2910671 to 3092686 tid 3 2024-09-15T03:57:24.114 INFO:tasks.rados.rados.0.smithi122.stdout:2508: delete oid 13 current snap is 286 2024-09-15T03:57:24.114 INFO:tasks.rados.rados.0.smithi122.stdout:2506: finishing copy_from to smithi12223906-12 2024-09-15T03:57:24.114 INFO:tasks.rados.rados.0.smithi122.stdout:2506: got expected ENOENT (src dne) 2024-09-15T03:57:24.114 INFO:tasks.rados.rados.0.smithi122.stdout:2506: finishing copy_from racing read to smithi12223906-12 2024-09-15T03:57:24.114 INFO:tasks.rados.rados.0.smithi122.stdout:2505: expect (ObjNum 876 snap 281 seq_num 876) 2024-09-15T03:57:24.206 INFO:tasks.rados.rados.0.smithi122.stdout:2507: finishing write tid 1 to smithi12223906-5 2024-09-15T03:57:24.206 INFO:tasks.rados.rados.0.smithi122.stdout:2507: finishing write tid 2 to smithi12223906-5 2024-09-15T03:57:24.206 INFO:tasks.rados.rados.0.smithi122.stdout:2507: finishing write tid 3 to smithi12223906-5 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2507: finishing write tid 4 to smithi12223906-5 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2507: finishing write tid 5 to smithi12223906-5 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 878 (ObjNum 898 snap 286 seq_num 898) dirty exists 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2507: left oid 5 (ObjNum 898 snap 286 seq_num 898) 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2503: done (4 left) 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2505: done (3 left) 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2506: done (2 left) 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2507: done (1 left) 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2508: done (0 left) 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2509: write oid 29 current snap is 286 2024-09-15T03:57:24.207 INFO:tasks.rados.rados.0.smithi122.stdout:2509: seq_num 900 ranges {726300=653754,1965794=429636,2965517=160783} 2024-09-15T03:57:24.214 INFO:tasks.rados.rados.0.smithi122.stdout:2509: writing smithi12223906-29 from 726300 to 1380054 tid 1 2024-09-15T03:57:24.220 INFO:tasks.rados.rados.0.smithi122.stdout:2509: writing smithi12223906-29 from 1965794 to 2395430 tid 2 2024-09-15T03:57:24.225 INFO:tasks.rados.rados.0.smithi122.stdout:2509: writing smithi12223906-29 from 2965517 to 3126300 tid 3 2024-09-15T03:57:24.225 INFO:tasks.rados.rados.0.smithi122.stdout:2510: copy_from oid 18 from oid 37 current snap is 286 2024-09-15T03:57:24.225 INFO:tasks.rados.rados.0.smithi122.stdout:2511: rollback oid 14 current snap is 286 2024-09-15T03:57:24.225 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 267 2024-09-15T03:57:24.225 INFO:tasks.rados.rados.0.smithi122.stdout:2512: write (excl) oid 24 current snap is 286 2024-09-15T03:57:24.225 INFO:tasks.rados.rados.0.smithi122.stdout:2512: seq_num 901 ranges {743107=729003,1907923=689647} 2024-09-15T03:57:24.233 INFO:tasks.rados.rados.0.smithi122.stdout:2512: writing smithi12223906-24 from 743107 to 1472110 tid 1 2024-09-15T03:57:24.240 INFO:tasks.rados.rados.0.smithi122.stdout:2512: writing smithi12223906-24 from 1907923 to 2597570 tid 2 2024-09-15T03:57:24.240 INFO:tasks.rados.rados.0.smithi122.stdout:2513: rollback oid 7 current snap is 286 2024-09-15T03:57:24.240 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 285 2024-09-15T03:57:24.240 INFO:tasks.rados.rados.0.smithi122.stdout:2509: finishing write tid 1 to smithi12223906-29 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2509: finishing write tid 2 to smithi12223906-29 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2509: finishing write tid 3 to smithi12223906-29 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2509: finishing write tid 4 to smithi12223906-29 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2509: finishing write tid 5 to smithi12223906-29 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 1573 (ObjNum 900 snap 286 seq_num 900) dirty exists 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2509: left oid 29 (ObjNum 900 snap 286 seq_num 900) 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2511: finishing rollback tid 0 to smithi12223906-14 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2509: done (4 left) 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2514: snap_create 2024-09-15T03:57:24.241 INFO:tasks.rados.rados.0.smithi122.stdout:2513: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:57:24.242 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1566 (ObjNum 993 snap 0 seq_num 1457508272) dirty dne 2024-09-15T03:57:24.242 INFO:tasks.rados.rados.0.smithi122.stdout:2511: finishing rollback tid 1 to smithi12223906-14 2024-09-15T03:57:24.247 INFO:tasks.rados.rados.0.smithi122.stdout:2511: finishing rollback tid 2 to smithi12223906-14 2024-09-15T03:57:24.247 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1831 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:57:24.249 INFO:tasks.rados.rados.0.smithi122.stdout:2512: finishing write tid 1 to smithi12223906-24 2024-09-15T03:57:24.249 INFO:tasks.rados.rados.0.smithi122.stdout:2510: finishing copy_from racing read to smithi12223906-18 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:2512: finishing write tid 2 to smithi12223906-24 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:2512: finishing write tid 3 to smithi12223906-24 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:2510: finishing copy_from to smithi12223906-18 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1049 (ObjNum 815 snap 267 seq_num 815) dirty exists 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:2512: finishing write tid 4 to smithi12223906-24 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 1834 (ObjNum 901 snap 286 seq_num 901) dirty exists 2024-09-15T03:57:24.253 INFO:tasks.rados.rados.0.smithi122.stdout:2512: left oid 24 (ObjNum 901 snap 286 seq_num 901) 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2510: done (4 left) 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2511: done (3 left) 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2512: done (2 left) 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2513: done (1 left) 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2514: done (0 left) 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2515: copy_from oid 47 from oid 20 current snap is 287 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2516: read oid 43 snap -1 2024-09-15T03:57:25.098 INFO:tasks.rados.rados.0.smithi122.stdout:2516: expect deleted 2024-09-15T03:57:25.099 INFO:tasks.rados.rados.0.smithi122.stdout:2517: snap_create 2024-09-15T03:57:25.115 INFO:tasks.rados.rados.0.smithi122.stdout:2515: finishing copy_from racing read to smithi12223906-47 2024-09-15T03:57:25.115 INFO:tasks.rados.rados.0.smithi122.stdout:2515: finishing copy_from to smithi12223906-47 2024-09-15T03:57:25.115 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 675 (ObjNum 894 snap 286 seq_num 894) dirty exists 2024-09-15T03:57:26.018 INFO:tasks.rados.rados.0.smithi122.stdout:2515: done (2 left) 2024-09-15T03:57:26.019 INFO:tasks.rados.rados.0.smithi122.stdout:2516: done (1 left) 2024-09-15T03:57:26.019 INFO:tasks.rados.rados.0.smithi122.stdout:2517: done (0 left) 2024-09-15T03:57:26.019 INFO:tasks.rados.rados.0.smithi122.stdout:2518: delete oid 38 current snap is 288 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2518: done (0 left) 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2519: rollback oid 32 current snap is 288 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 257 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2520: read oid 26 snap -1 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2520: expect (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2521: copy_from oid 35 from oid 14 current snap is 288 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2522: write (excl) oid 18 current snap is 288 2024-09-15T03:57:26.033 INFO:tasks.rados.rados.0.smithi122.stdout:2522: seq_num 903 ranges {776721=479501,2024887=693316} 2024-09-15T03:57:26.040 INFO:tasks.rados.rados.0.smithi122.stdout:2522: writing smithi12223906-18 from 776721 to 1256222 tid 1 2024-09-15T03:57:26.049 INFO:tasks.rados.rados.0.smithi122.stdout:2522: writing smithi12223906-18 from 2024887 to 2718203 tid 2 2024-09-15T03:57:26.049 INFO:tasks.rados.rados.0.smithi122.stdout:2523: snap_remove snap 252 2024-09-15T03:57:27.059 INFO:tasks.rados.rados.0.smithi122.stdout:2519: finishing rollback tid 0 to smithi12223906-32 2024-09-15T03:57:27.059 INFO:tasks.rados.rados.0.smithi122.stdout:2520: expect (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:2519: finishing rollback tid 1 to smithi12223906-32 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:2519: finishing rollback tid 2 to smithi12223906-32 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 1792 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:2521: finishing copy_from racing read to smithi12223906-35 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:2521: finishing copy_from to smithi12223906-35 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1831 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:2522: finishing write tid 1 to smithi12223906-18 2024-09-15T03:57:27.356 INFO:tasks.rados.rados.0.smithi122.stdout:2522: finishing write tid 2 to smithi12223906-18 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2522: finishing write tid 3 to smithi12223906-18 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2522: finishing write tid 4 to smithi12223906-18 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1796 (ObjNum 903 snap 288 seq_num 903) dirty exists 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2522: left oid 18 (ObjNum 903 snap 288 seq_num 903) 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2519: done (4 left) 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2520: done (3 left) 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2521: done (2 left) 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2522: done (1 left) 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2523: done (0 left) 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2524: rollback oid 34 current snap is 288 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 286 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2525: read oid 36 snap -1 2024-09-15T03:57:27.357 INFO:tasks.rados.rados.0.smithi122.stdout:2525: expect (ObjNum 876 snap 281 seq_num 876) 2024-09-15T03:57:27.358 INFO:tasks.rados.rados.0.smithi122.stdout:2526: read oid 43 snap -1 2024-09-15T03:57:27.358 INFO:tasks.rados.rados.0.smithi122.stdout:2526: expect deleted 2024-09-15T03:57:27.358 INFO:tasks.rados.rados.0.smithi122.stdout:2527: delete oid 20 current snap is 288 2024-09-15T03:57:27.363 INFO:tasks.rados.rados.0.smithi122.stdout:2527: done (3 left) 2024-09-15T03:57:27.363 INFO:tasks.rados.rados.0.smithi122.stdout:2528: copy_from oid 46 from oid 44 current snap is 288 2024-09-15T03:57:27.363 INFO:tasks.rados.rados.0.smithi122.stdout:2529: delete oid 37 current snap is 288 2024-09-15T03:57:27.364 INFO:tasks.rados.rados.0.smithi122.stdout:2525: expect (ObjNum 876 snap 281 seq_num 876) 2024-09-15T03:57:27.458 INFO:tasks.rados.rados.0.smithi122.stdout:2524: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:57:27.458 INFO:tasks.rados.rados.0.smithi122.stdout:2524: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:57:27.458 INFO:tasks.rados.rados.0.smithi122.stdout:2524: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 763 (ObjNum 809 snap 265 seq_num 809) dirty exists 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2528: finishing copy_from racing read to smithi12223906-46 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2528: finishing copy_from to smithi12223906-46 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 1562 (ObjNum 837 snap 271 seq_num 837) dirty exists 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2524: done (4 left) 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2525: done (3 left) 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2526: done (2 left) 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2528: done (1 left) 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2529: done (0 left) 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2530: rollback oid 7 current snap is 288 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 7 to 282 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2531: write (excl) oid 32 current snap is 288 2024-09-15T03:57:27.459 INFO:tasks.rados.rados.0.smithi122.stdout:2531: seq_num 906 ranges {427142=421601,1400333=440643,2613702=450326} 2024-09-15T03:57:27.469 INFO:tasks.rados.rados.0.smithi122.stdout:2531: writing smithi12223906-32 from 427142 to 848743 tid 1 2024-09-15T03:57:27.476 INFO:tasks.rados.rados.0.smithi122.stdout:2531: writing smithi12223906-32 from 1400333 to 1840976 tid 2 2024-09-15T03:57:27.483 INFO:tasks.rados.rados.0.smithi122.stdout:2531: writing smithi12223906-32 from 2613702 to 3064028 tid 3 2024-09-15T03:57:27.483 INFO:tasks.rados.rados.0.smithi122.stdout:2532: delete oid 48 current snap is 288 2024-09-15T03:57:27.483 INFO:tasks.rados.rados.0.smithi122.stdout:2530: finishing rollback tid 1 to smithi12223906-7 2024-09-15T03:57:27.483 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1566 (ObjNum 129 snap 0 seq_num 1842486088) dirty dne 2024-09-15T03:57:27.483 INFO:tasks.rados.rados.0.smithi122.stdout:2531: finishing write tid 1 to smithi12223906-32 2024-09-15T03:57:27.484 INFO:tasks.rados.rados.0.smithi122.stdout:2531: finishing write tid 2 to smithi12223906-32 2024-09-15T03:57:27.485 INFO:tasks.rados.rados.0.smithi122.stdout:2530: done (2 left) 2024-09-15T03:57:27.485 INFO:tasks.rados.rados.0.smithi122.stdout:2532: done (1 left) 2024-09-15T03:57:27.485 INFO:tasks.rados.rados.0.smithi122.stdout:2533: write (excl) oid 20 current snap is 288 2024-09-15T03:57:27.485 INFO:tasks.rados.rados.0.smithi122.stdout:2533: seq_num 908 ranges {460756=572099,1517297=444312,2668548=472870} 2024-09-15T03:57:27.491 INFO:tasks.rados.rados.0.smithi122.stdout:2533: writing smithi12223906-20 from 460756 to 1032855 tid 1 2024-09-15T03:57:27.496 INFO:tasks.rados.rados.0.smithi122.stdout:2533: writing smithi12223906-20 from 1517297 to 1961609 tid 2 2024-09-15T03:57:27.503 INFO:tasks.rados.rados.0.smithi122.stdout:2533: writing smithi12223906-20 from 2668548 to 3141418 tid 3 2024-09-15T03:57:27.503 INFO:tasks.rados.rados.0.smithi122.stdout:2534: write oid 48 current snap is 288 2024-09-15T03:57:27.503 INFO:tasks.rados.rados.0.smithi122.stdout:2534: seq_num 909 ranges {477563=647348,1859426=587970,2979618=297945} 2024-09-15T03:57:27.509 INFO:tasks.rados.rados.0.smithi122.stdout:2534: writing smithi12223906-48 from 477563 to 1124911 tid 1 2024-09-15T03:57:27.517 INFO:tasks.rados.rados.0.smithi122.stdout:2534: writing smithi12223906-48 from 1859426 to 2447396 tid 2 2024-09-15T03:57:27.522 INFO:tasks.rados.rados.0.smithi122.stdout:2534: writing smithi12223906-48 from 2979618 to 3277563 tid 3 2024-09-15T03:57:27.522 INFO:tasks.rados.rados.0.smithi122.stdout:2535: write (excl) oid 4 current snap is 288 2024-09-15T03:57:27.522 INFO:tasks.rados.rados.0.smithi122.stdout:2531: finishing write tid 3 to smithi12223906-32 2024-09-15T03:57:27.522 INFO:tasks.rados.rados.0.smithi122.stdout:2535: seq_num 910 ranges {494370=722597,1917908=447981,3007041=287329} 2024-09-15T03:57:27.529 INFO:tasks.rados.rados.0.smithi122.stdout:2535: writing smithi12223906-4 from 494370 to 1216967 tid 1 2024-09-15T03:57:27.536 INFO:tasks.rados.rados.0.smithi122.stdout:2535: writing smithi12223906-4 from 1917908 to 2365889 tid 2 2024-09-15T03:57:27.542 INFO:tasks.rados.rados.0.smithi122.stdout:2535: writing smithi12223906-4 from 3007041 to 3294370 tid 3 2024-09-15T03:57:27.542 INFO:tasks.rados.rados.0.smithi122.stdout:2536: copy_from oid 40 from oid 43 current snap is 288 2024-09-15T03:57:27.542 INFO:tasks.rados.rados.0.smithi122.stdout:2531: finishing write tid 4 to smithi12223906-32 2024-09-15T03:57:27.543 INFO:tasks.rados.rados.0.smithi122.stdout:2531: finishing write tid 5 to smithi12223906-32 2024-09-15T03:57:27.543 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 1812 (ObjNum 906 snap 288 seq_num 906) dirty exists 2024-09-15T03:57:27.543 INFO:tasks.rados.rados.0.smithi122.stdout:2531: left oid 32 (ObjNum 906 snap 288 seq_num 906) 2024-09-15T03:57:27.543 INFO:tasks.rados.rados.0.smithi122.stdout:2533: finishing write tid 1 to smithi12223906-20 2024-09-15T03:57:27.543 INFO:tasks.rados.rados.0.smithi122.stdout:2531: done (4 left) 2024-09-15T03:57:27.543 INFO:tasks.rados.rados.0.smithi122.stdout:2537: snap_remove snap 264 2024-09-15T03:57:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2537: done (4 left) 2024-09-15T03:57:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2538: read oid 19 snap -1 2024-09-15T03:57:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2538: expect (ObjNum 848 snap 276 seq_num 848) 2024-09-15T03:57:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2539: write oid 17 current snap is 288 2024-09-15T03:57:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2533: finishing write tid 2 to smithi12223906-20 2024-09-15T03:57:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2539: seq_num 911 ranges {511177=797846,1976390=591639,3034464=276713} 2024-09-15T03:57:28.079 INFO:tasks.rados.rados.0.smithi122.stdout:2539: writing smithi12223906-17 from 511177 to 1309023 tid 1 2024-09-15T03:57:28.087 INFO:tasks.rados.rados.0.smithi122.stdout:2539: writing smithi12223906-17 from 1976390 to 2568029 tid 2 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2539: writing smithi12223906-17 from 3034464 to 3311177 tid 3 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2540: delete oid 8 current snap is 288 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2533: finishing write tid 3 to smithi12223906-20 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2533: finishing write tid 4 to smithi12223906-20 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2533: finishing write tid 5 to smithi12223906-20 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 680 (ObjNum 908 snap 288 seq_num 908) dirty exists 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2533: left oid 20 (ObjNum 908 snap 288 seq_num 908) 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2534: finishing write tid 1 to smithi12223906-48 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2534: finishing write tid 2 to smithi12223906-48 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2534: finishing write tid 3 to smithi12223906-48 2024-09-15T03:57:28.092 INFO:tasks.rados.rados.0.smithi122.stdout:2534: finishing write tid 4 to smithi12223906-48 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2534: finishing write tid 5 to smithi12223906-48 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1059 (ObjNum 909 snap 288 seq_num 909) dirty exists 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2534: left oid 48 (ObjNum 909 snap 288 seq_num 909) 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2535: finishing write tid 1 to smithi12223906-4 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2536: finishing copy_from to smithi12223906-40 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2536: got expected ENOENT (src dne) 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2536: finishing copy_from racing read to smithi12223906-40 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2535: finishing write tid 2 to smithi12223906-4 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2535: finishing write tid 3 to smithi12223906-4 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2535: finishing write tid 4 to smithi12223906-4 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2535: finishing write tid 5 to smithi12223906-4 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 824 (ObjNum 910 snap 288 seq_num 910) dirty exists 2024-09-15T03:57:28.093 INFO:tasks.rados.rados.0.smithi122.stdout:2535: left oid 4 (ObjNum 910 snap 288 seq_num 910) 2024-09-15T03:57:28.094 INFO:tasks.rados.rados.0.smithi122.stdout:2538: expect (ObjNum 848 snap 276 seq_num 848) 2024-09-15T03:57:28.257 INFO:tasks.rados.rados.0.smithi122.stdout:2539: finishing write tid 1 to smithi12223906-17 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2539: finishing write tid 2 to smithi12223906-17 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2539: finishing write tid 3 to smithi12223906-17 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2539: finishing write tid 4 to smithi12223906-17 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2539: finishing write tid 5 to smithi12223906-17 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1044 (ObjNum 911 snap 288 seq_num 911) dirty exists 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2539: left oid 17 (ObjNum 911 snap 288 seq_num 911) 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2533: done (6 left) 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2534: done (5 left) 2024-09-15T03:57:28.258 INFO:tasks.rados.rados.0.smithi122.stdout:2535: done (4 left) 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2536: done (3 left) 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2538: done (2 left) 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2539: done (1 left) 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2540: done (0 left) 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2541: copy_from oid 29 from oid 12 current snap is 288 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2542: delete oid 43 current snap is 288 2024-09-15T03:57:28.259 INFO:tasks.rados.rados.0.smithi122.stdout:2541: finishing copy_from to smithi12223906-29 2024-09-15T03:57:28.260 INFO:tasks.rados.rados.0.smithi122.stdout:2541: got expected ENOENT (src dne) 2024-09-15T03:57:28.260 INFO:tasks.rados.rados.0.smithi122.stdout:2541: finishing copy_from racing read to smithi12223906-29 2024-09-15T03:57:28.262 INFO:tasks.rados.rados.0.smithi122.stdout:2541: done (1 left) 2024-09-15T03:57:28.262 INFO:tasks.rados.rados.0.smithi122.stdout:2542: done (0 left) 2024-09-15T03:57:28.262 INFO:tasks.rados.rados.0.smithi122.stdout:2543: snap_create 2024-09-15T03:57:29.079 INFO:tasks.rados.rados.0.smithi122.stdout:2543: done (0 left) 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2544: read oid 33 snap -1 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2544: expect deleted 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2545: copy_from oid 36 from oid 17 current snap is 289 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2546: read oid 39 snap -1 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2546: expect deleted 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2547: write oid 30 current snap is 289 2024-09-15T03:57:29.080 INFO:tasks.rados.rados.0.smithi122.stdout:2547: seq_num 914 ranges {561598=739946,1751836=455319,2716733=644865} 2024-09-15T03:57:29.087 INFO:tasks.rados.rados.0.smithi122.stdout:2547: writing smithi12223906-30 from 561598 to 1301544 tid 1 2024-09-15T03:57:29.095 INFO:tasks.rados.rados.0.smithi122.stdout:2547: writing smithi12223906-30 from 1751836 to 2207155 tid 2 2024-09-15T03:57:29.102 INFO:tasks.rados.rados.0.smithi122.stdout:2547: writing smithi12223906-30 from 2716733 to 3361598 tid 3 2024-09-15T03:57:29.102 INFO:tasks.rados.rados.0.smithi122.stdout:2548: write (excl) oid 7 current snap is 289 2024-09-15T03:57:29.102 INFO:tasks.rados.rados.0.smithi122.stdout:2548: seq_num 915 ranges {578405=415195,1410318=598977,2744156=584480} 2024-09-15T03:57:29.107 INFO:tasks.rados.rados.0.smithi122.stdout:2548: writing smithi12223906-7 from 578405 to 993600 tid 1 2024-09-15T03:57:29.113 INFO:tasks.rados.rados.0.smithi122.stdout:2548: writing smithi12223906-7 from 1410318 to 2009295 tid 2 2024-09-15T03:57:29.121 INFO:tasks.rados.rados.0.smithi122.stdout:2548: writing smithi12223906-7 from 2744156 to 3328636 tid 3 2024-09-15T03:57:29.121 INFO:tasks.rados.rados.0.smithi122.stdout:2549: read oid 23 snap -1 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2549: expect (ObjNum 896 snap 286 seq_num 896) 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2550: copy_from oid 19 from oid 49 current snap is 289 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2547: finishing write tid 1 to smithi12223906-30 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2547: finishing write tid 2 to smithi12223906-30 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2545: finishing copy_from racing read to smithi12223906-36 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2545: finishing copy_from to smithi12223906-36 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 1044 (ObjNum 911 snap 288 seq_num 911) dirty exists 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2547: finishing write tid 3 to smithi12223906-30 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2547: finishing write tid 4 to smithi12223906-30 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2547: finishing write tid 5 to smithi12223906-30 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 831 (ObjNum 914 snap 289 seq_num 914) dirty exists 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2547: left oid 30 (ObjNum 914 snap 289 seq_num 914) 2024-09-15T03:57:29.122 INFO:tasks.rados.rados.0.smithi122.stdout:2548: finishing write tid 1 to smithi12223906-7 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2548: finishing write tid 2 to smithi12223906-7 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2544: done (6 left) 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2545: done (5 left) 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2546: done (4 left) 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2547: done (3 left) 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2551: delete oid 25 current snap is 289 2024-09-15T03:57:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2550: finishing copy_from to smithi12223906-19 2024-09-15T03:57:29.124 INFO:tasks.rados.rados.0.smithi122.stdout:2550: got expected ENOENT (src dne) 2024-09-15T03:57:29.124 INFO:tasks.rados.rados.0.smithi122.stdout:2550: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:57:29.126 INFO:tasks.rados.rados.0.smithi122.stdout:2548: finishing write tid 3 to smithi12223906-7 2024-09-15T03:57:29.128 INFO:tasks.rados.rados.0.smithi122.stdout:2548: finishing write tid 4 to smithi12223906-7 2024-09-15T03:57:29.128 INFO:tasks.rados.rados.0.smithi122.stdout:2548: finishing write tid 5 to smithi12223906-7 2024-09-15T03:57:29.128 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1577 (ObjNum 915 snap 289 seq_num 915) dirty exists 2024-09-15T03:57:29.128 INFO:tasks.rados.rados.0.smithi122.stdout:2548: left oid 7 (ObjNum 915 snap 289 seq_num 915) 2024-09-15T03:57:29.129 INFO:tasks.rados.rados.0.smithi122.stdout:2548: done (3 left) 2024-09-15T03:57:29.129 INFO:tasks.rados.rados.0.smithi122.stdout:2550: done (2 left) 2024-09-15T03:57:29.129 INFO:tasks.rados.rados.0.smithi122.stdout:2551: done (1 left) 2024-09-15T03:57:29.129 INFO:tasks.rados.rados.0.smithi122.stdout:2552: write (excl) oid 5 current snap is 289 2024-09-15T03:57:29.129 INFO:tasks.rados.rados.0.smithi122.stdout:2552: seq_num 917 ranges {612019=565693,1810929=602646,3082649=329370} 2024-09-15T03:57:29.136 INFO:tasks.rados.rados.0.smithi122.stdout:2552: writing smithi12223906-5 from 612019 to 1177712 tid 1 2024-09-15T03:57:29.144 INFO:tasks.rados.rados.0.smithi122.stdout:2552: writing smithi12223906-5 from 1810929 to 2413575 tid 2 2024-09-15T03:57:29.150 INFO:tasks.rados.rados.0.smithi122.stdout:2552: writing smithi12223906-5 from 3082649 to 3412019 tid 3 2024-09-15T03:57:29.150 INFO:tasks.rados.rados.0.smithi122.stdout:2553: snap_remove snap 274 2024-09-15T03:57:30.091 INFO:tasks.rados.rados.0.smithi122.stdout:2553: done (2 left) 2024-09-15T03:57:30.092 INFO:tasks.rados.rados.0.smithi122.stdout:2554: read oid 2 snap -1 2024-09-15T03:57:30.092 INFO:tasks.rados.rados.0.smithi122.stdout:2554: expect deleted 2024-09-15T03:57:30.092 INFO:tasks.rados.rados.0.smithi122.stdout:2555: write oid 39 current snap is 289 2024-09-15T03:57:30.092 INFO:tasks.rados.rados.0.smithi122.stdout:2555: seq_num 918 ranges {628826=640942,1869411=462657,2826425=534649} 2024-09-15T03:57:30.099 INFO:tasks.rados.rados.0.smithi122.stdout:2555: writing smithi12223906-39 from 628826 to 1269768 tid 1 2024-09-15T03:57:30.105 INFO:tasks.rados.rados.0.smithi122.stdout:2555: writing smithi12223906-39 from 1869411 to 2332068 tid 2 2024-09-15T03:57:30.111 INFO:tasks.rados.rados.0.smithi122.stdout:2555: writing smithi12223906-39 from 2826425 to 3361074 tid 3 2024-09-15T03:57:30.112 INFO:tasks.rados.rados.0.smithi122.stdout:2556: rollback oid 44 current snap is 289 2024-09-15T03:57:30.112 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 230 2024-09-15T03:57:30.112 INFO:tasks.rados.rados.0.smithi122.stdout:2549: expect (ObjNum 896 snap 286 seq_num 896) 2024-09-15T03:57:30.383 INFO:tasks.rados.rados.0.smithi122.stdout:2552: finishing write tid 1 to smithi12223906-5 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2552: finishing write tid 2 to smithi12223906-5 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2552: finishing write tid 3 to smithi12223906-5 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2552: finishing write tid 4 to smithi12223906-5 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2552: finishing write tid 5 to smithi12223906-5 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 882 (ObjNum 917 snap 289 seq_num 917) dirty exists 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2552: left oid 5 (ObjNum 917 snap 289 seq_num 917) 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2555: finishing write tid 1 to smithi12223906-39 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2555: finishing write tid 2 to smithi12223906-39 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2555: finishing write tid 3 to smithi12223906-39 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2555: finishing write tid 4 to smithi12223906-39 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:2555: finishing write tid 5 to smithi12223906-39 2024-09-15T03:57:30.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 1885 (ObjNum 918 snap 289 seq_num 918) dirty exists 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2555: left oid 39 (ObjNum 918 snap 289 seq_num 918) 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2549: done (4 left) 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2552: done (3 left) 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2554: done (2 left) 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2555: done (1 left) 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2557: write oid 19 current snap is 289 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2556: finishing rollback tid 0 to smithi12223906-44 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2556: finishing rollback tid 1 to smithi12223906-44 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1833 (ObjNum 241 snap 0 seq_num 1843132720) dirty dne 2024-09-15T03:57:30.385 INFO:tasks.rados.rados.0.smithi122.stdout:2557: seq_num 919 ranges {645633=716191,1927893=606315,3137495=308138} 2024-09-15T03:57:30.401 INFO:tasks.rados.rados.0.smithi122.stdout:2557: writing smithi12223906-19 from 645633 to 1361824 tid 1 2024-09-15T03:57:30.407 INFO:tasks.rados.rados.0.smithi122.stdout:2557: writing smithi12223906-19 from 1927893 to 2534208 tid 2 2024-09-15T03:57:30.413 INFO:tasks.rados.rados.0.smithi122.stdout:2557: writing smithi12223906-19 from 3137495 to 3445633 tid 3 2024-09-15T03:57:30.413 INFO:tasks.rados.rados.0.smithi122.stdout:2556: done (1 left) 2024-09-15T03:57:30.413 INFO:tasks.rados.rados.0.smithi122.stdout:2558: write oid 34 current snap is 289 2024-09-15T03:57:30.413 INFO:tasks.rados.rados.0.smithi122.stdout:2558: seq_num 920 ranges {662440=507793,1702728=749973,2881271=557193} 2024-09-15T03:57:30.419 INFO:tasks.rados.rados.0.smithi122.stdout:2558: writing smithi12223906-34 from 662440 to 1170233 tid 1 2024-09-15T03:57:30.427 INFO:tasks.rados.rados.0.smithi122.stdout:2558: writing smithi12223906-34 from 1702728 to 2452701 tid 2 2024-09-15T03:57:30.433 INFO:tasks.rados.rados.0.smithi122.stdout:2558: writing smithi12223906-34 from 2881271 to 3438464 tid 3 2024-09-15T03:57:30.433 INFO:tasks.rados.rados.0.smithi122.stdout:2559: rollback oid 17 current snap is 289 2024-09-15T03:57:30.433 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 273 2024-09-15T03:57:30.433 INFO:tasks.rados.rados.0.smithi122.stdout:2557: finishing write tid 1 to smithi12223906-19 2024-09-15T03:57:30.433 INFO:tasks.rados.rados.0.smithi122.stdout:2557: finishing write tid 2 to smithi12223906-19 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2557: finishing write tid 3 to smithi12223906-19 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2557: finishing write tid 4 to smithi12223906-19 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2557: finishing write tid 5 to smithi12223906-19 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 1566 (ObjNum 919 snap 289 seq_num 919) dirty exists 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2557: left oid 19 (ObjNum 919 snap 289 seq_num 919) 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2558: finishing write tid 1 to smithi12223906-34 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2557: done (2 left) 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2560: rollback oid 49 current snap is 289 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 275 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2558: finishing write tid 2 to smithi12223906-34 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2561: copy_from oid 44 from oid 6 current snap is 289 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2562: rollback oid 9 current snap is 289 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 280 2024-09-15T03:57:30.434 INFO:tasks.rados.rados.0.smithi122.stdout:2563: snap_remove snap 282 2024-09-15T03:57:31.026 INFO:tasks.rados.rados.0.smithi122.stdout:2563: done (5 left) 2024-09-15T03:57:31.026 INFO:tasks.rados.rados.0.smithi122.stdout:2564: write oid 32 current snap is 289 2024-09-15T03:57:31.026 INFO:tasks.rados.rados.0.smithi122.stdout:2564: seq_num 921 ranges {679247=583042,2044857=609984,3192341=286906} 2024-09-15T03:57:31.035 INFO:tasks.rados.rados.0.smithi122.stdout:2564: writing smithi12223906-32 from 679247 to 1262289 tid 1 2024-09-15T03:57:31.043 INFO:tasks.rados.rados.0.smithi122.stdout:2564: writing smithi12223906-32 from 2044857 to 2654841 tid 2 2024-09-15T03:57:31.048 INFO:tasks.rados.rados.0.smithi122.stdout:2564: writing smithi12223906-32 from 3192341 to 3479247 tid 3 2024-09-15T03:57:31.048 INFO:tasks.rados.rados.0.smithi122.stdout:2565: read oid 26 snap 267 2024-09-15T03:57:31.048 INFO:tasks.rados.rados.0.smithi122.stdout:2565: expect (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:57:31.048 INFO:tasks.rados.rados.0.smithi122.stdout:2566: read oid 40 snap 267 2024-09-15T03:57:31.048 INFO:tasks.rados.rados.0.smithi122.stdout:2566: expect deleted 2024-09-15T03:57:31.049 INFO:tasks.rados.rados.0.smithi122.stdout:2567: write (excl) oid 35 current snap is 289 2024-09-15T03:57:31.049 INFO:tasks.rados.rados.0.smithi122.stdout:2567: seq_num 922 ranges {696054=658291,2103339=753642} 2024-09-15T03:57:31.055 INFO:tasks.rados.rados.0.smithi122.stdout:2567: writing smithi12223906-35 from 696054 to 1354345 tid 1 2024-09-15T03:57:31.064 INFO:tasks.rados.rados.0.smithi122.stdout:2567: writing smithi12223906-35 from 2103339 to 2856981 tid 2 2024-09-15T03:57:31.065 INFO:tasks.rados.rados.0.smithi122.stdout:2568: write oid 46 current snap is 289 2024-09-15T03:57:31.065 INFO:tasks.rados.rados.0.smithi122.stdout:2562: finishing rollback tid 1 to smithi12223906-9 2024-09-15T03:57:31.065 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 840 (ObjNum 33 snap 0 seq_num 1817658064) dirty dne 2024-09-15T03:57:31.065 INFO:tasks.rados.rados.0.smithi122.stdout:2558: finishing write tid 3 to smithi12223906-34 2024-09-15T03:57:31.065 INFO:tasks.rados.rados.0.smithi122.stdout:2558: finishing write tid 4 to smithi12223906-34 2024-09-15T03:57:31.065 INFO:tasks.rados.rados.0.smithi122.stdout:2568: seq_num 923 ranges {712861=733540,2161821=613653,3247187=265674} 2024-09-15T03:57:31.072 INFO:tasks.rados.rados.0.smithi122.stdout:2568: writing smithi12223906-46 from 712861 to 1446401 tid 1 2024-09-15T03:57:31.080 INFO:tasks.rados.rados.0.smithi122.stdout:2568: writing smithi12223906-46 from 2161821 to 2775474 tid 2 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2568: writing smithi12223906-46 from 3247187 to 3512861 tid 3 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2562: done (9 left) 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2569: snap_create 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2558: finishing write tid 5 to smithi12223906-34 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 767 (ObjNum 920 snap 289 seq_num 920) dirty exists 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2558: left oid 34 (ObjNum 920 snap 289 seq_num 920) 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2559: finishing rollback tid 0 to smithi12223906-17 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2559: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2559: finishing rollback tid 2 to smithi12223906-17 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1047 (ObjNum 758 snap 234 seq_num 758) dirty exists 2024-09-15T03:57:31.085 INFO:tasks.rados.rados.0.smithi122.stdout:2560: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:57:31.086 INFO:tasks.rados.rados.0.smithi122.stdout:2561: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:57:31.086 INFO:tasks.rados.rados.0.smithi122.stdout:2560: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:57:31.086 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1049 (ObjNum 820 snap 267 seq_num 820) dirty exists 2024-09-15T03:57:31.086 INFO:tasks.rados.rados.0.smithi122.stdout:2561: finishing copy_from to smithi12223906-44 2024-09-15T03:57:31.086 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 824 (ObjNum 844 snap 273 seq_num 844) dirty exists 2024-09-15T03:57:31.086 INFO:tasks.rados.rados.0.smithi122.stdout:2565: expect (ObjNum 779 snap 253 seq_num 779) 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2567: finishing write tid 1 to smithi12223906-35 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2564: finishing write tid 1 to smithi12223906-32 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2564: finishing write tid 2 to smithi12223906-32 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2564: finishing write tid 3 to smithi12223906-32 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2564: finishing write tid 4 to smithi12223906-32 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2564: finishing write tid 5 to smithi12223906-32 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 1843 (ObjNum 921 snap 289 seq_num 921) dirty exists 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2564: left oid 32 (ObjNum 921 snap 289 seq_num 921) 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2567: finishing write tid 2 to smithi12223906-35 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2567: finishing write tid 3 to smithi12223906-35 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:2567: finishing write tid 4 to smithi12223906-35 2024-09-15T03:57:31.386 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1834 (ObjNum 922 snap 289 seq_num 922) dirty exists 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2567: left oid 35 (ObjNum 922 snap 289 seq_num 922) 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2568: finishing write tid 1 to smithi12223906-46 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2568: finishing write tid 2 to smithi12223906-46 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2568: finishing write tid 3 to smithi12223906-46 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2568: finishing write tid 4 to smithi12223906-46 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2568: finishing write tid 5 to smithi12223906-46 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 1570 (ObjNum 923 snap 289 seq_num 923) dirty exists 2024-09-15T03:57:31.387 INFO:tasks.rados.rados.0.smithi122.stdout:2568: left oid 46 (ObjNum 923 snap 289 seq_num 923) 2024-09-15T03:57:32.099 INFO:tasks.rados.rados.0.smithi122.stdout:2558: done (9 left) 2024-09-15T03:57:32.099 INFO:tasks.rados.rados.0.smithi122.stdout:2559: done (8 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2560: done (7 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2561: done (6 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2564: done (5 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2565: done (4 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2566: done (3 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2567: done (2 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2568: done (1 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2569: done (0 left) 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2570: rollback oid 12 current snap is 290 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 272 2024-09-15T03:57:32.100 INFO:tasks.rados.rados.0.smithi122.stdout:2571: snap_create 2024-09-15T03:57:32.101 INFO:tasks.rados.rados.0.smithi122.stdout:2570: finishing rollback tid 1 to smithi12223906-12 2024-09-15T03:57:32.101 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 760 (ObjNum 49 snap 0 seq_num 1953066355) dirty dne 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2570: done (1 left) 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2571: done (0 left) 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2572: read oid 2 snap -1 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2572: expect deleted 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2573: read oid 16 snap -1 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2573: expect (ObjNum 857 snap 277 seq_num 857) 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2574: read oid 24 snap -1 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2574: expect (ObjNum 901 snap 286 seq_num 901) 2024-09-15T03:57:33.108 INFO:tasks.rados.rados.0.smithi122.stdout:2575: snap_remove snap 275 2024-09-15T03:57:34.108 INFO:tasks.rados.rados.0.smithi122.stdout:2575: done (3 left) 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:2576: rollback oid 27 current snap is 291 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 237 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:2577: copy_from oid 48 from oid 15 current snap is 291 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:2578: copy_from oid 22 from oid 17 current snap is 291 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:2579: rollback oid 49 current snap is 291 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 230 2024-09-15T03:57:34.109 INFO:tasks.rados.rados.0.smithi122.stdout:2580: snap_create 2024-09-15T03:57:34.110 INFO:tasks.rados.rados.0.smithi122.stdout:2573: expect (ObjNum 857 snap 277 seq_num 857) 2024-09-15T03:57:34.190 INFO:tasks.rados.rados.0.smithi122.stdout:2574: expect (ObjNum 901 snap 286 seq_num 901) 2024-09-15T03:57:34.339 INFO:tasks.rados.rados.0.smithi122.stdout:2576: finishing rollback tid 1 to smithi12223906-27 2024-09-15T03:57:34.339 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1713 (ObjNum 257 snap 0 seq_num 1842493936) dirty dne 2024-09-15T03:57:34.339 INFO:tasks.rados.rados.0.smithi122.stdout:2577: finishing copy_from to smithi12223906-48 2024-09-15T03:57:34.339 INFO:tasks.rados.rados.0.smithi122.stdout:2577: got expected ENOENT (src dne) 2024-09-15T03:57:34.339 INFO:tasks.rados.rados.0.smithi122.stdout:2577: finishing copy_from racing read to smithi12223906-48 2024-09-15T03:57:34.339 INFO:tasks.rados.rados.0.smithi122.stdout:2579: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:57:34.340 INFO:tasks.rados.rados.0.smithi122.stdout:2579: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:57:34.340 INFO:tasks.rados.rados.0.smithi122.stdout:2579: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:57:34.340 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1052 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:57:34.340 INFO:tasks.rados.rados.0.smithi122.stdout:2578: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:57:34.340 INFO:tasks.rados.rados.0.smithi122.stdout:2578: finishing copy_from to smithi12223906-22 2024-09-15T03:57:34.340 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1047 (ObjNum 758 snap 234 seq_num 758) dirty exists 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2572: done (7 left) 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2573: done (6 left) 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2574: done (5 left) 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2576: done (4 left) 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2577: done (3 left) 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2578: done (2 left) 2024-09-15T03:57:35.108 INFO:tasks.rados.rados.0.smithi122.stdout:2579: done (1 left) 2024-09-15T03:57:35.109 INFO:tasks.rados.rados.0.smithi122.stdout:2580: done (0 left) 2024-09-15T03:57:35.109 INFO:tasks.rados.rados.0.smithi122.stdout:2581: write (excl) oid 42 current snap is 292 2024-09-15T03:57:35.109 INFO:tasks.rados.rados.0.smithi122.stdout:2581: seq_num 924 ranges {729668=408789,1820303=757311,3274610=255058} 2024-09-15T03:57:35.116 INFO:tasks.rados.rados.0.smithi122.stdout:2581: writing smithi12223906-42 from 729668 to 1138457 tid 1 2024-09-15T03:57:35.124 INFO:tasks.rados.rados.0.smithi122.stdout:2581: writing smithi12223906-42 from 1820303 to 2577614 tid 2 2024-09-15T03:57:35.129 INFO:tasks.rados.rados.0.smithi122.stdout:2581: writing smithi12223906-42 from 3274610 to 3529668 tid 3 2024-09-15T03:57:35.129 INFO:tasks.rados.rados.0.smithi122.stdout:2582: snap_remove snap 280 2024-09-15T03:57:36.028 INFO:tasks.rados.rados.0.smithi122.stdout:2582: done (1 left) 2024-09-15T03:57:36.028 INFO:tasks.rados.rados.0.smithi122.stdout:2583: write oid 21 current snap is 292 2024-09-15T03:57:36.028 INFO:tasks.rados.rados.0.smithi122.stdout:2581: finishing write tid 1 to smithi12223906-42 2024-09-15T03:57:36.028 INFO:tasks.rados.rados.0.smithi122.stdout:2583: seq_num 925 ranges {746475=484038,1762432=617322,2785680=529906} 2024-09-15T03:57:36.036 INFO:tasks.rados.rados.0.smithi122.stdout:2583: writing smithi12223906-21 from 746475 to 1230513 tid 1 2024-09-15T03:57:36.043 INFO:tasks.rados.rados.0.smithi122.stdout:2583: writing smithi12223906-21 from 1762432 to 2379754 tid 2 2024-09-15T03:57:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2583: writing smithi12223906-21 from 2785680 to 3315586 tid 3 2024-09-15T03:57:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2581: finishing write tid 2 to smithi12223906-42 2024-09-15T03:57:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2584: read oid 3 snap 235 2024-09-15T03:57:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2584: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:57:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2585: write (excl) oid 22 current snap is 292 2024-09-15T03:57:36.050 INFO:tasks.rados.rados.0.smithi122.stdout:2585: seq_num 926 ranges {763282=559287,1820914=760980,3213103=350179} 2024-09-15T03:57:36.057 INFO:tasks.rados.rados.0.smithi122.stdout:2585: writing smithi12223906-22 from 763282 to 1322569 tid 1 2024-09-15T03:57:36.064 INFO:tasks.rados.rados.0.smithi122.stdout:2585: writing smithi12223906-22 from 1820914 to 2581894 tid 2 2024-09-15T03:57:36.070 INFO:tasks.rados.rados.0.smithi122.stdout:2585: writing smithi12223906-22 from 3213103 to 3563282 tid 3 2024-09-15T03:57:36.070 INFO:tasks.rados.rados.0.smithi122.stdout:2586: snap_create 2024-09-15T03:57:36.070 INFO:tasks.rados.rados.0.smithi122.stdout:2581: finishing write tid 3 to smithi12223906-42 2024-09-15T03:57:36.070 INFO:tasks.rados.rados.0.smithi122.stdout:2581: finishing write tid 4 to smithi12223906-42 2024-09-15T03:57:36.070 INFO:tasks.rados.rados.0.smithi122.stdout:2581: finishing write tid 5 to smithi12223906-42 2024-09-15T03:57:36.070 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 1914 (ObjNum 924 snap 292 seq_num 924) dirty exists 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2581: left oid 42 (ObjNum 924 snap 292 seq_num 924) 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2583: finishing write tid 1 to smithi12223906-21 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2583: finishing write tid 2 to smithi12223906-21 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2583: finishing write tid 3 to smithi12223906-21 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2583: finishing write tid 4 to smithi12223906-21 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2583: finishing write tid 5 to smithi12223906-21 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1063 (ObjNum 925 snap 292 seq_num 925) dirty exists 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2583: left oid 21 (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:36.071 INFO:tasks.rados.rados.0.smithi122.stdout:2584: expect (ObjNum 659 snap 201 seq_num 659) 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:2585: finishing write tid 1 to smithi12223906-22 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:2585: finishing write tid 2 to smithi12223906-22 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:2585: finishing write tid 3 to smithi12223906-22 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:2585: finishing write tid 4 to smithi12223906-22 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:2585: finishing write tid 5 to smithi12223906-22 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1875 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:57:36.306 INFO:tasks.rados.rados.0.smithi122.stdout:2585: left oid 22 (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:57:37.042 INFO:tasks.rados.rados.0.smithi122.stdout:2581: done (4 left) 2024-09-15T03:57:37.042 INFO:tasks.rados.rados.0.smithi122.stdout:2583: done (3 left) 2024-09-15T03:57:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:2584: done (2 left) 2024-09-15T03:57:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:2585: done (1 left) 2024-09-15T03:57:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:2586: done (0 left) 2024-09-15T03:57:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:2587: write oid 31 current snap is 293 2024-09-15T03:57:37.043 INFO:tasks.rados.rados.0.smithi122.stdout:2587: seq_num 927 ranges {780089=634536,1879396=620991,3240526=339563} 2024-09-15T03:57:37.054 INFO:tasks.rados.rados.0.smithi122.stdout:2587: writing smithi12223906-31 from 780089 to 1414625 tid 1 2024-09-15T03:57:37.060 INFO:tasks.rados.rados.0.smithi122.stdout:2587: writing smithi12223906-31 from 1879396 to 2500387 tid 2 2024-09-15T03:57:37.067 INFO:tasks.rados.rados.0.smithi122.stdout:2587: writing smithi12223906-31 from 3240526 to 3580089 tid 3 2024-09-15T03:57:37.067 INFO:tasks.rados.rados.0.smithi122.stdout:2588: snap_remove snap 277 2024-09-15T03:57:38.045 INFO:tasks.rados.rados.0.smithi122.stdout:2588: done (1 left) 2024-09-15T03:57:38.046 INFO:tasks.rados.rados.0.smithi122.stdout:2589: delete oid 40 current snap is 293 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:2587: finishing write tid 1 to smithi12223906-31 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:2587: finishing write tid 2 to smithi12223906-31 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:2587: finishing write tid 3 to smithi12223906-31 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:2587: finishing write tid 4 to smithi12223906-31 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:2587: finishing write tid 5 to smithi12223906-31 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 828 (ObjNum 927 snap 293 seq_num 927) dirty exists 2024-09-15T03:57:38.047 INFO:tasks.rados.rados.0.smithi122.stdout:2587: left oid 31 (ObjNum 927 snap 293 seq_num 927) 2024-09-15T03:57:38.056 INFO:tasks.rados.rados.0.smithi122.stdout:2587: done (1 left) 2024-09-15T03:57:38.056 INFO:tasks.rados.rados.0.smithi122.stdout:2589: done (0 left) 2024-09-15T03:57:38.056 INFO:tasks.rados.rados.0.smithi122.stdout:2590: write oid 30 current snap is 293 2024-09-15T03:57:38.056 INFO:tasks.rados.rados.0.smithi122.stdout:2590: seq_num 929 ranges {413703=501387,1596360=624660,2895372=691347} 2024-09-15T03:57:38.061 INFO:tasks.rados.rados.0.smithi122.stdout:2590: writing smithi12223906-30 from 413703 to 915090 tid 1 2024-09-15T03:57:38.069 INFO:tasks.rados.rados.0.smithi122.stdout:2590: writing smithi12223906-30 from 1596360 to 2221020 tid 2 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2590: writing smithi12223906-30 from 2895372 to 3586719 tid 3 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2591: read oid 5 snap 290 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2591: expect (ObjNum 917 snap 289 seq_num 917) 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2592: rollback oid 11 current snap is 293 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 11 to 281 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2590: finishing write tid 1 to smithi12223906-30 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2590: finishing write tid 2 to smithi12223906-30 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2593: write oid 18 current snap is 293 2024-09-15T03:57:38.078 INFO:tasks.rados.rados.0.smithi122.stdout:2593: seq_num 930 ranges {430510=576636,1654842=768318,2922795=502619} 2024-09-15T03:57:38.084 INFO:tasks.rados.rados.0.smithi122.stdout:2593: writing smithi12223906-18 from 430510 to 1007146 tid 1 2024-09-15T03:57:38.092 INFO:tasks.rados.rados.0.smithi122.stdout:2593: writing smithi12223906-18 from 1654842 to 2423160 tid 2 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2593: writing smithi12223906-18 from 2922795 to 3425414 tid 3 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2594: copy_from oid 23 from oid 37 current snap is 293 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2592: finishing rollback tid 1 to smithi12223906-11 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2595: rollback oid 49 current snap is 293 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 231 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2592: finishing rollback tid 2 to smithi12223906-11 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1318 (ObjNum 621 snap 185 seq_num 621) dirty exists 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2590: finishing write tid 3 to smithi12223906-30 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2590: finishing write tid 4 to smithi12223906-30 2024-09-15T03:57:38.098 INFO:tasks.rados.rados.0.smithi122.stdout:2590: finishing write tid 5 to smithi12223906-30 2024-09-15T03:57:38.099 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 835 (ObjNum 929 snap 293 seq_num 929) dirty exists 2024-09-15T03:57:38.099 INFO:tasks.rados.rados.0.smithi122.stdout:2590: left oid 30 (ObjNum 929 snap 293 seq_num 929) 2024-09-15T03:57:38.099 INFO:tasks.rados.rados.0.smithi122.stdout:2591: expect (ObjNum 917 snap 289 seq_num 917) 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2593: finishing write tid 1 to smithi12223906-18 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2593: finishing write tid 2 to smithi12223906-18 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2595: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2593: finishing write tid 3 to smithi12223906-18 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2593: finishing write tid 4 to smithi12223906-18 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2593: finishing write tid 5 to smithi12223906-18 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 1888 (ObjNum 930 snap 293 seq_num 930) dirty exists 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2593: left oid 18 (ObjNum 930 snap 293 seq_num 930) 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2594: finishing copy_from to smithi12223906-23 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2594: got expected ENOENT (src dne) 2024-09-15T03:57:38.301 INFO:tasks.rados.rados.0.smithi122.stdout:2595: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2594: finishing copy_from racing read to smithi12223906-23 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2595: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1055 (ObjNum 744 snap 226 seq_num 744) dirty exists 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2590: done (5 left) 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2591: done (4 left) 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2592: done (3 left) 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2593: done (2 left) 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2594: done (1 left) 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2595: done (0 left) 2024-09-15T03:57:38.302 INFO:tasks.rados.rados.0.smithi122.stdout:2596: snap_remove snap 233 2024-09-15T03:57:39.070 INFO:tasks.rados.rados.0.smithi122.stdout:2596: done (0 left) 2024-09-15T03:57:39.070 INFO:tasks.rados.rados.0.smithi122.stdout:2597: delete oid 28 current snap is 293 2024-09-15T03:57:39.079 INFO:tasks.rados.rados.0.smithi122.stdout:2597: done (0 left) 2024-09-15T03:57:39.080 INFO:tasks.rados.rados.0.smithi122.stdout:2598: delete oid 23 current snap is 293 2024-09-15T03:57:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:2598: done (0 left) 2024-09-15T03:57:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:2599: write oid 47 current snap is 293 2024-09-15T03:57:39.087 INFO:tasks.rados.rados.0.smithi122.stdout:2599: seq_num 933 ranges {480931=402383,1313935=515645,2488711=452788,3443230=237701} 2024-09-15T03:57:39.093 INFO:tasks.rados.rados.0.smithi122.stdout:2599: writing smithi12223906-47 from 480931 to 883314 tid 1 2024-09-15T03:57:39.099 INFO:tasks.rados.rados.0.smithi122.stdout:2599: writing smithi12223906-47 from 1313935 to 1829580 tid 2 2024-09-15T03:57:39.106 INFO:tasks.rados.rados.0.smithi122.stdout:2599: writing smithi12223906-47 from 2488711 to 2941499 tid 3 2024-09-15T03:57:39.111 INFO:tasks.rados.rados.0.smithi122.stdout:2599: writing smithi12223906-47 from 3443230 to 3680931 tid 4 2024-09-15T03:57:39.111 INFO:tasks.rados.rados.0.smithi122.stdout:2600: snap_create 2024-09-15T03:57:39.111 INFO:tasks.rados.rados.0.smithi122.stdout:2599: finishing write tid 1 to smithi12223906-47 2024-09-15T03:57:39.111 INFO:tasks.rados.rados.0.smithi122.stdout:2599: finishing write tid 2 to smithi12223906-47 2024-09-15T03:57:39.111 INFO:tasks.rados.rados.0.smithi122.stdout:2599: finishing write tid 3 to smithi12223906-47 2024-09-15T03:57:39.115 INFO:tasks.rados.rados.0.smithi122.stdout:2599: finishing write tid 4 to smithi12223906-47 2024-09-15T03:57:39.115 INFO:tasks.rados.rados.0.smithi122.stdout:2599: finishing write tid 5 to smithi12223906-47 2024-09-15T03:57:39.116 INFO:tasks.rados.rados.0.smithi122.stdout:2599: finishing write tid 6 to smithi12223906-47 2024-09-15T03:57:39.116 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 887 (ObjNum 933 snap 293 seq_num 933) dirty exists 2024-09-15T03:57:39.116 INFO:tasks.rados.rados.0.smithi122.stdout:2599: left oid 47 (ObjNum 933 snap 293 seq_num 933) 2024-09-15T03:57:40.075 INFO:tasks.rados.rados.0.smithi122.stdout:2599: done (1 left) 2024-09-15T03:57:40.075 INFO:tasks.rados.rados.0.smithi122.stdout:2600: done (0 left) 2024-09-15T03:57:40.075 INFO:tasks.rados.rados.0.smithi122.stdout:2601: copy_from oid 27 from oid 21 current snap is 294 2024-09-15T03:57:40.076 INFO:tasks.rados.rados.0.smithi122.stdout:2602: copy_from oid 6 from oid 1 current snap is 294 2024-09-15T03:57:40.076 INFO:tasks.rados.rados.0.smithi122.stdout:2603: read oid 9 snap -1 2024-09-15T03:57:40.076 INFO:tasks.rados.rados.0.smithi122.stdout:2603: expect deleted 2024-09-15T03:57:40.076 INFO:tasks.rados.rados.0.smithi122.stdout:2604: read oid 17 snap -1 2024-09-15T03:57:40.076 INFO:tasks.rados.rados.0.smithi122.stdout:2604: expect (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:57:40.076 INFO:tasks.rados.rados.0.smithi122.stdout:2605: read oid 3 snap -1 2024-09-15T03:57:40.079 INFO:tasks.rados.rados.0.smithi122.stdout:2605: expect (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:57:40.079 INFO:tasks.rados.rados.0.smithi122.stdout:2606: delete oid 40 current snap is 294 2024-09-15T03:57:40.079 INFO:tasks.rados.rados.0.smithi122.stdout:2602: finishing copy_from to smithi12223906-6 2024-09-15T03:57:40.079 INFO:tasks.rados.rados.0.smithi122.stdout:2602: got expected ENOENT (src dne) 2024-09-15T03:57:40.079 INFO:tasks.rados.rados.0.smithi122.stdout:2602: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:57:40.081 INFO:tasks.rados.rados.0.smithi122.stdout:2604: expect (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:57:40.274 INFO:tasks.rados.rados.0.smithi122.stdout:2605: expect (ObjNum 830 snap 271 seq_num 830) 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:2601: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:2601: finishing copy_from to smithi12223906-27 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 1063 (ObjNum 925 snap 292 seq_num 925) dirty exists 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:2601: done (5 left) 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:2602: done (4 left) 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:2603: done (3 left) 2024-09-15T03:57:40.565 INFO:tasks.rados.rados.0.smithi122.stdout:2604: done (2 left) 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2605: done (1 left) 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2606: done (0 left) 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2607: read oid 27 snap -1 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2607: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2608: read oid 38 snap -1 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2608: expect deleted 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2609: read oid 30 snap -1 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2609: expect (ObjNum 929 snap 293 seq_num 929) 2024-09-15T03:57:40.566 INFO:tasks.rados.rados.0.smithi122.stdout:2610: delete oid 17 current snap is 294 2024-09-15T03:57:40.570 INFO:tasks.rados.rados.0.smithi122.stdout:2608: done (3 left) 2024-09-15T03:57:40.570 INFO:tasks.rados.rados.0.smithi122.stdout:2610: done (2 left) 2024-09-15T03:57:40.570 INFO:tasks.rados.rados.0.smithi122.stdout:2611: snap_remove snap 292 2024-09-15T03:57:41.028 INFO:tasks.rados.rados.0.smithi122.stdout:2611: done (2 left) 2024-09-15T03:57:41.028 INFO:tasks.rados.rados.0.smithi122.stdout:2612: snap_create 2024-09-15T03:57:41.028 INFO:tasks.rados.rados.0.smithi122.stdout:2609: expect (ObjNum 929 snap 293 seq_num 929) 2024-09-15T03:57:41.238 INFO:tasks.rados.rados.0.smithi122.stdout:2607: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:42.030 INFO:tasks.rados.rados.0.smithi122.stdout:2607: done (2 left) 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2609: done (1 left) 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2612: done (0 left) 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2613: copy_from oid 4 from oid 19 current snap is 295 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2614: copy_from oid 34 from oid 41 current snap is 295 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2615: rollback oid 13 current snap is 295 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 287 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2616: write (excl) oid 7 current snap is 295 2024-09-15T03:57:42.031 INFO:tasks.rados.rados.0.smithi122.stdout:2616: seq_num 936 ranges {531352=744483,2005734=779325,3487333=244019} 2024-09-15T03:57:42.037 INFO:tasks.rados.rados.0.smithi122.stdout:2616: writing smithi12223906-7 from 531352 to 1275835 tid 1 2024-09-15T03:57:42.047 INFO:tasks.rados.rados.0.smithi122.stdout:2616: writing smithi12223906-7 from 2005734 to 2785059 tid 2 2024-09-15T03:57:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:2616: writing smithi12223906-7 from 3487333 to 3731352 tid 3 2024-09-15T03:57:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:2617: snap_remove snap 294 2024-09-15T03:57:43.039 INFO:tasks.rados.rados.0.smithi122.stdout:2615: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:57:43.039 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 876 (ObjNum 305 snap 0 seq_num 1) dirty dne 2024-09-15T03:57:43.039 INFO:tasks.rados.rados.0.smithi122.stdout:2613: finishing copy_from racing read to smithi12223906-4 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2614: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2614: finishing copy_from to smithi12223906-34 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 827 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2616: finishing write tid 1 to smithi12223906-7 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2613: finishing copy_from to smithi12223906-4 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1566 (ObjNum 919 snap 289 seq_num 919) dirty exists 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2616: finishing write tid 2 to smithi12223906-7 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2616: finishing write tid 3 to smithi12223906-7 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2616: finishing write tid 4 to smithi12223906-7 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2616: finishing write tid 5 to smithi12223906-7 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1582 (ObjNum 936 snap 295 seq_num 936) dirty exists 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2616: left oid 7 (ObjNum 936 snap 295 seq_num 936) 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2613: done (4 left) 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2614: done (3 left) 2024-09-15T03:57:43.040 INFO:tasks.rados.rados.0.smithi122.stdout:2615: done (2 left) 2024-09-15T03:57:43.041 INFO:tasks.rados.rados.0.smithi122.stdout:2616: done (1 left) 2024-09-15T03:57:43.041 INFO:tasks.rados.rados.0.smithi122.stdout:2617: done (0 left) 2024-09-15T03:57:43.041 INFO:tasks.rados.rados.0.smithi122.stdout:2618: write oid 10 current snap is 295 2024-09-15T03:57:43.041 INFO:tasks.rados.rados.0.smithi122.stdout:2618: seq_num 937 ranges {548159=419732,1664216=522983,2714756=614229} 2024-09-15T03:57:43.044 INFO:tasks.rados.rados.0.smithi122.stdout:2618: writing smithi12223906-10 from 548159 to 967891 tid 1 2024-09-15T03:57:43.051 INFO:tasks.rados.rados.0.smithi122.stdout:2618: writing smithi12223906-10 from 1664216 to 2187199 tid 2 2024-09-15T03:57:43.058 INFO:tasks.rados.rados.0.smithi122.stdout:2618: writing smithi12223906-10 from 2714756 to 3328985 tid 3 2024-09-15T03:57:43.058 INFO:tasks.rados.rados.0.smithi122.stdout:2619: read oid 21 snap -1 2024-09-15T03:57:43.058 INFO:tasks.rados.rados.0.smithi122.stdout:2619: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:43.058 INFO:tasks.rados.rados.0.smithi122.stdout:2620: delete oid 44 current snap is 295 2024-09-15T03:57:43.058 INFO:tasks.rados.rados.0.smithi122.stdout:2618: finishing write tid 1 to smithi12223906-10 2024-09-15T03:57:43.058 INFO:tasks.rados.rados.0.smithi122.stdout:2618: finishing write tid 2 to smithi12223906-10 2024-09-15T03:57:43.064 INFO:tasks.rados.rados.0.smithi122.stdout:2620: done (2 left) 2024-09-15T03:57:43.064 INFO:tasks.rados.rados.0.smithi122.stdout:2621: rollback oid 48 current snap is 295 2024-09-15T03:57:43.064 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 48 to 281 2024-09-15T03:57:43.064 INFO:tasks.rados.rados.0.smithi122.stdout:2622: rollback oid 14 current snap is 295 2024-09-15T03:57:43.064 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 267 2024-09-15T03:57:43.064 INFO:tasks.rados.rados.0.smithi122.stdout:2623: snap_remove snap 230 2024-09-15T03:57:44.052 INFO:tasks.rados.rados.0.smithi122.stdout:2619: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2618: finishing write tid 3 to smithi12223906-10 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2618: finishing write tid 4 to smithi12223906-10 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2618: finishing write tid 5 to smithi12223906-10 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1067 (ObjNum 937 snap 295 seq_num 937) dirty exists 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2618: left oid 10 (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2622: finishing rollback tid 0 to smithi12223906-14 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2622: finishing rollback tid 1 to smithi12223906-14 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2622: finishing rollback tid 2 to smithi12223906-14 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1954 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:57:44.171 INFO:tasks.rados.rados.0.smithi122.stdout:2621: finishing rollback tid 0 to smithi12223906-48 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2621: finishing rollback tid 1 to smithi12223906-48 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1069 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2618: done (4 left) 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2619: done (3 left) 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2621: done (2 left) 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2622: done (1 left) 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2623: done (0 left) 2024-09-15T03:57:44.172 INFO:tasks.rados.rados.0.smithi122.stdout:2624: snap_remove snap 291 2024-09-15T03:57:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2624: done (0 left) 2024-09-15T03:57:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2625: read oid 17 snap -1 2024-09-15T03:57:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2625: expect deleted 2024-09-15T03:57:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2626: write (excl) oid 50 current snap is 295 2024-09-15T03:57:45.062 INFO:tasks.rados.rados.0.smithi122.stdout:2626: seq_num 939 ranges {581773=570230,1664827=526652,2653249=636773} 2024-09-15T03:57:45.069 INFO:tasks.rados.rados.0.smithi122.stdout:2626: writing smithi12223906-50 from 581773 to 1152003 tid 1 2024-09-15T03:57:45.076 INFO:tasks.rados.rados.0.smithi122.stdout:2626: writing smithi12223906-50 from 1664827 to 2191479 tid 2 2024-09-15T03:57:45.083 INFO:tasks.rados.rados.0.smithi122.stdout:2626: writing smithi12223906-50 from 2653249 to 3290022 tid 3 2024-09-15T03:57:45.083 INFO:tasks.rados.rados.0.smithi122.stdout:2627: write oid 33 current snap is 295 2024-09-15T03:57:45.083 INFO:tasks.rados.rados.0.smithi122.stdout:2627: seq_num 940 ranges {598580=645479,1723309=786663,3080672=448045} 2024-09-15T03:57:45.092 INFO:tasks.rados.rados.0.smithi122.stdout:2627: writing smithi12223906-33 from 598580 to 1244059 tid 1 2024-09-15T03:57:45.097 INFO:tasks.rados.rados.0.smithi122.stdout:2627: writing smithi12223906-33 from 1723309 to 2509972 tid 2 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2627: writing smithi12223906-33 from 3080672 to 3528717 tid 3 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2628: copy_from oid 26 from oid 6 current snap is 295 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2626: finishing write tid 1 to smithi12223906-50 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2626: finishing write tid 2 to smithi12223906-50 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2626: finishing write tid 3 to smithi12223906-50 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2626: finishing write tid 4 to smithi12223906-50 2024-09-15T03:57:45.103 INFO:tasks.rados.rados.0.smithi122.stdout:2626: finishing write tid 5 to smithi12223906-50 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1934 (ObjNum 939 snap 295 seq_num 939) dirty exists 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2626: left oid 50 (ObjNum 939 snap 295 seq_num 939) 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2627: finishing write tid 1 to smithi12223906-33 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2625: done (3 left) 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2626: done (2 left) 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2629: read oid 5 snap -1 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2629: expect (ObjNum 917 snap 289 seq_num 917) 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2630: read oid 11 snap -1 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2630: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:57:45.104 INFO:tasks.rados.rados.0.smithi122.stdout:2631: snap_remove snap 257 2024-09-15T03:57:46.033 INFO:tasks.rados.rados.0.smithi122.stdout:2627: finishing write tid 2 to smithi12223906-33 2024-09-15T03:57:46.033 INFO:tasks.rados.rados.0.smithi122.stdout:2630: expect (ObjNum 621 snap 185 seq_num 621) 2024-09-15T03:57:46.172 INFO:tasks.rados.rados.0.smithi122.stdout:2627: finishing write tid 3 to smithi12223906-33 2024-09-15T03:57:46.172 INFO:tasks.rados.rados.0.smithi122.stdout:2627: finishing write tid 4 to smithi12223906-33 2024-09-15T03:57:46.172 INFO:tasks.rados.rados.0.smithi122.stdout:2627: finishing write tid 5 to smithi12223906-33 2024-09-15T03:57:46.172 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 1938 (ObjNum 940 snap 295 seq_num 940) dirty exists 2024-09-15T03:57:46.172 INFO:tasks.rados.rados.0.smithi122.stdout:2627: left oid 33 (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:57:46.172 INFO:tasks.rados.rados.0.smithi122.stdout:2629: expect (ObjNum 917 snap 289 seq_num 917) 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:2628: finishing copy_from racing read to smithi12223906-26 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:2628: finishing copy_from to smithi12223906-26 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 824 (ObjNum 844 snap 273 seq_num 844) dirty exists 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:2627: done (4 left) 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:2628: done (3 left) 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:2629: done (2 left) 2024-09-15T03:57:46.362 INFO:tasks.rados.rados.0.smithi122.stdout:2630: done (1 left) 2024-09-15T03:57:46.363 INFO:tasks.rados.rados.0.smithi122.stdout:2631: done (0 left) 2024-09-15T03:57:46.363 INFO:tasks.rados.rados.0.smithi122.stdout:2632: copy_from oid 35 from oid 41 current snap is 295 2024-09-15T03:57:46.363 INFO:tasks.rados.rados.0.smithi122.stdout:2633: rollback oid 6 current snap is 295 2024-09-15T03:57:46.363 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 6 to 283 2024-09-15T03:57:46.363 INFO:tasks.rados.rados.0.smithi122.stdout:2634: delete oid 23 current snap is 295 2024-09-15T03:57:46.368 INFO:tasks.rados.rados.0.smithi122.stdout:2634: done (2 left) 2024-09-15T03:57:46.368 INFO:tasks.rados.rados.0.smithi122.stdout:2635: write (excl) oid 48 current snap is 295 2024-09-15T03:57:46.368 INFO:tasks.rados.rados.0.smithi122.stdout:2635: seq_num 942 ranges {632194=795977,2123920=790332,3419165=413029} 2024-09-15T03:57:46.383 INFO:tasks.rados.rados.0.smithi122.stdout:2635: writing smithi12223906-48 from 632194 to 1428171 tid 1 2024-09-15T03:57:46.392 INFO:tasks.rados.rados.0.smithi122.stdout:2635: writing smithi12223906-48 from 2123920 to 2914252 tid 2 2024-09-15T03:57:46.397 INFO:tasks.rados.rados.0.smithi122.stdout:2635: writing smithi12223906-48 from 3419165 to 3832194 tid 3 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2636: delete oid 38 current snap is 295 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2633: finishing rollback tid 0 to smithi12223906-6 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2633: finishing rollback tid 1 to smithi12223906-6 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2633: finishing rollback tid 2 to smithi12223906-6 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 846 (ObjNum 844 snap 273 seq_num 844) dirty exists 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2632: finishing copy_from racing read to smithi12223906-35 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2632: finishing copy_from to smithi12223906-35 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 827 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2635: finishing write tid 1 to smithi12223906-48 2024-09-15T03:57:46.398 INFO:tasks.rados.rados.0.smithi122.stdout:2635: finishing write tid 2 to smithi12223906-48 2024-09-15T03:57:46.400 INFO:tasks.rados.rados.0.smithi122.stdout:2632: done (3 left) 2024-09-15T03:57:46.401 INFO:tasks.rados.rados.0.smithi122.stdout:2633: done (2 left) 2024-09-15T03:57:46.401 INFO:tasks.rados.rados.0.smithi122.stdout:2636: done (1 left) 2024-09-15T03:57:46.401 INFO:tasks.rados.rados.0.smithi122.stdout:2637: write oid 11 current snap is 295 2024-09-15T03:57:46.401 INFO:tasks.rados.rados.0.smithi122.stdout:2637: seq_num 944 ranges {665808=662828,1957237=677648,3074011=609486} 2024-09-15T03:57:46.408 INFO:tasks.rados.rados.0.smithi122.stdout:2637: writing smithi12223906-11 from 665808 to 1328636 tid 1 2024-09-15T03:57:46.415 INFO:tasks.rados.rados.0.smithi122.stdout:2637: writing smithi12223906-11 from 1957237 to 2634885 tid 2 2024-09-15T03:57:46.422 INFO:tasks.rados.rados.0.smithi122.stdout:2637: writing smithi12223906-11 from 3074011 to 3683497 tid 3 2024-09-15T03:57:46.422 INFO:tasks.rados.rados.0.smithi122.stdout:2638: read oid 21 snap -1 2024-09-15T03:57:46.422 INFO:tasks.rados.rados.0.smithi122.stdout:2638: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:46.422 INFO:tasks.rados.rados.0.smithi122.stdout:2639: copy_from oid 23 from oid 25 current snap is 295 2024-09-15T03:57:46.422 INFO:tasks.rados.rados.0.smithi122.stdout:2635: finishing write tid 3 to smithi12223906-48 2024-09-15T03:57:46.422 INFO:tasks.rados.rados.0.smithi122.stdout:2635: finishing write tid 4 to smithi12223906-48 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2635: finishing write tid 5 to smithi12223906-48 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1073 (ObjNum 942 snap 295 seq_num 942) dirty exists 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2635: left oid 48 (ObjNum 942 snap 295 seq_num 942) 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2637: finishing write tid 1 to smithi12223906-11 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2637: finishing write tid 2 to smithi12223906-11 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2635: done (3 left) 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2640: read oid 41 snap -1 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2640: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:57:46.423 INFO:tasks.rados.rados.0.smithi122.stdout:2641: delete oid 5 current snap is 295 2024-09-15T03:57:46.424 INFO:tasks.rados.rados.0.smithi122.stdout:2639: finishing copy_from to smithi12223906-23 2024-09-15T03:57:46.424 INFO:tasks.rados.rados.0.smithi122.stdout:2639: got expected ENOENT (src dne) 2024-09-15T03:57:46.424 INFO:tasks.rados.rados.0.smithi122.stdout:2639: finishing copy_from racing read to smithi12223906-23 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2637: finishing write tid 3 to smithi12223906-11 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2639: done (4 left) 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2641: done (3 left) 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2642: rollback oid 15 current snap is 295 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 272 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2637: finishing write tid 4 to smithi12223906-11 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2643: rollback oid 3 current snap is 295 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 283 2024-09-15T03:57:46.429 INFO:tasks.rados.rados.0.smithi122.stdout:2644: snap_create 2024-09-15T03:57:46.430 INFO:tasks.rados.rados.0.smithi122.stdout:2638: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:57:46.545 INFO:tasks.rados.rados.0.smithi122.stdout:2637: finishing write tid 5 to smithi12223906-11 2024-09-15T03:57:46.545 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1322 (ObjNum 944 snap 295 seq_num 944) dirty exists 2024-09-15T03:57:46.545 INFO:tasks.rados.rados.0.smithi122.stdout:2637: left oid 11 (ObjNum 944 snap 295 seq_num 944) 2024-09-15T03:57:46.545 INFO:tasks.rados.rados.0.smithi122.stdout:2640: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:57:46.629 INFO:tasks.rados.rados.0.smithi122.stdout:2642: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:57:46.629 INFO:tasks.rados.rados.0.smithi122.stdout:2642: finishing rollback tid 2 to smithi12223906-15 2024-09-15T03:57:46.629 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1324 (ObjNum 823 snap 269 seq_num 823) dirty exists 2024-09-15T03:57:46.629 INFO:tasks.rados.rados.0.smithi122.stdout:2643: finishing rollback tid 0 to smithi12223906-3 2024-09-15T03:57:46.630 INFO:tasks.rados.rados.0.smithi122.stdout:2643: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:57:46.630 INFO:tasks.rados.rados.0.smithi122.stdout:2643: finishing rollback tid 2 to smithi12223906-3 2024-09-15T03:57:46.630 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 1990 (ObjNum 830 snap 271 seq_num 830) dirty exists 2024-09-15T03:57:47.086 INFO:tasks.rados.rados.0.smithi122.stdout:2637: done (5 left) 2024-09-15T03:57:47.086 INFO:tasks.rados.rados.0.smithi122.stdout:2638: done (4 left) 2024-09-15T03:57:47.086 INFO:tasks.rados.rados.0.smithi122.stdout:2640: done (3 left) 2024-09-15T03:57:47.086 INFO:tasks.rados.rados.0.smithi122.stdout:2642: done (2 left) 2024-09-15T03:57:47.087 INFO:tasks.rados.rados.0.smithi122.stdout:2643: done (1 left) 2024-09-15T03:57:47.087 INFO:tasks.rados.rados.0.smithi122.stdout:2644: done (0 left) 2024-09-15T03:57:47.087 INFO:tasks.rados.rados.0.smithi122.stdout:2645: write oid 50 current snap is 296 2024-09-15T03:57:47.087 INFO:tasks.rados.rados.0.smithi122.stdout:2645: seq_num 946 ranges {699422=413326,1557848=681317,3012504=748383} 2024-09-15T03:57:47.093 INFO:tasks.rados.rados.0.smithi122.stdout:2645: writing smithi12223906-50 from 699422 to 1112748 tid 1 2024-09-15T03:57:47.099 INFO:tasks.rados.rados.0.smithi122.stdout:2645: writing smithi12223906-50 from 1557848 to 2239165 tid 2 2024-09-15T03:57:47.108 INFO:tasks.rados.rados.0.smithi122.stdout:2645: writing smithi12223906-50 from 3012504 to 3760887 tid 3 2024-09-15T03:57:47.108 INFO:tasks.rados.rados.0.smithi122.stdout:2646: delete oid 1 current snap is 296 2024-09-15T03:57:47.108 INFO:tasks.rados.rados.0.smithi122.stdout:2645: finishing write tid 1 to smithi12223906-50 2024-09-15T03:57:47.108 INFO:tasks.rados.rados.0.smithi122.stdout:2645: finishing write tid 2 to smithi12223906-50 2024-09-15T03:57:47.111 INFO:tasks.rados.rados.0.smithi122.stdout:2646: done (1 left) 2024-09-15T03:57:47.111 INFO:tasks.rados.rados.0.smithi122.stdout:2647: write (excl) oid 6 current snap is 296 2024-09-15T03:57:47.111 INFO:tasks.rados.rados.0.smithi122.stdout:2647: seq_num 948 ranges {733036=563824,2074812=684986,3183703=749333} 2024-09-15T03:57:47.118 INFO:tasks.rados.rados.0.smithi122.stdout:2647: writing smithi12223906-6 from 733036 to 1296860 tid 1 2024-09-15T03:57:47.127 INFO:tasks.rados.rados.0.smithi122.stdout:2647: writing smithi12223906-6 from 2074812 to 2759798 tid 2 2024-09-15T03:57:47.135 INFO:tasks.rados.rados.0.smithi122.stdout:2647: writing smithi12223906-6 from 3183703 to 3933036 tid 3 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2648: snap_create 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2645: finishing write tid 3 to smithi12223906-50 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2645: finishing write tid 4 to smithi12223906-50 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2645: finishing write tid 5 to smithi12223906-50 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1958 (ObjNum 946 snap 296 seq_num 946) dirty exists 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2645: left oid 50 (ObjNum 946 snap 296 seq_num 946) 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2647: finishing write tid 1 to smithi12223906-6 2024-09-15T03:57:47.136 INFO:tasks.rados.rados.0.smithi122.stdout:2647: finishing write tid 2 to smithi12223906-6 2024-09-15T03:57:47.140 INFO:tasks.rados.rados.0.smithi122.stdout:2647: finishing write tid 3 to smithi12223906-6 2024-09-15T03:57:47.141 INFO:tasks.rados.rados.0.smithi122.stdout:2647: finishing write tid 4 to smithi12223906-6 2024-09-15T03:57:47.141 INFO:tasks.rados.rados.0.smithi122.stdout:2647: finishing write tid 5 to smithi12223906-6 2024-09-15T03:57:47.141 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 850 (ObjNum 948 snap 296 seq_num 948) dirty exists 2024-09-15T03:57:47.141 INFO:tasks.rados.rados.0.smithi122.stdout:2647: left oid 6 (ObjNum 948 snap 296 seq_num 948) 2024-09-15T03:57:48.094 INFO:tasks.rados.rados.0.smithi122.stdout:2645: done (2 left) 2024-09-15T03:57:48.094 INFO:tasks.rados.rados.0.smithi122.stdout:2647: done (1 left) 2024-09-15T03:57:48.094 INFO:tasks.rados.rados.0.smithi122.stdout:2648: done (0 left) 2024-09-15T03:57:48.094 INFO:tasks.rados.rados.0.smithi122.stdout:2649: read oid 17 snap 287 2024-09-15T03:57:48.094 INFO:tasks.rados.rados.0.smithi122.stdout:2649: expect (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:57:48.094 INFO:tasks.rados.rados.0.smithi122.stdout:2650: snap_remove snap 283 2024-09-15T03:57:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:2650: done (1 left) 2024-09-15T03:57:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:2651: snap_create 2024-09-15T03:57:49.098 INFO:tasks.rados.rados.0.smithi122.stdout:2649: expect (ObjNum 758 snap 234 seq_num 758) 2024-09-15T03:57:50.097 INFO:tasks.rados.rados.0.smithi122.stdout:2649: done (1 left) 2024-09-15T03:57:50.097 INFO:tasks.rados.rados.0.smithi122.stdout:2651: done (0 left) 2024-09-15T03:57:50.097 INFO:tasks.rados.rados.0.smithi122.stdout:2652: rollback oid 25 current snap is 298 2024-09-15T03:57:50.097 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 295 2024-09-15T03:57:50.097 INFO:tasks.rados.rados.0.smithi122.stdout:2653: write (excl) oid 16 current snap is 298 2024-09-15T03:57:50.097 INFO:tasks.rados.rados.0.smithi122.stdout:2653: seq_num 949 ranges {749843=639073,2133294=544997,3211126=582199} 2024-09-15T03:57:50.110 INFO:tasks.rados.rados.0.smithi122.stdout:2653: writing smithi12223906-16 from 749843 to 1388916 tid 1 2024-09-15T03:57:50.118 INFO:tasks.rados.rados.0.smithi122.stdout:2653: writing smithi12223906-16 from 2133294 to 2678291 tid 2 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2653: writing smithi12223906-16 from 3211126 to 3793325 tid 3 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2654: copy_from oid 13 from oid 31 current snap is 298 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2652: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 1867 (ObjNum 129 snap 0 seq_num 1842552712) dirty dne 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2653: finishing write tid 1 to smithi12223906-16 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2652: done (2 left) 2024-09-15T03:57:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2655: snap_remove snap 290 2024-09-15T03:57:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:2655: done (2 left) 2024-09-15T03:57:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:2656: snap_create 2024-09-15T03:57:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:2653: finishing write tid 2 to smithi12223906-16 2024-09-15T03:57:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:2653: finishing write tid 3 to smithi12223906-16 2024-09-15T03:57:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:2653: finishing write tid 4 to smithi12223906-16 2024-09-15T03:57:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:2653: finishing write tid 5 to smithi12223906-16 2024-09-15T03:57:51.035 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1328 (ObjNum 949 snap 298 seq_num 949) dirty exists 2024-09-15T03:57:51.035 INFO:tasks.rados.rados.0.smithi122.stdout:2653: left oid 16 (ObjNum 949 snap 298 seq_num 949) 2024-09-15T03:57:51.035 INFO:tasks.rados.rados.0.smithi122.stdout:2654: finishing copy_from racing read to smithi12223906-13 2024-09-15T03:57:51.035 INFO:tasks.rados.rados.0.smithi122.stdout:2654: finishing copy_from to smithi12223906-13 2024-09-15T03:57:51.035 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 828 (ObjNum 927 snap 293 seq_num 927) dirty exists 2024-09-15T03:57:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2653: done (2 left) 2024-09-15T03:57:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2654: done (1 left) 2024-09-15T03:57:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2656: done (0 left) 2024-09-15T03:57:52.039 INFO:tasks.rados.rados.0.smithi122.stdout:2657: snap_create 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2657: done (0 left) 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2658: read oid 5 snap -1 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2658: expect deleted 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2659: read oid 24 snap 251 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2659: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2660: write (excl) oid 50 current snap is 300 2024-09-15T03:57:53.059 INFO:tasks.rados.rados.0.smithi122.stdout:2660: seq_num 950 ranges {766650=714322,2075423=688655,3522196=444454} 2024-09-15T03:57:53.071 INFO:tasks.rados.rados.0.smithi122.stdout:2660: writing smithi12223906-50 from 766650 to 1480972 tid 1 2024-09-15T03:57:53.079 INFO:tasks.rados.rados.0.smithi122.stdout:2660: writing smithi12223906-50 from 2075423 to 2764078 tid 2 2024-09-15T03:57:53.086 INFO:tasks.rados.rados.0.smithi122.stdout:2660: writing smithi12223906-50 from 3522196 to 3966650 tid 3 2024-09-15T03:57:53.086 INFO:tasks.rados.rados.0.smithi122.stdout:2661: snap_create 2024-09-15T03:57:53.087 INFO:tasks.rados.rados.0.smithi122.stdout:2659: expect (ObjNum 551 snap 163 seq_num 551) 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:2660: finishing write tid 1 to smithi12223906-50 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:2660: finishing write tid 2 to smithi12223906-50 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:2660: finishing write tid 3 to smithi12223906-50 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:2660: finishing write tid 4 to smithi12223906-50 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:2660: finishing write tid 5 to smithi12223906-50 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1985 (ObjNum 950 snap 300 seq_num 950) dirty exists 2024-09-15T03:57:53.332 INFO:tasks.rados.rados.0.smithi122.stdout:2660: left oid 50 (ObjNum 950 snap 300 seq_num 950) 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2658: done (3 left) 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2659: done (2 left) 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2660: done (1 left) 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2661: done (0 left) 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2662: read oid 33 snap -1 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2662: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:57:54.067 INFO:tasks.rados.rados.0.smithi122.stdout:2663: delete oid 23 current snap is 301 2024-09-15T03:57:54.085 INFO:tasks.rados.rados.0.smithi122.stdout:2662: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:57:54.206 INFO:tasks.rados.rados.0.smithi122.stdout:2662: done (1 left) 2024-09-15T03:57:54.206 INFO:tasks.rados.rados.0.smithi122.stdout:2663: done (0 left) 2024-09-15T03:57:54.206 INFO:tasks.rados.rados.0.smithi122.stdout:2664: delete oid 5 current snap is 301 2024-09-15T03:57:54.210 INFO:tasks.rados.rados.0.smithi122.stdout:2664: done (0 left) 2024-09-15T03:57:54.210 INFO:tasks.rados.rados.0.smithi122.stdout:2665: snap_create 2024-09-15T03:57:55.077 INFO:tasks.rados.rados.0.smithi122.stdout:2665: done (0 left) 2024-09-15T03:57:55.078 INFO:tasks.rados.rados.0.smithi122.stdout:2666: write oid 1 current snap is 302 2024-09-15T03:57:55.078 INFO:tasks.rados.rados.0.smithi122.stdout:2666: seq_num 953 ranges {417071=656422,1567222=449849} 2024-09-15T03:57:55.085 INFO:tasks.rados.rados.0.smithi122.stdout:2666: writing smithi12223906-1 from 417071 to 1073493 tid 1 2024-09-15T03:57:55.093 INFO:tasks.rados.rados.0.smithi122.stdout:2666: writing smithi12223906-1 from 1567222 to 2017071 tid 2 2024-09-15T03:57:55.093 INFO:tasks.rados.rados.0.smithi122.stdout:2667: write oid 9 current snap is 302 2024-09-15T03:57:55.093 INFO:tasks.rados.rados.0.smithi122.stdout:2667: seq_num 954 ranges {433878=731671,1909351=124527} 2024-09-15T03:57:55.100 INFO:tasks.rados.rados.0.smithi122.stdout:2667: writing smithi12223906-9 from 433878 to 1165549 tid 1 2024-09-15T03:57:55.105 INFO:tasks.rados.rados.0.smithi122.stdout:2667: writing smithi12223906-9 from 1909351 to 2033878 tid 2 2024-09-15T03:57:55.105 INFO:tasks.rados.rados.0.smithi122.stdout:2668: snap_create 2024-09-15T03:57:55.105 INFO:tasks.rados.rados.0.smithi122.stdout:2666: finishing write tid 1 to smithi12223906-1 2024-09-15T03:57:55.105 INFO:tasks.rados.rados.0.smithi122.stdout:2666: finishing write tid 2 to smithi12223906-1 2024-09-15T03:57:55.105 INFO:tasks.rados.rados.0.smithi122.stdout:2666: finishing write tid 3 to smithi12223906-1 2024-09-15T03:57:55.106 INFO:tasks.rados.rados.0.smithi122.stdout:2666: finishing write tid 4 to smithi12223906-1 2024-09-15T03:57:55.106 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1331 (ObjNum 953 snap 302 seq_num 953) dirty exists 2024-09-15T03:57:55.106 INFO:tasks.rados.rados.0.smithi122.stdout:2666: left oid 1 (ObjNum 953 snap 302 seq_num 953) 2024-09-15T03:57:55.106 INFO:tasks.rados.rados.0.smithi122.stdout:2667: finishing write tid 1 to smithi12223906-9 2024-09-15T03:57:55.107 INFO:tasks.rados.rados.0.smithi122.stdout:2667: finishing write tid 2 to smithi12223906-9 2024-09-15T03:57:55.109 INFO:tasks.rados.rados.0.smithi122.stdout:2667: finishing write tid 3 to smithi12223906-9 2024-09-15T03:57:55.109 INFO:tasks.rados.rados.0.smithi122.stdout:2667: finishing write tid 4 to smithi12223906-9 2024-09-15T03:57:55.109 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 853 (ObjNum 954 snap 302 seq_num 954) dirty exists 2024-09-15T03:57:55.109 INFO:tasks.rados.rados.0.smithi122.stdout:2667: left oid 9 (ObjNum 954 snap 302 seq_num 954) 2024-09-15T03:57:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2666: done (2 left) 2024-09-15T03:57:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2667: done (1 left) 2024-09-15T03:57:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2668: done (0 left) 2024-09-15T03:57:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2669: write oid 3 current snap is 303 2024-09-15T03:57:56.038 INFO:tasks.rados.rados.0.smithi122.stdout:2669: seq_num 955 ranges {450685=406920,1567833=482852} 2024-09-15T03:57:56.043 INFO:tasks.rados.rados.0.smithi122.stdout:2669: writing smithi12223906-3 from 450685 to 857605 tid 1 2024-09-15T03:57:56.051 INFO:tasks.rados.rados.0.smithi122.stdout:2669: writing smithi12223906-3 from 1567833 to 2050685 tid 2 2024-09-15T03:57:56.051 INFO:tasks.rados.rados.0.smithi122.stdout:2670: read oid 31 snap -1 2024-09-15T03:57:56.051 INFO:tasks.rados.rados.0.smithi122.stdout:2670: expect (ObjNum 927 snap 293 seq_num 927) 2024-09-15T03:57:56.051 INFO:tasks.rados.rados.0.smithi122.stdout:2671: delete oid 24 current snap is 303 2024-09-15T03:57:56.062 INFO:tasks.rados.rados.0.smithi122.stdout:2669: finishing write tid 1 to smithi12223906-3 2024-09-15T03:57:56.063 INFO:tasks.rados.rados.0.smithi122.stdout:2670: expect (ObjNum 927 snap 293 seq_num 927) 2024-09-15T03:57:56.186 INFO:tasks.rados.rados.0.smithi122.stdout:2669: finishing write tid 2 to smithi12223906-3 2024-09-15T03:57:56.186 INFO:tasks.rados.rados.0.smithi122.stdout:2669: finishing write tid 3 to smithi12223906-3 2024-09-15T03:57:56.186 INFO:tasks.rados.rados.0.smithi122.stdout:2669: finishing write tid 4 to smithi12223906-3 2024-09-15T03:57:56.186 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2011 (ObjNum 955 snap 303 seq_num 955) dirty exists 2024-09-15T03:57:56.186 INFO:tasks.rados.rados.0.smithi122.stdout:2669: left oid 3 (ObjNum 955 snap 303 seq_num 955) 2024-09-15T03:57:56.186 INFO:tasks.rados.rados.0.smithi122.stdout:2669: done (2 left) 2024-09-15T03:57:56.187 INFO:tasks.rados.rados.0.smithi122.stdout:2670: done (1 left) 2024-09-15T03:57:56.187 INFO:tasks.rados.rados.0.smithi122.stdout:2671: done (0 left) 2024-09-15T03:57:56.187 INFO:tasks.rados.rados.0.smithi122.stdout:2672: snap_create 2024-09-15T03:57:57.100 INFO:tasks.rados.rados.0.smithi122.stdout:2672: done (0 left) 2024-09-15T03:57:57.100 INFO:tasks.rados.rados.0.smithi122.stdout:2673: copy_from oid 9 from oid 2 current snap is 304 2024-09-15T03:57:57.100 INFO:tasks.rados.rados.0.smithi122.stdout:2674: snap_remove snap 289 2024-09-15T03:57:58.115 INFO:tasks.rados.rados.0.smithi122.stdout:2674: done (1 left) 2024-09-15T03:57:58.115 INFO:tasks.rados.rados.0.smithi122.stdout:2675: write (excl) oid 31 current snap is 304 2024-09-15T03:57:58.115 INFO:tasks.rados.rados.0.smithi122.stdout:2673: finishing copy_from to smithi12223906-9 2024-09-15T03:57:58.115 INFO:tasks.rados.rados.0.smithi122.stdout:2673: got expected ENOENT (src dne) 2024-09-15T03:57:58.115 INFO:tasks.rados.rados.0.smithi122.stdout:2675: seq_num 957 ranges {484299=557418,1684797=399502} 2024-09-15T03:57:58.122 INFO:tasks.rados.rados.0.smithi122.stdout:2675: writing smithi12223906-31 from 484299 to 1041717 tid 1 2024-09-15T03:57:58.128 INFO:tasks.rados.rados.0.smithi122.stdout:2675: writing smithi12223906-31 from 1684797 to 2084299 tid 2 2024-09-15T03:57:58.128 INFO:tasks.rados.rados.0.smithi122.stdout:2676: write oid 8 current snap is 304 2024-09-15T03:57:58.128 INFO:tasks.rados.rados.0.smithi122.stdout:2673: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:57:58.128 INFO:tasks.rados.rados.0.smithi122.stdout:2676: seq_num 958 ranges {501106=749020,1743279=357827} 2024-09-15T03:57:58.135 INFO:tasks.rados.rados.0.smithi122.stdout:2676: writing smithi12223906-8 from 501106 to 1250126 tid 1 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2676: writing smithi12223906-8 from 1743279 to 2101106 tid 2 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2673: done (2 left) 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2677: read oid 13 snap -1 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2677: expect (ObjNum 927 snap 293 seq_num 927) 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2678: rollback oid 26 current snap is 304 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2675: finishing write tid 1 to smithi12223906-31 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2675: finishing write tid 2 to smithi12223906-31 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2675: finishing write tid 3 to smithi12223906-31 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2675: finishing write tid 4 to smithi12223906-31 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1569 (ObjNum 957 snap 304 seq_num 957) dirty exists 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2675: left oid 31 (ObjNum 957 snap 304 seq_num 957) 2024-09-15T03:57:58.142 INFO:tasks.rados.rados.0.smithi122.stdout:2676: finishing write tid 1 to smithi12223906-8 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 302 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2675: done (3 left) 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2679: read oid 50 snap -1 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2679: expect (ObjNum 950 snap 300 seq_num 950) 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2680: read oid 10 snap -1 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2680: expect (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2681: copy_from oid 38 from oid 21 current snap is 304 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2682: read oid 48 snap -1 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2682: expect (ObjNum 942 snap 295 seq_num 942) 2024-09-15T03:57:58.143 INFO:tasks.rados.rados.0.smithi122.stdout:2683: snap_create 2024-09-15T03:57:58.148 INFO:tasks.rados.rados.0.smithi122.stdout:2676: finishing write tid 2 to smithi12223906-8 2024-09-15T03:57:58.148 INFO:tasks.rados.rados.0.smithi122.stdout:2676: finishing write tid 3 to smithi12223906-8 2024-09-15T03:57:58.149 INFO:tasks.rados.rados.0.smithi122.stdout:2676: finishing write tid 4 to smithi12223906-8 2024-09-15T03:57:58.149 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1290 (ObjNum 958 snap 304 seq_num 958) dirty exists 2024-09-15T03:57:58.149 INFO:tasks.rados.rados.0.smithi122.stdout:2676: left oid 8 (ObjNum 958 snap 304 seq_num 958) 2024-09-15T03:57:58.149 INFO:tasks.rados.rados.0.smithi122.stdout:2678: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:57:58.153 INFO:tasks.rados.rados.0.smithi122.stdout:2678: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:57:58.153 INFO:tasks.rados.rados.0.smithi122.stdout:2678: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:57:58.153 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1293 (ObjNum 844 snap 273 seq_num 844) dirty exists 2024-09-15T03:57:58.153 INFO:tasks.rados.rados.0.smithi122.stdout:2680: expect (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:57:58.458 INFO:tasks.rados.rados.0.smithi122.stdout:2677: expect (ObjNum 927 snap 293 seq_num 927) 2024-09-15T03:57:58.566 INFO:tasks.rados.rados.0.smithi122.stdout:2682: expect (ObjNum 942 snap 295 seq_num 942) 2024-09-15T03:57:58.677 INFO:tasks.rados.rados.0.smithi122.stdout:2679: expect (ObjNum 950 snap 300 seq_num 950) 2024-09-15T03:57:58.879 INFO:tasks.rados.rados.0.smithi122.stdout:2681: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:57:58.879 INFO:tasks.rados.rados.0.smithi122.stdout:2681: finishing copy_from to smithi12223906-38 2024-09-15T03:57:58.879 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1063 (ObjNum 925 snap 292 seq_num 925) dirty exists 2024-09-15T03:57:59.116 INFO:tasks.rados.rados.0.smithi122.stdout:2676: done (7 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2677: done (6 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2678: done (5 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2679: done (4 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2680: done (3 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2681: done (2 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2682: done (1 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2683: done (0 left) 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2684: copy_from oid 20 from oid 47 current snap is 305 2024-09-15T03:57:59.117 INFO:tasks.rados.rados.0.smithi122.stdout:2685: snap_remove snap 271 2024-09-15T03:58:00.119 INFO:tasks.rados.rados.0.smithi122.stdout:2685: done (1 left) 2024-09-15T03:58:00.119 INFO:tasks.rados.rados.0.smithi122.stdout:2686: write oid 41 current snap is 305 2024-09-15T03:58:00.119 INFO:tasks.rados.rados.0.smithi122.stdout:2686: seq_num 959 ranges {517913=424269,1401761=446989} 2024-09-15T03:58:00.127 INFO:tasks.rados.rados.0.smithi122.stdout:2686: writing smithi12223906-41 from 517913 to 942182 tid 1 2024-09-15T03:58:00.133 INFO:tasks.rados.rados.0.smithi122.stdout:2686: writing smithi12223906-41 from 1401761 to 1848750 tid 2 2024-09-15T03:58:00.133 INFO:tasks.rados.rados.0.smithi122.stdout:2684: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:2684: finishing copy_from to smithi12223906-20 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 887 (ObjNum 933 snap 293 seq_num 933) dirty exists 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:2687: read oid 22 snap -1 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:2687: expect (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:2684: done (2 left) 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:2688: write (excl) oid 18 current snap is 305 2024-09-15T03:58:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:2688: seq_num 960 ranges {534720=499518,1460243=674477} 2024-09-15T03:58:00.140 INFO:tasks.rados.rados.0.smithi122.stdout:2688: writing smithi12223906-18 from 534720 to 1034238 tid 1 2024-09-15T03:58:00.147 INFO:tasks.rados.rados.0.smithi122.stdout:2688: writing smithi12223906-18 from 1460243 to 2134720 tid 2 2024-09-15T03:58:00.147 INFO:tasks.rados.rados.0.smithi122.stdout:2689: write (excl) oid 26 current snap is 305 2024-09-15T03:58:00.148 INFO:tasks.rados.rados.0.smithi122.stdout:2686: finishing write tid 1 to smithi12223906-41 2024-09-15T03:58:00.148 INFO:tasks.rados.rados.0.smithi122.stdout:2689: seq_num 961 ranges {551527=574767,1918725=232802} 2024-09-15T03:58:00.153 INFO:tasks.rados.rados.0.smithi122.stdout:2689: writing smithi12223906-26 from 551527 to 1126294 tid 1 2024-09-15T03:58:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:2689: writing smithi12223906-26 from 1918725 to 2151527 tid 2 2024-09-15T03:58:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:2690: read oid 32 snap -1 2024-09-15T03:58:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:2690: expect (ObjNum 921 snap 289 seq_num 921) 2024-09-15T03:58:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:2691: rollback oid 13 current snap is 305 2024-09-15T03:58:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 13 to 287 2024-09-15T03:58:00.161 INFO:tasks.rados.rados.0.smithi122.stdout:2686: finishing write tid 2 to smithi12223906-41 2024-09-15T03:58:00.161 INFO:tasks.rados.rados.0.smithi122.stdout:2686: finishing write tid 3 to smithi12223906-41 2024-09-15T03:58:00.161 INFO:tasks.rados.rados.0.smithi122.stdout:2686: finishing write tid 4 to smithi12223906-41 2024-09-15T03:58:00.161 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 838 (ObjNum 959 snap 305 seq_num 959) dirty exists 2024-09-15T03:58:00.161 INFO:tasks.rados.rados.0.smithi122.stdout:2686: left oid 41 (ObjNum 959 snap 305 seq_num 959) 2024-09-15T03:58:00.161 INFO:tasks.rados.rados.0.smithi122.stdout:2687: expect (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:58:00.429 INFO:tasks.rados.rados.0.smithi122.stdout:2688: finishing write tid 1 to smithi12223906-18 2024-09-15T03:58:00.429 INFO:tasks.rados.rados.0.smithi122.stdout:2688: finishing write tid 2 to smithi12223906-18 2024-09-15T03:58:00.429 INFO:tasks.rados.rados.0.smithi122.stdout:2688: finishing write tid 3 to smithi12223906-18 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2688: finishing write tid 4 to smithi12223906-18 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2008 (ObjNum 960 snap 305 seq_num 960) dirty exists 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2688: left oid 18 (ObjNum 960 snap 305 seq_num 960) 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2689: finishing write tid 1 to smithi12223906-26 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2689: finishing write tid 2 to smithi12223906-26 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2689: finishing write tid 3 to smithi12223906-26 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2689: finishing write tid 4 to smithi12223906-26 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1296 (ObjNum 961 snap 305 seq_num 961) dirty exists 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2689: left oid 26 (ObjNum 961 snap 305 seq_num 961) 2024-09-15T03:58:00.430 INFO:tasks.rados.rados.0.smithi122.stdout:2686: done (5 left) 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:2687: done (4 left) 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:2688: done (3 left) 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:2689: done (2 left) 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:2692: rollback oid 38 current snap is 305 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 267 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:2691: finishing rollback tid 0 to smithi12223906-13 2024-09-15T03:58:00.431 INFO:tasks.rados.rados.0.smithi122.stdout:2690: expect (ObjNum 921 snap 289 seq_num 921) 2024-09-15T03:58:00.641 INFO:tasks.rados.rados.0.smithi122.stdout:2691: finishing rollback tid 1 to smithi12223906-13 2024-09-15T03:58:00.641 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 1572 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2692: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2692: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2692: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1299 (ObjNum 726 snap 221 seq_num 726) dirty exists 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2690: done (2 left) 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2691: done (1 left) 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2692: done (0 left) 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2693: read oid 39 snap -1 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2693: expect (ObjNum 918 snap 289 seq_num 918) 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2694: write oid 45 current snap is 305 2024-09-15T03:58:00.642 INFO:tasks.rados.rados.0.smithi122.stdout:2694: seq_num 962 ranges {568334=650016,1860854=307480} 2024-09-15T03:58:00.657 INFO:tasks.rados.rados.0.smithi122.stdout:2694: writing smithi12223906-45 from 568334 to 1218350 tid 1 2024-09-15T03:58:00.663 INFO:tasks.rados.rados.0.smithi122.stdout:2694: writing smithi12223906-45 from 1860854 to 2168334 tid 2 2024-09-15T03:58:00.663 INFO:tasks.rados.rados.0.smithi122.stdout:2695: write (excl) oid 26 current snap is 305 2024-09-15T03:58:00.663 INFO:tasks.rados.rados.0.smithi122.stdout:2695: seq_num 963 ranges {585141=725265,1919336=265805} 2024-09-15T03:58:00.670 INFO:tasks.rados.rados.0.smithi122.stdout:2695: writing smithi12223906-26 from 585141 to 1310406 tid 1 2024-09-15T03:58:00.675 INFO:tasks.rados.rados.0.smithi122.stdout:2695: writing smithi12223906-26 from 1919336 to 2185141 tid 2 2024-09-15T03:58:00.675 INFO:tasks.rados.rados.0.smithi122.stdout:2696: write (excl) oid 47 current snap is 305 2024-09-15T03:58:00.675 INFO:tasks.rados.rados.0.smithi122.stdout:2696: seq_num 964 ranges {601948=400514,1577818=624130} 2024-09-15T03:58:00.681 INFO:tasks.rados.rados.0.smithi122.stdout:2696: writing smithi12223906-47 from 601948 to 1002462 tid 1 2024-09-15T03:58:00.688 INFO:tasks.rados.rados.0.smithi122.stdout:2696: writing smithi12223906-47 from 1577818 to 2201948 tid 2 2024-09-15T03:58:00.688 INFO:tasks.rados.rados.0.smithi122.stdout:2697: read oid 10 snap -1 2024-09-15T03:58:00.688 INFO:tasks.rados.rados.0.smithi122.stdout:2697: expect (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:58:00.688 INFO:tasks.rados.rados.0.smithi122.stdout:2698: delete oid 23 current snap is 305 2024-09-15T03:58:00.688 INFO:tasks.rados.rados.0.smithi122.stdout:2693: expect (ObjNum 918 snap 289 seq_num 918) 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2694: finishing write tid 1 to smithi12223906-45 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2694: finishing write tid 2 to smithi12223906-45 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2694: finishing write tid 3 to smithi12223906-45 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2694: finishing write tid 4 to smithi12223906-45 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 1575 (ObjNum 962 snap 305 seq_num 962) dirty exists 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2694: left oid 45 (ObjNum 962 snap 305 seq_num 962) 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2695: finishing write tid 1 to smithi12223906-26 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2695: finishing write tid 2 to smithi12223906-26 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2693: done (5 left) 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2694: done (4 left) 2024-09-15T03:58:00.798 INFO:tasks.rados.rados.0.smithi122.stdout:2698: done (3 left) 2024-09-15T03:58:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:2699: write oid 9 current snap is 305 2024-09-15T03:58:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:2695: finishing write tid 3 to smithi12223906-26 2024-09-15T03:58:00.799 INFO:tasks.rados.rados.0.smithi122.stdout:2699: seq_num 966 ranges {635562=667365,2094782=140780} 2024-09-15T03:58:00.812 INFO:tasks.rados.rados.0.smithi122.stdout:2699: writing smithi12223906-9 from 635562 to 1302927 tid 1 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2699: writing smithi12223906-9 from 2094782 to 2235562 tid 2 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2700: rollback oid 2 current snap is 305 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 302 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2695: finishing write tid 4 to smithi12223906-26 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1302 (ObjNum 963 snap 305 seq_num 963) dirty exists 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2695: left oid 26 (ObjNum 963 snap 305 seq_num 963) 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2696: finishing write tid 1 to smithi12223906-47 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2696: finishing write tid 2 to smithi12223906-47 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2696: finishing write tid 3 to smithi12223906-47 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:2696: finishing write tid 4 to smithi12223906-47 2024-09-15T03:58:00.818 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 891 (ObjNum 964 snap 305 seq_num 964) dirty exists 2024-09-15T03:58:00.819 INFO:tasks.rados.rados.0.smithi122.stdout:2696: left oid 47 (ObjNum 964 snap 305 seq_num 964) 2024-09-15T03:58:00.819 INFO:tasks.rados.rados.0.smithi122.stdout:2697: expect (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:58:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:2700: finishing rollback tid 1 to smithi12223906-2 2024-09-15T03:58:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1039 (ObjNum 289 snap 0 seq_num 1) dirty dne 2024-09-15T03:58:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:2699: finishing write tid 1 to smithi12223906-9 2024-09-15T03:58:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:2699: finishing write tid 2 to smithi12223906-9 2024-09-15T03:58:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:2695: done (4 left) 2024-09-15T03:58:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:2696: done (3 left) 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2697: done (2 left) 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2700: done (1 left) 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2699: finishing write tid 3 to smithi12223906-9 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2699: finishing write tid 4 to smithi12223906-9 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 856 (ObjNum 966 snap 305 seq_num 966) dirty exists 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2699: left oid 9 (ObjNum 966 snap 305 seq_num 966) 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2701: read oid 44 snap 295 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2701: expect deleted 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2699: done (1 left) 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2702: rollback oid 14 current snap is 305 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 295 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2703: copy_from oid 34 from oid 12 current snap is 305 2024-09-15T03:58:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:2704: snap_create 2024-09-15T03:58:01.110 INFO:tasks.rados.rados.0.smithi122.stdout:2703: finishing copy_from to smithi12223906-34 2024-09-15T03:58:01.110 INFO:tasks.rados.rados.0.smithi122.stdout:2703: got expected ENOENT (src dne) 2024-09-15T03:58:01.110 INFO:tasks.rados.rados.0.smithi122.stdout:2703: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:58:01.116 INFO:tasks.rados.rados.0.smithi122.stdout:2702: finishing rollback tid 0 to smithi12223906-14 2024-09-15T03:58:01.116 INFO:tasks.rados.rados.0.smithi122.stdout:2702: finishing rollback tid 1 to smithi12223906-14 2024-09-15T03:58:01.116 INFO:tasks.rados.rados.0.smithi122.stdout:2702: finishing rollback tid 2 to smithi12223906-14 2024-09-15T03:58:01.116 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 2039 (ObjNum 760 snap 235 seq_num 760) dirty exists 2024-09-15T03:58:02.080 INFO:tasks.rados.rados.0.smithi122.stdout:2701: done (3 left) 2024-09-15T03:58:02.080 INFO:tasks.rados.rados.0.smithi122.stdout:2702: done (2 left) 2024-09-15T03:58:02.080 INFO:tasks.rados.rados.0.smithi122.stdout:2703: done (1 left) 2024-09-15T03:58:02.080 INFO:tasks.rados.rados.0.smithi122.stdout:2704: done (0 left) 2024-09-15T03:58:02.080 INFO:tasks.rados.rados.0.smithi122.stdout:2705: delete oid 9 current snap is 306 2024-09-15T03:58:02.089 INFO:tasks.rados.rados.0.smithi122.stdout:2705: done (0 left) 2024-09-15T03:58:02.089 INFO:tasks.rados.rados.0.smithi122.stdout:2706: read oid 36 snap -1 2024-09-15T03:58:02.089 INFO:tasks.rados.rados.0.smithi122.stdout:2706: expect (ObjNum 911 snap 288 seq_num 911) 2024-09-15T03:58:02.089 INFO:tasks.rados.rados.0.smithi122.stdout:2707: write (excl) oid 14 current snap is 306 2024-09-15T03:58:02.089 INFO:tasks.rados.rados.0.smithi122.stdout:2707: seq_num 968 ranges {669176=417863,1811746=457430} 2024-09-15T03:58:02.096 INFO:tasks.rados.rados.0.smithi122.stdout:2707: writing smithi12223906-14 from 669176 to 1087039 tid 1 2024-09-15T03:58:02.106 INFO:tasks.rados.rados.0.smithi122.stdout:2707: writing smithi12223906-14 from 1811746 to 2269176 tid 2 2024-09-15T03:58:02.106 INFO:tasks.rados.rados.0.smithi122.stdout:2708: write oid 10 current snap is 306 2024-09-15T03:58:02.106 INFO:tasks.rados.rados.0.smithi122.stdout:2708: seq_num 969 ranges {685983=493112,1870228=415755} 2024-09-15T03:58:02.113 INFO:tasks.rados.rados.0.smithi122.stdout:2708: writing smithi12223906-10 from 685983 to 1179095 tid 1 2024-09-15T03:58:02.119 INFO:tasks.rados.rados.0.smithi122.stdout:2708: writing smithi12223906-10 from 1870228 to 2285983 tid 2 2024-09-15T03:58:02.120 INFO:tasks.rados.rados.0.smithi122.stdout:2709: write (excl) oid 6 current snap is 306 2024-09-15T03:58:02.120 INFO:tasks.rados.rados.0.smithi122.stdout:2706: expect (ObjNum 911 snap 288 seq_num 911) 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:2707: finishing write tid 1 to smithi12223906-14 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:2707: finishing write tid 2 to smithi12223906-14 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:2707: finishing write tid 3 to smithi12223906-14 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:2707: finishing write tid 4 to smithi12223906-14 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 2043 (ObjNum 968 snap 306 seq_num 968) dirty exists 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:2707: left oid 14 (ObjNum 968 snap 306 seq_num 968) 2024-09-15T03:58:02.364 INFO:tasks.rados.rados.0.smithi122.stdout:2708: finishing write tid 1 to smithi12223906-10 2024-09-15T03:58:02.365 INFO:tasks.rados.rados.0.smithi122.stdout:2708: finishing write tid 2 to smithi12223906-10 2024-09-15T03:58:02.365 INFO:tasks.rados.rados.0.smithi122.stdout:2708: finishing write tid 3 to smithi12223906-10 2024-09-15T03:58:02.365 INFO:tasks.rados.rados.0.smithi122.stdout:2708: finishing write tid 4 to smithi12223906-10 2024-09-15T03:58:02.365 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1076 (ObjNum 969 snap 306 seq_num 969) dirty exists 2024-09-15T03:58:02.365 INFO:tasks.rados.rados.0.smithi122.stdout:2708: left oid 10 (ObjNum 969 snap 306 seq_num 969) 2024-09-15T03:58:02.365 INFO:tasks.rados.rados.0.smithi122.stdout:2709: seq_num 970 ranges {702790=568361,1812357=490433} 2024-09-15T03:58:02.380 INFO:tasks.rados.rados.0.smithi122.stdout:2709: writing smithi12223906-6 from 702790 to 1271151 tid 1 2024-09-15T03:58:02.386 INFO:tasks.rados.rados.0.smithi122.stdout:2709: writing smithi12223906-6 from 1812357 to 2302790 tid 2 2024-09-15T03:58:02.386 INFO:tasks.rados.rados.0.smithi122.stdout:2706: done (3 left) 2024-09-15T03:58:02.386 INFO:tasks.rados.rados.0.smithi122.stdout:2707: done (2 left) 2024-09-15T03:58:02.386 INFO:tasks.rados.rados.0.smithi122.stdout:2708: done (1 left) 2024-09-15T03:58:02.386 INFO:tasks.rados.rados.0.smithi122.stdout:2710: delete oid 41 current snap is 306 2024-09-15T03:58:02.394 INFO:tasks.rados.rados.0.smithi122.stdout:2709: finishing write tid 1 to smithi12223906-6 2024-09-15T03:58:02.394 INFO:tasks.rados.rados.0.smithi122.stdout:2709: finishing write tid 2 to smithi12223906-6 2024-09-15T03:58:02.394 INFO:tasks.rados.rados.0.smithi122.stdout:2710: done (1 left) 2024-09-15T03:58:02.394 INFO:tasks.rados.rados.0.smithi122.stdout:2711: write (excl) oid 21 current snap is 306 2024-09-15T03:58:02.394 INFO:tasks.rados.rados.0.smithi122.stdout:2711: seq_num 972 ranges {736404=718859,1929321=407083} 2024-09-15T03:58:02.401 INFO:tasks.rados.rados.0.smithi122.stdout:2711: writing smithi12223906-21 from 736404 to 1455263 tid 1 2024-09-15T03:58:02.407 INFO:tasks.rados.rados.0.smithi122.stdout:2711: writing smithi12223906-21 from 1929321 to 2336404 tid 2 2024-09-15T03:58:02.407 INFO:tasks.rados.rados.0.smithi122.stdout:2712: snap_remove snap 300 2024-09-15T03:58:03.092 INFO:tasks.rados.rados.0.smithi122.stdout:2712: done (2 left) 2024-09-15T03:58:03.092 INFO:tasks.rados.rados.0.smithi122.stdout:2713: snap_create 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2709: finishing write tid 3 to smithi12223906-6 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2709: finishing write tid 4 to smithi12223906-6 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 860 (ObjNum 970 snap 306 seq_num 970) dirty exists 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2709: left oid 6 (ObjNum 970 snap 306 seq_num 970) 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2711: finishing write tid 1 to smithi12223906-21 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2711: finishing write tid 2 to smithi12223906-21 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2711: finishing write tid 3 to smithi12223906-21 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2711: finishing write tid 4 to smithi12223906-21 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1079 (ObjNum 972 snap 306 seq_num 972) dirty exists 2024-09-15T03:58:03.093 INFO:tasks.rados.rados.0.smithi122.stdout:2711: left oid 21 (ObjNum 972 snap 306 seq_num 972) 2024-09-15T03:58:04.102 INFO:tasks.rados.rados.0.smithi122.stdout:2709: done (2 left) 2024-09-15T03:58:04.102 INFO:tasks.rados.rados.0.smithi122.stdout:2711: done (1 left) 2024-09-15T03:58:04.103 INFO:tasks.rados.rados.0.smithi122.stdout:2713: done (0 left) 2024-09-15T03:58:04.103 INFO:tasks.rados.rados.0.smithi122.stdout:2714: write (excl) oid 3 current snap is 307 2024-09-15T03:58:04.103 INFO:tasks.rados.rados.0.smithi122.stdout:2714: seq_num 973 ranges {753211=794108,1987803=365408} 2024-09-15T03:58:04.114 INFO:tasks.rados.rados.0.smithi122.stdout:2714: writing smithi12223906-3 from 753211 to 1547319 tid 1 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:2714: writing smithi12223906-3 from 1987803 to 2353211 tid 2 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:2715: copy_from oid 4 from oid 22 current snap is 307 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:2716: read oid 16 snap 235 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:2716: expect (ObjNum 625 snap 187 seq_num 625) 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:2717: rollback oid 39 current snap is 307 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 39 to 293 2024-09-15T03:58:04.119 INFO:tasks.rados.rados.0.smithi122.stdout:2718: write oid 46 current snap is 307 2024-09-15T03:58:04.120 INFO:tasks.rados.rados.0.smithi122.stdout:2718: seq_num 974 ranges {770018=585710,2046285=323733} 2024-09-15T03:58:04.126 INFO:tasks.rados.rados.0.smithi122.stdout:2718: writing smithi12223906-46 from 770018 to 1355728 tid 1 2024-09-15T03:58:04.133 INFO:tasks.rados.rados.0.smithi122.stdout:2718: writing smithi12223906-46 from 2046285 to 2370018 tid 2 2024-09-15T03:58:04.133 INFO:tasks.rados.rados.0.smithi122.stdout:2719: read oid 38 snap -1 2024-09-15T03:58:04.133 INFO:tasks.rados.rados.0.smithi122.stdout:2719: expect (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:58:04.133 INFO:tasks.rados.rados.0.smithi122.stdout:2720: rollback oid 6 current snap is 307 2024-09-15T03:58:04.133 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 6 to 297 2024-09-15T03:58:04.133 INFO:tasks.rados.rados.0.smithi122.stdout:2716: expect (ObjNum 625 snap 187 seq_num 625) 2024-09-15T03:58:04.301 INFO:tasks.rados.rados.0.smithi122.stdout:2714: finishing write tid 1 to smithi12223906-3 2024-09-15T03:58:04.301 INFO:tasks.rados.rados.0.smithi122.stdout:2714: finishing write tid 2 to smithi12223906-3 2024-09-15T03:58:04.301 INFO:tasks.rados.rados.0.smithi122.stdout:2714: finishing write tid 3 to smithi12223906-3 2024-09-15T03:58:04.301 INFO:tasks.rados.rados.0.smithi122.stdout:2717: finishing rollback tid 0 to smithi12223906-39 2024-09-15T03:58:04.301 INFO:tasks.rados.rados.0.smithi122.stdout:2714: finishing write tid 4 to smithi12223906-3 2024-09-15T03:58:04.301 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2053 (ObjNum 973 snap 307 seq_num 973) dirty exists 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2714: left oid 3 (ObjNum 973 snap 307 seq_num 973) 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2717: finishing rollback tid 1 to smithi12223906-39 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2717: finishing rollback tid 2 to smithi12223906-39 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2057 (ObjNum 918 snap 289 seq_num 918) dirty exists 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2714: done (6 left) 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2716: done (5 left) 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2717: done (4 left) 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2721: delete oid 34 current snap is 307 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2720: finishing rollback tid 0 to smithi12223906-6 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2720: finishing rollback tid 1 to smithi12223906-6 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2720: finishing rollback tid 2 to smithi12223906-6 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 863 (ObjNum 948 snap 296 seq_num 948) dirty exists 2024-09-15T03:58:04.302 INFO:tasks.rados.rados.0.smithi122.stdout:2718: finishing write tid 1 to smithi12223906-46 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:2718: finishing write tid 2 to smithi12223906-46 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:2718: finishing write tid 3 to smithi12223906-46 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:2718: finishing write tid 4 to smithi12223906-46 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 1578 (ObjNum 974 snap 307 seq_num 974) dirty exists 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:2718: left oid 46 (ObjNum 974 snap 307 seq_num 974) 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:2715: finishing copy_from racing read to smithi12223906-4 2024-09-15T03:58:04.303 INFO:tasks.rados.rados.0.smithi122.stdout:2719: expect (ObjNum 726 snap 221 seq_num 726) 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2715: finishing copy_from to smithi12223906-4 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1875 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2715: done (4 left) 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2718: done (3 left) 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2719: done (2 left) 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2720: done (1 left) 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2721: done (0 left) 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:2722: rollback oid 39 current snap is 307 2024-09-15T03:58:04.410 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 39 to 266 2024-09-15T03:58:04.411 INFO:tasks.rados.rados.0.smithi122.stdout:2723: delete oid 17 current snap is 307 2024-09-15T03:58:04.413 INFO:tasks.rados.rados.0.smithi122.stdout:2723: done (1 left) 2024-09-15T03:58:04.413 INFO:tasks.rados.rados.0.smithi122.stdout:2724: copy_from oid 19 from oid 16 current snap is 307 2024-09-15T03:58:04.413 INFO:tasks.rados.rados.0.smithi122.stdout:2725: write oid 14 current snap is 307 2024-09-15T03:58:04.413 INFO:tasks.rados.rados.0.smithi122.stdout:2725: seq_num 977 ranges {420439=411457,1421731=480010} 2024-09-15T03:58:04.423 INFO:tasks.rados.rados.0.smithi122.stdout:2725: writing smithi12223906-14 from 420439 to 831896 tid 1 2024-09-15T03:58:04.430 INFO:tasks.rados.rados.0.smithi122.stdout:2725: writing smithi12223906-14 from 1421731 to 1901741 tid 2 2024-09-15T03:58:04.430 INFO:tasks.rados.rados.0.smithi122.stdout:2726: copy_from oid 38 from oid 35 current snap is 307 2024-09-15T03:58:04.431 INFO:tasks.rados.rados.0.smithi122.stdout:2722: finishing rollback tid 0 to smithi12223906-39 2024-09-15T03:58:04.431 INFO:tasks.rados.rados.0.smithi122.stdout:2722: finishing rollback tid 1 to smithi12223906-39 2024-09-15T03:58:04.431 INFO:tasks.rados.rados.0.smithi122.stdout:2722: finishing rollback tid 2 to smithi12223906-39 2024-09-15T03:58:04.431 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2060 (ObjNum 596 snap 180 seq_num 596) dirty exists 2024-09-15T03:58:04.431 INFO:tasks.rados.rados.0.smithi122.stdout:2722: done (3 left) 2024-09-15T03:58:04.431 INFO:tasks.rados.rados.0.smithi122.stdout:2727: delete oid 37 current snap is 307 2024-09-15T03:58:04.436 INFO:tasks.rados.rados.0.smithi122.stdout:2727: done (3 left) 2024-09-15T03:58:04.436 INFO:tasks.rados.rados.0.smithi122.stdout:2728: copy_from oid 18 from oid 40 current snap is 307 2024-09-15T03:58:04.436 INFO:tasks.rados.rados.0.smithi122.stdout:2729: delete oid 12 current snap is 307 2024-09-15T03:58:04.437 INFO:tasks.rados.rados.0.smithi122.stdout:2724: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:58:04.437 INFO:tasks.rados.rados.0.smithi122.stdout:2728: finishing copy_from to smithi12223906-18 2024-09-15T03:58:04.437 INFO:tasks.rados.rados.0.smithi122.stdout:2728: got expected ENOENT (src dne) 2024-09-15T03:58:04.438 INFO:tasks.rados.rados.0.smithi122.stdout:2728: finishing copy_from racing read to smithi12223906-18 2024-09-15T03:58:04.439 INFO:tasks.rados.rados.0.smithi122.stdout:2728: done (4 left) 2024-09-15T03:58:04.439 INFO:tasks.rados.rados.0.smithi122.stdout:2729: done (3 left) 2024-09-15T03:58:04.439 INFO:tasks.rados.rados.0.smithi122.stdout:2730: snap_create 2024-09-15T03:58:04.440 INFO:tasks.rados.rados.0.smithi122.stdout:2725: finishing write tid 1 to smithi12223906-14 2024-09-15T03:58:04.440 INFO:tasks.rados.rados.0.smithi122.stdout:2724: finishing copy_from to smithi12223906-19 2024-09-15T03:58:04.440 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 1328 (ObjNum 949 snap 298 seq_num 949) dirty exists 2024-09-15T03:58:04.440 INFO:tasks.rados.rados.0.smithi122.stdout:2725: finishing write tid 2 to smithi12223906-14 2024-09-15T03:58:04.441 INFO:tasks.rados.rados.0.smithi122.stdout:2725: finishing write tid 3 to smithi12223906-14 2024-09-15T03:58:04.441 INFO:tasks.rados.rados.0.smithi122.stdout:2725: finishing write tid 4 to smithi12223906-14 2024-09-15T03:58:04.441 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 2064 (ObjNum 977 snap 307 seq_num 977) dirty exists 2024-09-15T03:58:04.441 INFO:tasks.rados.rados.0.smithi122.stdout:2725: left oid 14 (ObjNum 977 snap 307 seq_num 977) 2024-09-15T03:58:04.442 INFO:tasks.rados.rados.0.smithi122.stdout:2726: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:58:04.446 INFO:tasks.rados.rados.0.smithi122.stdout:2726: finishing copy_from to smithi12223906-38 2024-09-15T03:58:04.446 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 827 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T03:58:05.112 INFO:tasks.rados.rados.0.smithi122.stdout:2724: done (3 left) 2024-09-15T03:58:05.112 INFO:tasks.rados.rados.0.smithi122.stdout:2725: done (2 left) 2024-09-15T03:58:05.113 INFO:tasks.rados.rados.0.smithi122.stdout:2726: done (1 left) 2024-09-15T03:58:05.113 INFO:tasks.rados.rados.0.smithi122.stdout:2730: done (0 left) 2024-09-15T03:58:05.113 INFO:tasks.rados.rados.0.smithi122.stdout:2731: read oid 23 snap -1 2024-09-15T03:58:05.113 INFO:tasks.rados.rados.0.smithi122.stdout:2731: expect deleted 2024-09-15T03:58:05.113 INFO:tasks.rados.rados.0.smithi122.stdout:2732: snap_create 2024-09-15T03:58:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2731: done (1 left) 2024-09-15T03:58:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2732: done (0 left) 2024-09-15T03:58:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2733: copy_from oid 13 from oid 24 current snap is 309 2024-09-15T03:58:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2734: copy_from oid 44 from oid 5 current snap is 309 2024-09-15T03:58:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2735: read oid 31 snap -1 2024-09-15T03:58:06.050 INFO:tasks.rados.rados.0.smithi122.stdout:2735: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-15T03:58:06.051 INFO:tasks.rados.rados.0.smithi122.stdout:2736: delete oid 4 current snap is 309 2024-09-15T03:58:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:2734: finishing copy_from to smithi12223906-44 2024-09-15T03:58:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:2734: got expected ENOENT (src dne) 2024-09-15T03:58:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:2733: finishing copy_from to smithi12223906-13 2024-09-15T03:58:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:2733: got expected ENOENT (src dne) 2024-09-15T03:58:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:2734: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:58:06.054 INFO:tasks.rados.rados.0.smithi122.stdout:2733: finishing copy_from racing read to smithi12223906-13 2024-09-15T03:58:06.074 INFO:tasks.rados.rados.0.smithi122.stdout:2735: expect (ObjNum 957 snap 304 seq_num 957) 2024-09-15T03:58:06.168 INFO:tasks.rados.rados.0.smithi122.stdout:2733: done (3 left) 2024-09-15T03:58:06.168 INFO:tasks.rados.rados.0.smithi122.stdout:2734: done (2 left) 2024-09-15T03:58:06.168 INFO:tasks.rados.rados.0.smithi122.stdout:2735: done (1 left) 2024-09-15T03:58:06.168 INFO:tasks.rados.rados.0.smithi122.stdout:2736: done (0 left) 2024-09-15T03:58:06.169 INFO:tasks.rados.rados.0.smithi122.stdout:2737: snap_remove snap 299 2024-09-15T03:58:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:2737: done (0 left) 2024-09-15T03:58:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:2738: read oid 46 snap -1 2024-09-15T03:58:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:2738: expect (ObjNum 974 snap 307 seq_num 974) 2024-09-15T03:58:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:2739: copy_from oid 32 from oid 38 current snap is 309 2024-09-15T03:58:07.133 INFO:tasks.rados.rados.0.smithi122.stdout:2740: snap_remove snap 238 2024-09-15T03:58:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2740: done (2 left) 2024-09-15T03:58:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:2741: snap_remove snap 293 2024-09-15T03:58:09.157 INFO:tasks.rados.rados.0.smithi122.stdout:2741: done (2 left) 2024-09-15T03:58:09.158 INFO:tasks.rados.rados.0.smithi122.stdout:2742: read oid 15 snap -1 2024-09-15T03:58:09.158 INFO:tasks.rados.rados.0.smithi122.stdout:2742: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:58:09.158 INFO:tasks.rados.rados.0.smithi122.stdout:2743: snap_create 2024-09-15T03:58:09.158 INFO:tasks.rados.rados.0.smithi122.stdout:2738: expect (ObjNum 974 snap 307 seq_num 974) 2024-09-15T03:58:09.372 INFO:tasks.rados.rados.0.smithi122.stdout:2739: finishing copy_from racing read to smithi12223906-32 2024-09-15T03:58:09.372 INFO:tasks.rados.rados.0.smithi122.stdout:2739: finishing copy_from to smithi12223906-32 2024-09-15T03:58:09.372 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 827 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T03:58:09.372 INFO:tasks.rados.rados.0.smithi122.stdout:2742: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:58:10.160 INFO:tasks.rados.rados.0.smithi122.stdout:2738: done (3 left) 2024-09-15T03:58:10.160 INFO:tasks.rados.rados.0.smithi122.stdout:2739: done (2 left) 2024-09-15T03:58:10.161 INFO:tasks.rados.rados.0.smithi122.stdout:2742: done (1 left) 2024-09-15T03:58:10.161 INFO:tasks.rados.rados.0.smithi122.stdout:2743: done (0 left) 2024-09-15T03:58:10.161 INFO:tasks.rados.rados.0.smithi122.stdout:2744: read oid 3 snap -1 2024-09-15T03:58:10.161 INFO:tasks.rados.rados.0.smithi122.stdout:2744: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T03:58:10.161 INFO:tasks.rados.rados.0.smithi122.stdout:2745: write oid 30 current snap is 310 2024-09-15T03:58:10.161 INFO:tasks.rados.rados.0.smithi122.stdout:2745: seq_num 981 ranges {487667=428806,1655659=770995} 2024-09-15T03:58:10.166 INFO:tasks.rados.rados.0.smithi122.stdout:2745: writing smithi12223906-30 from 487667 to 916473 tid 1 2024-09-15T03:58:10.177 INFO:tasks.rados.rados.0.smithi122.stdout:2745: writing smithi12223906-30 from 1655659 to 2426654 tid 2 2024-09-15T03:58:10.178 INFO:tasks.rados.rados.0.smithi122.stdout:2746: snap_remove snap 266 2024-09-15T03:58:11.055 INFO:tasks.rados.rados.0.smithi122.stdout:2746: done (2 left) 2024-09-15T03:58:11.055 INFO:tasks.rados.rados.0.smithi122.stdout:2747: copy_from oid 34 from oid 29 current snap is 310 2024-09-15T03:58:11.055 INFO:tasks.rados.rados.0.smithi122.stdout:2744: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T03:58:11.189 INFO:tasks.rados.rados.0.smithi122.stdout:2745: finishing write tid 1 to smithi12223906-30 2024-09-15T03:58:11.189 INFO:tasks.rados.rados.0.smithi122.stdout:2745: finishing write tid 2 to smithi12223906-30 2024-09-15T03:58:11.189 INFO:tasks.rados.rados.0.smithi122.stdout:2745: finishing write tid 3 to smithi12223906-30 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2745: finishing write tid 4 to smithi12223906-30 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 842 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2745: left oid 30 (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2747: finishing copy_from racing read to smithi12223906-34 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2747: finishing copy_from to smithi12223906-34 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1573 (ObjNum 900 snap 286 seq_num 900) dirty exists 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2744: done (2 left) 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2745: done (1 left) 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2747: done (0 left) 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2748: read oid 5 snap -1 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2748: expect deleted 2024-09-15T03:58:11.190 INFO:tasks.rados.rados.0.smithi122.stdout:2749: snap_remove snap 239 2024-09-15T03:58:12.176 INFO:tasks.rados.rados.0.smithi122.stdout:2749: done (1 left) 2024-09-15T03:58:12.176 INFO:tasks.rados.rados.0.smithi122.stdout:2750: read oid 15 snap -1 2024-09-15T03:58:12.176 INFO:tasks.rados.rados.0.smithi122.stdout:2750: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:58:12.176 INFO:tasks.rados.rados.0.smithi122.stdout:2751: snap_create 2024-09-15T03:58:12.185 INFO:tasks.rados.rados.0.smithi122.stdout:2750: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:58:13.180 INFO:tasks.rados.rados.0.smithi122.stdout:2748: done (2 left) 2024-09-15T03:58:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:2750: done (1 left) 2024-09-15T03:58:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:2751: done (0 left) 2024-09-15T03:58:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:2752: write (excl) oid 34 current snap is 311 2024-09-15T03:58:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:2752: seq_num 982 ranges {504474=504055,1714141=631006} 2024-09-15T03:58:13.187 INFO:tasks.rados.rados.0.smithi122.stdout:2752: writing smithi12223906-34 from 504474 to 1008529 tid 1 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2752: writing smithi12223906-34 from 1714141 to 2345147 tid 2 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2753: rollback oid 15 current snap is 311 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 295 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2752: finishing write tid 1 to smithi12223906-34 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2754: read oid 9 snap -1 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2754: expect deleted 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2755: write (excl) oid 31 current snap is 311 2024-09-15T03:58:13.197 INFO:tasks.rados.rados.0.smithi122.stdout:2755: seq_num 983 ranges {521281=579304,1656270=774664} 2024-09-15T03:58:13.203 INFO:tasks.rados.rados.0.smithi122.stdout:2755: writing smithi12223906-31 from 521281 to 1100585 tid 1 2024-09-15T03:58:13.211 INFO:tasks.rados.rados.0.smithi122.stdout:2755: writing smithi12223906-31 from 1656270 to 2430934 tid 2 2024-09-15T03:58:13.211 INFO:tasks.rados.rados.0.smithi122.stdout:2756: copy_from oid 50 from oid 29 current snap is 311 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2752: finishing write tid 2 to smithi12223906-34 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2753: finishing rollback tid 0 to smithi12223906-15 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2753: finishing rollback tid 1 to smithi12223906-15 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2752: finishing write tid 3 to smithi12223906-34 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2753: finishing rollback tid 2 to smithi12223906-15 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1334 (ObjNum 823 snap 269 seq_num 823) dirty exists 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2752: finishing write tid 4 to smithi12223906-34 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1576 (ObjNum 982 snap 311 seq_num 982) dirty exists 2024-09-15T03:58:13.212 INFO:tasks.rados.rados.0.smithi122.stdout:2752: left oid 34 (ObjNum 982 snap 311 seq_num 982) 2024-09-15T03:58:13.213 INFO:tasks.rados.rados.0.smithi122.stdout:2752: done (4 left) 2024-09-15T03:58:13.213 INFO:tasks.rados.rados.0.smithi122.stdout:2753: done (3 left) 2024-09-15T03:58:13.213 INFO:tasks.rados.rados.0.smithi122.stdout:2754: done (2 left) 2024-09-15T03:58:13.213 INFO:tasks.rados.rados.0.smithi122.stdout:2757: write oid 46 current snap is 311 2024-09-15T03:58:13.213 INFO:tasks.rados.rados.0.smithi122.stdout:2757: seq_num 984 ranges {538088=654553,1714752=634675} 2024-09-15T03:58:13.218 INFO:tasks.rados.rados.0.smithi122.stdout:2757: writing smithi12223906-46 from 538088 to 1192641 tid 1 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2757: writing smithi12223906-46 from 1714752 to 2349427 tid 2 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2758: delete oid 48 current snap is 311 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2755: finishing write tid 1 to smithi12223906-31 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2755: finishing write tid 2 to smithi12223906-31 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2755: finishing write tid 3 to smithi12223906-31 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2755: finishing write tid 4 to smithi12223906-31 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1879 (ObjNum 983 snap 311 seq_num 983) dirty exists 2024-09-15T03:58:13.225 INFO:tasks.rados.rados.0.smithi122.stdout:2755: left oid 31 (ObjNum 983 snap 311 seq_num 983) 2024-09-15T03:58:13.233 INFO:tasks.rados.rados.0.smithi122.stdout:2756: finishing copy_from racing read to smithi12223906-50 2024-09-15T03:58:13.233 INFO:tasks.rados.rados.0.smithi122.stdout:2757: finishing write tid 1 to smithi12223906-46 2024-09-15T03:58:13.234 INFO:tasks.rados.rados.0.smithi122.stdout:2755: done (3 left) 2024-09-15T03:58:13.234 INFO:tasks.rados.rados.0.smithi122.stdout:2758: done (2 left) 2024-09-15T03:58:13.235 INFO:tasks.rados.rados.0.smithi122.stdout:2759: write (excl) oid 10 current snap is 311 2024-09-15T03:58:13.235 INFO:tasks.rados.rados.0.smithi122.stdout:2759: seq_num 986 ranges {571702=405051,1431716=638344,2560365=11337} 2024-09-15T03:58:13.240 INFO:tasks.rados.rados.0.smithi122.stdout:2759: writing smithi12223906-10 from 571702 to 976753 tid 1 2024-09-15T03:58:13.249 INFO:tasks.rados.rados.0.smithi122.stdout:2759: writing smithi12223906-10 from 1431716 to 2070060 tid 2 2024-09-15T03:58:13.252 INFO:tasks.rados.rados.0.smithi122.stdout:2759: writing smithi12223906-10 from 2560365 to 2571702 tid 3 2024-09-15T03:58:13.252 INFO:tasks.rados.rados.0.smithi122.stdout:2760: snap_remove snap 296 2024-09-15T03:58:14.189 INFO:tasks.rados.rados.0.smithi122.stdout:2760: done (3 left) 2024-09-15T03:58:14.190 INFO:tasks.rados.rados.0.smithi122.stdout:2761: snap_remove snap 310 2024-09-15T03:58:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2761: done (3 left) 2024-09-15T03:58:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2762: read oid 34 snap -1 2024-09-15T03:58:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2762: expect (ObjNum 982 snap 311 seq_num 982) 2024-09-15T03:58:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2763: read oid 45 snap -1 2024-09-15T03:58:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2763: expect (ObjNum 962 snap 305 seq_num 962) 2024-09-15T03:58:15.207 INFO:tasks.rados.rados.0.smithi122.stdout:2764: snap_remove snap 284 2024-09-15T03:58:16.056 INFO:tasks.rados.rados.0.smithi122.stdout:2764: done (5 left) 2024-09-15T03:58:16.056 INFO:tasks.rados.rados.0.smithi122.stdout:2765: snap_remove snap 273 2024-09-15T03:58:17.063 INFO:tasks.rados.rados.0.smithi122.stdout:2765: done (5 left) 2024-09-15T03:58:17.063 INFO:tasks.rados.rados.0.smithi122.stdout:2766: delete oid 48 current snap is 311 2024-09-15T03:58:17.063 INFO:tasks.rados.rados.0.smithi122.stdout:2756: finishing copy_from to smithi12223906-50 2024-09-15T03:58:17.063 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1573 (ObjNum 900 snap 286 seq_num 900) dirty exists 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2757: finishing write tid 2 to smithi12223906-46 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2757: finishing write tid 3 to smithi12223906-46 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2757: finishing write tid 4 to smithi12223906-46 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 1581 (ObjNum 984 snap 311 seq_num 984) dirty exists 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2757: left oid 46 (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2759: finishing write tid 1 to smithi12223906-10 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2759: finishing write tid 2 to smithi12223906-10 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2759: finishing write tid 3 to smithi12223906-10 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2759: finishing write tid 4 to smithi12223906-10 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2759: finishing write tid 5 to smithi12223906-10 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1084 (ObjNum 986 snap 311 seq_num 986) dirty exists 2024-09-15T03:58:17.064 INFO:tasks.rados.rados.0.smithi122.stdout:2759: left oid 10 (ObjNum 986 snap 311 seq_num 986) 2024-09-15T03:58:17.065 INFO:tasks.rados.rados.0.smithi122.stdout:2763: expect (ObjNum 962 snap 305 seq_num 962) 2024-09-15T03:58:17.212 INFO:tasks.rados.rados.0.smithi122.stdout:2762: expect (ObjNum 982 snap 311 seq_num 982) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2756: done (5 left) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2757: done (4 left) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2759: done (3 left) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2762: done (2 left) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2763: done (1 left) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2766: done (0 left) 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2767: read oid 23 snap -1 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2767: expect deleted 2024-09-15T03:58:17.363 INFO:tasks.rados.rados.0.smithi122.stdout:2768: delete oid 12 current snap is 311 2024-09-15T03:58:17.366 INFO:tasks.rados.rados.0.smithi122.stdout:2767: done (1 left) 2024-09-15T03:58:17.366 INFO:tasks.rados.rados.0.smithi122.stdout:2768: done (0 left) 2024-09-15T03:58:17.366 INFO:tasks.rados.rados.0.smithi122.stdout:2769: write oid 31 current snap is 311 2024-09-15T03:58:17.366 INFO:tasks.rados.rados.0.smithi122.stdout:2769: seq_num 989 ranges {622123=747151,2007162=614961} 2024-09-15T03:58:17.379 INFO:tasks.rados.rados.0.smithi122.stdout:2769: writing smithi12223906-31 from 622123 to 1369274 tid 1 2024-09-15T03:58:17.386 INFO:tasks.rados.rados.0.smithi122.stdout:2769: writing smithi12223906-31 from 2007162 to 2622123 tid 2 2024-09-15T03:58:17.387 INFO:tasks.rados.rados.0.smithi122.stdout:2770: read oid 46 snap -1 2024-09-15T03:58:17.387 INFO:tasks.rados.rados.0.smithi122.stdout:2770: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:17.387 INFO:tasks.rados.rados.0.smithi122.stdout:2771: write oid 50 current snap is 311 2024-09-15T03:58:17.387 INFO:tasks.rados.rados.0.smithi122.stdout:2771: seq_num 990 ranges {638930=422400,1665644=645682} 2024-09-15T03:58:17.392 INFO:tasks.rados.rados.0.smithi122.stdout:2771: writing smithi12223906-50 from 638930 to 1061330 tid 1 2024-09-15T03:58:17.400 INFO:tasks.rados.rados.0.smithi122.stdout:2771: writing smithi12223906-50 from 1665644 to 2311326 tid 2 2024-09-15T03:58:17.400 INFO:tasks.rados.rados.0.smithi122.stdout:2772: copy_from oid 10 from oid 13 current snap is 311 2024-09-15T03:58:17.400 INFO:tasks.rados.rados.0.smithi122.stdout:2769: finishing write tid 1 to smithi12223906-31 2024-09-15T03:58:17.400 INFO:tasks.rados.rados.0.smithi122.stdout:2773: read oid 21 snap -1 2024-09-15T03:58:17.400 INFO:tasks.rados.rados.0.smithi122.stdout:2773: expect (ObjNum 972 snap 306 seq_num 972) 2024-09-15T03:58:17.400 INFO:tasks.rados.rados.0.smithi122.stdout:2774: write (excl) oid 4 current snap is 311 2024-09-15T03:58:17.401 INFO:tasks.rados.rados.0.smithi122.stdout:2774: seq_num 991 ranges {655737=497649,1607773=789340} 2024-09-15T03:58:17.406 INFO:tasks.rados.rados.0.smithi122.stdout:2774: writing smithi12223906-4 from 655737 to 1153386 tid 1 2024-09-15T03:58:17.414 INFO:tasks.rados.rados.0.smithi122.stdout:2774: writing smithi12223906-4 from 1607773 to 2397113 tid 2 2024-09-15T03:58:17.414 INFO:tasks.rados.rados.0.smithi122.stdout:2775: read oid 16 snap 231 2024-09-15T03:58:17.415 INFO:tasks.rados.rados.0.smithi122.stdout:2775: expect (ObjNum 744 snap 226 seq_num 744) 2024-09-15T03:58:17.415 INFO:tasks.rados.rados.0.smithi122.stdout:2776: read oid 44 snap -1 2024-09-15T03:58:17.415 INFO:tasks.rados.rados.0.smithi122.stdout:2776: expect deleted 2024-09-15T03:58:17.415 INFO:tasks.rados.rados.0.smithi122.stdout:2777: write (excl) oid 36 current snap is 311 2024-09-15T03:58:17.415 INFO:tasks.rados.rados.0.smithi122.stdout:2769: finishing write tid 2 to smithi12223906-31 2024-09-15T03:58:17.415 INFO:tasks.rados.rados.0.smithi122.stdout:2777: seq_num 992 ranges {672544=572898,1666255=532998} 2024-09-15T03:58:17.421 INFO:tasks.rados.rados.0.smithi122.stdout:2777: writing smithi12223906-36 from 672544 to 1245442 tid 1 2024-09-15T03:58:17.427 INFO:tasks.rados.rados.0.smithi122.stdout:2777: writing smithi12223906-36 from 1666255 to 2199253 tid 2 2024-09-15T03:58:17.427 INFO:tasks.rados.rados.0.smithi122.stdout:2778: read oid 24 snap -1 2024-09-15T03:58:17.427 INFO:tasks.rados.rados.0.smithi122.stdout:2778: expect deleted 2024-09-15T03:58:17.427 INFO:tasks.rados.rados.0.smithi122.stdout:2779: read oid 27 snap -1 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:2779: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:2780: write oid 25 current snap is 311 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:2769: finishing write tid 3 to smithi12223906-31 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:2769: finishing write tid 4 to smithi12223906-31 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1882 (ObjNum 989 snap 311 seq_num 989) dirty exists 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:2769: left oid 31 (ObjNum 989 snap 311 seq_num 989) 2024-09-15T03:58:17.428 INFO:tasks.rados.rados.0.smithi122.stdout:2780: seq_num 993 ranges {689351=648147,2124737=564614} 2024-09-15T03:58:17.434 INFO:tasks.rados.rados.0.smithi122.stdout:2780: writing smithi12223906-25 from 689351 to 1337498 tid 1 2024-09-15T03:58:17.442 INFO:tasks.rados.rados.0.smithi122.stdout:2780: writing smithi12223906-25 from 2124737 to 2689351 tid 2 2024-09-15T03:58:17.442 INFO:tasks.rados.rados.0.smithi122.stdout:2769: done (11 left) 2024-09-15T03:58:17.442 INFO:tasks.rados.rados.0.smithi122.stdout:2781: copy_from oid 2 from oid 30 current snap is 311 2024-09-15T03:58:17.442 INFO:tasks.rados.rados.0.smithi122.stdout:2770: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2771: finishing write tid 1 to smithi12223906-50 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2770: done (11 left) 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2782: delete oid 17 current snap is 311 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2771: finishing write tid 2 to smithi12223906-50 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2771: finishing write tid 3 to smithi12223906-50 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2771: finishing write tid 4 to smithi12223906-50 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2086 (ObjNum 990 snap 311 seq_num 990) dirty exists 2024-09-15T03:58:17.633 INFO:tasks.rados.rados.0.smithi122.stdout:2771: left oid 50 (ObjNum 990 snap 311 seq_num 990) 2024-09-15T03:58:17.634 INFO:tasks.rados.rados.0.smithi122.stdout:2772: finishing copy_from to smithi12223906-10 2024-09-15T03:58:17.634 INFO:tasks.rados.rados.0.smithi122.stdout:2772: got expected ENOENT (src dne) 2024-09-15T03:58:17.634 INFO:tasks.rados.rados.0.smithi122.stdout:2772: finishing copy_from racing read to smithi12223906-10 2024-09-15T03:58:17.634 INFO:tasks.rados.rados.0.smithi122.stdout:2774: finishing write tid 1 to smithi12223906-4 2024-09-15T03:58:17.634 INFO:tasks.rados.rados.0.smithi122.stdout:2773: expect (ObjNum 972 snap 306 seq_num 972) 2024-09-15T03:58:17.739 INFO:tasks.rados.rados.0.smithi122.stdout:2774: finishing write tid 2 to smithi12223906-4 2024-09-15T03:58:17.739 INFO:tasks.rados.rados.0.smithi122.stdout:2774: finishing write tid 3 to smithi12223906-4 2024-09-15T03:58:17.739 INFO:tasks.rados.rados.0.smithi122.stdout:2774: finishing write tid 4 to smithi12223906-4 2024-09-15T03:58:17.739 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1885 (ObjNum 991 snap 311 seq_num 991) dirty exists 2024-09-15T03:58:17.739 INFO:tasks.rados.rados.0.smithi122.stdout:2774: left oid 4 (ObjNum 991 snap 311 seq_num 991) 2024-09-15T03:58:17.739 INFO:tasks.rados.rados.0.smithi122.stdout:2775: expect (ObjNum 744 snap 226 seq_num 744) 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:2777: finishing write tid 1 to smithi12223906-36 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:2777: finishing write tid 2 to smithi12223906-36 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:2777: finishing write tid 3 to smithi12223906-36 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:2777: finishing write tid 4 to smithi12223906-36 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 2154 (ObjNum 992 snap 311 seq_num 992) dirty exists 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:2777: left oid 36 (ObjNum 992 snap 311 seq_num 992) 2024-09-15T03:58:17.926 INFO:tasks.rados.rados.0.smithi122.stdout:2779: expect (ObjNum 925 snap 292 seq_num 925) 2024-09-15T03:58:18.033 INFO:tasks.rados.rados.0.smithi122.stdout:2780: finishing write tid 1 to smithi12223906-25 2024-09-15T03:58:18.033 INFO:tasks.rados.rados.0.smithi122.stdout:2780: finishing write tid 2 to smithi12223906-25 2024-09-15T03:58:18.033 INFO:tasks.rados.rados.0.smithi122.stdout:2780: finishing write tid 3 to smithi12223906-25 2024-09-15T03:58:18.033 INFO:tasks.rados.rados.0.smithi122.stdout:2780: finishing write tid 4 to smithi12223906-25 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2157 (ObjNum 993 snap 311 seq_num 993) dirty exists 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2780: left oid 25 (ObjNum 993 snap 311 seq_num 993) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2781: finishing copy_from racing read to smithi12223906-2 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2781: finishing copy_from to smithi12223906-2 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 842 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2771: done (11 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2772: done (10 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2773: done (9 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2774: done (8 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2775: done (7 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2776: done (6 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2777: done (5 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2778: done (4 left) 2024-09-15T03:58:18.034 INFO:tasks.rados.rados.0.smithi122.stdout:2779: done (3 left) 2024-09-15T03:58:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2780: done (2 left) 2024-09-15T03:58:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2781: done (1 left) 2024-09-15T03:58:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2782: done (0 left) 2024-09-15T03:58:18.035 INFO:tasks.rados.rados.0.smithi122.stdout:2783: snap_create 2024-09-15T03:58:18.071 INFO:tasks.rados.rados.0.smithi122.stdout:2783: done (0 left) 2024-09-15T03:58:18.071 INFO:tasks.rados.rados.0.smithi122.stdout:2784: snap_remove snap 235 2024-09-15T03:58:19.078 INFO:tasks.rados.rados.0.smithi122.stdout:2784: done (0 left) 2024-09-15T03:58:19.078 INFO:tasks.rados.rados.0.smithi122.stdout:2785: snap_remove snap 267 2024-09-15T03:58:20.082 INFO:tasks.rados.rados.0.smithi122.stdout:2785: done (0 left) 2024-09-15T03:58:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:2786: read oid 22 snap -1 2024-09-15T03:58:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:2786: expect (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:58:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:2787: write (excl) oid 48 current snap is 312 2024-09-15T03:58:20.083 INFO:tasks.rados.rados.0.smithi122.stdout:2787: seq_num 995 ranges {722965=798645,2125348=597617} 2024-09-15T03:58:20.091 INFO:tasks.rados.rados.0.smithi122.stdout:2787: writing smithi12223906-48 from 722965 to 1521610 tid 1 2024-09-15T03:58:20.099 INFO:tasks.rados.rados.0.smithi122.stdout:2787: writing smithi12223906-48 from 2125348 to 2722965 tid 2 2024-09-15T03:58:20.099 INFO:tasks.rados.rados.0.smithi122.stdout:2788: write oid 49 current snap is 312 2024-09-15T03:58:20.100 INFO:tasks.rados.rados.0.smithi122.stdout:2788: seq_num 996 ranges {739772=590247,1900183=540336} 2024-09-15T03:58:20.106 INFO:tasks.rados.rados.0.smithi122.stdout:2788: writing smithi12223906-49 from 739772 to 1330019 tid 1 2024-09-15T03:58:20.116 INFO:tasks.rados.rados.0.smithi122.stdout:2788: writing smithi12223906-49 from 1900183 to 2440519 tid 2 2024-09-15T03:58:20.116 INFO:tasks.rados.rados.0.smithi122.stdout:2789: rollback oid 32 current snap is 312 2024-09-15T03:58:20.116 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 237 2024-09-15T03:58:20.116 INFO:tasks.rados.rados.0.smithi122.stdout:2787: finishing write tid 1 to smithi12223906-48 2024-09-15T03:58:20.116 INFO:tasks.rados.rados.0.smithi122.stdout:2786: expect (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:58:20.379 INFO:tasks.rados.rados.0.smithi122.stdout:2787: finishing write tid 2 to smithi12223906-48 2024-09-15T03:58:20.380 INFO:tasks.rados.rados.0.smithi122.stdout:2787: finishing write tid 3 to smithi12223906-48 2024-09-15T03:58:20.380 INFO:tasks.rados.rados.0.smithi122.stdout:2787: finishing write tid 4 to smithi12223906-48 2024-09-15T03:58:20.380 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1087 (ObjNum 995 snap 312 seq_num 995) dirty exists 2024-09-15T03:58:20.380 INFO:tasks.rados.rados.0.smithi122.stdout:2787: left oid 48 (ObjNum 995 snap 312 seq_num 995) 2024-09-15T03:58:20.381 INFO:tasks.rados.rados.0.smithi122.stdout:2788: finishing write tid 1 to smithi12223906-49 2024-09-15T03:58:20.381 INFO:tasks.rados.rados.0.smithi122.stdout:2788: finishing write tid 2 to smithi12223906-49 2024-09-15T03:58:20.381 INFO:tasks.rados.rados.0.smithi122.stdout:2789: finishing rollback tid 0 to smithi12223906-32 2024-09-15T03:58:20.381 INFO:tasks.rados.rados.0.smithi122.stdout:2789: finishing rollback tid 1 to smithi12223906-32 2024-09-15T03:58:20.381 INFO:tasks.rados.rados.0.smithi122.stdout:2788: finishing write tid 3 to smithi12223906-49 2024-09-15T03:58:20.381 INFO:tasks.rados.rados.0.smithi122.stdout:2789: finishing rollback tid 2 to smithi12223906-32 2024-09-15T03:58:20.382 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 2110 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T03:58:20.382 INFO:tasks.rados.rados.0.smithi122.stdout:2788: finishing write tid 4 to smithi12223906-49 2024-09-15T03:58:20.382 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1059 (ObjNum 996 snap 312 seq_num 996) dirty exists 2024-09-15T03:58:20.382 INFO:tasks.rados.rados.0.smithi122.stdout:2788: left oid 49 (ObjNum 996 snap 312 seq_num 996) 2024-09-15T03:58:20.382 INFO:tasks.rados.rados.0.smithi122.stdout:2786: done (3 left) 2024-09-15T03:58:20.382 INFO:tasks.rados.rados.0.smithi122.stdout:2787: done (2 left) 2024-09-15T03:58:20.383 INFO:tasks.rados.rados.0.smithi122.stdout:2788: done (1 left) 2024-09-15T03:58:20.383 INFO:tasks.rados.rados.0.smithi122.stdout:2789: done (0 left) 2024-09-15T03:58:20.383 INFO:tasks.rados.rados.0.smithi122.stdout:2790: write oid 39 current snap is 312 2024-09-15T03:58:20.383 INFO:tasks.rados.rados.0.smithi122.stdout:2790: seq_num 997 ranges {756579=665496,1958665=400347} 2024-09-15T03:58:20.397 INFO:tasks.rados.rados.0.smithi122.stdout:2790: writing smithi12223906-39 from 756579 to 1422075 tid 1 2024-09-15T03:58:20.403 INFO:tasks.rados.rados.0.smithi122.stdout:2790: writing smithi12223906-39 from 1958665 to 2359012 tid 2 2024-09-15T03:58:20.403 INFO:tasks.rados.rados.0.smithi122.stdout:2791: write oid 16 current snap is 312 2024-09-15T03:58:20.403 INFO:tasks.rados.rados.0.smithi122.stdout:2791: seq_num 998 ranges {773386=740745,2017147=544005} 2024-09-15T03:58:20.411 INFO:tasks.rados.rados.0.smithi122.stdout:2791: writing smithi12223906-16 from 773386 to 1514131 tid 1 2024-09-15T03:58:20.417 INFO:tasks.rados.rados.0.smithi122.stdout:2791: writing smithi12223906-16 from 2017147 to 2561152 tid 2 2024-09-15T03:58:20.417 INFO:tasks.rados.rados.0.smithi122.stdout:2792: snap_remove snap 309 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2792: done (2 left) 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2793: snap_remove snap 285 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2790: finishing write tid 1 to smithi12223906-39 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2790: finishing write tid 2 to smithi12223906-39 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2790: finishing write tid 3 to smithi12223906-39 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2790: finishing write tid 4 to smithi12223906-39 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2185 (ObjNum 997 snap 312 seq_num 997) dirty exists 2024-09-15T03:58:21.065 INFO:tasks.rados.rados.0.smithi122.stdout:2790: left oid 39 (ObjNum 997 snap 312 seq_num 997) 2024-09-15T03:58:21.066 INFO:tasks.rados.rados.0.smithi122.stdout:2791: finishing write tid 1 to smithi12223906-16 2024-09-15T03:58:21.066 INFO:tasks.rados.rados.0.smithi122.stdout:2791: finishing write tid 2 to smithi12223906-16 2024-09-15T03:58:21.066 INFO:tasks.rados.rados.0.smithi122.stdout:2791: finishing write tid 3 to smithi12223906-16 2024-09-15T03:58:21.066 INFO:tasks.rados.rados.0.smithi122.stdout:2791: finishing write tid 4 to smithi12223906-16 2024-09-15T03:58:21.066 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1337 (ObjNum 998 snap 312 seq_num 998) dirty exists 2024-09-15T03:58:21.066 INFO:tasks.rados.rados.0.smithi122.stdout:2791: left oid 16 (ObjNum 998 snap 312 seq_num 998) 2024-09-15T03:58:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2790: done (2 left) 2024-09-15T03:58:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2791: done (1 left) 2024-09-15T03:58:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2793: done (0 left) 2024-09-15T03:58:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2794: write (excl) oid 28 current snap is 312 2024-09-15T03:58:22.064 INFO:tasks.rados.rados.0.smithi122.stdout:2794: seq_num 999 ranges {790193=415994,1959276=404016} 2024-09-15T03:58:22.071 INFO:tasks.rados.rados.0.smithi122.stdout:2794: writing smithi12223906-28 from 790193 to 1206187 tid 1 2024-09-15T03:58:22.078 INFO:tasks.rados.rados.0.smithi122.stdout:2794: writing smithi12223906-28 from 1959276 to 2363292 tid 2 2024-09-15T03:58:22.078 INFO:tasks.rados.rados.0.smithi122.stdout:2795: read oid 15 snap 308 2024-09-15T03:58:22.078 INFO:tasks.rados.rados.0.smithi122.stdout:2795: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:58:22.078 INFO:tasks.rados.rados.0.smithi122.stdout:2796: write (excl) oid 21 current snap is 312 2024-09-15T03:58:22.078 INFO:tasks.rados.rados.0.smithi122.stdout:2794: finishing write tid 1 to smithi12223906-28 2024-09-15T03:58:22.079 INFO:tasks.rados.rados.0.smithi122.stdout:2796: seq_num 1000 ranges {407000=491243,1617758=547674,2595228=211772} 2024-09-15T03:58:22.084 INFO:tasks.rados.rados.0.smithi122.stdout:2796: writing smithi12223906-21 from 407000 to 898243 tid 1 2024-09-15T03:58:22.092 INFO:tasks.rados.rados.0.smithi122.stdout:2796: writing smithi12223906-21 from 1617758 to 2165432 tid 2 2024-09-15T03:58:22.096 INFO:tasks.rados.rados.0.smithi122.stdout:2796: writing smithi12223906-21 from 2595228 to 2807000 tid 3 2024-09-15T03:58:22.096 INFO:tasks.rados.rados.0.smithi122.stdout:2797: write (excl) oid 1 current snap is 312 2024-09-15T03:58:22.096 INFO:tasks.rados.rados.0.smithi122.stdout:2797: seq_num 1001 ranges {423807=566492,1676240=407685,2739004=84803} 2024-09-15T03:58:22.101 INFO:tasks.rados.rados.0.smithi122.stdout:2797: writing smithi12223906-1 from 423807 to 990299 tid 1 2024-09-15T03:58:22.108 INFO:tasks.rados.rados.0.smithi122.stdout:2797: writing smithi12223906-1 from 1676240 to 2083925 tid 2 2024-09-15T03:58:22.112 INFO:tasks.rados.rados.0.smithi122.stdout:2797: writing smithi12223906-1 from 2739004 to 2823807 tid 3 2024-09-15T03:58:22.112 INFO:tasks.rados.rados.0.smithi122.stdout:2798: snap_create 2024-09-15T03:58:22.113 INFO:tasks.rados.rados.0.smithi122.stdout:2794: finishing write tid 2 to smithi12223906-28 2024-09-15T03:58:22.113 INFO:tasks.rados.rados.0.smithi122.stdout:2794: finishing write tid 3 to smithi12223906-28 2024-09-15T03:58:22.113 INFO:tasks.rados.rados.0.smithi122.stdout:2794: finishing write tid 4 to smithi12223906-28 2024-09-15T03:58:22.113 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 1585 (ObjNum 999 snap 312 seq_num 999) dirty exists 2024-09-15T03:58:22.113 INFO:tasks.rados.rados.0.smithi122.stdout:2794: left oid 28 (ObjNum 999 snap 312 seq_num 999) 2024-09-15T03:58:22.113 INFO:tasks.rados.rados.0.smithi122.stdout:2795: expect (ObjNum 823 snap 269 seq_num 823) 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2796: finishing write tid 1 to smithi12223906-21 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2796: finishing write tid 2 to smithi12223906-21 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2796: finishing write tid 3 to smithi12223906-21 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2796: finishing write tid 4 to smithi12223906-21 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2796: finishing write tid 5 to smithi12223906-21 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1091 (ObjNum 1000 snap 312 seq_num 1000) dirty exists 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2796: left oid 21 (ObjNum 1000 snap 312 seq_num 1000) 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2797: finishing write tid 1 to smithi12223906-1 2024-09-15T03:58:22.244 INFO:tasks.rados.rados.0.smithi122.stdout:2797: finishing write tid 2 to smithi12223906-1 2024-09-15T03:58:22.245 INFO:tasks.rados.rados.0.smithi122.stdout:2797: finishing write tid 3 to smithi12223906-1 2024-09-15T03:58:22.245 INFO:tasks.rados.rados.0.smithi122.stdout:2797: finishing write tid 4 to smithi12223906-1 2024-09-15T03:58:22.245 INFO:tasks.rados.rados.0.smithi122.stdout:2797: finishing write tid 5 to smithi12223906-1 2024-09-15T03:58:22.245 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1341 (ObjNum 1001 snap 312 seq_num 1001) dirty exists 2024-09-15T03:58:22.245 INFO:tasks.rados.rados.0.smithi122.stdout:2797: left oid 1 (ObjNum 1001 snap 312 seq_num 1001) 2024-09-15T03:58:22.834 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T03:58:22.834 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-15T03:58:22.834 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T03:58:22.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T03:58:22.837 INFO:tasks.ceph.mon.i:Started 2024-09-15T03:58:22.837 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:58:22.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:58:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: get_monmap_and_config 2024-09-15T03:58:22.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: build_initial_monmap 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: monmap: 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:22.945226+0000 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:22.945226+0000 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:22.950 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 1 Processor -- start 2024-09-15T03:58:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 1 -- start start 2024-09-15T03:58:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: init 2024-09-15T03:58:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: _add_conns ranks=[8,3,1,4,5,7,6,0,2] 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient(hunting): picked mon.noname-e con 0x7f87f40a6a80 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient(hunting): picked mon.noname-g con 0x7f87f409fb80 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient(hunting): picked mon.noname-f con 0x7f87f4138d30 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f404ca10 con 0x7f87f4138d30 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40a0f50 con 0x7f87f409fb80 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f4133680 con 0x7f87f40a6a80 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient(hunting): authenticate will time out at 1647.374606s 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f975b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f87f40a6a80 0x7f87f4132f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33454/0 (socket says 172.21.15.26:33454) 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f975b700 1 -- 172.21.15.26:0/968677689 learned_addr learned my addr 172.21.15.26:0/968677689 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:22.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858221218 0 0) 0x7f87f4133680 con 0x7f87f40a6a80 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): my global_id is 135251 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87dc003570 con 0x7f87f40a6a80 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844978566 0 0) 0x7f87f40a0f50 con 0x7f87f409fb80 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): my global_id is 204330 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f4133680 con 0x7f87f409fb80 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147005866 0 0) 0x7f87f404ca10 con 0x7f87f4138d30 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient(hunting): my global_id is 204319 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f40a0f50 con 0x7f87f4138d30 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1411174427 0 0) 0x7f87dc003570 con 0x7f87f40a6a80 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 >> v1:172.21.15.122:6790/0 conn(0x7f87f409fb80 legacy=0x7f87f409ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 >> v1:172.21.15.122:6789/0 conn(0x7f87f4138d30 legacy=0x7f87f413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 monclient: found mon.noname-e 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:58:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40a6e50 con 0x7f87f40a6a80 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: _finish_auth 0 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87ec001b30 con 0x7f87f40a6a80 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: dump: 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:22.959 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:22.960 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 5 monclient: authenticate success, global_id 135251 2024-09-15T03:58:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f87ec001e10 con 0x7f87f40a6a80 2024-09-15T03:58:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 1 -- 172.21.15.26:0/968677689 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87ec002000 con 0x7f87f40a6a80 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87f3fff700 10 monclient: dump: 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.940+0000 7f87fa75d700 4 set_mon_vals no callback set 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 10 monclient: shutdown 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 -- 172.21.15.26:0/968677689 >> v1:172.21.15.26:6793/0 conn(0x7f87f40a6a80 legacy=0x7f87f4132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 -- 172.21.15.26:0/968677689 shutdown_connections 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 -- 172.21.15.26:0/968677689 >> 172.21.15.26:0/968677689 conn(0x7f87f406e380 msgr2=0x7f87f40a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 -- 172.21.15.26:0/968677689 shutdown_connections 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 -- 172.21.15.26:0/968677689 wait complete. 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 10 monclient: build_initial_monmap 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 10 monclient: monmap: 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:22.964 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 Processor -- start 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 1 -- start start 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.944+0000 7f87fa75d700 10 monclient: init 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient: _add_conns ranks=[1,5,6,3,7,2,8,0,4] 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient(hunting): picked mon.noname-f con 0x7f87f409fb80 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient(hunting): picked mon.noname-h con 0x7f87f40a6a80 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient(hunting): picked mon.noname-d con 0x7f87f4138d30 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f40a1920 con 0x7f87f409fb80 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f409cfc0 con 0x7f87f40a6a80 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f411dd20 con 0x7f87f4138d30 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient(hunting): authenticate will time out at 1647.381048s 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f975b700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f87f409fb80 0x7f87f4099b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:52018/0 (socket says 172.21.15.26:52018) 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f975b700 1 -- 172.21.15.26:0/1390491882 learned_addr learned my addr 172.21.15.26:0/1390491882 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f9f5c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f87f4138d30 0x7f87f414a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43756/0 (socket says 172.21.15.26:43756) 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f8f5a700 1 --1- 172.21.15.26:0/1390491882 >> v1:172.21.15.122:6791/0 conn(0x7f87f40a6a80 0x7f87f409a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:41374/0 (socket says 172.21.15.26:41374) 2024-09-15T03:58:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730011021 0 0) 0x7f87f411dd20 con 0x7f87f4138d30 2024-09-15T03:58:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): my global_id is 194559 2024-09-15T03:58:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e0003570 con 0x7f87f4138d30 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174382093 0 0) 0x7f87f40a1920 con 0x7f87f409fb80 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): my global_id is 204328 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f411dd20 con 0x7f87f409fb80 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945344577 0 0) 0x7f87f409cfc0 con 0x7f87f40a6a80 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:22.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): my global_id is 5846 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f40a1920 con 0x7f87f40a6a80 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1384265918 0 0) 0x7f87e0003570 con 0x7f87f4138d30 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87f409cfc0 con 0x7f87f4138d30 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4131239618 0 0) 0x7f87f40a1920 con 0x7f87f40a6a80 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87e0003570 con 0x7f87f40a6a80 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87e40019c0 con 0x7f87f4138d30 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): dump: 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 302662282 0 0) 0x7f87f411dd20 con 0x7f87f409fb80 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f87f40a1920 con 0x7f87f409fb80 2024-09-15T03:58:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87e80019e0 con 0x7f87f40a6a80 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): dump: 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87ec001af0 con 0x7f87f409fb80 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient(hunting): dump: 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2633884311 0 0) 0x7f87f409cfc0 con 0x7f87f4138d30 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 >> v1:172.21.15.122:6791/0 conn(0x7f87f40a6a80 legacy=0x7f87f409a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 >> v1:172.21.15.122:6789/0 conn(0x7f87f409fb80 legacy=0x7f87f4099b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 monclient: found mon.d 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f411ca90 con 0x7f87f4138d30 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 5 monclient: authenticate success, global_id 194559 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient: _renew_subs 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f409bfd0 con 0x7f87f4138d30 2024-09-15T03:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient: _renew_subs 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f409a950 con 0x7f87f4138d30 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f87e4002d80 con 0x7f87f4138d30 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f87e4002250 con 0x7f87f4138d30 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f2ffd700 4 set_mon_vals no callback set 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 10 monclient: dump: 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.948+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f87e4002430 con 0x7f87f4138d30 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.952+0000 7f87fa75d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.952+0000 7f87fa75d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.952+0000 7f87fa75d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.952+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d40050b0 con 0x7f87f4138d30 2024-09-15T03:58:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.952+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(629..629 src has 435..629) v4 ==== 3987+0+0 (unknown 3959629008 0 0) 0x7f87e4043de0 con 0x7f87f4138d30 2024-09-15T03:58:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.952+0000 7f87f975b700 10 monclient: get_auth_request con 0x7f87e0038090 auth_method 0 2024-09-15T03:58:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.956+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f87e4014c20 con 0x7f87f4138d30 2024-09-15T03:58:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.956+0000 7f87f17fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:22.956+0000 7f87f17fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2794: done (4 left) 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2795: done (3 left) 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2796: done (2 left) 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2797: done (1 left) 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2798: done (0 left) 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2799: write (excl) oid 26 current snap is 313 2024-09-15T03:58:23.105 INFO:tasks.rados.rados.0.smithi122.stdout:2799: seq_num 1002 ranges {440614=641741,1734722=551343} 2024-09-15T03:58:23.111 INFO:tasks.rados.rados.0.smithi122.stdout:2799: writing smithi12223906-26 from 440614 to 1082355 tid 1 2024-09-15T03:58:23.119 INFO:tasks.rados.rados.0.smithi122.stdout:2799: writing smithi12223906-26 from 1734722 to 2286065 tid 2 2024-09-15T03:58:23.119 INFO:tasks.rados.rados.0.smithi122.stdout:2800: snap_create 2024-09-15T03:58:23.130 INFO:tasks.rados.rados.0.smithi122.stdout:2799: finishing write tid 1 to smithi12223906-26 2024-09-15T03:58:23.130 INFO:tasks.rados.rados.0.smithi122.stdout:2799: finishing write tid 2 to smithi12223906-26 2024-09-15T03:58:23.130 INFO:tasks.rados.rados.0.smithi122.stdout:2799: finishing write tid 3 to smithi12223906-26 2024-09-15T03:58:23.130 INFO:tasks.rados.rados.0.smithi122.stdout:2799: finishing write tid 4 to smithi12223906-26 2024-09-15T03:58:23.131 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1305 (ObjNum 1002 snap 313 seq_num 1002) dirty exists 2024-09-15T03:58:23.131 INFO:tasks.rados.rados.0.smithi122.stdout:2799: left oid 26 (ObjNum 1002 snap 313 seq_num 1002) 2024-09-15T03:58:23.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87fa75d700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87fa75d700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87fa75d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f87d40058e0 con 0x7f87f4138d30 2024-09-15T03:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87f17fa700 1 -- 172.21.15.26:0/1390491882 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 2686940480) 0x7f87e4014620 con 0x7f87f4138d30 2024-09-15T03:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87f17fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.208+0000 7f87f17fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:23.220 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":94,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:58:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 >> v1:172.21.15.122:6812/20895 conn(0x7f87e0038090 legacy=0x7f87e003a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 10 monclient: shutdown 2024-09-15T03:58:23.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 >> v1:172.21.15.26:6792/0 conn(0x7f87f4138d30 legacy=0x7f87f414a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 shutdown_connections 2024-09-15T03:58:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 >> 172.21.15.26:0/1390491882 conn(0x7f87f406e380 msgr2=0x7f87f409db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 shutdown_connections 2024-09-15T03:58:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.212+0000 7f87fa75d700 1 -- 172.21.15.26:0/1390491882 wait complete. 2024-09-15T03:58:23.233 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:58:23.233 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:58:23.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:58:23.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.328+0000 7f845682a700 10 monclient: get_monmap_and_config 2024-09-15T03:58:23.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.328+0000 7f845682a700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.328+0000 7f845682a700 10 monclient: monmap: 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:23.334703+0000 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:23.334703+0000 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.340 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.328+0000 7f845682a700 1 Processor -- start 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.328+0000 7f845682a700 1 -- start start 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.328+0000 7f845682a700 10 monclient: init 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient: _add_conns ranks=[7,3,6,5,4,0,8,2,1] 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient(hunting): picked mon.noname-i con 0x7f845009e440 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient(hunting): picked mon.noname-g con 0x7f845012ffc0 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient(hunting): picked mon.noname-d con 0x7f8450130390 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845004ca10 con 0x7f845012ffc0 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845009a0f0 con 0x7f8450130390 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845009e810 con 0x7f845009e440 2024-09-15T03:58:23.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient(hunting): authenticate will time out at 1647.764760s 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f8454dc7700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8450130390 0x7f8450138310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43762/0 (socket says 172.21.15.26:43762) 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f8454dc7700 1 -- 172.21.15.26:0/4135845004 learned_addr learned my addr 172.21.15.26:0/4135845004 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331643483 0 0) 0x7f845009a0f0 con 0x7f8450130390 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient(hunting): my global_id is 194568 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8438003570 con 0x7f8450130390 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007334141 0 0) 0x7f845004ca10 con 0x7f845012ffc0 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient(hunting): my global_id is 204339 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f845009a0f0 con 0x7f845012ffc0 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238316036 0 0) 0x7f8438003570 con 0x7f8450130390 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 >> v1:172.21.15.122:6792/0 conn(0x7f845009e440 legacy=0x7f845012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 >> v1:172.21.15.122:6790/0 conn(0x7f845012ffc0 legacy=0x7f8450134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 monclient: found mon.noname-d 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845009c2a0 con 0x7f8450130390 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 5 monclient: authenticate success, global_id 194568 2024-09-15T03:58:23.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8448001b30 con 0x7f8450130390 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: dump: 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8448001e10 con 0x7f8450130390 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 1 -- 172.21.15.26:0/4135845004 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8448004fb0 con 0x7f8450130390 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 4 set_mon_vals no callback set 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f844effd700 10 monclient: dump: 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.350 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient: shutdown 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- 172.21.15.26:0/4135845004 >> v1:172.21.15.26:6792/0 conn(0x7f8450130390 legacy=0x7f8450138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- 172.21.15.26:0/4135845004 shutdown_connections 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- 172.21.15.26:0/4135845004 >> 172.21.15.26:0/4135845004 conn(0x7f8450063f30 msgr2=0x7f84500a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- 172.21.15.26:0/4135845004 shutdown_connections 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 1 -- 172.21.15.26:0/4135845004 wait complete. 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.332+0000 7f845682a700 10 monclient: monmap: 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 1 Processor -- start 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 1 -- start start 2024-09-15T03:58:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient: init 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient: _add_conns ranks=[0,2,4,8,5,1,3,7,6] 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient(hunting): picked mon.noname-a con 0x7f845009e440 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient(hunting): picked mon.noname-b con 0x7f845012ffc0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient(hunting): picked mon.noname-c con 0x7f8450130390 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84500994f0 con 0x7f845009e440 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845011c090 con 0x7f845012ffc0 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845011bdb0 con 0x7f8450130390 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f845682a700 10 monclient(hunting): authenticate will time out at 1647.769040s 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f844f7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f845012ffc0 0x7f8450147210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52698/0 (socket says 172.21.15.26:52698) 2024-09-15T03:58:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.336+0000 7f844f7fe700 1 -- 172.21.15.26:0/992263413 learned_addr learned my addr 172.21.15.26:0/992263413 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818582174 0 0) 0x7f84500994f0 con 0x7f845009e440 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): my global_id is 194526 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f843c003570 con 0x7f845009e440 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3746121658 0 0) 0x7f843c003570 con 0x7f845009e440 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f84500994f0 con 0x7f845009e440 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8440001a20 con 0x7f845009e440 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): dump: 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.359 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2397493088 0 0) 0x7f84500994f0 con 0x7f845009e440 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6791/0 conn(0x7f8450130390 legacy=0x7f845014a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6790/0 conn(0x7f845012ffc0 legacy=0x7f8450147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 monclient: found mon.a 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450142d90 con 0x7f845009e440 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 5 monclient: authenticate success, global_id 194526 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 10 monclient: _renew_subs 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8450119fc0 con 0x7f845009e440 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 10 monclient: _renew_subs 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f845012f0c0 con 0x7f845009e440 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8440001ff0 con 0x7f845009e440 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8440002520 con 0x7f845009e440 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f844e7fc700 4 set_mon_vals no callback set 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.340+0000 7f8437fff700 10 monclient: dump: 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f845682a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8440002700 con 0x7f845009e440 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8430006590 con 0x7f8430002840 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(630..630 src has 435..630) v4 ==== 3959+0+0 (unknown 1257062899 0 0) 0x7f8440041670 con 0x7f845009e440 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269770427 0 0) 0x7f8430006590 con 0x7f8430002840 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient(hunting): my global_id is 194535 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84500994f0 con 0x7f8430002840 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8454dc7700 10 monclient: get_auth_request con 0x7f843c037520 auth_method 0 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3279758607 0 0) 0x7f84500994f0 con 0x7f8430002840 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8430006590 con 0x7f8430002840 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f84440018e0 con 0x7f8430002840 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient: dump: 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1339571641 0 0) 0x7f8430006590 con 0x7f8430002840 2024-09-15T03:58:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f843000a130 con 0x7f8430002840 2024-09-15T03:58:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8444005810 con 0x7f8430002840 2024-09-15T03:58:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6789/0 conn(0x7f8430002840 legacy=0x7f8430004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.344+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6789/0 conn(0x7f8430002840 legacy=0x7f8430004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8430009640 con 0x7f84300090b0 2024-09-15T03:58:23.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176074516 0 0) 0x7f8430009640 con 0x7f84300090b0 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient(hunting): my global_id is 194544 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8430006590 con 0x7f84300090b0 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4250412776 0 0) 0x7f8430006590 con 0x7f84300090b0 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8430009640 con 0x7f84300090b0 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8444002270 con 0x7f84300090b0 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient: dump: 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 665485426 0 0) 0x7f8430009640 con 0x7f84300090b0 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f84300098f0 con 0x7f84300090b0 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 113134501) 0x7f8444002790 con 0x7f84300090b0 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6789/0 conn(0x7f84300090b0 legacy=0x7f8430008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8437fff700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6789/0 conn(0x7f84300090b0 legacy=0x7f8430008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:23.371 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.372 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.373 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:23.374 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.375 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:23.376 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.377 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.380 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.381 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.382 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.385 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:23.386 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:23.387 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.388 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.389 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:23.390 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.391 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.391 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:23.391 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.122:6812/20895 conn(0x7f843c037520 legacy=0x7f843c0399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 10 monclient: shutdown 2024-09-15T03:58:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 >> v1:172.21.15.26:6789/0 conn(0x7f845009e440 legacy=0x7f845012e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f8454dc7700 1 -- 172.21.15.26:0/992263413 reap_dead start 2024-09-15T03:58:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 shutdown_connections 2024-09-15T03:58:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 >> 172.21.15.26:0/992263413 conn(0x7f8450063f30 msgr2=0x7f845006d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 shutdown_connections 2024-09-15T03:58:23.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.352+0000 7f845682a700 1 -- 172.21.15.26:0/992263413 wait complete. 2024-09-15T03:58:23.393 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:23.396 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:58:23.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient: get_monmap_and_config 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient: monmap: 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:23.504496+0000 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:23.504496+0000 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.509 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 1 Processor -- start 2024-09-15T03:58:23.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 1 -- start start 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient: init 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient: _add_conns ranks=[5,7,3,1,2,8,4,6,0] 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient(hunting): picked mon.noname-h con 0x7f6a3c09b560 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient(hunting): picked mon.noname-i con 0x7f6a3c098170 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient(hunting): picked mon.noname-g con 0x7f6a3c135f20 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c04ca10 con 0x7f6a3c135f20 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c099590 con 0x7f6a3c09b560 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c099870 con 0x7f6a3c098170 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 10 monclient(hunting): authenticate will time out at 1647.934197s 2024-09-15T03:58:23.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a416e0700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f6a3c09b560 0x7f6a3c097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:41382/0 (socket says 172.21.15.26:41382) 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a416e0700 1 -- 172.21.15.26:0/2981134135 learned_addr learned my addr 172.21.15.26:0/2981134135 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509705056 0 0) 0x7f6a3c099590 con 0x7f6a3c09b560 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): my global_id is 5855 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a28003570 con 0x7f6a3c09b560 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848406062 0 0) 0x7f6a3c099870 con 0x7f6a3c098170 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): my global_id is 234106 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c099590 con 0x7f6a3c098170 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681024655 0 0) 0x7f6a3c04ca10 con 0x7f6a3c135f20 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient(hunting): my global_id is 204348 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c099870 con 0x7f6a3c135f20 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1243831080 0 0) 0x7f6a28003570 con 0x7f6a3c09b560 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 >> v1:172.21.15.122:6792/0 conn(0x7f6a3c098170 legacy=0x7f6a3c134800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 >> v1:172.21.15.122:6790/0 conn(0x7f6a3c135f20 legacy=0x7f6a3c138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 monclient: found mon.noname-h 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c098c60 con 0x7f6a3c09b560 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a2c0017b0 con 0x7f6a3c09b560 2024-09-15T03:58:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 5 monclient: authenticate success, global_id 5855 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a33fff700 10 monclient: dump: 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.515 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.500+0000 7f6a43944700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a2c001a20 con 0x7f6a3c09b560 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a33fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 4 set_mon_vals no callback set 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a33fff700 1 -- 172.21.15.26:0/2981134135 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a2c004c30 con 0x7f6a3c09b560 2024-09-15T03:58:23.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a33fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a33fff700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a33fff700 10 monclient: dump: 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.517 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.518 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: shutdown 2024-09-15T03:58:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- 172.21.15.26:0/2981134135 >> v1:172.21.15.122:6791/0 conn(0x7f6a3c09b560 legacy=0x7f6a3c097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- 172.21.15.26:0/2981134135 shutdown_connections 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- 172.21.15.26:0/2981134135 >> 172.21.15.26:0/2981134135 conn(0x7f6a3c063f30 msgr2=0x7f6a3c0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- 172.21.15.26:0/2981134135 shutdown_connections 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- 172.21.15.26:0/2981134135 wait complete. 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: monmap: 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:23.524 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 Processor -- start 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- start start 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: init 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient: _add_conns ranks=[7,5,3,0,4,6,1,2,8] 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient(hunting): picked mon.noname-i con 0x7f6a3c098170 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient(hunting): picked mon.noname-h con 0x7f6a3c09b560 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient(hunting): picked mon.noname-g con 0x7f6a3c135f20 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c06c1d0 con 0x7f6a3c135f20 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c06d160 con 0x7f6a3c09b560 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c134d20 con 0x7f6a3c098170 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.504+0000 7f6a43944700 10 monclient(hunting): authenticate will time out at 1647.939909s 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a41ee1700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f6a3c135f20 0x7f6a3c1483e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:53232/0 (socket says 172.21.15.26:53232) 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a416e0700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f6a3c098170 0x7f6a3c133f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:51208/0 (socket says 172.21.15.26:51208) 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a40edf700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f6a3c09b560 0x7f6a3c134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:41392/0 (socket says 172.21.15.26:41392) 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a41ee1700 1 -- 172.21.15.26:0/2543583950 learned_addr learned my addr 172.21.15.26:0/2543583950 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017100354 0 0) 0x7f6a3c134d20 con 0x7f6a3c098170 2024-09-15T03:58:23.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): my global_id is 234115 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a20003570 con 0x7f6a3c098170 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235954593 0 0) 0x7f6a3c06c1d0 con 0x7f6a3c135f20 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): my global_id is 204357 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c134d20 con 0x7f6a3c135f20 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309938063 0 0) 0x7f6a3c06d160 con 0x7f6a3c09b560 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): my global_id is 5864 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c06c1d0 con 0x7f6a3c09b560 2024-09-15T03:58:23.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3113847742 0 0) 0x7f6a3c134d20 con 0x7f6a3c135f20 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a3c06d160 con 0x7f6a3c135f20 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2047757522 0 0) 0x7f6a20003570 con 0x7f6a3c098170 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a3c134d20 con 0x7f6a3c098170 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a34001dd0 con 0x7f6a3c135f20 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): dump: 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a2c001830 con 0x7f6a3c098170 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): dump: 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.530 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 416625261 0 0) 0x7f6a3c06c1d0 con 0x7f6a3c09b560 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a20003570 con 0x7f6a3c09b560 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a38001960 con 0x7f6a3c09b560 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient(hunting): dump: 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3346816483 0 0) 0x7f6a3c06d160 con 0x7f6a3c135f20 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.122:6792/0 conn(0x7f6a3c098170 legacy=0x7f6a3c133f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.122:6791/0 conn(0x7f6a3c09b560 legacy=0x7f6a3c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 monclient: found mon.g 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c06bef0 con 0x7f6a3c135f20 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 5 monclient: authenticate success, global_id 204357 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 10 monclient: _renew_subs 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c09afe0 con 0x7f6a3c135f20 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 10 monclient: _renew_subs 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T03:58:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.122:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c06c8d0 con 0x7f6a3c135f20 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6a34002360 con 0x7f6a3c135f20 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a34002890 con 0x7f6a3c135f20 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a32ffd700 4 set_mon_vals no callback set 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 10 monclient: dump: 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6a1c006590 con 0x7f6a1c002840 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.508+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6a340029f0 con 0x7f6a3c135f20 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.3 v1:172.21.15.122:6790/0 8 ==== osd_map(630..630 src has 435..630) v4 ==== 3959+0+0 (unknown 1257062899 0 0) 0x7f6a34044190 con 0x7f6a3c135f20 2024-09-15T03:58:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062861039 0 0) 0x7f6a1c006590 con 0x7f6a1c002840 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient(hunting): my global_id is 204347 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c06d160 con 0x7f6a1c002840 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a40edf700 10 monclient: get_auth_request con 0x7f6a20038070 auth_method 0 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2408870180 0 0) 0x7f6a3c06d160 con 0x7f6a1c002840 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a1c006590 con 0x7f6a1c002840 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a2c004f00 con 0x7f6a1c002840 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient: dump: 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.536 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2299620932 0 0) 0x7f6a1c006590 con 0x7f6a1c002840 2024-09-15T03:58:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a1c00a130 con 0x7f6a1c002840 2024-09-15T03:58:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6a2c007190 con 0x7f6a1c002840 2024-09-15T03:58:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.26:6790/0 conn(0x7f6a1c002840 legacy=0x7f6a1c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.512+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.26:6790/0 conn(0x7f6a1c002840 legacy=0x7f6a1c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a43944700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6a1c009640 con 0x7f6a1c0090b0 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879773534 0 0) 0x7f6a1c009640 con 0x7f6a1c0090b0 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 10 monclient(hunting): my global_id is 204356 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a1c006590 con 0x7f6a1c0090b0 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1407700035 0 0) 0x7f6a1c006590 con 0x7f6a1c0090b0 2024-09-15T03:58:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a1c009640 con 0x7f6a1c0090b0 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6a2c005990 con 0x7f6a1c0090b0 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.524+0000 7f6a317fa700 10 monclient: dump: 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 931069810 0 0) 0x7f6a1c009640 con 0x7f6a1c0090b0 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6a1c0098f0 con 0x7f6a1c0090b0 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 88087777) 0x7f6a2c006f10 con 0x7f6a1c0090b0 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.26:6790/0 conn(0x7f6a1c0090b0 legacy=0x7f6a1c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a317fa700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.26:6790/0 conn(0x7f6a1c0090b0 legacy=0x7f6a1c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:23.543 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.544 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.546 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.547 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.550 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.551 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.552 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:23.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.554 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.555 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:23.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.560 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:23.561 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:23.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.122:6812/20895 conn(0x7f6a20038070 legacy=0x7f6a2003a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 10 monclient: shutdown 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 >> v1:172.21.15.122:6790/0 conn(0x7f6a3c135f20 legacy=0x7f6a3c1483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a41ee1700 1 -- 172.21.15.26:0/2543583950 reap_dead start 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a41ee1700 1 -- 172.21.15.26:0/2543583950 reap_dead start 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 shutdown_connections 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 >> 172.21.15.26:0/2543583950 conn(0x7f6a3c063f30 msgr2=0x7f6a3c06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 shutdown_connections 2024-09-15T03:58:23.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.528+0000 7f6a43944700 1 -- 172.21.15.26:0/2543583950 wait complete. 2024-09-15T03:58:23.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:23.564 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:58:23.564 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:58:23.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.668+0000 7f3e88f04700 10 monclient: get_monmap_and_config 2024-09-15T03:58:23.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.668+0000 7f3e88f04700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.668+0000 7f3e88f04700 10 monclient: monmap: 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:23.674179+0000 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:23.674179+0000 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.678 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.679 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.679 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.679 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.668+0000 7f3e88f04700 1 Processor -- start 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.668+0000 7f3e88f04700 1 -- start start 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.668+0000 7f3e88f04700 10 monclient: init 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient: _add_conns ranks=[0,8,3,6,1,2,7,5,4] 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient(hunting): picked mon.noname-a con 0x7f3e84132b40 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient(hunting): picked mon.noname-e con 0x7f3e8409faa0 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient(hunting): picked mon.noname-g con 0x7f3e84138d40 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8404ca10 con 0x7f3e84132b40 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8409d5d0 con 0x7f3e84138d40 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e84133620 con 0x7f3e8409faa0 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient(hunting): authenticate will time out at 1648.104443s 2024-09-15T03:58:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e837fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3e84132b40 0x7f3e84132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44758/0 (socket says 172.21.15.26:44758) 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e82ffd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f3e8409faa0 0x7f3e8409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33462/0 (socket says 172.21.15.26:33462) 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e837fe700 1 -- 172.21.15.26:0/559271057 learned_addr learned my addr 172.21.15.26:0/559271057 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787677068 0 0) 0x7f3e84133620 con 0x7f3e8409faa0 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): my global_id is 135260 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e70003570 con 0x7f3e8409faa0 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573563128 0 0) 0x7f3e8409d5d0 con 0x7f3e84138d40 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): my global_id is 204366 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e84133620 con 0x7f3e84138d40 2024-09-15T03:58:23.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637409657 0 0) 0x7f3e8404ca10 con 0x7f3e84132b40 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient(hunting): my global_id is 194553 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8409d5d0 con 0x7f3e84132b40 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1925780738 0 0) 0x7f3e70003570 con 0x7f3e8409faa0 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 >> v1:172.21.15.122:6790/0 conn(0x7f3e84138d40 legacy=0x7f3e8413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 >> v1:172.21.15.26:6789/0 conn(0x7f3e84132b40 legacy=0x7f3e84132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 monclient: found mon.noname-e 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840a0cc0 con 0x7f3e8409faa0 2024-09-15T03:58:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 5 monclient: authenticate success, global_id 135260 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e6c0017b0 con 0x7f3e8409faa0 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: dump: 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.685 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3e6c001a20 con 0x7f3e8409faa0 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 1 -- 172.21.15.26:0/559271057 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e6c004b70 con 0x7f3e8409faa0 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 4 set_mon_vals no callback set 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e827fc700 10 monclient: dump: 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.687 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.688 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 10 monclient: shutdown 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- 172.21.15.26:0/559271057 >> v1:172.21.15.26:6793/0 conn(0x7f3e8409faa0 legacy=0x7f3e8409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- 172.21.15.26:0/559271057 shutdown_connections 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- 172.21.15.26:0/559271057 >> 172.21.15.26:0/559271057 conn(0x7f3e8406e370 msgr2=0x7f3e840a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- 172.21.15.26:0/559271057 shutdown_connections 2024-09-15T03:58:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.672+0000 7f3e88f04700 1 -- 172.21.15.26:0/559271057 wait complete. 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: monmap: 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 1 Processor -- start 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 1 -- start start 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: init 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: _add_conns ranks=[5,6,2,0,1,7,4,3,8] 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient(hunting): picked mon.noname-h con 0x7f3e8409faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient(hunting): picked mon.noname-d con 0x7f3e84132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient(hunting): picked mon.noname-b con 0x7f3e84138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:23.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8411d990 con 0x7f3e84138d40 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8409ccc0 con 0x7f3e8409faa0 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8409cfa0 con 0x7f3e84132b40 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient(hunting): authenticate will time out at 1648.110065s 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e82ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f3e84132b40 0x7f3e841473e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43776/0 (socket says 172.21.15.26:43776) 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e82ffd700 1 -- 172.21.15.26:0/1375178274 learned_addr learned my addr 172.21.15.26:0/1375178274 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765000957 0 0) 0x7f3e8409cfa0 con 0x7f3e84132b40 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): my global_id is 194577 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e64003570 con 0x7f3e84132b40 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163748078 0 0) 0x7f3e8411d990 con 0x7f3e84138d40 2024-09-15T03:58:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): my global_id is 204365 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8409cfa0 con 0x7f3e84138d40 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822943462 0 0) 0x7f3e8409ccc0 con 0x7f3e8409faa0 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): my global_id is 5873 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8411d990 con 0x7f3e8409faa0 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1252924710 0 0) 0x7f3e64003570 con 0x7f3e84132b40 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e8409ccc0 con 0x7f3e84132b40 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e6c001770 con 0x7f3e84132b40 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): dump: 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1727750801 0 0) 0x7f3e8409cfa0 con 0x7f3e84138d40 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e64003570 con 0x7f3e84138d40 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e7c001d00 con 0x7f3e84138d40 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient(hunting): dump: 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 84092419 0 0) 0x7f3e8411d990 con 0x7f3e8409faa0 2024-09-15T03:58:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e8409cfa0 con 0x7f3e8409faa0 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1725858267 0 0) 0x7f3e8409ccc0 con 0x7f3e84132b40 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.122:6791/0 conn(0x7f3e8409faa0 legacy=0x7f3e841423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.26:6790/0 conn(0x7f3e84138d40 legacy=0x7f3e8414ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 monclient: found mon.d 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e841377a0 con 0x7f3e84132b40 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e7b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 5 monclient: authenticate success, global_id 194577 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: _renew_subs 2024-09-15T03:58:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.676+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8411cd30 con 0x7f3e84132b40 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e88f04700 10 monclient: _renew_subs 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e88f04700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e84137e60 con 0x7f3e84132b40 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3e6c001c70 con 0x7f3e84132b40 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e6c005430 con 0x7f3e84132b40 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e81ffb700 4 set_mon_vals no callback set 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: dump: 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.703 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3e6c00fdf0 con 0x7f3e84132b40 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e88f04700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(630..630 src has 435..630) v4 ==== 3959+0+0 (unknown 1257062899 0 0) 0x7f3e6c044260 con 0x7f3e84132b40 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868844644 0 0) 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient(hunting): my global_id is 204358 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8409ccc0 con 0x7f3e68002840 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e83fff700 10 monclient: get_auth_request con 0x7f3e64037f50 auth_method 0 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1395292747 0 0) 0x7f3e8409ccc0 con 0x7f3e68002840 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:58:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e740015a0 con 0x7f3e68002840 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: dump: 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1251351172 0 0) 0x7f3e68006590 con 0x7f3e68002840 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e6800a130 con 0x7f3e68002840 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3e74004f00 con 0x7f3e68002840 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.26:6791/0 conn(0x7f3e68002840 legacy=0x7f3e68004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.680+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.26:6791/0 conn(0x7f3e68002840 legacy=0x7f3e68004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.688+0000 7f3e88f04700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.688+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3e68007520 con 0x7f3e680068d0 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777205047 0 0) 0x7f3e68007520 con 0x7f3e680068d0 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient(hunting): my global_id is 204367 2024-09-15T03:58:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e68006590 con 0x7f3e680068d0 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 673624905 0 0) 0x7f3e68006590 con 0x7f3e680068d0 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e68007520 con 0x7f3e680068d0 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3e74005710 con 0x7f3e680068d0 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient: dump: 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3533292365 0 0) 0x7f3e68007520 con 0x7f3e680068d0 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3e68008760 con 0x7f3e680068d0 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 2073205242) 0x7f3e740058c0 con 0x7f3e680068d0 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.26:6791/0 conn(0x7f3e680068d0 legacy=0x7f3e68006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e7b7fe700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.26:6791/0 conn(0x7f3e680068d0 legacy=0x7f3e68006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:23.712 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:23.713 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:23.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:23.715 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:23.716 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.717 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:23.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.724 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:23.729 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:23.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.731 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:23.732 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.122:6812/20895 conn(0x7f3e64037f50 legacy=0x7f3e6403a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 10 monclient: shutdown 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 >> v1:172.21.15.26:6792/0 conn(0x7f3e84132b40 legacy=0x7f3e841473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e83fff700 1 -- 172.21.15.26:0/1375178274 reap_dead start 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 shutdown_connections 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 >> 172.21.15.26:0/1375178274 conn(0x7f3e8406e370 msgr2=0x7f3e8409da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 shutdown_connections 2024-09-15T03:58:23.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.692+0000 7f3e88f04700 1 -- 172.21.15.26:0/1375178274 wait complete. 2024-09-15T03:58:23.734 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:23.734 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:58:23.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.840+0000 7f47a9a6d700 10 monclient: get_monmap_and_config 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.840+0000 7f47a9a6d700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.840+0000 7f47a9a6d700 10 monclient: monmap: 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:23.847446+0000 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:23.847446+0000 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 1 Processor -- start 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 1 -- start start 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient: init 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient: _add_conns ranks=[3,5,8,2,1,7,4,6,0] 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient(hunting): picked mon.noname-g con 0x7f47a4132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient(hunting): picked mon.noname-h con 0x7f47a409faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient(hunting): picked mon.noname-e con 0x7f47a4138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a404ca10 con 0x7f47a4132b40 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a409d5d0 con 0x7f47a409faa0 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a4133620 con 0x7f47a4138d40 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient(hunting): authenticate will time out at 1648.277730s 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a926c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f47a4138d40 0x7f47a413b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33470/0 (socket says 172.21.15.26:33470) 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a926c700 1 -- 172.21.15.26:0/3291568525 learned_addr learned my addr 172.21.15.26:0/3291568525 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283237437 0 0) 0x7f47a404ca10 con 0x7f47a4132b40 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): my global_id is 204375 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4790003570 con 0x7f47a4132b40 2024-09-15T03:58:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423079964 0 0) 0x7f47a4133620 con 0x7f47a4138d40 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): my global_id is 135269 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a404ca10 con 0x7f47a4138d40 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181484177 0 0) 0x7f47a409d5d0 con 0x7f47a409faa0 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient(hunting): my global_id is 5882 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a4133620 con 0x7f47a409faa0 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850709919 0 0) 0x7f4790003570 con 0x7f47a4132b40 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 >> v1:172.21.15.26:6793/0 conn(0x7f47a4138d40 legacy=0x7f47a413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 >> v1:172.21.15.122:6791/0 conn(0x7f47a409faa0 legacy=0x7f47a409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 monclient: found mon.noname-g 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a40a0cc0 con 0x7f47a4132b40 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47940017b0 con 0x7f47a4132b40 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 5 monclient: authenticate success, global_id 204375 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: dump: 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4794001a20 con 0x7f47a4132b40 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 4 set_mon_vals no callback set 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 1 -- 172.21.15.26:0/3291568525 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4794001f50 con 0x7f47a4132b40 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a37fe700 10 monclient: dump: 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.861 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 10 monclient: shutdown 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.844+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3291568525 >> v1:172.21.15.122:6790/0 conn(0x7f47a4132b40 legacy=0x7f47a4132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3291568525 shutdown_connections 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3291568525 >> 172.21.15.26:0/3291568525 conn(0x7f47a406e370 msgr2=0x7f47a40a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3291568525 shutdown_connections 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3291568525 wait complete. 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: build_initial_monmap 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: monmap: 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 Processor -- start 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- start start 2024-09-15T03:58:23.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: init 2024-09-15T03:58:23.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:23.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: _add_conns ranks=[2,5,1,4,7,8,3,0,6] 2024-09-15T03:58:23.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient(hunting): picked mon.noname-b con 0x7f47a409faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:23.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient(hunting): picked mon.noname-h con 0x7f47a4132b40 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:23.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient(hunting): picked mon.noname-f con 0x7f47a4138d40 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a4137a20 con 0x7f47a4138d40 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a409ccc0 con 0x7f47a409faa0 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a409cfa0 con 0x7f47a4132b40 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient(hunting): authenticate will time out at 1648.281659s 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a8a6b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f47a409faa0 0x7f47a41354c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52750/0 (socket says 172.21.15.26:52750) 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a8a6b700 1 -- 172.21.15.26:0/3276148332 learned_addr learned my addr 172.21.15.26:0/3276148332 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405695338 0 0) 0x7f47a409ccc0 con 0x7f47a409faa0 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): my global_id is 204374 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4784003570 con 0x7f47a409faa0 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590742759 0 0) 0x7f47a4137a20 con 0x7f47a4138d40 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): my global_id is 204337 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a409ccc0 con 0x7f47a4138d40 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104369192 0 0) 0x7f47a409cfa0 con 0x7f47a4132b40 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): my global_id is 5891 2024-09-15T03:58:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a4137a20 con 0x7f47a4132b40 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1143639070 0 0) 0x7f47a409ccc0 con 0x7f47a4138d40 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47a409cfa0 con 0x7f47a4138d40 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f479c001d10 con 0x7f47a4138d40 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): dump: 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2002403703 0 0) 0x7f4784003570 con 0x7f47a409faa0 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47a409ccc0 con 0x7f47a409faa0 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4794001770 con 0x7f47a409faa0 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): dump: 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.874 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1153003130 0 0) 0x7f47a4137a20 con 0x7f47a4132b40 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4784003570 con 0x7f47a4132b40 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f478c001a60 con 0x7f47a4132b40 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient(hunting): dump: 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3479201834 0 0) 0x7f47a409ccc0 con 0x7f47a409faa0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.122:6791/0 conn(0x7f47a4132b40 legacy=0x7f47a4135bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.122:6789/0 conn(0x7f47a4138d40 legacy=0x7f47a41362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 monclient: found mon.b 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a4137020 con 0x7f47a409faa0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: _finish_auth 0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 5 monclient: authenticate success, global_id 204374 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: _renew_subs 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a4137530 con 0x7f47a409faa0 2024-09-15T03:58:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: _renew_subs 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a4132560 con 0x7f47a409faa0 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4794002590 con 0x7f47a409faa0 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4794002a40 con 0x7f47a409faa0 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a2ffd700 4 set_mon_vals no callback set 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.848+0000 7f47a0ff9700 10 monclient: dump: 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a9a6d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4788006590 con 0x7f4788002840 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4794002ba0 con 0x7f47a409faa0 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(630..630 src has 435..630) v4 ==== 3959+0+0 (unknown 1257062899 0 0) 0x7f4794043f90 con 0x7f47a409faa0 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662240731 0 0) 0x7f4788006590 con 0x7f4788002840 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient(hunting): my global_id is 194586 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a409ccc0 con 0x7f4788002840 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a926c700 10 monclient: get_auth_request con 0x7f47840380e0 auth_method 0 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1777066758 0 0) 0x7f47a409ccc0 con 0x7f4788002840 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4788006590 con 0x7f4788002840 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f478c0021b0 con 0x7f4788002840 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient: dump: 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.881 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3479011238 0 0) 0x7f4788006590 con 0x7f4788002840 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f478800a130 con 0x7f4788002840 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f478c002750 con 0x7f4788002840 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.26:6792/0 conn(0x7f4788002840 legacy=0x7f4788004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.852+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.26:6792/0 conn(0x7f4788002840 legacy=0x7f4788004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a9a6d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4788009640 con 0x7f47880090b0 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579635588 0 0) 0x7f4788009640 con 0x7f47880090b0 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient(hunting): my global_id is 194595 2024-09-15T03:58:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4788006590 con 0x7f47880090b0 2024-09-15T03:58:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1384362955 0 0) 0x7f4788006590 con 0x7f47880090b0 2024-09-15T03:58:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4788009640 con 0x7f47880090b0 2024-09-15T03:58:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f478c002bf0 con 0x7f47880090b0 2024-09-15T03:58:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient: dump: 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:23.885 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3924194562 0 0) 0x7f4788009640 con 0x7f47880090b0 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f47880098f0 con 0x7f47880090b0 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 945527631) 0x7f478c003480 con 0x7f47880090b0 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.26:6792/0 conn(0x7f47880090b0 legacy=0x7f4788008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.860+0000 7f47a0ff9700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.26:6792/0 conn(0x7f47880090b0 legacy=0x7f4788008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:23.886 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:23.887 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.888 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.889 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.890 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:23.891 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:23.892 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.900 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:23.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.904 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.906 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:23.907 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:23.908 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:23.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.122:6812/20895 conn(0x7f47840380e0 legacy=0x7f478403a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 10 monclient: shutdown 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 >> v1:172.21.15.26:6790/0 conn(0x7f47a409faa0 legacy=0x7f47a41354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a926c700 1 -- 172.21.15.26:0/3276148332 reap_dead start 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a926c700 1 -- 172.21.15.26:0/3276148332 reap_dead start 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 shutdown_connections 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 >> 172.21.15.26:0/3276148332 conn(0x7f47a406e370 msgr2=0x7f47a409f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 shutdown_connections 2024-09-15T03:58:23.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:23.864+0000 7f47a9a6d700 1 -- 172.21.15.26:0/3276148332 wait complete. 2024-09-15T03:58:23.910 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:23.910 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:58:23.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.020+0000 7fafc1670700 10 monclient: get_monmap_and_config 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.020+0000 7fafc1670700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.020+0000 7fafc1670700 10 monclient: monmap: 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:24.025447+0000 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:24.025447+0000 2024-09-15T03:58:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.020+0000 7fafc1670700 1 Processor -- start 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.020+0000 7fafc1670700 1 -- start start 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.020+0000 7fafc1670700 10 monclient: init 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient: _add_conns ranks=[4,2,6,7,3,1,8,0,5] 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient(hunting): picked mon.noname-c con 0x7fafbc070fb0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient(hunting): picked mon.noname-b con 0x7fafbc09a9f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient(hunting): picked mon.noname-d con 0x7fafbc135b80 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc04ca10 con 0x7fafbc09a9f0 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc09afa0 con 0x7fafbc070fb0 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc130660 con 0x7fafbc135b80 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient(hunting): authenticate will time out at 1648.456484s 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbbfff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fafbc070fb0 0x7fafbc12f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:58000/0 (socket says 172.21.15.26:58000) 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbbfff700 1 -- 172.21.15.26:0/1808445508 learned_addr learned my addr 172.21.15.26:0/1808445508 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc0e6f700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fafbc135b80 0x7fafbc137f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43804/0 (socket says 172.21.15.26:43804) 2024-09-15T03:58:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafb3fff700 1 --1- 172.21.15.26:0/1808445508 >> v1:172.21.15.26:6790/0 conn(0x7fafbc09a9f0 0x7fafbc12ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52760/0 (socket says 172.21.15.26:52760) 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488771899 0 0) 0x7fafbc09afa0 con 0x7fafbc070fb0 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): my global_id is 204376 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa8003570 con 0x7fafbc070fb0 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797829010 0 0) 0x7fafbc130660 con 0x7fafbc135b80 2024-09-15T03:58:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): my global_id is 194604 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafbc09afa0 con 0x7fafbc135b80 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876516094 0 0) 0x7fafbc04ca10 con 0x7fafbc09a9f0 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient(hunting): my global_id is 204383 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafbc130660 con 0x7fafbc09a9f0 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576326829 0 0) 0x7fafbc09afa0 con 0x7fafbc135b80 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 >> v1:172.21.15.26:6791/0 conn(0x7fafbc070fb0 legacy=0x7fafbc12f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 >> v1:172.21.15.26:6790/0 conn(0x7fafbc09a9f0 legacy=0x7fafbc12ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 monclient: found mon.noname-d 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc09f600 con 0x7fafbc135b80 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 5 monclient: authenticate success, global_id 194604 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafb4001b10 con 0x7fafbc135b80 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: dump: 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fafb4001d40 con 0x7fafbc135b80 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 1 -- 172.21.15.26:0/1808445508 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafb4004bb0 con 0x7fafbc135b80 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 4 set_mon_vals no callback set 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafbb7fe700 10 monclient: dump: 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 10 monclient: shutdown 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- 172.21.15.26:0/1808445508 >> v1:172.21.15.26:6792/0 conn(0x7fafbc135b80 legacy=0x7fafbc137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- 172.21.15.26:0/1808445508 shutdown_connections 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- 172.21.15.26:0/1808445508 >> 172.21.15.26:0/1808445508 conn(0x7fafbc0a2ea0 msgr2=0x7fafbc09eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- 172.21.15.26:0/1808445508 shutdown_connections 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.024+0000 7fafc1670700 1 -- 172.21.15.26:0/1808445508 wait complete. 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient: monmap: 2024-09-15T03:58:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.044 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 1 Processor -- start 2024-09-15T03:58:24.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 1 -- start start 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient: init 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient: _add_conns ranks=[3,7,8,0,2,1,4,6,5] 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient(hunting): picked mon.noname-g con 0x7fafbc09a9f0 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient(hunting): picked mon.noname-i con 0x7fafbc135b80 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient(hunting): picked mon.noname-e con 0x7fafbc132590 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc132d40 con 0x7fafbc09a9f0 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc09a0b0 con 0x7fafbc135b80 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafbc133d90 con 0x7fafbc132590 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc1670700 10 monclient(hunting): authenticate will time out at 1648.462935s 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc0e6f700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fafbc132590 0x7fafbc148040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33484/0 (socket says 172.21.15.26:33484) 2024-09-15T03:58:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafc0e6f700 1 -- 172.21.15.26:0/2104393538 learned_addr learned my addr 172.21.15.26:0/2104393538 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb3fff700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6792/0 conn(0x7fafbc135b80 legacy=0x7fafbc130fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb3fff700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6792/0 conn(0x7fafbc135b80 legacy=0x7fafbc130fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb3fff700 1 --1- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6792/0 conn(0x7fafbc135b80 0x7fafbc130fd0 :-1 s=OPENED pgs=394 cs=1 l=1).handle_message read tag failed 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb3fff700 1 --1- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6792/0 conn(0x7fafbc135b80 0x7fafbc130fd0 :-1 s=OPENED pgs=394 cs=1 l=1).fault on lossy channel, failing 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.122:6792/0 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471878138 0 0) 0x7fafbc133d90 con 0x7fafbc132590 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): my global_id is 135278 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf9c003570 con 0x7fafbc132590 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865025844 0 0) 0x7fafbc132d40 con 0x7fafbc09a9f0 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 10 monclient(hunting): my global_id is 204384 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.028+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafbc133d90 con 0x7fafbc09a9f0 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1523255059 0 0) 0x7faf9c003570 con 0x7fafbc132590 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafbc132d40 con 0x7fafbc132590 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafb4001ad0 con 0x7fafbc132590 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): dump: 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2297313682 0 0) 0x7fafbc133d90 con 0x7fafbc09a9f0 2024-09-15T03:58:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf9c003570 con 0x7fafbc09a9f0 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafac001570 con 0x7fafbc09a9f0 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): dump: 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1622975068 0 0) 0x7fafbc132d40 con 0x7fafbc132590 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6792/0 conn(0x7fafbc135b80 legacy=0x7fafbc130fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6790/0 conn(0x7fafbc09a9f0 legacy=0x7fafbc1308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 monclient: found mon.e 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc132a60 con 0x7fafbc132590 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 5 monclient: authenticate success, global_id 135278 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 10 monclient: _renew_subs 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc133020 con 0x7fafbc132590 2024-09-15T03:58:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 10 monclient: _renew_subs 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc0998e0 con 0x7fafbc132590 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fafb4005090 con 0x7fafbc132590 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafb40055c0 con 0x7fafbc132590 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafbaffd700 4 set_mon_vals no callback set 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: dump: 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fafa0006590 con 0x7fafa0002840 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fafb40057a0 con 0x7fafbc132590 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(630..630 src has 435..630) v4 ==== 3959+0+0 (unknown 1257062899 0 0) 0x7fafb4044470 con 0x7fafbc132590 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595453012 0 0) 0x7fafa0006590 con 0x7fafa0002840 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient(hunting): my global_id is 135287 2024-09-15T03:58:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafbc132d40 con 0x7fafa0002840 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb3fff700 10 monclient: get_auth_request con 0x7faf9c037bb0 auth_method 0 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3395132126 0 0) 0x7fafbc132d40 con 0x7fafa0002840 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafa0006590 con 0x7fafa0002840 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafac004b70 con 0x7fafa0002840 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 10 monclient: dump: 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 624873640 0 0) 0x7fafa0006590 con 0x7fafa0002840 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.032+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafa000a130 con 0x7fafa0002840 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.036+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fafac005000 con 0x7fafa0002840 2024-09-15T03:58:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.036+0000 7fafb8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.036+0000 7fafb8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.036+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.26:6793/0 conn(0x7fafa0002840 legacy=0x7fafa0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.036+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.26:6793/0 conn(0x7fafa0002840 legacy=0x7fafa0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafc1670700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fafa0009640 con 0x7fafa00090b0 2024-09-15T03:58:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240552800 0 0) 0x7fafa0009640 con 0x7fafa00090b0 2024-09-15T03:58:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafb8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafb8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafb8ff9700 10 monclient(hunting): my global_id is 135296 2024-09-15T03:58:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.040+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa0006590 con 0x7fafa00090b0 2024-09-15T03:58:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2319396515 0 0) 0x7fafa0006590 con 0x7fafa00090b0 2024-09-15T03:58:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafa0009640 con 0x7fafa00090b0 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fafac005620 con 0x7fafa00090b0 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 10 monclient: dump: 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2223116432 0 0) 0x7fafa0009640 con 0x7fafa00090b0 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fafa00098b0 con 0x7fafa00090b0 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1811644933) 0x7fafac005990 con 0x7fafa00090b0 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:24.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.26:6793/0 conn(0x7fafa00090b0 legacy=0x7fafa0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafb8ff9700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.26:6793/0 conn(0x7fafa00090b0 legacy=0x7fafa0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:24.068 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:24.069 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:24.070 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:24.071 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:24.072 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.073 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:24.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.076 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:24.077 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:24.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.079 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.081 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:24.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.086 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.088 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:24.089 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.122:6812/20895 conn(0x7faf9c037bb0 legacy=0x7faf9c03a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 10 monclient: shutdown 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 >> v1:172.21.15.26:6793/0 conn(0x7fafbc132590 legacy=0x7fafbc148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc0e6f700 1 -- 172.21.15.26:0/2104393538 reap_dead start 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc0e6f700 1 -- 172.21.15.26:0/2104393538 reap_dead start 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 shutdown_connections 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 >> 172.21.15.26:0/2104393538 conn(0x7fafbc0a2ea0 msgr2=0x7fafbc0a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 shutdown_connections 2024-09-15T03:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.044+0000 7fafc1670700 1 -- 172.21.15.26:0/2104393538 wait complete. 2024-09-15T03:58:24.092 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:24.092 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:58:24.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:58:24.116 INFO:tasks.rados.rados.0.smithi122.stdout:2799: done (1 left) 2024-09-15T03:58:24.116 INFO:tasks.rados.rados.0.smithi122.stdout:2800: done (0 left) 2024-09-15T03:58:24.116 INFO:tasks.rados.rados.0.smithi122.stdout:2801: rollback oid 19 current snap is 314 2024-09-15T03:58:24.116 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 288 2024-09-15T03:58:24.116 INFO:tasks.rados.rados.0.smithi122.stdout:2802: copy_from oid 18 from oid 30 current snap is 314 2024-09-15T03:58:24.117 INFO:tasks.rados.rados.0.smithi122.stdout:2803: rollback oid 43 current snap is 314 2024-09-15T03:58:24.117 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 313 2024-09-15T03:58:24.117 INFO:tasks.rados.rados.0.smithi122.stdout:2804: read oid 46 snap -1 2024-09-15T03:58:24.117 INFO:tasks.rados.rados.0.smithi122.stdout:2804: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:24.117 INFO:tasks.rados.rados.0.smithi122.stdout:2805: delete oid 37 current snap is 314 2024-09-15T03:58:24.120 INFO:tasks.rados.rados.0.smithi122.stdout:2803: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:58:24.120 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 1706 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T03:58:24.122 INFO:tasks.rados.rados.0.smithi122.stdout:2803: done (4 left) 2024-09-15T03:58:24.122 INFO:tasks.rados.rados.0.smithi122.stdout:2805: done (3 left) 2024-09-15T03:58:24.122 INFO:tasks.rados.rados.0.smithi122.stdout:2806: snap_remove snap 295 2024-09-15T03:58:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: get_monmap_and_config 2024-09-15T03:58:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: monmap: 2024-09-15T03:58:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.204 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:24.200451+0000 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:24.200451+0000 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.205 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 1 Processor -- start 2024-09-15T03:58:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 1 -- start start 2024-09-15T03:58:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: init 2024-09-15T03:58:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: _add_conns ranks=[1,6,2,8,3,7,4,0,5] 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient(hunting): picked mon.noname-f con 0x7fe108132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient(hunting): picked mon.noname-d con 0x7fe10809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient(hunting): picked mon.noname-b con 0x7fe108138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10804ca10 con 0x7fe108132b40 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10809d5d0 con 0x7fe108138d40 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe108133620 con 0x7fe10809faa0 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient(hunting): authenticate will time out at 1648.630675s 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe106ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fe10809faa0 0x7fe10809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43820/0 (socket says 172.21.15.26:43820) 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe106ffd700 1 -- 172.21.15.26:0/3394877859 learned_addr learned my addr 172.21.15.26:0/3394877859 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe107fff700 1 --1- 172.21.15.26:0/3394877859 >> v1:172.21.15.26:6790/0 conn(0x7fe108138d40 0x7fe10813b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52766/0 (socket says 172.21.15.26:52766) 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989686574 0 0) 0x7fe108133620 con 0x7fe10809faa0 2024-09-15T03:58:24.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): my global_id is 194613 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0003570 con 0x7fe10809faa0 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357981941 0 0) 0x7fe10809d5d0 con 0x7fe108138d40 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): my global_id is 204392 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe108133620 con 0x7fe108138d40 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321868351 0 0) 0x7fe10804ca10 con 0x7fe108132b40 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient(hunting): my global_id is 204346 2024-09-15T03:58:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10809d5d0 con 0x7fe108132b40 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2584294421 0 0) 0x7fe0f0003570 con 0x7fe10809faa0 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 >> v1:172.21.15.26:6790/0 conn(0x7fe108138d40 legacy=0x7fe10813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 >> v1:172.21.15.122:6789/0 conn(0x7fe108132b40 legacy=0x7fe108132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 monclient: found mon.noname-d 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080a0cc0 con 0x7fe10809faa0 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0fc0017b0 con 0x7fe10809faa0 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 5 monclient: authenticate success, global_id 194613 2024-09-15T03:58:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: dump: 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.214 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0fc001a20 con 0x7fe10809faa0 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 1 -- 172.21.15.26:0/3394877859 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0fc004c30 con 0x7fe10809faa0 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe10cb8a700 4 set_mon_vals no callback set 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.196+0000 7fe1067fc700 10 monclient: dump: 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient: shutdown 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- 172.21.15.26:0/3394877859 >> v1:172.21.15.26:6792/0 conn(0x7fe10809faa0 legacy=0x7fe10809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- 172.21.15.26:0/3394877859 shutdown_connections 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- 172.21.15.26:0/3394877859 >> 172.21.15.26:0/3394877859 conn(0x7fe10806e370 msgr2=0x7fe1080a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- 172.21.15.26:0/3394877859 shutdown_connections 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- 172.21.15.26:0/3394877859 wait complete. 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient: monmap: 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 Processor -- start 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- start start 2024-09-15T03:58:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient: init 2024-09-15T03:58:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient: _add_conns ranks=[7,3,2,8,4,0,1,5,6] 2024-09-15T03:58:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient(hunting): picked mon.noname-i con 0x7fe10809faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient(hunting): picked mon.noname-g con 0x7fe108132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient(hunting): picked mon.noname-b con 0x7fe108138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10809cfa0 con 0x7fe108138d40 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10809ccc0 con 0x7fe108132b40 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10811c640 con 0x7fe10809faa0 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe10cb8a700 10 monclient(hunting): authenticate will time out at 1648.635524s 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe107fff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe108138d40 0x7fe10814ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52782/0 (socket says 172.21.15.26:52782) 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.200+0000 7fe107fff700 1 -- 172.21.15.26:0/2209829074 learned_addr learned my addr 172.21.15.26:0/2209829074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911388106 0 0) 0x7fe10809cfa0 con 0x7fe108138d40 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): my global_id is 204401 2024-09-15T03:58:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f4003570 con 0x7fe108138d40 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485294455 0 0) 0x7fe10811c640 con 0x7fe10809faa0 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): my global_id is 234124 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10809cfa0 con 0x7fe10809faa0 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150469119 0 0) 0x7fe10809ccc0 con 0x7fe108132b40 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): my global_id is 204393 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10811c640 con 0x7fe108132b40 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3815455489 0 0) 0x7fe0f4003570 con 0x7fe108138d40 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe10809ccc0 con 0x7fe108138d40 2024-09-15T03:58:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe100001d60 con 0x7fe108138d40 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): dump: 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3449007024 0 0) 0x7fe10809cfa0 con 0x7fe10809faa0 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0f4003570 con 0x7fe10809faa0 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0f8001a60 con 0x7fe10809faa0 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): dump: 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2397537010 0 0) 0x7fe10809ccc0 con 0x7fe108138d40 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6792/0 conn(0x7fe10809faa0 legacy=0x7fe108142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6790/0 conn(0x7fe108132b40 legacy=0x7fe1081473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 monclient: found mon.b 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10809b2c0 con 0x7fe108138d40 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 5 monclient: authenticate success, global_id 204401 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 10 monclient: _renew_subs 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10811c940 con 0x7fe108138d40 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 10 monclient: _renew_subs 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe108142f30 con 0x7fe108138d40 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe1000020f0 con 0x7fe108138d40 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe1000051a0 con 0x7fe108138d40 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe1057fa700 4 set_mon_vals no callback set 2024-09-15T03:58:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: dump: 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0e8006590 con 0x7fe0e8002840 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe10000fbe0 con 0x7fe108138d40 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(631..631 src has 435..631) v4 ==== 3959+0+0 (unknown 655598167 0 0) 0x7fe100044280 con 0x7fe108138d40 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856315682 0 0) 0x7fe0e8006590 con 0x7fe0e8002840 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient(hunting): my global_id is 204355 2024-09-15T03:58:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10809ccc0 con 0x7fe0e8002840 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe106ffd700 10 monclient: get_auth_request con 0x7fe0f4037c40 auth_method 0 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2799235386 0 0) 0x7fe10809ccc0 con 0x7fe0e8002840 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0e8006590 con 0x7fe0e8002840 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0f8002170 con 0x7fe0e8002840 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.204+0000 7fe0ef7fe700 10 monclient: dump: 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3398499386 0 0) 0x7fe0e8006590 con 0x7fe0e8002840 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0e800a130 con 0x7fe0e8002840 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe0f8002710 con 0x7fe0e8002840 2024-09-15T03:58:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6789/0 conn(0x7fe0e8002840 legacy=0x7fe0e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.208+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6789/0 conn(0x7fe0e8002840 legacy=0x7fe0e8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe10cb8a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0e8007520 con 0x7fe0e80068d0 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256828414 0 0) 0x7fe0e8007520 con 0x7fe0e80068d0 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 10 monclient(hunting): my global_id is 204364 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0e8006590 con 0x7fe0e80068d0 2024-09-15T03:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2421726168 0 0) 0x7fe0e8006590 con 0x7fe0e80068d0 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0e8007520 con 0x7fe0e80068d0 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0f8003000 con 0x7fe0e80068d0 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 10 monclient: dump: 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1337492171 0 0) 0x7fe0e8007520 con 0x7fe0e80068d0 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.216+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe0e8008760 con 0x7fe0e80068d0 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1927655997) 0x7fe0f80052a0 con 0x7fe0e80068d0 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe0ef7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe0ef7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6789/0 conn(0x7fe0e80068d0 legacy=0x7fe0e8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe0ef7fe700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6789/0 conn(0x7fe0e80068d0 legacy=0x7fe0e8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:24.240 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:24.241 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.242 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:24.243 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.244 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:24.247 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.249 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:24.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.253 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.254 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.256 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.257 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.258 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:24.259 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.122:6812/20895 conn(0x7fe0f4037c40 legacy=0x7fe0f403a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 10 monclient: shutdown 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe107fff700 1 -- 172.21.15.26:0/2209829074 reap_dead start 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 >> v1:172.21.15.26:6790/0 conn(0x7fe108138d40 legacy=0x7fe10814ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 shutdown_connections 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 >> 172.21.15.26:0/2209829074 conn(0x7fe10806e370 msgr2=0x7fe1080669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 shutdown_connections 2024-09-15T03:58:24.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.220+0000 7fe10cb8a700 1 -- 172.21.15.26:0/2209829074 wait complete. 2024-09-15T03:58:24.262 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:24.262 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:58:24.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:58:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: get_monmap_and_config 2024-09-15T03:58:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: monmap: 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:24.368022+0000 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:24.368022+0000 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.373 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.373 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.373 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 Processor -- start 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- start start 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: init 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: _add_conns ranks=[1,6,2,8,7,3,5,0,4] 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient(hunting): picked mon.noname-f con 0x7f0a6c132b40 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient(hunting): picked mon.noname-d con 0x7f0a6c09faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient(hunting): picked mon.noname-b con 0x7f0a6c138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c04ca10 con 0x7f0a6c132b40 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c09d5d0 con 0x7f0a6c138d40 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c133620 con 0x7f0a6c09faa0 2024-09-15T03:58:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient(hunting): authenticate will time out at 1648.797480s 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6ad9d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0a6c138d40 0x7f0a6c13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52794/0 (socket says 172.21.15.26:52794) 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a69d9b700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f0a6c09faa0 0x7f0a6c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43822/0 (socket says 172.21.15.26:43822) 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6ad9d700 1 -- 172.21.15.26:0/3319557660 learned_addr learned my addr 172.21.15.26:0/3319557660 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952134522 0 0) 0x7f0a6c133620 con 0x7f0a6c09faa0 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): my global_id is 194622 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a58003570 con 0x7f0a6c09faa0 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668147776 0 0) 0x7f0a6c09d5d0 con 0x7f0a6c138d40 2024-09-15T03:58:24.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): my global_id is 204410 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c133620 con 0x7f0a6c138d40 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629704134 0 0) 0x7f0a6c04ca10 con 0x7f0a6c132b40 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient(hunting): my global_id is 204373 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c09d5d0 con 0x7f0a6c132b40 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1305584842 0 0) 0x7f0a58003570 con 0x7f0a6c09faa0 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 >> v1:172.21.15.26:6790/0 conn(0x7f0a6c138d40 legacy=0x7f0a6c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 >> v1:172.21.15.122:6789/0 conn(0x7f0a6c132b40 legacy=0x7f0a6c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 monclient: found mon.noname-d 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0a0cc0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a540017b0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: dump: 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.379 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.380 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 5 monclient: authenticate success, global_id 194622 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a540019a0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 4 set_mon_vals no callback set 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 1 -- 172.21.15.26:0/3319557660 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a54004830 con 0x7f0a6c09faa0 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6959a700 10 monclient: dump: 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.381 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.382 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 10 monclient: shutdown 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- 172.21.15.26:0/3319557660 >> v1:172.21.15.26:6792/0 conn(0x7f0a6c09faa0 legacy=0x7f0a6c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- 172.21.15.26:0/3319557660 shutdown_connections 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- 172.21.15.26:0/3319557660 >> 172.21.15.26:0/3319557660 conn(0x7f0a6c06e370 msgr2=0x7f0a6c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- 172.21.15.26:0/3319557660 shutdown_connections 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.364+0000 7f0a6b59e700 1 -- 172.21.15.26:0/3319557660 wait complete. 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient: monmap: 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:24.383 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.384 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.385 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.385 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 1 Processor -- start 2024-09-15T03:58:24.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 1 -- start start 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient: init 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient: _add_conns ranks=[1,2,7,3,8,4,5,0,6] 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient(hunting): picked mon.noname-f con 0x7f0a6c09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient(hunting): picked mon.noname-b con 0x7f0a6c132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient(hunting): picked mon.noname-i con 0x7f0a6c138d40 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c09ccc0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c09b2f0 con 0x7f0a6c132b40 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6c1348c0 con 0x7f0a6c138d40 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a6b59e700 10 monclient(hunting): authenticate will time out at 1648.802576s 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a69d9b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0a6c132b40 0x7f0a6c09abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52806/0 (socket says 172.21.15.26:52806) 2024-09-15T03:58:24.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a69d9b700 1 -- 172.21.15.26:0/444248387 learned_addr learned my addr 172.21.15.26:0/444248387 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509577030 0 0) 0x7f0a6c09b2f0 con 0x7f0a6c132b40 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): my global_id is 204419 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a4c003570 con 0x7f0a6c132b40 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133830906 0 0) 0x7f0a6c09ccc0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): my global_id is 204382 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c09b2f0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551110562 0 0) 0x7f0a6c1348c0 con 0x7f0a6c138d40 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): my global_id is 234133 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c09ccc0 con 0x7f0a6c138d40 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 246024548 0 0) 0x7f0a4c003570 con 0x7f0a6c132b40 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a6c1348c0 con 0x7f0a6c132b40 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a54001770 con 0x7f0a6c132b40 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): dump: 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 158238343 0 0) 0x7f0a6c09ccc0 con 0x7f0a6c138d40 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a4c003570 con 0x7f0a6c138d40 2024-09-15T03:58:24.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a64001900 con 0x7f0a6c138d40 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): dump: 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.394 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 139479290 0 0) 0x7f0a6c09b2f0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a6c09ccc0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a5c001ce0 con 0x7f0a6c09faa0 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.368+0000 7f0a627fc700 10 monclient(hunting): dump: 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.395 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.396 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 525372665 0 0) 0x7f0a6c1348c0 con 0x7f0a6c132b40 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6792/0 conn(0x7f0a6c138d40 legacy=0x7f0a6c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6789/0 conn(0x7f0a6c09faa0 legacy=0x7f0a6c09a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 monclient: found mon.b 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c134ba0 con 0x7f0a6c132b40 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 5 monclient: authenticate success, global_id 204419 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 10 monclient: _renew_subs 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6c138820 con 0x7f0a6c132b40 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 10 monclient: _renew_subs 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6c135720 con 0x7f0a6c132b40 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a54004db0 con 0x7f0a6c132b40 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a54005260 con 0x7f0a6c132b40 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a68d99700 4 set_mon_vals no callback set 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: dump: 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.398 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0a50006590 con 0x7f0a50002840 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0a540054c0 con 0x7f0a6c132b40 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(631..631 src has 435..631) v4 ==== 3959+0+0 (unknown 655598167 0 0) 0x7f0a54044200 con 0x7f0a6c132b40 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292869199 0 0) 0x7f0a50006590 con 0x7f0a50002840 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient(hunting): my global_id is 204402 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c1348c0 con 0x7f0a50002840 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a6ad9d700 10 monclient: get_auth_request con 0x7f0a4c0380a0 auth_method 0 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1711235508 0 0) 0x7f0a6c1348c0 con 0x7f0a50002840 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a50006590 con 0x7f0a50002840 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a5c002470 con 0x7f0a50002840 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: dump: 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.401 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2335443232 0 0) 0x7f0a50006590 con 0x7f0a50002840 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a5000a130 con 0x7f0a50002840 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0a5c004770 con 0x7f0a50002840 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6790/0 conn(0x7f0a50002840 legacy=0x7f0a50004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.372+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6790/0 conn(0x7f0a50002840 legacy=0x7f0a50004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0a50009640 con 0x7f0a500090b0 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296167323 0 0) 0x7f0a50009640 con 0x7f0a500090b0 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient(hunting): my global_id is 204411 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a50006590 con 0x7f0a500090b0 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1833264490 0 0) 0x7f0a50006590 con 0x7f0a500090b0 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a50009640 con 0x7f0a500090b0 2024-09-15T03:58:24.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a5c002fc0 con 0x7f0a500090b0 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient: dump: 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.405 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3849931274 0 0) 0x7f0a50009640 con 0x7f0a500090b0 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a500098f0 con 0x7f0a500090b0 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1418155936) 0x7f0a5c004980 con 0x7f0a500090b0 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6790/0 conn(0x7f0a500090b0 legacy=0x7f0a50008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a627fc700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6790/0 conn(0x7f0a500090b0 legacy=0x7f0a50008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:24.407 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.408 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.409 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.411 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:24.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.418 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.419 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.420 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:24.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.422 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:24.425 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:24.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.122:6812/20895 conn(0x7f0a4c0380a0 legacy=0x7f0a4c03a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 10 monclient: shutdown 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 >> v1:172.21.15.26:6790/0 conn(0x7f0a6c132b40 legacy=0x7f0a6c09abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6ad9d700 1 -- 172.21.15.26:0/444248387 reap_dead start 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6ad9d700 1 -- 172.21.15.26:0/444248387 reap_dead start 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.388+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 shutdown_connections 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.392+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 >> 172.21.15.26:0/444248387 conn(0x7f0a6c06e370 msgr2=0x7f0a6c132110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.392+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 shutdown_connections 2024-09-15T03:58:24.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.392+0000 7f0a6b59e700 1 -- 172.21.15.26:0/444248387 wait complete. 2024-09-15T03:58:24.428 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:24.428 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:58:24.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:58:24.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: get_monmap_and_config 2024-09-15T03:58:24.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: monmap: 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:24.544182+0000 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:24.544182+0000 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.548 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.549 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.549 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.549 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.549 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 Processor -- start 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 -- start start 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: init 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: _add_conns ranks=[0,6,8,7,5,4,3,2,1] 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient(hunting): picked mon.noname-a con 0x7f28000a48e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient(hunting): picked mon.noname-d con 0x7f280009aca0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient(hunting): picked mon.noname-e con 0x7f280009a5e0 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280004ca10 con 0x7f28000a48e0 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28000696a0 con 0x7f280009aca0 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2800068d20 con 0x7f280009a5e0 2024-09-15T03:58:24.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient(hunting): authenticate will time out at 1648.973575s 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fffff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f280009a5e0 0x7f280013b340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33500/0 (socket says 172.21.15.26:33500) 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fffff700 1 -- 172.21.15.26:0/3454841629 learned_addr learned my addr 172.21.15.26:0/3454841629 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27feffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f280009aca0 0x7f2800137c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43830/0 (socket says 172.21.15.26:43830) 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27ff7fe700 1 --1- 172.21.15.26:0/3454841629 >> v1:172.21.15.26:6789/0 conn(0x7f28000a48e0 0x7f2800099ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44762/0 (socket says 172.21.15.26:44762) 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812395681 0 0) 0x7f2800068d20 con 0x7f280009a5e0 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): my global_id is 135305 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e8003570 con 0x7f280009a5e0 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151299355 0 0) 0x7f28000696a0 con 0x7f280009aca0 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): my global_id is 194631 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800068d20 con 0x7f280009aca0 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150195362 0 0) 0x7f280004ca10 con 0x7f28000a48e0 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient(hunting): my global_id is 194562 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28000696a0 con 0x7f28000a48e0 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31978375 0 0) 0x7f28000696a0 con 0x7f28000a48e0 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 >> v1:172.21.15.26:6793/0 conn(0x7f280009a5e0 legacy=0x7f280013b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 >> v1:172.21.15.26:6792/0 conn(0x7f280009aca0 legacy=0x7f2800137c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 monclient: found mon.noname-a 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800067640 con 0x7f28000a48e0 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f40016d0 con 0x7f28000a48e0 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: dump: 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.555 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 5 monclient: authenticate success, global_id 194562 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27f40018c0 con 0x7f28000a48e0 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 1 -- 172.21.15.26:0/3454841629 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f4004830 con 0x7f28000a48e0 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 4 set_mon_vals no callback set 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f27fe7fc700 10 monclient: dump: 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 10 monclient: shutdown 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 -- 172.21.15.26:0/3454841629 >> v1:172.21.15.26:6789/0 conn(0x7f28000a48e0 legacy=0x7f2800099ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.540+0000 7f2804b07700 1 -- 172.21.15.26:0/3454841629 shutdown_connections 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- 172.21.15.26:0/3454841629 >> 172.21.15.26:0/3454841629 conn(0x7f28000638b0 msgr2=0x7f28000a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- 172.21.15.26:0/3454841629 shutdown_connections 2024-09-15T03:58:24.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- 172.21.15.26:0/3454841629 wait complete. 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: monmap: 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.560 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 Processor -- start 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- start start 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: init 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: _add_conns ranks=[2,1,0,8,4,6,5,3,7] 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient(hunting): picked mon.noname-b con 0x7f280009a5e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient(hunting): picked mon.noname-f con 0x7f280009aca0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient(hunting): picked mon.noname-a con 0x7f28000a48e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280006e680 con 0x7f28000a48e0 2024-09-15T03:58:24.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280013dfc0 con 0x7f280009aca0 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280006e2b0 con 0x7f280009a5e0 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient(hunting): authenticate will time out at 1648.978214s 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27fffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28000a48e0 0x7f280014a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44766/0 (socket says 172.21.15.26:44766) 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27fffff700 1 -- 172.21.15.26:0/2116735139 learned_addr learned my addr 172.21.15.26:0/2116735139 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033132406 0 0) 0x7f280006e2b0 con 0x7f280009a5e0 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): my global_id is 204428 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ec003570 con 0x7f280009a5e0 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110771893 0 0) 0x7f280006e680 con 0x7f28000a48e0 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): my global_id is 194571 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f280006e2b0 con 0x7f28000a48e0 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798103941 0 0) 0x7f280013dfc0 con 0x7f280009aca0 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): my global_id is 204391 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f280006e680 con 0x7f280009aca0 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3585477830 0 0) 0x7f27ec003570 con 0x7f280009a5e0 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f280013dfc0 con 0x7f280009a5e0 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f4001690 con 0x7f280009a5e0 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): dump: 2024-09-15T03:58:24.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.567 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3974211630 0 0) 0x7f280006e2b0 con 0x7f28000a48e0 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27ec003570 con 0x7f28000a48e0 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f8001d40 con 0x7f28000a48e0 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient(hunting): dump: 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.568 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 332261475 0 0) 0x7f280013dfc0 con 0x7f280009a5e0 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.122:6789/0 conn(0x7f280009aca0 legacy=0x7f2800146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.26:6789/0 conn(0x7f28000a48e0 legacy=0x7f280014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 monclient: found mon.b 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800122950 con 0x7f280009a5e0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f27e77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 5 monclient: authenticate success, global_id 204428 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: _renew_subs 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2800122c30 con 0x7f280009a5e0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: _renew_subs 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.544+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f280013ee60 con 0x7f280009a5e0 2024-09-15T03:58:24.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27f4004db0 con 0x7f280009a5e0 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f4005260 con 0x7f280009a5e0 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27fdffb700 4 set_mon_vals no callback set 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: dump: 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.571 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f2804b07700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f27e0006590 con 0x7f27e0002840 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f27f400fc20 con 0x7f280009a5e0 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(631..631 src has 435..631) v4 ==== 3959+0+0 (unknown 655598167 0 0) 0x7f27f4044150 con 0x7f280009a5e0 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27fffff700 10 monclient: get_auth_request con 0x7f27ec037d00 auth_method 0 2024-09-15T03:58:24.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758027610 0 0) 0x7f27e0006590 con 0x7f27e0002840 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient(hunting): my global_id is 5900 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f280013dfc0 con 0x7f27e0002840 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3842030748 0 0) 0x7f280013dfc0 con 0x7f27e0002840 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27e0006590 con 0x7f27e0002840 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f0001510 con 0x7f27e0002840 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 10 monclient: dump: 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.573 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4093869320 0 0) 0x7f27e0006590 con 0x7f27e0002840 2024-09-15T03:58:24.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.548+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27e000a130 con 0x7f27e0002840 2024-09-15T03:58:24.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.552+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f27f0004830 con 0x7f27e0002840 2024-09-15T03:58:24.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.552+0000 7f27e77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.552+0000 7f27e77fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:24.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.552+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.122:6791/0 conn(0x7f27e0002840 legacy=0x7f27e0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.552+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.122:6791/0 conn(0x7f27e0002840 legacy=0x7f27e0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f2804b07700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:24.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f27e0009640 con 0x7f27e00090b0 2024-09-15T03:58:24.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938615056 0 0) 0x7f27e0009640 con 0x7f27e00090b0 2024-09-15T03:58:24.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 10 monclient(hunting): my global_id is 5909 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e0006590 con 0x7f27e00090b0 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1002573537 0 0) 0x7f27e0006590 con 0x7f27e00090b0 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27e0009640 con 0x7f27e00090b0 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27f0004f90 con 0x7f27e00090b0 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 10 monclient: dump: 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.577 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3169084487 0 0) 0x7f27e0009640 con 0x7f27e00090b0 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.560+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f27e00098f0 con 0x7f27e00090b0 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7740 (unknown 0 0 409341027) 0x7f27f00072e0 con 0x7f27e00090b0 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27e77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27e77fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.122:6791/0 conn(0x7f27e00090b0 legacy=0x7f27e0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27e77fe700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.122:6791/0 conn(0x7f27e00090b0 legacy=0x7f27e0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:24.579 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.580 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.581 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.582 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:24.583 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.584 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:24.585 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:24.586 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.587 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.589 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.590 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.591 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.592 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.593 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.598 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.599 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:24.600 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:24.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.122:6812/20895 conn(0x7f27ec037d00 legacy=0x7f27ec03a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 10 monclient: shutdown 2024-09-15T03:58:24.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 >> v1:172.21.15.26:6790/0 conn(0x7f280009a5e0 legacy=0x7f280013d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27fffff700 1 -- 172.21.15.26:0/2116735139 reap_dead start 2024-09-15T03:58:24.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f27fffff700 1 -- 172.21.15.26:0/2116735139 reap_dead start 2024-09-15T03:58:24.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 shutdown_connections 2024-09-15T03:58:24.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 >> 172.21.15.26:0/2116735139 conn(0x7f28000638b0 msgr2=0x7f2800098d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 shutdown_connections 2024-09-15T03:58:24.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.564+0000 7f2804b07700 1 -- 172.21.15.26:0/2116735139 wait complete. 2024-09-15T03:58:24.602 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:24.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:58:24.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:58:24.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient: get_monmap_and_config 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient: monmap: 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:24.714298+0000 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:24.714298+0000 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.718 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.719 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.719 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 1 Processor -- start 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 1 -- start start 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient: init 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient: _add_conns ranks=[0,6,2,3,4,8,5,1,7] 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient(hunting): picked mon.noname-a con 0x7ff52c09b8d0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient(hunting): picked mon.noname-d con 0x7ff52c09c250 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient(hunting): picked mon.noname-b con 0x7ff52c09e490 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c04ca10 con 0x7ff52c09b8d0 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c09d8a0 con 0x7ff52c09e490 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c1317e0 con 0x7ff52c09c250 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.708+0000 7ff53109d700 10 monclient(hunting): authenticate will time out at 1649.143830s 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff52b59e700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ff52c09e490 0x7ff52c138f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:52824/0 (socket says 172.21.15.26:52824) 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff52ad9d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff52c09b8d0 0x7ff52c1309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44770/0 (socket says 172.21.15.26:44770) 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff52b59e700 1 -- 172.21.15.26:0/1176850421 learned_addr learned my addr 172.21.15.26:0/1176850421 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797418659 0 0) 0x7ff52c04ca10 con 0x7ff52c09b8d0 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): my global_id is 194580 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff518003570 con 0x7ff52c09b8d0 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915152161 0 0) 0x7ff52c1317e0 con 0x7ff52c09c250 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): my global_id is 194640 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c04ca10 con 0x7ff52c09c250 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124799013 0 0) 0x7ff52c09d8a0 con 0x7ff52c09e490 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient(hunting): my global_id is 204437 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c1317e0 con 0x7ff52c09e490 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 141926311 0 0) 0x7ff52c04ca10 con 0x7ff52c09c250 2024-09-15T03:58:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 >> v1:172.21.15.26:6790/0 conn(0x7ff52c09e490 legacy=0x7ff52c138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 >> v1:172.21.15.26:6789/0 conn(0x7ff52c09b8d0 legacy=0x7ff52c1309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 monclient: found mon.noname-d 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c09d5c0 con 0x7ff52c09c250 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff514001790 con 0x7ff52c09c250 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: dump: 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.725 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 5 monclient: authenticate success, global_id 194640 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff514001a30 con 0x7ff52c09c250 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 1 -- 172.21.15.26:0/1176850421 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff514001f60 con 0x7ff52c09c250 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 4 set_mon_vals no callback set 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff529d9b700 10 monclient: dump: 2024-09-15T03:58:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.728 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 10 monclient: shutdown 2024-09-15T03:58:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 1 -- 172.21.15.26:0/1176850421 >> v1:172.21.15.26:6792/0 conn(0x7ff52c09c250 legacy=0x7ff52c1310d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 1 -- 172.21.15.26:0/1176850421 shutdown_connections 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 1 -- 172.21.15.26:0/1176850421 >> 172.21.15.26:0/1176850421 conn(0x7ff52c063f30 msgr2=0x7ff52c0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 1 -- 172.21.15.26:0/1176850421 shutdown_connections 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 1 -- 172.21.15.26:0/1176850421 wait complete. 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 10 monclient: monmap: 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.712+0000 7ff53109d700 1 Processor -- start 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- start start 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: init 2024-09-15T03:58:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: _add_conns ranks=[6,5,1,8,4,3,7,0,2] 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient(hunting): picked mon.noname-d con 0x7ff52c09c250 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient(hunting): picked mon.noname-h con 0x7ff52c09e490 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient(hunting): picked mon.noname-f con 0x7ff52c09a580 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff52ad9d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ff52c09c250 0x7ff52c141f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43850/0 (socket says 172.21.15.26:43850) 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff52ad9d700 1 -- 172.21.15.26:0/396805536 learned_addr learned my addr 172.21.15.26:0/396805536 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c06e140 con 0x7ff52c09a580 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c09cd20 con 0x7ff52c09e490 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c11eb20 con 0x7ff52c09c250 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient(hunting): authenticate will time out at 1649.148967s 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898346831 0 0) 0x7ff52c11eb20 con 0x7ff52c09c250 2024-09-15T03:58:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): my global_id is 194649 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff50c003570 con 0x7ff52c09c250 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2003145378 0 0) 0x7ff50c003570 con 0x7ff52c09c250 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff52c11eb20 con 0x7ff52c09c250 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff51c001530 con 0x7ff52c09c250 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164156909 0 0) 0x7ff52c09cd20 con 0x7ff52c09e490 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): my global_id is 5918 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff50c003570 con 0x7ff52c09e490 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875350458 0 0) 0x7ff52c06e140 con 0x7ff52c09a580 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient(hunting): my global_id is 204400 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c09cd20 con 0x7ff52c09a580 2024-09-15T03:58:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2806916893 0 0) 0x7ff52c11eb20 con 0x7ff52c09c250 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6791/0 conn(0x7ff52c09e490 legacy=0x7ff52c146f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6789/0 conn(0x7ff52c09a580 legacy=0x7ff52c14a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 monclient: found mon.d 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c11d8b0 con 0x7ff52c09c250 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 5 monclient: authenticate success, global_id 194649 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: _renew_subs 2024-09-15T03:58:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c09b0a0 con 0x7ff52c09c250 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: _renew_subs 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c11f780 con 0x7ff52c09c250 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff51c004ae0 con 0x7ff52c09c250 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff51c004f90 con 0x7ff52c09c250 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff52959a700 4 set_mon_vals no callback set 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff522ffd700 10 monclient: dump: 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.740 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.741 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.716+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff510006590 con 0x7ff510002840 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff51c00f9d0 con 0x7ff52c09c250 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(631..631 src has 435..631) v4 ==== 3959+0+0 (unknown 655598167 0 0) 0x7ff51c044040 con 0x7ff52c09c250 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100712026 0 0) 0x7ff510006590 con 0x7ff510002840 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient(hunting): my global_id is 234142 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c11eb20 con 0x7ff510002840 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff52b59e700 10 monclient: get_auth_request con 0x7ff50c037670 auth_method 0 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2376880524 0 0) 0x7ff52c11eb20 con 0x7ff510002840 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff510006590 con 0x7ff510002840 2024-09-15T03:58:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff514001ee0 con 0x7ff510002840 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient: dump: 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.743 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1650868753 0 0) 0x7ff510006590 con 0x7ff510002840 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff51000a130 con 0x7ff510002840 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff514002230 con 0x7ff510002840 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6792/0 conn(0x7ff510002840 legacy=0x7ff510004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.720+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6792/0 conn(0x7ff510002840 legacy=0x7ff510004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff510007520 con 0x7ff5100068d0 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956171503 0 0) 0x7ff510007520 con 0x7ff5100068d0 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient(hunting): my global_id is 234151 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff510006590 con 0x7ff5100068d0 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 49807401 0 0) 0x7ff510006590 con 0x7ff5100068d0 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff510007520 con 0x7ff5100068d0 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff514002b40 con 0x7ff5100068d0 2024-09-15T03:58:24.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient: dump: 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.747 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3310324033 0 0) 0x7ff510007520 con 0x7ff5100068d0 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff510008760 con 0x7ff5100068d0 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 1635738898) 0x7ff514005220 con 0x7ff5100068d0 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:24.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6792/0 conn(0x7ff5100068d0 legacy=0x7ff510006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff522ffd700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6792/0 conn(0x7ff5100068d0 legacy=0x7ff510006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:24.749 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:24.750 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:24.751 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.752 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:24.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.754 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.756 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.758 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:24.759 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.760 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:24.761 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.762 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.763 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.764 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.765 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:24.766 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:24.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:24.768 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:24.769 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.122:6812/20895 conn(0x7ff50c037670 legacy=0x7ff50c039b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 10 monclient: shutdown 2024-09-15T03:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 >> v1:172.21.15.26:6792/0 conn(0x7ff52c09c250 legacy=0x7ff52c141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff52b59e700 1 -- 172.21.15.26:0/396805536 reap_dead start 2024-09-15T03:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff52b59e700 1 -- 172.21.15.26:0/396805536 reap_dead start 2024-09-15T03:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 shutdown_connections 2024-09-15T03:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 >> 172.21.15.26:0/396805536 conn(0x7ff52c063f30 msgr2=0x7ff52c131110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 shutdown_connections 2024-09-15T03:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.732+0000 7ff53109d700 1 -- 172.21.15.26:0/396805536 wait complete. 2024-09-15T03:58:24.771 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:24.771 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T03:58:24.771 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.880+0000 7fbd5d473700 10 monclient: get_monmap_and_config 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.880+0000 7fbd5d473700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.880+0000 7fbd5d473700 10 monclient: monmap: 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:24.887362+0000 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:24.887362+0000 2024-09-15T03:58:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.892 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 Processor -- start 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- start start 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: init 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: _add_conns ranks=[4,1,6,7,5,0,2,8,3] 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient(hunting): picked mon.noname-c con 0x7fbd58099310 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient(hunting): picked mon.noname-f con 0x7fbd58132fa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient(hunting): picked mon.noname-d con 0x7fbd581333f0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5804ca10 con 0x7fbd58132fa0 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd58097e50 con 0x7fbd58099310 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd580a1410 con 0x7fbd581333f0 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient(hunting): authenticate will time out at 1649.317350s 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd56ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fbd58099310 0x7fbd580996e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:58012/0 (socket says 172.21.15.26:58012) 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd56ffd700 1 -- 172.21.15.26:0/624921508 learned_addr learned my addr 172.21.15.26:0/624921508 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd577fe700 1 --1- 172.21.15.26:0/624921508 >> v1:172.21.15.26:6792/0 conn(0x7fbd581333f0 0x7fbd5813b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43862/0 (socket says 172.21.15.26:43862) 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106121077 0 0) 0x7fbd58097e50 con 0x7fbd58099310 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): my global_id is 204385 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd44003570 con 0x7fbd58099310 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609090782 0 0) 0x7fbd580a1410 con 0x7fbd581333f0 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): my global_id is 194658 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd58097e50 con 0x7fbd581333f0 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894074452 0 0) 0x7fbd5804ca10 con 0x7fbd58132fa0 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient(hunting): my global_id is 204409 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd580a1410 con 0x7fbd58132fa0 2024-09-15T03:58:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3441569861 0 0) 0x7fbd44003570 con 0x7fbd58099310 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 >> v1:172.21.15.26:6792/0 conn(0x7fbd581333f0 legacy=0x7fbd5813b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 >> v1:172.21.15.122:6789/0 conn(0x7fbd58132fa0 legacy=0x7fbd581380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 monclient: found mon.noname-c 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580a2710 con 0x7fbd58099310 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd400017b0 con 0x7fbd58099310 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: dump: 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd40001a20 con 0x7fbd58099310 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 5 monclient: authenticate success, global_id 204385 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 4 set_mon_vals no callback set 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 1 -- 172.21.15.26:0/624921508 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd40004c30 con 0x7fbd58099310 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd55ffb700 10 monclient: dump: 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: shutdown 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- 172.21.15.26:0/624921508 >> v1:172.21.15.26:6791/0 conn(0x7fbd58099310 legacy=0x7fbd580996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- 172.21.15.26:0/624921508 shutdown_connections 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- 172.21.15.26:0/624921508 >> 172.21.15.26:0/624921508 conn(0x7fbd580a6c30 msgr2=0x7fbd5806dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- 172.21.15.26:0/624921508 shutdown_connections 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 1 -- 172.21.15.26:0/624921508 wait complete. 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: build_initial_monmap 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.884+0000 7fbd5d473700 10 monclient: monmap: 2024-09-15T03:58:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 Processor -- start 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 -- start start 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: init 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: _add_conns ranks=[5,6,0,2,4,7,3,8,1] 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient(hunting): picked mon.noname-h con 0x7fbd58099310 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient(hunting): picked mon.noname-d con 0x7fbd58132fa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient(hunting): picked mon.noname-a con 0x7fbd581333f0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd58120120 con 0x7fbd581333f0 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5811ef50 con 0x7fbd58099310 2024-09-15T03:58:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5811fe40 con 0x7fbd58132fa0 2024-09-15T03:58:24.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:24.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient(hunting): authenticate will time out at 1649.320960s 2024-09-15T03:58:24.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd567fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fbd58132fa0 0x7fbd58147110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:43872/0 (socket says 172.21.15.26:43872) 2024-09-15T03:58:24.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd567fc700 1 -- 172.21.15.26:0/1333414318 learned_addr learned my addr 172.21.15.26:0/1333414318 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:24.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089747116 0 0) 0x7fbd5811fe40 con 0x7fbd58132fa0 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): my global_id is 194667 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd38003570 con 0x7fbd58132fa0 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089300429 0 0) 0x7fbd58120120 con 0x7fbd581333f0 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): my global_id is 194589 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5811fe40 con 0x7fbd581333f0 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580437103 0 0) 0x7fbd5811ef50 con 0x7fbd58099310 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): my global_id is 5927 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd58120120 con 0x7fbd58099310 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3909335970 0 0) 0x7fbd38003570 con 0x7fbd58132fa0 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd5811ef50 con 0x7fbd58132fa0 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd50002e10 con 0x7fbd58132fa0 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): dump: 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.910 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:24.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1467024636 0 0) 0x7fbd5811fe40 con 0x7fbd581333f0 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd38003570 con 0x7fbd581333f0 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd48001940 con 0x7fbd581333f0 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): dump: 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2413575036 0 0) 0x7fbd58120120 con 0x7fbd58099310 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd5811fe40 con 0x7fbd58099310 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd40001770 con 0x7fbd58099310 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient(hunting): dump: 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1222226683 0 0) 0x7fbd5811ef50 con 0x7fbd58132fa0 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 >> v1:172.21.15.122:6791/0 conn(0x7fbd58099310 legacy=0x7fbd58142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 >> v1:172.21.15.26:6789/0 conn(0x7fbd581333f0 legacy=0x7fbd5814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 monclient: found mon.d 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5811ea80 con 0x7fbd58132fa0 2024-09-15T03:58:24.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: _finish_auth 0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 5 monclient: authenticate success, global_id 194667 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: _renew_subs 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58120ff0 con 0x7fbd58132fa0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: _renew_subs 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd580a1f10 con 0x7fbd58132fa0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbd500018e0 con 0x7fbd58132fa0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbd50001e10 con 0x7fbd58132fa0 2024-09-15T03:58:24.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 10 monclient: dump: 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:24.917 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.888+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbd50001ff0 con 0x7fbd58132fa0 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd5d473700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(631..631 src has 435..631) v4 ==== 3959+0+0 (unknown 655598167 0 0) 0x7fbd50044240 con 0x7fbd58132fa0 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd557fa700 4 set_mon_vals no callback set 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd5d473700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd5d473700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd58120550 con 0x7fbd58132fa0 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.892+0000 7fbd56ffd700 10 monclient: get_auth_request con 0x7fbd38037fb0 auth_method 0 2024-09-15T03:58:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.896+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fbd50015040 con 0x7fbd58132fa0 2024-09-15T03:58:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.896+0000 7fbd4effd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:24.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:24.896+0000 7fbd4effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:25.127 INFO:tasks.rados.rados.0.smithi122.stdout:2806: done (3 left) 2024-09-15T03:58:25.464 INFO:tasks.rados.rados.0.smithi122.stdout:2807: write oid 48 current snap is 314 2024-09-15T03:58:25.465 INFO:tasks.rados.rados.0.smithi122.stdout:2807: seq_num 1004 ranges {474228=508592,1451686=555012,2704920=169308} 2024-09-15T03:58:25.466 INFO:tasks.rados.rados.0.smithi122.stdout:2807: writing smithi12223906-48 from 474228 to 982820 tid 1 2024-09-15T03:58:25.467 INFO:tasks.rados.rados.0.smithi122.stdout:2807: writing smithi12223906-48 from 1451686 to 2006698 tid 2 2024-09-15T03:58:25.467 INFO:tasks.rados.rados.0.smithi122.stdout:2807: writing smithi12223906-48 from 2704920 to 2874228 tid 3 2024-09-15T03:58:25.467 INFO:tasks.rados.rados.0.smithi122.stdout:2808: delete oid 7 current snap is 314 2024-09-15T03:58:25.467 INFO:tasks.rados.rados.0.smithi122.stdout:2804: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.132+0000 7fbd5d473700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T03:58:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.132+0000 7fbd5d473700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T03:58:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.132+0000 7fbd5d473700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.132+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fbd5804bbf0 con 0x7fbd58132fa0 2024-09-15T03:58:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd4effd700 1 -- 172.21.15.26:0/1333414318 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fbd50014a40 con 0x7fbd58132fa0 2024-09-15T03:58:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd4effd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd4effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 >> v1:172.21.15.122:6812/20895 conn(0x7fbd38037fb0 legacy=0x7fbd3803a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd5d473700 10 monclient: shutdown 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd5d473700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 >> v1:172.21.15.26:6792/0 conn(0x7fbd58132fa0 legacy=0x7fbd58147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.140+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 shutdown_connections 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.144+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 >> 172.21.15.26:0/1333414318 conn(0x7fbd580a6c30 msgr2=0x7fbd58067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.144+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 shutdown_connections 2024-09-15T03:58:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:25.144+0000 7fbd5d473700 1 -- 172.21.15.26:0/1333414318 wait complete. 2024-09-15T03:58:25.471 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T03:58:25.584 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.584 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=4217777436,config=3140514009,health=3235959426,kv=2519098299,logm=2395491384}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1574338708,config=3140514009,health=3235959426,kv=1680502341,logm=1228050128}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=22,config=2,health=12,kv=8,logm=56} crc {auth=3599622574,config=3140514009,health=3235959426,kv=2519098299,logm=509550246}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=3843316640,config=3140514009,health=1919741848,kv=2519098299,logm=728760742}) 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.585 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=23,config=2,health=10,kv=8,logm=57} crc {auth=2420154103,config=3140514009,health=3796970882,kv=2519098299,logm=390894438}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=25,config=2,health=12,kv=8,logm=53} crc {auth=1878055205,config=3140514009,health=3235959426,kv=2519098299,logm=142616432}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=2702584865,config=3140514009,health=3235959426,kv=1555229470,logm=1440353815}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=865605565,config=3140514009,health=3235959426,kv=1247634456,logm=514457256}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.146+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=12,kv=7,logm=53} crc {auth=4042660056,config=3140514009,health=3235959426,kv=4086678316,logm=1610758782}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.586 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=526439280}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3064492879}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=915405170}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3271956703}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.587 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2171473618}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3331695354}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3593244332}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=443024603}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.150+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933484555}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2196164223}) 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.588 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4015681057}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1667959753}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3747094593}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4078505514}) 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.589 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2737731088}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2207123886}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=870403235}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=728123317}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3610601376}) 2024-09-15T03:58:25.590 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=20246358}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=495767084}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3160264997}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2569765663}) 2024-09-15T03:58:25.591 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3300066424}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3946029119}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=773216305}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.154+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454037480}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2384344681}) 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.592 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2106820232}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3274906503}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2055550662}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1165836817}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.593 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3821317172}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3858843435}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2159865537}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2094089763}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2234663926}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=688325771}) 2024-09-15T03:58:25.594 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2276606480}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1428860525}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1285980637}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1201119981}) 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.595 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1747594427}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=697474330}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.158+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=773395313}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2428694806}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=811801034}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1277086977}) 2024-09-15T03:58:25.596 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3641136645}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=39659842}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3084552016}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3371337549}) 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.597 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3818641041}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=534497772}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2726649124}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2712842553}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4274196674}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.598 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2221640509}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3808696148}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2336478406}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1852366501}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1173718515}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.162+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3563983585}) 2024-09-15T03:58:25.599 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2293566847}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1277124244}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2790408363}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=70690677}) 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.600 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4089908616}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2622842345}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4038394683}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3488047012}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3184612298}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4074993513}) 2024-09-15T03:58:25.601 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3010306836}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3024417693}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2885645811}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2486298484}) 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.602 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2647429885}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=801214322}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2083679209}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.166+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=146517535}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2080946044}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.603 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1460390730}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1697427225}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1015684377}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3939543856}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2522396979}) 2024-09-15T03:58:25.604 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4257224221}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=529588693}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3408611579}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1433132731}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3583075626}) 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.605 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3105583309}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1335012594}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4046196078}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1412229181}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.606 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2183200643}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2358092484}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.170+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318623389}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1176535164}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1921325622}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4086387444}) 2024-09-15T03:58:25.607 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=155023356}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3821769025}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3971798892}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3782689335}) 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.608 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2764510355}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3992258957}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=337807423}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.174+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2432573450}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2672825889}) 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.609 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4015213678}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1680085763}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1277458047}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3328391778}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=776520372}) 2024-09-15T03:58:25.610 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4276064293}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2889407126}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1221903344}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=708125811}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3501990783}) 2024-09-15T03:58:25.611 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4194643592}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1991864612}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=813901587}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=310206687}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.178+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2427246210}) 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.612 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1928787696}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3127728278}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1450976127}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3758115713}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.613 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1461981542}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=854592847}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1745528851}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3689008133}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1666263141}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2195840429}) 2024-09-15T03:58:25.614 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2434790469}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2045713637}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=795433773}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=657128638}) 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.615 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2243236210}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3801629900}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2940701329}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.182+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=51446739}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3149304298}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.616 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1200934401}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3942317582}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=796971875}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1777460482}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3554632177}) 2024-09-15T03:58:25.617 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2250788315}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1973126397}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4181249180}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2089528530}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1197836434}) 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.618 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3161695252}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=865144853}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2447562410}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1166412749}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=935676095}) 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.619 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1381019578}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.186+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2561111145}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4215421462}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2096442425}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=558977855}) 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.620 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2445317150}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=277834533}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2518569929}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1062045411}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2273010930}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1141433749}) 2024-09-15T03:58:25.621 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1664829464}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=117408305}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3575116435}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3588494979}) 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.622 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3460700468}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2561776465}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2127754568}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=37154030}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.190+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1090165182}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.623 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3750930247}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=95527239}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1095948645}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3670618001}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=287126140}) 2024-09-15T03:58:25.624 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1457841708}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3094667486}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3802347878}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=14318174}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1529634455}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.625 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=803965305}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1865501926}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2046315686}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3870102274}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.194+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2010802688}) 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.626 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2669828042}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11870616}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2115977951}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3314213895}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2622025533}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.627 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1716053196}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3589994589}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3387100282}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1889796754}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4066652389}) 2024-09-15T03:58:25.628 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1658631345}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=203702368}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1210114776}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2438159192}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4181778406}) 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.629 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2647488804}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2706005401}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.198+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2089225132}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2975787571}) 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.630 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2089620243}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4087054566}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3285743922}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1161664856}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1990974321}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.631 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2405333701}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=46485256}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2841369246}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3955291285}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1798729046}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.202+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=404507813}) 2024-09-15T03:58:25.632 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=291586422}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1769354350}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=245550085}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2933917077}) 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.633 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1548492980}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3514249733}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3628083756}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2115765186}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3746116109}) 2024-09-15T03:58:25.634 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3934974207}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3078926724}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3213364966}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2919884444}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.635 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2800448106}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2383379968}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1092000678}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3387001798}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.206+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1423323163}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4081602206}) 2024-09-15T03:58:25.636 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=278051158}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1504727908}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2351862622}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1334835986}) 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.637 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=9864471}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3240724866}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=93584007}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2061024156}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1395184350}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.638 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4193363842}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=547602994}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1976781099}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1535838952}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4062444391}) 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.639 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3956112959}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=708791608}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.210+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3555269348}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1346283542}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=803317114}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.640 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4015708040}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1845144048}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2626394508}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3711066013}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4016804655}) 2024-09-15T03:58:25.641 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1665050595}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3094594592}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2834837439}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2500171257}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1767333227}) 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.642 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1037772989}) 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1181236886}) 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.643 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4066477649}) 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4032360237}) 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4238245994}) 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.214+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2211013767}) 2024-09-15T03:58:25.644 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4282209676}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1212702231}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3286576295}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2919832250}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.645 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2997746819}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3559597630}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=134860182}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=432514592}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2724241930}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=245270223}) 2024-09-15T03:58:25.646 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3505021491}) 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2386400936}) 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3874442055}) 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.647 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3557758348}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.218+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=279861116}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4162667016}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=732534370}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2474393193}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.648 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3027805143}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4251918987}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1228192868}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1634534302}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1470650422}) 2024-09-15T03:58:25.649 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2876224651}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=303716890}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2664031260}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.222+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=109545618}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2436733348}) 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.650 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2716222625}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1330091800}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2595465318}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4117032360}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3363126747}) 2024-09-15T03:58:25.651 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2789900340}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1238412895}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=231982573}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2899353637}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2918195002}) 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.652 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3783652913}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=313500359}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1929365231}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1976431213}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.653 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1982736268}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2905542233}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.226+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1394152496}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3447899819}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1274913742}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2986028880}) 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.654 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1327484194}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3704237602}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2707703647}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=114246752}) 2024-09-15T03:58:25.655 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=619894615}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1168301028}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3587910925}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1351896007}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.656 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3757623822}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1885812261}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4190290038}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4286331800}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3927623596}) 2024-09-15T03:58:25.657 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2505042387}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.230+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1355158311}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=581377542}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=738378713}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=218659299}) 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.658 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=912191675}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=327282580}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1403066191}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2009997312}) 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.659 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1116567952}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1164291767}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3300411246}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2806588033}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=383009023}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.660 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=372653899}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1615112703}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=378559197}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=19462391}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2788875952}) 2024-09-15T03:58:25.661 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.234+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2899203332}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1962723092}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2896294717}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3327182716}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=941495374}) 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.662 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3549180835}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1544754475}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1488455289}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491175634}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.238+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454975954}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.663 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2801006154}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2394853292}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=623615233}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=533724434}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2557996226}) 2024-09-15T03:58:25.664 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1578675564}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=810275610}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2555207527}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1111149066}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2978169675}) 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.665 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2440925038}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.242+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4188088409}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3083262965}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2013230475}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.666 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213241051}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3188447924}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=423237462}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1243774155}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3095543929}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3540178775}) 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.667 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=450191978}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=640780308}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3607370731}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=457631293}) 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.668 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=312851114}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3161483498}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.246+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2883120941}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3822756556}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=49056481}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2348970369}) 2024-09-15T03:58:25.669 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=887205076}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2106161049}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3572380340}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3869269277}) 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.670 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=438517524}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.250+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3909143454}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2325220596}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2029306560}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1531148044}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3327741034}) 2024-09-15T03:58:25.671 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=964752014}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2579894004}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=553191216}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797485646}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3138420479}) 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.672 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=973232910}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=933691252}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4053131147}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4237224819}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.673 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2153905854}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=76523453}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3223418279}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2694646795}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.254+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1985532875}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1926010898}) 2024-09-15T03:58:25.674 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3469313233}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=44469642}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2037102938}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2929193295}) 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.675 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4008882180}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=86249299}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2435198865}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.258+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810245336}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3191728682}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.676 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2755976498}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1619677269}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2600783728}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=685824460}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=288250397}) 2024-09-15T03:58:25.677 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=475744656}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1340800968}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.262+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=335610106}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3473718698}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=734686392}) 2024-09-15T03:58:25.678 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=917179996}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1267944771}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2215201395}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3719718211}) 2024-09-15T03:58:25.679 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2288666406}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=184063226}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=622754797}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2073562744}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519782896}) 2024-09-15T03:58:25.680 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1612121897}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.266+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3415453201}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=125327432}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2671762536}) 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.681 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1298591359}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2394334717}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3363227786}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=79,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=393355257,mdsmap=1531151339,mgr=747136936,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1293206348}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=13} crc {logm=191021619,mdsmap=1531151339,mgr=1568706250,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=237538744}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.682 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=79,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=605680684,mdsmap=1531151339,mgr=2192755073,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=3712274809}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12} crc {logm=1858732548,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2327794006}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=75,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12} crc {logm=3173235475,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1560535839}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=72,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=4} crc {logm=4144097434,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=506903188,mon_config_key=3784792071}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=75,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=1} crc {logm=2276717808,mdsmap=1531151339,mgr=73913752,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=506903188,mon_config_key=551709721}) 2024-09-15T03:58:25.683 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=13,mon_config_key=1} crc {logm=1800750211,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1060954187,mon_config_key=551709721}) 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.270+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=77,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10} crc {logm=3233380165,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2337791586}) 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3386915503}) 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=87} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2459566930,osd_pg_creating=3187683209,osd_snap=2522488106}) 2024-09-15T03:58:25.684 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=87} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3187683209,osd_snap=3379201390}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=2223603198}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=50045983}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=87} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=231491462,osd_pg_creating=3187683209,osd_snap=3002834595}) 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.685 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=4117553718}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3915734792}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.274+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=4,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=86} crc {mon_config_key=2113949861,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3187683209,osd_snap=3705385995}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=32,osdmap=68} crc {osd_snap=3029516641,osdmap=2221705423}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=2882243157,osdmap=945929107}) 2024-09-15T03:58:25.686 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=33,osdmap=67} crc {osd_snap=3961113408,osdmap=682824051}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=2694676211,osdmap=2354420520}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=32,osdmap=68} crc {osd_snap=912025553,osdmap=140446360}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=32,osdmap=68} crc {osd_snap=2368903381,osdmap=475189227}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.687 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=30,osdmap=70} crc {osd_snap=4170395961,osdmap=683416598}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=4250019740,osdmap=1043021905}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=31,osdmap=69} crc {osd_snap=143039609,osdmap=3995164261}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2525158388}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=642834531}) 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.688 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=830421657}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3158031034}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3033345178}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1388864569}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.689 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4155174057}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3096415113}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.278+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=842612239}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=519788879}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2675190269}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1088137916}) 2024-09-15T03:58:25.690 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1795026563}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3090683359}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=113236324}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.691 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1004957301}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4010957408}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.282+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=746827764}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=35} crc {osdmap=1741567315}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=35} crc {osdmap=678368036}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.692 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=36} crc {osdmap=3003606024}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=36} crc {osdmap=2103839036}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=36} crc {osdmap=1421601136}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=33} crc {osdmap=141512012}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=35} crc {osdmap=1750150680}) 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T03:58:25.693 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=35} crc {osdmap=3745407125}) 2024-09-15T03:58:25.694 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T03:58:25.286+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=35} crc {osdmap=4271515328}) 2024-09-15T03:58:25.694 INFO:tasks.rados.rados.0.smithi122.stdout:2801: finishing rollback tid 0 to smithi12223906-19 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2801: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2801: finishing rollback tid 2 to smithi12223906-19 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 1584 (ObjNum 848 snap 276 seq_num 848) dirty exists 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2802: finishing copy_from racing read to smithi12223906-18 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2802: finishing copy_from to smithi12223906-18 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 842 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2807: finishing write tid 1 to smithi12223906-48 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2807: finishing write tid 2 to smithi12223906-48 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2807: finishing write tid 3 to smithi12223906-48 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2807: finishing write tid 4 to smithi12223906-48 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:2807: finishing write tid 5 to smithi12223906-48 2024-09-15T03:58:25.695 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1095 (ObjNum 1004 snap 314 seq_num 1004) dirty exists 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2807: left oid 48 (ObjNum 1004 snap 314 seq_num 1004) 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2801: done (4 left) 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2802: done (3 left) 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2804: done (2 left) 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2807: done (1 left) 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2808: done (0 left) 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2809: rollback oid 42 current snap is 314 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 237 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2810: snap_create 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2809: finishing rollback tid 0 to smithi12223906-42 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2809: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:2809: finishing rollback tid 2 to smithi12223906-42 2024-09-15T03:58:25.696 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 2212 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:58:26.062 INFO:tasks.rados.rados.0.smithi122.stdout:2809: done (1 left) 2024-09-15T03:58:26.062 INFO:tasks.rados.rados.0.smithi122.stdout:2810: done (0 left) 2024-09-15T03:58:26.062 INFO:tasks.rados.rados.0.smithi122.stdout:2811: snap_remove snap 272 2024-09-15T03:58:26.472 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T03:58:26.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:58:26.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient: get_monmap_and_config 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient: build_initial_monmap 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient: monmap: 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:26.581296+0000 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:26.581296+0000 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:26.585 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:26.586 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 1 Processor -- start 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 1 -- start start 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient: init 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient: _add_conns ranks=[3,0,1,8,6,5,4,7,2] 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient(hunting): picked mon.noname-g con 0x7f683c09aa90 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:26.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient(hunting): picked mon.noname-a con 0x7f683c06b5e0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient(hunting): picked mon.noname-f con 0x7f683c09f830 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c04ca10 con 0x7f683c06b5e0 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c0685f0 con 0x7f683c09f830 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c06e490 con 0x7f683c09aa90 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.576+0000 7f684356d700 10 monclient(hunting): authenticate will time out at 1651.011740s 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6840b08700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f683c06b5e0 0x7f683c09f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47652/0 (socket says 172.21.15.26:47652) 2024-09-15T03:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6840b08700 1 -- 172.21.15.26:0/545746552 learned_addr learned my addr 172.21.15.26:0/545746552 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887441764 0 0) 0x7f683c04ca10 con 0x7f683c06b5e0 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): my global_id is 194598 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6828003570 con 0x7f683c06b5e0 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474754256 0 0) 0x7f683c06e490 con 0x7f683c09aa90 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): my global_id is 204420 2024-09-15T03:58:26.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c04ca10 con 0x7f683c09aa90 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657115499 0 0) 0x7f683c0685f0 con 0x7f683c09f830 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient(hunting): my global_id is 204418 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c06e490 con 0x7f683c09f830 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4093484935 0 0) 0x7f6828003570 con 0x7f683c06b5e0 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 >> v1:172.21.15.122:6790/0 conn(0x7f683c09aa90 legacy=0x7f683c09ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 >> v1:172.21.15.122:6789/0 conn(0x7f683c09f830 legacy=0x7f683c13bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 monclient: found mon.noname-a 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:58:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c06b9b0 con 0x7f683c06b5e0 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: _finish_auth 0 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f682c001730 con 0x7f683c06b5e0 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 5 monclient: authenticate success, global_id 194598 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: dump: 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f682c001950 con 0x7f683c06b5e0 2024-09-15T03:58:26.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 4 set_mon_vals no callback set 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 1 -- 172.21.15.26:0/545746552 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f682c0048f0 con 0x7f683c06b5e0 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f6833fff700 10 monclient: dump: 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.596 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 10 monclient: shutdown 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 1 -- 172.21.15.26:0/545746552 >> v1:172.21.15.26:6789/0 conn(0x7f683c06b5e0 legacy=0x7f683c09f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 1 -- 172.21.15.26:0/545746552 shutdown_connections 2024-09-15T03:58:26.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 1 -- 172.21.15.26:0/545746552 >> 172.21.15.26:0/545746552 conn(0x7f683c063bb0 msgr2=0x7f683c0a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 1 -- 172.21.15.26:0/545746552 shutdown_connections 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 1 -- 172.21.15.26:0/545746552 wait complete. 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 10 monclient: build_initial_monmap 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.580+0000 7f684356d700 10 monclient: monmap: 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 Processor -- start 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 -- start start 2024-09-15T03:58:26.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: init 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: _add_conns ranks=[7,0,3,4,1,6,2,8,5] 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient(hunting): picked mon.noname-i con 0x7f683c06b5e0 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient(hunting): picked mon.noname-a con 0x7f683c09aa90 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient(hunting): picked mon.noname-g con 0x7f683c09f830 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c123000 con 0x7f683c09aa90 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c13e390 con 0x7f683c09f830 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c122d20 con 0x7f683c06b5e0 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient(hunting): authenticate will time out at 1651.017744s 2024-09-15T03:58:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f6840b08700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f683c09aa90 0x7f683c146ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47668/0 (socket says 172.21.15.26:47668) 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f6840b08700 1 -- 172.21.15.26:0/1373645074 learned_addr learned my addr 172.21.15.26:0/1373645074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163764135 0 0) 0x7f683c123000 con 0x7f683c09aa90 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): my global_id is 194607 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6820003570 con 0x7f683c09aa90 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763934733 0 0) 0x7f683c122d20 con 0x7f683c06b5e0 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): my global_id is 234160 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c123000 con 0x7f683c06b5e0 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095760018 0 0) 0x7f683c13e390 con 0x7f683c09f830 2024-09-15T03:58:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): my global_id is 204429 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c122d20 con 0x7f683c09f830 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 476464637 0 0) 0x7f6820003570 con 0x7f683c09aa90 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f683c13e390 con 0x7f683c09aa90 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f682c0016b0 con 0x7f683c09aa90 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): dump: 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1032691392 0 0) 0x7f683c13e390 con 0x7f683c09aa90 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 >> v1:172.21.15.122:6792/0 conn(0x7f683c06b5e0 legacy=0x7f683c13dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 >> v1:172.21.15.122:6790/0 conn(0x7f683c09f830 legacy=0x7f683c14a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 monclient: found mon.a 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c120af0 con 0x7f683c09aa90 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 5 monclient: authenticate success, global_id 194607 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: _renew_subs 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c13f050 con 0x7f683c09aa90 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: _renew_subs 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c121ff0 con 0x7f683c09aa90 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f682c004e70 con 0x7f683c09aa90 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f682c005320 con 0x7f683c09aa90 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f6832ffd700 4 set_mon_vals no callback set 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.584+0000 7f68317fa700 10 monclient: dump: 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f682c00fce0 con 0x7f683c09aa90 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f684356d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f684356d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f684356d700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c0050b0 con 0x7f683c09aa90 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(633..633 src has 435..633) v4 ==== 3987+0+0 (unknown 3216915949 0 0) 0x7f682c044190 con 0x7f683c09aa90 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.588+0000 7f6841309700 10 monclient: get_auth_request con 0x7f6820037500 auth_method 0 2024-09-15T03:58:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.596+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f682c015030 con 0x7f683c09aa90 2024-09-15T03:58:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.596+0000 7f68317fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.596+0000 7f68317fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:26.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:58:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f681c0058e0 con 0x7f683c09aa90 2024-09-15T03:58:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f68317fa700 1 -- 172.21.15.26:0/1373645074 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 756388831) 0x7f682c014a30 con 0x7f683c09aa90 2024-09-15T03:58:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f68317fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:26.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f68317fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:26.829 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":94,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 >> v1:172.21.15.122:6812/20895 conn(0x7f6820037500 legacy=0x7f68200399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 10 monclient: shutdown 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 >> v1:172.21.15.26:6789/0 conn(0x7f683c09aa90 legacy=0x7f683c146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 shutdown_connections 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 >> 172.21.15.26:0/1373645074 conn(0x7f683c063bb0 msgr2=0x7f683c0a2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 shutdown_connections 2024-09-15T03:58:26.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.820+0000 7f684356d700 1 -- 172.21.15.26:0/1373645074 wait complete. 2024-09-15T03:58:26.842 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T03:58:26.842 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T03:58:26.842 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T03:58:26.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:58:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 10 monclient: get_monmap_and_config 2024-09-15T03:58:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 10 monclient: build_initial_monmap 2024-09-15T03:58:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 10 monclient: monmap: 2024-09-15T03:58:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:26.946733+0000 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:26.946733+0000 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:26.952 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 1 Processor -- start 2024-09-15T03:58:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 1 -- start start 2024-09-15T03:58:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 10 monclient: init 2024-09-15T03:58:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.940+0000 7f89db56f700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient: _add_conns ranks=[1,4,8,6,3,5,7,0,2] 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient(hunting): picked mon.noname-f con 0x7f89d40a6880 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient(hunting): picked mon.noname-c con 0x7f89d409faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient(hunting): picked mon.noname-e con 0x7f89d4138d30 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d404ca10 con 0x7f89d40a6880 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d40a0580 con 0x7f89d409faa0 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d4133780 con 0x7f89d4138d30 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient(hunting): authenticate will time out at 1651.376203s 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d9d6c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f89d409faa0 0x7f89d409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40350/0 (socket says 172.21.15.26:40350) 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d9d6c700 1 -- 172.21.15.26:0/4230700687 learned_addr learned my addr 172.21.15.26:0/4230700687 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89dad6e700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f89d4138d30 0x7f89d413b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:58982/0 (socket says 172.21.15.26:58982) 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282251274 0 0) 0x7f89d0001110 con 0x7f89d409faa0 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): my global_id is 204394 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89c0003570 con 0x7f89d409faa0 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703603323 0 0) 0x7f89d4133780 con 0x7f89d4138d30 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): my global_id is 135314 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d0001110 con 0x7f89d4138d30 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354740003 0 0) 0x7f89d404ca10 con 0x7f89d40a6880 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient(hunting): my global_id is 204427 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d4133780 con 0x7f89d40a6880 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3642117587 0 0) 0x7f89c0003570 con 0x7f89d409faa0 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 >> v1:172.21.15.26:6793/0 conn(0x7f89d4138d30 legacy=0x7f89d413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 >> v1:172.21.15.122:6789/0 conn(0x7f89d40a6880 legacy=0x7f89d4133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: _finish_hunting 0 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 monclient: found mon.noname-c 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d40a6c50 con 0x7f89d409faa0 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: _finish_auth 0 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 5 monclient: authenticate success, global_id 204394 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89d0001940 con 0x7f89d409faa0 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: dump: 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f89d0001c20 con 0x7f89d409faa0 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 4 set_mon_vals no callback set 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 1 -- 172.21.15.26:0/4230700687 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89d0004c30 con 0x7f89d409faa0 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89d956b700 10 monclient: dump: 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient: shutdown 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- 172.21.15.26:0/4230700687 >> v1:172.21.15.26:6791/0 conn(0x7f89d409faa0 legacy=0x7f89d409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- 172.21.15.26:0/4230700687 shutdown_connections 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- 172.21.15.26:0/4230700687 >> 172.21.15.26:0/4230700687 conn(0x7f89d406e2a0 msgr2=0x7f89d40a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- 172.21.15.26:0/4230700687 shutdown_connections 2024-09-15T03:58:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 1 -- 172.21.15.26:0/4230700687 wait complete. 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient: build_initial_monmap 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.944+0000 7f89db56f700 10 monclient: monmap: 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 Processor -- start 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 -- start start 2024-09-15T03:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: init 2024-09-15T03:58:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: _add_conns ranks=[6,4,2,0,5,1,7,3,8] 2024-09-15T03:58:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient(hunting): picked mon.noname-d con 0x7f89d409faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient(hunting): picked mon.noname-c con 0x7f89d40a6880 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient(hunting): picked mon.noname-b con 0x7f89d4138d30 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d40a18c0 con 0x7f89d4138d30 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d409cf10 con 0x7f89d40a6880 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d411cb30 con 0x7f89d409faa0 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient(hunting): authenticate will time out at 1651.381254s 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89da56d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f89d409faa0 0x7f89d4135670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37794/0 (socket says 172.21.15.26:37794) 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89da56d700 1 -- 172.21.15.26:0/1479534925 learned_addr learned my addr 172.21.15.26:0/1479534925 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616070829 0 0) 0x7f89d409cf10 con 0x7f89d40a6880 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): my global_id is 204403 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89b8003570 con 0x7f89d40a6880 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278667400 0 0) 0x7f89d411cb30 con 0x7f89d409faa0 2024-09-15T03:58:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): my global_id is 194676 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d409cf10 con 0x7f89d409faa0 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660246181 0 0) 0x7f89d40a18c0 con 0x7f89d4138d30 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): my global_id is 204446 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d411cb30 con 0x7f89d4138d30 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2196708352 0 0) 0x7f89d409cf10 con 0x7f89d409faa0 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89d40a18c0 con 0x7f89d409faa0 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89c4001b20 con 0x7f89d409faa0 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): dump: 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 873724588 0 0) 0x7f89b8003570 con 0x7f89d40a6880 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89d409cf10 con 0x7f89d40a6880 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4010727149 0 0) 0x7f89d411cb30 con 0x7f89d4138d30 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89b8003570 con 0x7f89d4138d30 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89d0004b70 con 0x7f89d40a6880 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): dump: 2024-09-15T03:58:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89cc001bb0 con 0x7f89d4138d30 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient(hunting): dump: 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 72987216 0 0) 0x7f89d40a18c0 con 0x7f89d409faa0 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6791/0 conn(0x7f89d40a6880 legacy=0x7f89d4135d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6790/0 conn(0x7f89d4138d30 legacy=0x7f89d4136490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 monclient: found mon.d 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4137220 con 0x7f89d409faa0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 5 monclient: authenticate success, global_id 194676 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: _renew_subs 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d411c7c0 con 0x7f89d409faa0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: _renew_subs 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d41329e0 con 0x7f89d409faa0 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f89c4002100 con 0x7f89d409faa0 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89c4002630 con 0x7f89d409faa0 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89d8d6a700 4 set_mon_vals no callback set 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.948+0000 7f89ca7fc700 10 monclient: dump: 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89db56f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f89bc006590 con 0x7f89bc002840 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488011373 0 0) 0x7f89bc006590 con 0x7f89bc002840 2024-09-15T03:58:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient(hunting): my global_id is 194616 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d40a18c0 con 0x7f89bc002840 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f89c4002810 con 0x7f89d409faa0 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(633..633 src has 435..633) v4 ==== 3987+0+0 (unknown 3216915949 0 0) 0x7f89c4043f90 con 0x7f89d409faa0 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4264727947 0 0) 0x7f89d40a18c0 con 0x7f89bc002840 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89bc006590 con 0x7f89bc002840 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89d0001d90 con 0x7f89bc002840 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient: dump: 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.980 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89dad6e700 10 monclient: get_auth_request con 0x7f89b80380e0 auth_method 0 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 381450098 0 0) 0x7f89bc006590 con 0x7f89bc002840 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89bc00a130 con 0x7f89bc002840 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f89d00055e0 con 0x7f89bc002840 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6789/0 conn(0x7f89bc002840 legacy=0x7f89bc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.952+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6789/0 conn(0x7f89bc002840 legacy=0x7f89bc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f89bc009620 con 0x7f89bc009090 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944833012 0 0) 0x7f89bc009620 con 0x7f89bc009090 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient(hunting): my global_id is 194625 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89bc006590 con 0x7f89bc009090 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2882304193 0 0) 0x7f89bc006590 con 0x7f89bc009090 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89bc009620 con 0x7f89bc009090 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f89d0005e20 con 0x7f89bc009090 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient: dump: 2024-09-15T03:58:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2260665378 0 0) 0x7f89bc009620 con 0x7f89bc009090 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f89bc0098d0 con 0x7f89bc009090 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 1346808778) 0x7f89d0008130 con 0x7f89bc009090 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6789/0 conn(0x7f89bc009090 legacy=0x7f89bc008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89ca7fc700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6789/0 conn(0x7f89bc009090 legacy=0x7f89bc008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.005 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:27.006 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.122:6812/20895 conn(0x7f89b80380e0 legacy=0x7f89b803a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 10 monclient: shutdown 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 >> v1:172.21.15.26:6792/0 conn(0x7f89d409faa0 legacy=0x7f89d4135670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89dad6e700 1 -- 172.21.15.26:0/1479534925 reap_dead start 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 shutdown_connections 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 >> 172.21.15.26:0/1479534925 conn(0x7f89d406e2a0 msgr2=0x7f89d40a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 shutdown_connections 2024-09-15T03:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:26.964+0000 7f89db56f700 1 -- 172.21.15.26:0/1479534925 wait complete. 2024-09-15T03:58:27.009 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:27.009 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T03:58:27.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:58:27.066 INFO:tasks.rados.rados.0.smithi122.stdout:2811: done (0 left) 2024-09-15T03:58:27.066 INFO:tasks.rados.rados.0.smithi122.stdout:2812: rollback oid 4 current snap is 315 2024-09-15T03:58:27.066 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 308 2024-09-15T03:58:27.066 INFO:tasks.rados.rados.0.smithi122.stdout:2813: snap_create 2024-09-15T03:58:27.080 INFO:tasks.rados.rados.0.smithi122.stdout:2812: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:58:27.082 INFO:tasks.rados.rados.0.smithi122.stdout:2812: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:58:27.082 INFO:tasks.rados.rados.0.smithi122.stdout:2812: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:58:27.082 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1888 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:58:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient: get_monmap_and_config 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient: monmap: 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:27.122396+0000 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:27.122396+0000 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.127 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 1 Processor -- start 2024-09-15T03:58:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 1 -- start start 2024-09-15T03:58:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient: init 2024-09-15T03:58:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient: _add_conns ranks=[7,4,1,5,6,0,8,3,2] 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient(hunting): picked mon.noname-i con 0x7f9fd8132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient(hunting): picked mon.noname-c con 0x7f9fd809faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient(hunting): picked mon.noname-f con 0x7f9fd8138d40 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd804ca10 con 0x7f9fd8138d40 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd809d5d0 con 0x7f9fd809faa0 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd8133620 con 0x7f9fd8132b40 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.116+0000 7f9fde519700 10 monclient(hunting): authenticate will time out at 1651.551886s 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fdcd16700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f9fd809faa0 0x7f9fd809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40368/0 (socket says 172.21.15.26:40368) 2024-09-15T03:58:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fdcd16700 1 -- 172.21.15.26:0/1395029238 learned_addr learned my addr 172.21.15.26:0/1395029238 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962331935 0 0) 0x7f9fd809d5d0 con 0x7f9fd809faa0 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient(hunting): my global_id is 204412 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc4003570 con 0x7f9fd809faa0 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424985621 0 0) 0x7f9fd804ca10 con 0x7f9fd8138d40 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient(hunting): my global_id is 204436 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd809d5d0 con 0x7f9fd8138d40 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 520869447 0 0) 0x7f9fc4003570 con 0x7f9fd809faa0 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 >> v1:172.21.15.122:6792/0 conn(0x7f9fd8132b40 legacy=0x7f9fd8132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 >> v1:172.21.15.122:6789/0 conn(0x7f9fd8138d40 legacy=0x7f9fd813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 monclient: found mon.noname-c 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80a0cc0 con 0x7f9fd809faa0 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9fd40017b0 con 0x7f9fd809faa0 2024-09-15T03:58:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: dump: 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.133 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 5 monclient: authenticate success, global_id 204412 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9fd4001a20 con 0x7f9fd809faa0 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 1 -- 172.21.15.26:0/1395029238 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9fd4001c10 con 0x7f9fd809faa0 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:27.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fcffff700 10 monclient: dump: 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.135 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 4 set_mon_vals no callback set 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 10 monclient: shutdown 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 -- 172.21.15.26:0/1395029238 >> v1:172.21.15.26:6791/0 conn(0x7f9fd809faa0 legacy=0x7f9fd809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 -- 172.21.15.26:0/1395029238 shutdown_connections 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 -- 172.21.15.26:0/1395029238 >> 172.21.15.26:0/1395029238 conn(0x7f9fd806e370 msgr2=0x7f9fd80a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 -- 172.21.15.26:0/1395029238 shutdown_connections 2024-09-15T03:58:27.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 -- 172.21.15.26:0/1395029238 wait complete. 2024-09-15T03:58:27.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 10 monclient: monmap: 2024-09-15T03:58:27.141 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 Processor -- start 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 1 -- start start 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.120+0000 7f9fde519700 10 monclient: init 2024-09-15T03:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient: _add_conns ranks=[5,3,2,4,6,0,7,8,1] 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient(hunting): picked mon.noname-h con 0x7f9fd809faa0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient(hunting): picked mon.noname-g con 0x7f9fd8132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient(hunting): picked mon.noname-b con 0x7f9fd8138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd809aa70 con 0x7f9fd8138d40 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd8098880 con 0x7f9fd8132b40 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd809a790 con 0x7f9fd809faa0 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient(hunting): authenticate will time out at 1651.557014s 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fdcd16700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f9fd8132b40 0x7f9fd8098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:35766/0 (socket says 172.21.15.26:35766) 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fdcd16700 1 -- 172.21.15.26:0/1093268601 learned_addr learned my addr 172.21.15.26:0/1093268601 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fddd18700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f9fd8138d40 0x7f9fd814a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42822/0 (socket says 172.21.15.26:42822) 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834236544 0 0) 0x7f9fd809aa70 con 0x7f9fd8138d40 2024-09-15T03:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): my global_id is 204455 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fbc0035a0 con 0x7f9fd8138d40 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563400464 0 0) 0x7f9fd8098880 con 0x7f9fd8132b40 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): my global_id is 204438 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd809aa70 con 0x7f9fd8132b40 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020899433 0 0) 0x7f9fd809a790 con 0x7f9fd809faa0 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): my global_id is 5936 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd8098880 con 0x7f9fd809faa0 2024-09-15T03:58:27.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4095347477 0 0) 0x7f9fbc0035a0 con 0x7f9fd8138d40 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9fd809a790 con 0x7f9fd8138d40 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9fd0001d00 con 0x7f9fd8138d40 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): dump: 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3939041128 0 0) 0x7f9fd809a790 con 0x7f9fd8138d40 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.122:6791/0 conn(0x7f9fd809faa0 legacy=0x7f9fd8097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.122:6790/0 conn(0x7f9fd8132b40 legacy=0x7f9fd8098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 monclient: found mon.b 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd809ad50 con 0x7f9fd8138d40 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 5 monclient: authenticate success, global_id 204455 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient: _renew_subs 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd809b9c0 con 0x7f9fd8138d40 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient: _renew_subs 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd80999d0 con 0x7f9fd8138d40 2024-09-15T03:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9fd00030c0 con 0x7f9fd8138d40 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9fd0002510 con 0x7f9fd8138d40 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcf7fe700 4 set_mon_vals no callback set 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 10 monclient: dump: 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9fd00026f0 con 0x7f9fd8138d40 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9fb8006590 con 0x7f9fb8002840 2024-09-15T03:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.124+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(634..634 src has 435..634) v4 ==== 3955+0+0 (unknown 3131246357 0 0) 0x7f9fd00440d0 con 0x7f9fd8138d40 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fdcd16700 10 monclient: get_auth_request con 0x7f9fbc0376e0 auth_method 0 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914306346 0 0) 0x7f9fb8006590 con 0x7f9fb8002840 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient(hunting): my global_id is 204464 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd809a790 con 0x7f9fb8002840 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3158208942 0 0) 0x7f9fd809a790 con 0x7f9fb8002840 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9fb8006590 con 0x7f9fb8002840 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9fc80019b0 con 0x7f9fb8002840 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient: dump: 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.152 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1409642538 0 0) 0x7f9fb8006590 con 0x7f9fb8002840 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fb800a130 con 0x7f9fb8002840 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f9fc8001f70 con 0x7f9fb8002840 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.26:6790/0 conn(0x7f9fb8002840 legacy=0x7f9fb8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.128+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.26:6790/0 conn(0x7f9fb8002840 legacy=0x7f9fb8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fde519700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f9fb8007520 con 0x7f9fb80068d0 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037379881 0 0) 0x7f9fb8007520 con 0x7f9fb80068d0 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 10 monclient(hunting): my global_id is 204473 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb8006590 con 0x7f9fb80068d0 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 366161923 0 0) 0x7f9fb8006590 con 0x7f9fb80068d0 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9fb8007520 con 0x7f9fb80068d0 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9fc8002780 con 0x7f9fb80068d0 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 10 monclient: dump: 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2915268320 0 0) 0x7f9fb8007520 con 0x7f9fb80068d0 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.136+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9fb8008760 con 0x7f9fb80068d0 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3368092975) 0x7f9fc8002ba0 con 0x7f9fb80068d0 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fcd7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fcd7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.26:6790/0 conn(0x7f9fb80068d0 legacy=0x7f9fb8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fcd7fa700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.26:6790/0 conn(0x7f9fb80068d0 legacy=0x7f9fb8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.157 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:27.158 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.159 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.160 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:27.161 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.162 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.163 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.164 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.166 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.167 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:27.168 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.169 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.170 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:27.171 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:27.172 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.174 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:27.176 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.122:6812/20895 conn(0x7f9fbc0376e0 legacy=0x7f9fbc039b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 10 monclient: shutdown 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 >> v1:172.21.15.26:6790/0 conn(0x7f9fd8138d40 legacy=0x7f9fd814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fddd18700 1 -- 172.21.15.26:0/1093268601 reap_dead start 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 shutdown_connections 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 >> 172.21.15.26:0/1093268601 conn(0x7f9fd806e370 msgr2=0x7f9fd812ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 shutdown_connections 2024-09-15T03:58:27.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.140+0000 7f9fde519700 1 -- 172.21.15.26:0/1093268601 wait complete. 2024-09-15T03:58:27.178 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:27.179 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T03:58:27.179 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.284+0000 7fdc3f997700 10 monclient: get_monmap_and_config 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.284+0000 7fdc3f997700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.284+0000 7fdc3f997700 10 monclient: monmap: 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:27.291552+0000 2024-09-15T03:58:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:27.291552+0000 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 1 Processor -- start 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 1 -- start start 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient: init 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient: _add_conns ranks=[0,4,2,5,3,7,1,8,6] 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient(hunting): picked mon.noname-a con 0x7fdc38067b20 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient(hunting): picked mon.noname-c con 0x7fdc3806e310 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient(hunting): picked mon.noname-b con 0x7fdc3806e6e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3804ca10 con 0x7fdc38067b20 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3809e700 con 0x7fdc3806e6e0 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38068e30 con 0x7fdc3806e310 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient(hunting): authenticate will time out at 1651.721374s 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3cf32700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fdc3806e310 0x7fdc38137e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40378/0 (socket says 172.21.15.26:40378) 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3df34700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fdc3806e6e0 0x7fdc3813b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42848/0 (socket says 172.21.15.26:42848) 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3cf32700 1 -- 172.21.15.26:0/2381202506 learned_addr learned my addr 172.21.15.26:0/2381202506 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3d733700 1 --1- 172.21.15.26:0/2381202506 >> v1:172.21.15.26:6789/0 conn(0x7fdc38067b20 0x7fdc38067ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47698/0 (socket says 172.21.15.26:47698) 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392686637 0 0) 0x7fdc3809e700 con 0x7fdc3806e6e0 2024-09-15T03:58:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): my global_id is 204482 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc24003570 con 0x7fdc3806e6e0 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952437522 0 0) 0x7fdc3804ca10 con 0x7fdc38067b20 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): my global_id is 194634 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc3809e700 con 0x7fdc38067b20 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547219052 0 0) 0x7fdc38068e30 con 0x7fdc3806e310 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient(hunting): my global_id is 204421 2024-09-15T03:58:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc3804ca10 con 0x7fdc3806e310 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3896331634 0 0) 0x7fdc24003570 con 0x7fdc3806e6e0 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 >> v1:172.21.15.26:6791/0 conn(0x7fdc3806e310 legacy=0x7fdc38137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 >> v1:172.21.15.26:6789/0 conn(0x7fdc38067b20 legacy=0x7fdc38067ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 monclient: found mon.noname-b 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380a0a40 con 0x7fdc3806e6e0 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 5 monclient: authenticate success, global_id 204482 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdc30001b30 con 0x7fdc3806e6e0 2024-09-15T03:58:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: dump: 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdc30001d50 con 0x7fdc3806e6e0 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc3f997700 4 set_mon_vals no callback set 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 1 -- 172.21.15.26:0/2381202506 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdc30004bb0 con 0x7fdc3806e6e0 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.288+0000 7fdc2ffff700 10 monclient: dump: 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient: shutdown 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- 172.21.15.26:0/2381202506 >> v1:172.21.15.26:6790/0 conn(0x7fdc3806e6e0 legacy=0x7fdc3813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- 172.21.15.26:0/2381202506 shutdown_connections 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- 172.21.15.26:0/2381202506 >> 172.21.15.26:0/2381202506 conn(0x7fdc38063d10 msgr2=0x7fdc380a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- 172.21.15.26:0/2381202506 shutdown_connections 2024-09-15T03:58:27.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- 172.21.15.26:0/2381202506 wait complete. 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient: monmap: 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 Processor -- start 2024-09-15T03:58:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- start start 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient: init 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient: _add_conns ranks=[3,6,2,0,8,5,1,4,7] 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient(hunting): picked mon.noname-g con 0x7fdc38067b20 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient(hunting): picked mon.noname-d con 0x7fdc3806e310 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient(hunting): picked mon.noname-b con 0x7fdc3806e6e0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38123810 con 0x7fdc3806e6e0 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3813e600 con 0x7fdc38067b20 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc38123250 con 0x7fdc3806e310 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3f997700 10 monclient(hunting): authenticate will time out at 1651.727627s 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3df34700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fdc3806e6e0 0x7fdc3814a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42854/0 (socket says 172.21.15.26:42854) 2024-09-15T03:58:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.292+0000 7fdc3df34700 1 -- 172.21.15.26:0/3153608281 learned_addr learned my addr 172.21.15.26:0/3153608281 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026575026 0 0) 0x7fdc38123810 con 0x7fdc3806e6e0 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): my global_id is 204491 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc1c003570 con 0x7fdc3806e6e0 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618149404 0 0) 0x7fdc38123250 con 0x7fdc3806e310 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): my global_id is 194685 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc38123810 con 0x7fdc3806e310 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3969305272 0 0) 0x7fdc1c003570 con 0x7fdc3806e6e0 2024-09-15T03:58:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc38123250 con 0x7fdc3806e6e0 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdc30001a30 con 0x7fdc3806e6e0 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): dump: 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.315 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153119927 0 0) 0x7fdc3813e600 con 0x7fdc38067b20 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): my global_id is 204447 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc1c003570 con 0x7fdc38067b20 2024-09-15T03:58:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2219665424 0 0) 0x7fdc38123250 con 0x7fdc3806e6e0 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.26:6792/0 conn(0x7fdc3806e310 legacy=0x7fdc38147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.122:6790/0 conn(0x7fdc38067b20 legacy=0x7fdc3813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 monclient: found mon.b 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38122f70 con 0x7fdc3806e6e0 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 5 monclient: authenticate success, global_id 204491 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 10 monclient: _renew_subs 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38123530 con 0x7fdc3806e6e0 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 10 monclient: _renew_subs 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc381220a0 con 0x7fdc3806e6e0 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdc30005010 con 0x7fdc3806e6e0 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdc300054c0 con 0x7fdc3806e6e0 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2f7fe700 4 set_mon_vals no callback set 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: dump: 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdc18006590 con 0x7fdc18002840 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128840348 0 0) 0x7fdc18006590 con 0x7fdc18002840 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient(hunting): my global_id is 204430 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc38123250 con 0x7fdc18002840 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdc3000fe80 con 0x7fdc3806e6e0 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(634..634 src has 435..634) v4 ==== 3955+0+0 (unknown 3131246357 0 0) 0x7fdc30044440 con 0x7fdc3806e6e0 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 890294672 0 0) 0x7fdc38123250 con 0x7fdc18002840 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc18006590 con 0x7fdc18002840 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdc34001510 con 0x7fdc18002840 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: dump: 2024-09-15T03:58:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3616959573 0 0) 0x7fdc18006590 con 0x7fdc18002840 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc1800a130 con 0x7fdc18002840 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc3cf32700 10 monclient: get_auth_request con 0x7fdc1c037520 auth_method 0 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdc34004830 con 0x7fdc18002840 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.26:6791/0 conn(0x7fdc18002840 legacy=0x7fdc18004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.296+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.26:6791/0 conn(0x7fdc18002840 legacy=0x7fdc18004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc3f997700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdc18007500 con 0x7fdc180068b0 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557300981 0 0) 0x7fdc18007500 con 0x7fdc180068b0 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 10 monclient(hunting): my global_id is 204439 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc18006590 con 0x7fdc180068b0 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4249025362 0 0) 0x7fdc18006590 con 0x7fdc180068b0 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc18007500 con 0x7fdc180068b0 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdc34005070 con 0x7fdc180068b0 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 10 monclient: dump: 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.324 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.325 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3287280935 0 0) 0x7fdc18007500 con 0x7fdc180068b0 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.304+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdc18008760 con 0x7fdc180068b0 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 3935676433) 0x7fdc34007360 con 0x7fdc180068b0 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc2d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc2d7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.26:6791/0 conn(0x7fdc180068b0 legacy=0x7fdc18006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc2d7fa700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.26:6791/0 conn(0x7fdc180068b0 legacy=0x7fdc18006d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:27.326 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.327 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.328 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:27.330 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.331 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.332 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.334 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.335 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:27.338 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.339 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.340 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:27.341 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.342 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.343 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.344 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.345 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.345 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.345 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:27.345 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.122:6812/20895 conn(0x7fdc1c037520 legacy=0x7fdc1c0399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 10 monclient: shutdown 2024-09-15T03:58:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3df34700 1 -- 172.21.15.26:0/3153608281 reap_dead start 2024-09-15T03:58:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 >> v1:172.21.15.26:6790/0 conn(0x7fdc3806e6e0 legacy=0x7fdc3814a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 shutdown_connections 2024-09-15T03:58:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 >> 172.21.15.26:0/3153608281 conn(0x7fdc38063d10 msgr2=0x7fdc380a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 shutdown_connections 2024-09-15T03:58:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.308+0000 7fdc3f997700 1 -- 172.21.15.26:0/3153608281 wait complete. 2024-09-15T03:58:27.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:27.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T03:58:27.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:58:27.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.452+0000 7f6ae540b700 10 monclient: get_monmap_and_config 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.452+0000 7f6ae540b700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.452+0000 7f6ae540b700 10 monclient: monmap: 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:27.459169+0000 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:27.459169+0000 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.464 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 1 Processor -- start 2024-09-15T03:58:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 1 -- start start 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient: init 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient: _add_conns ranks=[1,6,4,0,7,3,2,5,8] 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient(hunting): picked mon.noname-f con 0x7f6ae00a3890 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient(hunting): picked mon.noname-d con 0x7f6ae0099360 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient(hunting): picked mon.noname-c con 0x7f6ae0138ee0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae004ca10 con 0x7f6ae00a3890 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae006e6a0 con 0x7f6ae0138ee0 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae009e190 con 0x7f6ae0099360 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient(hunting): authenticate will time out at 1651.889139s 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ad7fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f6ae0099360 0x7f6ae0099730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37810/0 (socket says 172.21.15.26:37810) 2024-09-15T03:58:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ad7fff700 1 -- 172.21.15.26:0/610461239 learned_addr learned my addr 172.21.15.26:0/610461239 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027766437 0 0) 0x7f6ae009e190 con 0x7f6ae0099360 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): my global_id is 194694 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6acc003570 con 0x7f6ae0099360 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169358881 0 0) 0x7f6ae004ca10 con 0x7f6ae00a3890 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): my global_id is 204445 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae009e190 con 0x7f6ae00a3890 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824259620 0 0) 0x7f6ae006e6a0 con 0x7f6ae0138ee0 2024-09-15T03:58:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient(hunting): my global_id is 204448 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae004ca10 con 0x7f6ae0138ee0 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271698067 0 0) 0x7f6acc003570 con 0x7f6ae0099360 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 >> v1:172.21.15.26:6791/0 conn(0x7f6ae0138ee0 legacy=0x7f6ae013b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 >> v1:172.21.15.122:6789/0 conn(0x7f6ae00a3890 legacy=0x7f6ae01331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 monclient: found mon.noname-d 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae01338c0 con 0x7f6ae0099360 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ac80017b0 con 0x7f6ae0099360 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 5 monclient: authenticate success, global_id 194694 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: dump: 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ac8001a20 con 0x7f6ae0099360 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 1 -- 172.21.15.26:0/610461239 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ac8004c30 con 0x7f6ae0099360 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ade7fc700 10 monclient: dump: 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.473 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.473 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:27.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.456+0000 7f6ae540b700 4 set_mon_vals no callback set 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient: shutdown 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 -- 172.21.15.26:0/610461239 >> v1:172.21.15.26:6792/0 conn(0x7f6ae0099360 legacy=0x7f6ae0099730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 -- 172.21.15.26:0/610461239 shutdown_connections 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 -- 172.21.15.26:0/610461239 >> 172.21.15.26:0/610461239 conn(0x7f6ae0071350 msgr2=0x7f6ae00a2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 -- 172.21.15.26:0/610461239 shutdown_connections 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 -- 172.21.15.26:0/610461239 wait complete. 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient: monmap: 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.479 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 Processor -- start 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 1 -- start start 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient: init 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient: _add_conns ranks=[6,1,2,8,0,7,3,5,4] 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient(hunting): picked mon.noname-d con 0x7f6ae0099360 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient(hunting): picked mon.noname-f con 0x7f6ae0138ee0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.460+0000 7f6ae540b700 10 monclient(hunting): picked mon.noname-b con 0x7f6ae0131830 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae009ec80 con 0x7f6ae0138ee0 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae009e8c0 con 0x7f6ae0131830 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae0134780 con 0x7f6ae0099360 2024-09-15T03:58:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 monclient(hunting): authenticate will time out at 1651.896086s 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6adeffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f6ae0099360 0x7f6ae012f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37816/0 (socket says 172.21.15.26:37816) 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6adeffd700 1 -- 172.21.15.26:0/3588366890 learned_addr learned my addr 172.21.15.26:0/3588366890 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6adf7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6ae0131830 0x7f6ae012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42864/0 (socket says 172.21.15.26:42864) 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475797102 0 0) 0x7f6ae0134780 con 0x7f6ae0099360 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): my global_id is 194703 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac0003570 con 0x7f6ae0099360 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139598152 0 0) 0x7f6ae009e8c0 con 0x7f6ae0131830 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): my global_id is 204500 2024-09-15T03:58:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae0134780 con 0x7f6ae0131830 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410653818 0 0) 0x7f6ae009ec80 con 0x7f6ae0138ee0 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): my global_id is 204454 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae009e8c0 con 0x7f6ae0138ee0 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2836270270 0 0) 0x7f6ac0003570 con 0x7f6ae0099360 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ae009ec80 con 0x7f6ae0099360 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ad0002b10 con 0x7f6ae0099360 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.483 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.484 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3455854739 0 0) 0x7f6ae0134780 con 0x7f6ae0131830 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ac0003570 con 0x7f6ae0131830 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ad8001cc0 con 0x7f6ae0131830 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.486 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3700507102 0 0) 0x7f6ae009ec80 con 0x7f6ae0099360 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.26:6790/0 conn(0x7f6ae0131830 legacy=0x7f6ae012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.122:6789/0 conn(0x7f6ae0138ee0 legacy=0x7f6ae012f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 monclient: found mon.d 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae0133fa0 con 0x7f6ae0099360 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 5 monclient: authenticate success, global_id 194703 2024-09-15T03:58:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 monclient: _renew_subs 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae0133cc0 con 0x7f6ae0099360 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 monclient: _renew_subs 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae014b3a0 con 0x7f6ae0099360 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6ad00015e0 con 0x7f6ae0099360 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ad0001a90 con 0x7f6ae0099360 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient: dump: 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.491 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6addffb700 4 set_mon_vals no callback set 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6ad0001c70 con 0x7f6ae0099360 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ae004bbf0 con 0x7f6ae00a3890 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(634..634 src has 435..634) v4 ==== 3955+0+0 (unknown 3131246357 0 0) 0x7f6ad0043f40 con 0x7f6ae0099360 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593112656 0 0) 0x7f6ae004bbf0 con 0x7f6ae00a3890 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.464+0000 7f6ad6ffd700 10 monclient(hunting): my global_id is 194712 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae009ec80 con 0x7f6ae00a3890 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6adf7fe700 10 monclient: get_auth_request con 0x7f6ac0037d80 auth_method 0 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2621964639 0 0) 0x7f6ae009ec80 con 0x7f6ae00a3890 2024-09-15T03:58:27.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ae004bbf0 con 0x7f6ae00a3890 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ac8001770 con 0x7f6ae00a3890 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 10 monclient: dump: 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 384243632 0 0) 0x7f6ae004bbf0 con 0x7f6ae00a3890 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ae006fac0 con 0x7f6ae00a3890 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6ac80049f0 con 0x7f6ae00a3890 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.26:6792/0 conn(0x7f6ae00a3890 legacy=0x7f6ae009a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.468+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.26:6792/0 conn(0x7f6ae00a3890 legacy=0x7f6ae009a390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ae540b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ae0067f00 con 0x7f6ae002ad10 2024-09-15T03:58:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775232438 0 0) 0x7f6ae0067f00 con 0x7f6ae002ad10 2024-09-15T03:58:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient(hunting): my global_id is 194721 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae004bbf0 con 0x7f6ae002ad10 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1915022119 0 0) 0x7f6ae004bbf0 con 0x7f6ae002ad10 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ae0067f00 con 0x7f6ae002ad10 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6ac8005530 con 0x7f6ae002ad10 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient: dump: 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2228749930 0 0) 0x7f6ae0067f00 con 0x7f6ae002ad10 2024-09-15T03:58:27.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ae009b1b0 con 0x7f6ae002ad10 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 3895152155) 0x7f6ac80076d0 con 0x7f6ae002ad10 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.26:6792/0 conn(0x7f6ae002ad10 legacy=0x7f6ae009aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.480+0000 7f6ad6ffd700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.26:6792/0 conn(0x7f6ae002ad10 legacy=0x7f6ae009aaa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:58:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:27.502 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:27.503 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:27.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.505 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.506 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:27.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.508 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.509 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.510 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:27.511 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.513 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.514 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.515 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.516 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.517 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:27.518 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.122:6812/20895 conn(0x7f6ac0037d80 legacy=0x7f6ac003a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 10 monclient: shutdown 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 >> v1:172.21.15.26:6792/0 conn(0x7f6ae0099360 legacy=0x7f6ae012f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6adf7fe700 1 -- 172.21.15.26:0/3588366890 reap_dead start 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6adf7fe700 1 -- 172.21.15.26:0/3588366890 reap_dead start 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 shutdown_connections 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 >> 172.21.15.26:0/3588366890 conn(0x7f6ae0071350 msgr2=0x7f6ae0098990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 shutdown_connections 2024-09-15T03:58:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.484+0000 7f6ae540b700 1 -- 172.21.15.26:0/3588366890 wait complete. 2024-09-15T03:58:27.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:27.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T03:58:27.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.628+0000 7f7015412700 10 monclient: get_monmap_and_config 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.628+0000 7f7015412700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.628+0000 7f7015412700 10 monclient: monmap: 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:27.635555+0000 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:27.635555+0000 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.640 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 Processor -- start 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- start start 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient: init 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient: _add_conns ranks=[6,3,5,7,1,2,8,4,0] 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient(hunting): picked mon.noname-d con 0x7f7010132b40 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient(hunting): picked mon.noname-g con 0x7f701009faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:27.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient(hunting): picked mon.noname-h con 0x7f7010138d40 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701004ca10 con 0x7f701009faa0 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701009d5d0 con 0x7f7010138d40 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7010133620 con 0x7f7010132b40 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient(hunting): authenticate will time out at 1652.065511s 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700ffff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f7010132b40 0x7f7010132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37850/0 (socket says 172.21.15.26:37850) 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700ffff700 1 -- 172.21.15.26:0/3896433457 learned_addr learned my addr 172.21.15.26:0/3896433457 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114067481 0 0) 0x7f7010133620 con 0x7f7010132b40 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient(hunting): my global_id is 194730 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff8003570 con 0x7f7010132b40 2024-09-15T03:58:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1379380926 0 0) 0x7f6ff8003570 con 0x7f7010132b40 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 >> v1:172.21.15.122:6791/0 conn(0x7f7010138d40 legacy=0x7f701013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 >> v1:172.21.15.122:6790/0 conn(0x7f701009faa0 legacy=0x7f701009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 monclient: found mon.noname-d 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100a0cc0 con 0x7f7010132b40 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 5 monclient: authenticate success, global_id 194730 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7000001b30 con 0x7f7010132b40 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: dump: 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.646 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7000001de0 con 0x7f7010132b40 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 1 -- 172.21.15.26:0/3896433457 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7000004fb0 con 0x7f7010132b40 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 4 set_mon_vals no callback set 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f700effd700 10 monclient: dump: 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 10 monclient: shutdown 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- 172.21.15.26:0/3896433457 >> v1:172.21.15.26:6792/0 conn(0x7f7010132b40 legacy=0x7f7010132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- 172.21.15.26:0/3896433457 shutdown_connections 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- 172.21.15.26:0/3896433457 >> 172.21.15.26:0/3896433457 conn(0x7f701006e370 msgr2=0x7f70100a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- 172.21.15.26:0/3896433457 shutdown_connections 2024-09-15T03:58:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.632+0000 7f7015412700 1 -- 172.21.15.26:0/3896433457 wait complete. 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: monmap: 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:27.654 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 Processor -- start 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 -- start start 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: init 2024-09-15T03:58:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: _add_conns ranks=[7,2,6,5,3,4,1,8,0] 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient(hunting): picked mon.noname-i con 0x7f701009faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient(hunting): picked mon.noname-b con 0x7f7010132b40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient(hunting): picked mon.noname-d con 0x7f7010138d40 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701009cfa0 con 0x7f7010132b40 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701009c8d0 con 0x7f7010138d40 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701011c6e0 con 0x7f701009faa0 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient(hunting): authenticate will time out at 1652.069860s 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f700f7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f7010132b40 0x7f70100999a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42880/0 (socket says 172.21.15.26:42880) 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f700f7fe700 1 -- 172.21.15.26:0/3528636831 learned_addr learned my addr 172.21.15.26:0/3528636831 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862917673 0 0) 0x7f701009cfa0 con 0x7f7010132b40 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): my global_id is 204509 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ffc003570 con 0x7f7010132b40 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017758011 0 0) 0x7f701009c8d0 con 0x7f7010138d40 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): my global_id is 194739 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701009cfa0 con 0x7f7010138d40 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824146452 0 0) 0x7f701011c6e0 con 0x7f701009faa0 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): my global_id is 234178 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701009c8d0 con 0x7f701009faa0 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1798101462 0 0) 0x7f6ffc003570 con 0x7f7010132b40 2024-09-15T03:58:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f701011c6e0 con 0x7f7010132b40 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7004001b30 con 0x7f7010132b40 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): dump: 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.658 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1022657861 0 0) 0x7f701009cfa0 con 0x7f7010138d40 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ffc003570 con 0x7f7010138d40 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7008001820 con 0x7f7010138d40 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): dump: 2024-09-15T03:58:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3636640763 0 0) 0x7f701009c8d0 con 0x7f701009faa0 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f701009cfa0 con 0x7f701009faa0 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7000001af0 con 0x7f701009faa0 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient(hunting): dump: 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.661 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4128982824 0 0) 0x7f701011c6e0 con 0x7f7010132b40 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.122:6792/0 conn(0x7f701009faa0 legacy=0x7f7010099290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.26:6792/0 conn(0x7f7010138d40 legacy=0x7f701009a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 monclient: found mon.b 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701009cb40 con 0x7f7010132b40 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f6ff7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 5 monclient: authenticate success, global_id 204509 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: _renew_subs 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7010132510 con 0x7f7010132b40 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: _renew_subs 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.636+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f701014b3a0 con 0x7f7010132b40 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f70040021b0 con 0x7f7010132b40 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f700e7fc700 4 set_mon_vals no callback set 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f70040026e0 con 0x7f7010132b40 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: dump: 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f7015412700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ff0006590 con 0x7f6ff0002840 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f70040028c0 con 0x7f7010132b40 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(634..634 src has 435..634) v4 ==== 3955+0+0 (unknown 3131246357 0 0) 0x7f7004043f80 con 0x7f7010132b40 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19884316 0 0) 0x7f6ff0006590 con 0x7f6ff0002840 2024-09-15T03:58:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient(hunting): my global_id is 135323 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f701011c6e0 con 0x7f6ff0002840 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f7014c11700 10 monclient: get_auth_request con 0x7f6ffc0380b0 auth_method 0 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1756172243 0 0) 0x7f701011c6e0 con 0x7f6ff0002840 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ff0006590 con 0x7f6ff0002840 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7000001580 con 0x7f6ff0002840 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: dump: 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4291850958 0 0) 0x7f6ff0006590 con 0x7f6ff0002840 2024-09-15T03:58:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ff000a130 con 0x7f6ff0002840 2024-09-15T03:58:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7000005640 con 0x7f6ff0002840 2024-09-15T03:58:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.26:6793/0 conn(0x7f6ff0002840 legacy=0x7f6ff0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.640+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.26:6793/0 conn(0x7f6ff0002840 legacy=0x7f6ff0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f7015412700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6ff0009640 con 0x7f6ff00090b0 2024-09-15T03:58:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263235235 0 0) 0x7f6ff0009640 con 0x7f6ff00090b0 2024-09-15T03:58:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f6ff7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f6ff7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f6ff7fff700 10 monclient(hunting): my global_id is 135332 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.648+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff0006590 con 0x7f6ff00090b0 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3679390421 0 0) 0x7f6ff0006590 con 0x7f6ff00090b0 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6ff0009640 con 0x7f6ff00090b0 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7000005e00 con 0x7f6ff00090b0 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 10 monclient: dump: 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3463027194 0 0) 0x7f6ff0009640 con 0x7f6ff00090b0 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ff00098f0 con 0x7f6ff00090b0 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7568 (unknown 0 0 3181449367) 0x7f70000080d0 con 0x7f6ff00090b0 2024-09-15T03:58:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.26:6793/0 conn(0x7f6ff00090b0 legacy=0x7f6ff0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:27.672 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.673 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:27.674 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:27.675 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:27.676 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.678 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.680 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.682 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.689 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.690 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:27.691 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f6ff7fff700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.26:6793/0 conn(0x7f6ff00090b0 legacy=0x7f6ff0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.122:6812/20895 conn(0x7f6ffc0380b0 legacy=0x7f6ffc03a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 10 monclient: shutdown 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7014c11700 1 -- 172.21.15.26:0/3528636831 reap_dead start 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 >> v1:172.21.15.26:6790/0 conn(0x7f7010132b40 legacy=0x7f70100999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 shutdown_connections 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 >> 172.21.15.26:0/3528636831 conn(0x7f701006e370 msgr2=0x7f701009e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 shutdown_connections 2024-09-15T03:58:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.652+0000 7f7015412700 1 -- 172.21.15.26:0/3528636831 wait complete. 2024-09-15T03:58:27.694 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:27.694 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T03:58:27.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.800+0000 7fd8ba941700 10 monclient: get_monmap_and_config 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.800+0000 7fd8ba941700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.800+0000 7fd8ba941700 10 monclient: monmap: 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:27.807672+0000 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:27.807672+0000 2024-09-15T03:58:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.812 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 Processor -- start 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- start start 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient: init 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient: _add_conns ranks=[5,6,7,2,4,3,1,0,8] 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient(hunting): picked mon.noname-h con 0x7fd8b40a6880 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient(hunting): picked mon.noname-d con 0x7fd8b409faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient(hunting): picked mon.noname-i con 0x7fd8b4138d30 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b404ca10 con 0x7fd8b40a6880 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40a0580 con 0x7fd8b409faa0 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b4133780 con 0x7fd8b4138d30 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient(hunting): authenticate will time out at 1652.237074s 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b913e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd8b409faa0 0x7fd8b409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37870/0 (socket says 172.21.15.26:37870) 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b913e700 1 -- 172.21.15.26:0/3315168527 learned_addr learned my addr 172.21.15.26:0/3315168527 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930148992 0 0) 0x7fd8a4001490 con 0x7fd8b409faa0 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): my global_id is 194748 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c003570 con 0x7fd8b409faa0 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776255687 0 0) 0x7fd8b404ca10 con 0x7fd8b40a6880 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): my global_id is 5954 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a4001490 con 0x7fd8b40a6880 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149945164 0 0) 0x7fd8b4133780 con 0x7fd8b4138d30 2024-09-15T03:58:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient(hunting): my global_id is 234187 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b404ca10 con 0x7fd8b4138d30 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574932939 0 0) 0x7fd89c003570 con 0x7fd8b409faa0 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 >> v1:172.21.15.122:6792/0 conn(0x7fd8b4138d30 legacy=0x7fd8b413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 >> v1:172.21.15.122:6791/0 conn(0x7fd8b40a6880 legacy=0x7fd8b4133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 monclient: found mon.noname-d 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40a6c50 con 0x7fd8b409faa0 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 5 monclient: authenticate success, global_id 194748 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a4001cc0 con 0x7fd8b409faa0 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: dump: 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd8a4001fa0 con 0x7fd8b409faa0 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 4 set_mon_vals no callback set 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 1 -- 172.21.15.26:0/3315168527 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a4004fb0 con 0x7fd8b409faa0 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8b893d700 10 monclient: dump: 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 10 monclient: shutdown 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- 172.21.15.26:0/3315168527 >> v1:172.21.15.26:6792/0 conn(0x7fd8b409faa0 legacy=0x7fd8b409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- 172.21.15.26:0/3315168527 shutdown_connections 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- 172.21.15.26:0/3315168527 >> 172.21.15.26:0/3315168527 conn(0x7fd8b406e2a0 msgr2=0x7fd8b40a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- 172.21.15.26:0/3315168527 shutdown_connections 2024-09-15T03:58:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.804+0000 7fd8ba941700 1 -- 172.21.15.26:0/3315168527 wait complete. 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: monmap: 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.824 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 Processor -- start 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 -- start start 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: init 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: _add_conns ranks=[2,6,7,5,4,1,0,3,8] 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient(hunting): picked mon.noname-b con 0x7fd8b409faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient(hunting): picked mon.noname-d con 0x7fd8b40a6880 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient(hunting): picked mon.noname-i con 0x7fd8b4138d30 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40a18c0 con 0x7fd8b409faa0 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b409cf10 con 0x7fd8b40a6880 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b4131c20 con 0x7fd8b4138d30 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient(hunting): authenticate will time out at 1652.241472s 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b993f700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fd8b409faa0 0x7fd8b4135440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42890/0 (socket says 172.21.15.26:42890) 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b913e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd8b40a6880 0x7fd8b4135b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37872/0 (socket says 172.21.15.26:37872) 2024-09-15T03:58:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b993f700 1 -- 172.21.15.26:0/3571287152 learned_addr learned my addr 172.21.15.26:0/3571287152 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207416400 0 0) 0x7fd8b409cf10 con 0x7fd8b40a6880 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): my global_id is 194757 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a0003570 con 0x7fd8b40a6880 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531327914 0 0) 0x7fd8b40a18c0 con 0x7fd8b409faa0 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): my global_id is 204518 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b409cf10 con 0x7fd8b409faa0 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961249165 0 0) 0x7fd8b4131c20 con 0x7fd8b4138d30 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): my global_id is 234196 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b40a18c0 con 0x7fd8b4138d30 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3158337479 0 0) 0x7fd8a0003570 con 0x7fd8b40a6880 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8b4131c20 con 0x7fd8b40a6880 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a4004ef0 con 0x7fd8b40a6880 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): dump: 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2855881163 0 0) 0x7fd8b409cf10 con 0x7fd8b409faa0 2024-09-15T03:58:27.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd8a0003570 con 0x7fd8b409faa0 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a8001aa0 con 0x7fd8b409faa0 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient(hunting): dump: 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1862202248 0 0) 0x7fd8b4131c20 con 0x7fd8b40a6880 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.122:6792/0 conn(0x7fd8b4138d30 legacy=0x7fd8b4136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.26:6790/0 conn(0x7fd8b409faa0 legacy=0x7fd8b4135440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 monclient: found mon.d 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4137550 con 0x7fd8b40a6880 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 5 monclient: authenticate success, global_id 194757 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: _renew_subs 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b4137040 con 0x7fd8b40a6880 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: _renew_subs 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b4137c10 con 0x7fd8b40a6880 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd8a4002110 con 0x7fd8b40a6880 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a4005870 con 0x7fd8b40a6880 2024-09-15T03:58:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b3fff700 4 set_mon_vals no callback set 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.808+0000 7fd8b1ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient: dump: 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.837 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd8a4010230 con 0x7fd8b40a6880 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8ba941700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd894006590 con 0x7fd894002840 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(634..634 src has 435..634) v4 ==== 3955+0+0 (unknown 3131246357 0 0) 0x7fd8a4044540 con 0x7fd8b40a6880 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8ba140700 10 monclient: get_auth_request con 0x7fd8a0037c80 auth_method 0 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522911826 0 0) 0x7fd894006590 con 0x7fd894002840 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient(hunting): my global_id is 204463 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b4131c20 con 0x7fd894002840 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1056762398 0 0) 0x7fd8b4131c20 con 0x7fd894002840 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd894006590 con 0x7fd894002840 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a8002000 con 0x7fd894002840 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 10 monclient: dump: 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3359372043 0 0) 0x7fd894006590 con 0x7fd894002840 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.812+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd89400a130 con 0x7fd894002840 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.816+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd8a8002560 con 0x7fd894002840 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.816+0000 7fd8b1ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.816+0000 7fd8b1ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:27.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.816+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.122:6789/0 conn(0x7fd894002840 legacy=0x7fd894004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.816+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.122:6789/0 conn(0x7fd894002840 legacy=0x7fd894004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.824+0000 7fd8ba941700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.824+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd894009640 con 0x7fd8940090b0 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536272435 0 0) 0x7fd894009640 con 0x7fd8940090b0 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient(hunting): my global_id is 204472 2024-09-15T03:58:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd894006590 con 0x7fd8940090b0 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2746315530 0 0) 0x7fd894006590 con 0x7fd8940090b0 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd894009640 con 0x7fd8940090b0 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8a8002a60 con 0x7fd8940090b0 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient: dump: 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2590693904 0 0) 0x7fd894009640 con 0x7fd8940090b0 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd8940098f0 con 0x7fd8940090b0 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 70150504) 0x7fd8a8007cf0 con 0x7fd8940090b0 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.122:6789/0 conn(0x7fd8940090b0 legacy=0x7fd894008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8b1ffb700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.122:6789/0 conn(0x7fd8940090b0 legacy=0x7fd894008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:27.845 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.846 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.847 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:27.848 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:27.849 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:27.850 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.852 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.854 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.856 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:27.857 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.858 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.859 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.860 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.861 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.862 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:27.863 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.122:6812/20895 conn(0x7fd8a0037c80 legacy=0x7fd8a003a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 10 monclient: shutdown 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 >> v1:172.21.15.26:6792/0 conn(0x7fd8b40a6880 legacy=0x7fd8b4135b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba140700 1 -- 172.21.15.26:0/3571287152 reap_dead start 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba140700 1 -- 172.21.15.26:0/3571287152 reap_dead start 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 shutdown_connections 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 >> 172.21.15.26:0/3571287152 conn(0x7fd8b406e2a0 msgr2=0x7fd8b40a4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 shutdown_connections 2024-09-15T03:58:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.828+0000 7fd8ba941700 1 -- 172.21.15.26:0/3571287152 wait complete. 2024-09-15T03:58:27.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:27.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T03:58:27.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.976+0000 7ff3651d4700 10 monclient: get_monmap_and_config 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.976+0000 7ff3651d4700 10 monclient: build_initial_monmap 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.976+0000 7ff3651d4700 10 monclient: monmap: 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:27.983456+0000 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:27.983456+0000 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:27.988 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 Processor -- start 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- start start 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient: init 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient: _add_conns ranks=[0,4,7,5,6,8,3,1,2] 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient(hunting): picked mon.noname-a con 0x7ff3600a6880 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient(hunting): picked mon.noname-c con 0x7ff36009faa0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient(hunting): picked mon.noname-i con 0x7ff360138d30 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36004ca10 con 0x7ff3600a6880 2024-09-15T03:58:27.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3600a0580 con 0x7ff36009faa0 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff360133780 con 0x7ff360138d30 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient(hunting): authenticate will time out at 1652.413133s 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff3600a6880 0x7ff360133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47708/0 (socket says 172.21.15.26:47708) 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35ffff700 1 -- 172.21.15.26:0/1579591809 learned_addr learned my addr 172.21.15.26:0/1579591809 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff357fff700 1 --1- 172.21.15.26:0/1579591809 >> v1:172.21.15.26:6791/0 conn(0x7ff36009faa0 0x7ff36009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40410/0 (socket says 172.21.15.26:40410) 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127972239 0 0) 0x7ff348001110 con 0x7ff36009faa0 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): my global_id is 204457 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff34c003570 con 0x7ff36009faa0 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211630853 0 0) 0x7ff36004ca10 con 0x7ff3600a6880 2024-09-15T03:58:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): my global_id is 194643 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff348001110 con 0x7ff3600a6880 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983622961 0 0) 0x7ff360133780 con 0x7ff360138d30 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient(hunting): my global_id is 234205 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff36004ca10 con 0x7ff360138d30 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2872389046 0 0) 0x7ff34c003570 con 0x7ff36009faa0 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 >> v1:172.21.15.122:6792/0 conn(0x7ff360138d30 legacy=0x7ff36013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 >> v1:172.21.15.26:6789/0 conn(0x7ff3600a6880 legacy=0x7ff360133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 monclient: found mon.noname-c 2024-09-15T03:58:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3600a6c50 con 0x7ff36009faa0 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 5 monclient: authenticate success, global_id 204457 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff348001940 con 0x7ff36009faa0 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: dump: 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff348001c20 con 0x7ff36009faa0 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 1 -- 172.21.15.26:0/1579591809 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff348004b70 con 0x7ff36009faa0 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 4 set_mon_vals no callback set 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff35f7fe700 10 monclient: dump: 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 10 monclient: shutdown 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- 172.21.15.26:0/1579591809 >> v1:172.21.15.26:6791/0 conn(0x7ff36009faa0 legacy=0x7ff36009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- 172.21.15.26:0/1579591809 shutdown_connections 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- 172.21.15.26:0/1579591809 >> 172.21.15.26:0/1579591809 conn(0x7ff36006e2a0 msgr2=0x7ff3600a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- 172.21.15.26:0/1579591809 shutdown_connections 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.980+0000 7ff3651d4700 1 -- 172.21.15.26:0/1579591809 wait complete. 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: monmap: 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.002 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 Processor -- start 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 -- start start 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: init 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: _add_conns ranks=[1,3,4,0,5,8,6,2,7] 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient(hunting): picked mon.noname-f con 0x7ff36009faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient(hunting): picked mon.noname-g con 0x7ff3600a6880 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient(hunting): picked mon.noname-c con 0x7ff360138d30 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3600a18c0 con 0x7ff36009faa0 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36009cf10 con 0x7ff3600a6880 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff360136340 con 0x7ff360138d30 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient(hunting): authenticate will time out at 1652.418461s 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3649d3700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7ff360138d30 0x7ff36014a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40412/0 (socket says 172.21.15.26:40412) 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3649d3700 1 -- 172.21.15.26:0/520654089 learned_addr learned my addr 172.21.15.26:0/520654089 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138971666 0 0) 0x7ff360136340 con 0x7ff360138d30 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): my global_id is 204466 2024-09-15T03:58:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff340003570 con 0x7ff360138d30 2024-09-15T03:58:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3980159455 0 0) 0x7ff340003570 con 0x7ff360138d30 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff360136340 con 0x7ff360138d30 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff358002d90 con 0x7ff360138d30 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): dump: 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151193136 0 0) 0x7ff3600a18c0 con 0x7ff36009faa0 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): my global_id is 204481 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff340003570 con 0x7ff36009faa0 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154602664 0 0) 0x7ff36009cf10 con 0x7ff3600a6880 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient(hunting): my global_id is 204465 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3600a18c0 con 0x7ff3600a6880 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2004550330 0 0) 0x7ff360136340 con 0x7ff360138d30 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6790/0 conn(0x7ff3600a6880 legacy=0x7ff360135c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6789/0 conn(0x7ff36009faa0 legacy=0x7ff360135520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 monclient: found mon.c 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360137580 con 0x7ff360138d30 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 5 monclient: authenticate success, global_id 204466 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: _renew_subs 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff360137070 con 0x7ff360138d30 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: _renew_subs 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36011d2a0 con 0x7ff360138d30 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff358001940 con 0x7ff360138d30 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff358001e70 con 0x7ff360138d30 2024-09-15T03:58:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35cff9700 10 monclient: dump: 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.984+0000 7ff35effd700 4 set_mon_vals no callback set 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff358001fd0 con 0x7ff360138d30 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff3651d4700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(634..634 src has 435..634) v4 ==== 3955+0+0 (unknown 3131246357 0 0) 0x7ff358044190 con 0x7ff360138d30 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff344006590 con 0x7ff344002840 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35ffff700 10 monclient: get_auth_request con 0x7ff340037620 auth_method 0 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230018527 0 0) 0x7ff344006590 con 0x7ff344002840 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient(hunting): my global_id is 204474 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff360136340 con 0x7ff344002840 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 630950390 0 0) 0x7ff360136340 con 0x7ff344002840 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff344006590 con 0x7ff344002840 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff348004b70 con 0x7ff344002840 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient: dump: 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1817048612 0 0) 0x7ff344006590 con 0x7ff344002840 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff34400a130 con 0x7ff344002840 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff348001860 con 0x7ff344002840 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6790/0 conn(0x7ff344002840 legacy=0x7ff344004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.988+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6790/0 conn(0x7ff344002840 legacy=0x7ff344004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff3651d4700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff344009600 con 0x7ff344009070 2024-09-15T03:58:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134648650 0 0) 0x7ff344009600 con 0x7ff344009070 2024-09-15T03:58:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff35cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff35cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff35cff9700 10 monclient(hunting): my global_id is 204483 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:27.996+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff344006590 con 0x7ff344009070 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3248404176 0 0) 0x7ff344006590 con 0x7ff344009070 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff344009600 con 0x7ff344009070 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff348005530 con 0x7ff344009070 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 10 monclient: dump: 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 259357080 0 0) 0x7ff344009600 con 0x7ff344009070 2024-09-15T03:58:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3440098b0 con 0x7ff344009070 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 575442165) 0x7ff3480077b0 con 0x7ff344009070 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6790/0 conn(0x7ff344009070 legacy=0x7ff344008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff35cff9700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6790/0 conn(0x7ff344009070 legacy=0x7ff344008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:28.019 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.020 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.021 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.022 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:28.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.024 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.025 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:28.028 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.031 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.032 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.034 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.035 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.036 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.037 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.038 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:28.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.038 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.038 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:28.038 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.122:6812/20895 conn(0x7ff340037620 legacy=0x7ff340039ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 10 monclient: shutdown 2024-09-15T03:58:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 >> v1:172.21.15.26:6791/0 conn(0x7ff360138d30 legacy=0x7ff36014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3649d3700 1 -- 172.21.15.26:0/520654089 reap_dead start 2024-09-15T03:58:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3649d3700 1 -- 172.21.15.26:0/520654089 reap_dead start 2024-09-15T03:58:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 shutdown_connections 2024-09-15T03:58:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 >> 172.21.15.26:0/520654089 conn(0x7ff36006e2a0 msgr2=0x7ff360068110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 shutdown_connections 2024-09-15T03:58:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.000+0000 7ff3651d4700 1 -- 172.21.15.26:0/520654089 wait complete. 2024-09-15T03:58:28.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:28.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T03:58:28.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:58:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2812: done (1 left) 2024-09-15T03:58:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2813: done (0 left) 2024-09-15T03:58:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2814: copy_from oid 9 from oid 15 current snap is 316 2024-09-15T03:58:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2815: read oid 11 snap -1 2024-09-15T03:58:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2815: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T03:58:28.068 INFO:tasks.rados.rados.0.smithi122.stdout:2816: delete oid 39 current snap is 316 2024-09-15T03:58:28.074 INFO:tasks.rados.rados.0.smithi122.stdout:2816: done (2 left) 2024-09-15T03:58:28.074 INFO:tasks.rados.rados.0.smithi122.stdout:2817: delete oid 1 current snap is 316 2024-09-15T03:58:28.076 INFO:tasks.rados.rados.0.smithi122.stdout:2815: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: get_monmap_and_config 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: monmap: 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:28.153237+0000 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:28.153237+0000 2024-09-15T03:58:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 1 Processor -- start 2024-09-15T03:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 1 -- start start 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: init 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: _add_conns ranks=[6,4,1,5,7,8,2,3,0] 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient(hunting): picked mon.noname-d con 0x7ffa580a4920 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient(hunting): picked mon.noname-c con 0x7ffa58132dc0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient(hunting): picked mon.noname-f con 0x7ffa581338d0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5804ca10 con 0x7ffa581338d0 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5806a1d0 con 0x7ffa58132dc0 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa58069300 con 0x7ffa580a4920 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient(hunting): authenticate will time out at 1652.582543s 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa567fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7ffa58132dc0 0x7ffa581331c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40414/0 (socket says 172.21.15.26:40414) 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa56ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ffa580a4920 0x7ffa58068bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37884/0 (socket says 172.21.15.26:37884) 2024-09-15T03:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa567fc700 1 -- 172.21.15.26:0/562992785 learned_addr learned my addr 172.21.15.26:0/562992785 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070467733 0 0) 0x7ffa58069300 con 0x7ffa580a4920 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): my global_id is 194766 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa40003570 con 0x7ffa580a4920 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494965877 0 0) 0x7ffa5806a1d0 con 0x7ffa58132dc0 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): my global_id is 204475 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58069300 con 0x7ffa58132dc0 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590396301 0 0) 0x7ffa5804ca10 con 0x7ffa581338d0 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient(hunting): my global_id is 204490 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5806a1d0 con 0x7ffa581338d0 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3098079961 0 0) 0x7ffa40003570 con 0x7ffa580a4920 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 >> v1:172.21.15.26:6791/0 conn(0x7ffa58132dc0 legacy=0x7ffa581331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 >> v1:172.21.15.122:6789/0 conn(0x7ffa581338d0 legacy=0x7ffa5813b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 monclient: found mon.noname-d 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580a4cf0 con 0x7ffa580a4920 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 5 monclient: authenticate success, global_id 194766 2024-09-15T03:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa480017b0 con 0x7ffa580a4920 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: dump: 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa480019d0 con 0x7ffa580a4920 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa57fff700 4 set_mon_vals no callback set 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 1 -- 172.21.15.26:0/562992785 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa48004830 con 0x7ffa580a4920 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.148+0000 7ffa55ffb700 10 monclient: dump: 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient: shutdown 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- 172.21.15.26:0/562992785 >> v1:172.21.15.26:6792/0 conn(0x7ffa580a4920 legacy=0x7ffa58068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- 172.21.15.26:0/562992785 shutdown_connections 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- 172.21.15.26:0/562992785 >> 172.21.15.26:0/562992785 conn(0x7ffa58069a70 msgr2=0x7ffa580a4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- 172.21.15.26:0/562992785 shutdown_connections 2024-09-15T03:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- 172.21.15.26:0/562992785 wait complete. 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient: monmap: 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 Processor -- start 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- start start 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient: init 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient: _add_conns ranks=[2,6,4,7,3,5,8,1,0] 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient(hunting): picked mon.noname-b con 0x7ffa580a4920 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient(hunting): picked mon.noname-d con 0x7ffa58132dc0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient(hunting): picked mon.noname-c con 0x7ffa581338d0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa580a01b0 con 0x7ffa580a4920 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa58069530 con 0x7ffa581338d0 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa58122310 con 0x7ffa58132dc0 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa57fff700 10 monclient(hunting): authenticate will time out at 1652.587752s 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa567fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ffa58132dc0 0x7ffa5811fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37898/0 (socket says 172.21.15.26:37898) 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa567fc700 1 -- 172.21.15.26:0/412732240 learned_addr learned my addr 172.21.15.26:0/412732240 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.152+0000 7ffa56ffd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ffa580a4920 0x7ffa581420e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42904/0 (socket says 172.21.15.26:42904) 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118957833 0 0) 0x7ffa58122310 con 0x7ffa58132dc0 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): my global_id is 194775 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa440035a0 con 0x7ffa58132dc0 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609548549 0 0) 0x7ffa580a01b0 con 0x7ffa580a4920 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): my global_id is 204527 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58122310 con 0x7ffa580a4920 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029363613 0 0) 0x7ffa58069530 con 0x7ffa581338d0 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): my global_id is 204484 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa580a01b0 con 0x7ffa581338d0 2024-09-15T03:58:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2600481163 0 0) 0x7ffa440035a0 con 0x7ffa58132dc0 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa58069530 con 0x7ffa58132dc0 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa4c001b20 con 0x7ffa58132dc0 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): dump: 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3292681828 0 0) 0x7ffa580a01b0 con 0x7ffa581338d0 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa440035a0 con 0x7ffa581338d0 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa50001c80 con 0x7ffa581338d0 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): dump: 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4287375683 0 0) 0x7ffa58122310 con 0x7ffa580a4920 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa580a01b0 con 0x7ffa580a4920 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 193206396 0 0) 0x7ffa58069530 con 0x7ffa58132dc0 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.26:6791/0 conn(0x7ffa581338d0 legacy=0x7ffa5814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.26:6790/0 conn(0x7ffa580a4920 legacy=0x7ffa581420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 monclient: found mon.d 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:28.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58120df0 con 0x7ffa58132dc0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 5 monclient: authenticate success, global_id 194775 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 10 monclient: _renew_subs 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa58122f70 con 0x7ffa58132dc0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 10 monclient: _renew_subs 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa58121d50 con 0x7ffa58132dc0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa4c004b60 con 0x7ffa58132dc0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa4c005010 con 0x7ffa58132dc0 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa557fa700 4 set_mon_vals no callback set 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient: dump: 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffa4c00f9d0 con 0x7ffa58132dc0 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7ffa4c043f60 con 0x7ffa58132dc0 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffa38006590 con 0x7ffa38002840 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa56ffd700 10 monclient: get_auth_request con 0x7ffa44038030 auth_method 0 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638498416 0 0) 0x7ffa38006590 con 0x7ffa38002840 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 10 monclient(hunting): my global_id is 5963 2024-09-15T03:58:28.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.156+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58069530 con 0x7ffa38002840 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3114946907 0 0) 0x7ffa58069530 con 0x7ffa38002840 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa38006590 con 0x7ffa38002840 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa50002410 con 0x7ffa38002840 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 10 monclient: dump: 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.184 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 685351358 0 0) 0x7ffa38006590 con 0x7ffa38002840 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa3800a130 con 0x7ffa38002840 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ffa500029c0 con 0x7ffa38002840 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:28.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.122:6791/0 conn(0x7ffa38002840 legacy=0x7ffa38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.160+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.122:6791/0 conn(0x7ffa38002840 legacy=0x7ffa38004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa57fff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffa38007520 con 0x7ffa380068d0 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477683172 0 0) 0x7ffa38007520 con 0x7ffa380068d0 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa3effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa3effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa3effd700 10 monclient(hunting): my global_id is 5972 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.168+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa38006590 con 0x7ffa380068d0 2024-09-15T03:58:28.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1701800672 0 0) 0x7ffa38006590 con 0x7ffa380068d0 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa38007520 con 0x7ffa380068d0 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa500032b0 con 0x7ffa380068d0 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 10 monclient: dump: 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1248773943 0 0) 0x7ffa38007520 con 0x7ffa380068d0 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffa38008760 con 0x7ffa380068d0 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7740 (unknown 0 0 2265597704) 0x7ffa500062a0 con 0x7ffa380068d0 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.122:6791/0 conn(0x7ffa380068d0 legacy=0x7ffa38006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa3effd700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.122:6791/0 conn(0x7ffa380068d0 legacy=0x7ffa38006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:28.190 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:28.191 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.192 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:28.193 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:28.194 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.195 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:28.200 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.202 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:28.204 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.206 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:28.207 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.209 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:28.210 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.122:6812/20895 conn(0x7ffa44038030 legacy=0x7ffa4403a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 10 monclient: shutdown 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 >> v1:172.21.15.26:6792/0 conn(0x7ffa58132dc0 legacy=0x7ffa5811fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa577fe700 1 -- 172.21.15.26:0/412732240 reap_dead start 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 shutdown_connections 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 >> 172.21.15.26:0/412732240 conn(0x7ffa58069a70 msgr2=0x7ffa580a35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 shutdown_connections 2024-09-15T03:58:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.172+0000 7ffa57fff700 1 -- 172.21.15.26:0/412732240 wait complete. 2024-09-15T03:58:28.212 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:28.213 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T03:58:28.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:58:28.284 INFO:tasks.rados.rados.0.smithi122.stdout:2814: finishing copy_from racing read to smithi12223906-9 2024-09-15T03:58:28.285 INFO:tasks.rados.rados.0.smithi122.stdout:2814: finishing copy_from to smithi12223906-9 2024-09-15T03:58:28.285 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 1334 (ObjNum 823 snap 269 seq_num 823) dirty exists 2024-09-15T03:58:28.285 INFO:tasks.rados.rados.0.smithi122.stdout:2814: done (2 left) 2024-09-15T03:58:28.285 INFO:tasks.rados.rados.0.smithi122.stdout:2815: done (1 left) 2024-09-15T03:58:28.285 INFO:tasks.rados.rados.0.smithi122.stdout:2817: done (0 left) 2024-09-15T03:58:28.286 INFO:tasks.rados.rados.0.smithi122.stdout:2818: snap_remove snap 237 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.312+0000 7f799e66c700 10 monclient: get_monmap_and_config 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.312+0000 7f799e66c700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.312+0000 7f799e66c700 10 monclient: monmap: 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:28.319735+0000 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:28.319735+0000 2024-09-15T03:58:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.325 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 1 Processor -- start 2024-09-15T03:58:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 1 -- start start 2024-09-15T03:58:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient: init 2024-09-15T03:58:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient: _add_conns ranks=[4,5,0,6,7,3,2,8,1] 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient(hunting): picked mon.noname-c con 0x7f7998099e40 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient(hunting): picked mon.noname-h con 0x7f799812f5d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient(hunting): picked mon.noname-a con 0x7f79981300b0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799804ca10 con 0x7f79981300b0 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799809c910 con 0x7f7998099e40 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799809a210 con 0x7f799812f5d0 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799d66a700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f7998099e40 0x7f799809b610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40432/0 (socket says 172.21.15.26:40432) 2024-09-15T03:58:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799d66a700 1 -- 172.21.15.26:0/1134481427 learned_addr learned my addr 172.21.15.26:0/1134481427 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f799e66c700 10 monclient(hunting): authenticate will time out at 1652.751054s 2024-09-15T03:58:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101069425 0 0) 0x7f799809c910 con 0x7f7998099e40 2024-09-15T03:58:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): my global_id is 204493 2024-09-15T03:58:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7984004680 con 0x7f7998099e40 2024-09-15T03:58:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012387818 0 0) 0x7f799804ca10 con 0x7f79981300b0 2024-09-15T03:58:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): my global_id is 194652 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799809c910 con 0x7f79981300b0 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173447436 0 0) 0x7f799809a210 con 0x7f799812f5d0 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 10 monclient(hunting): my global_id is 5981 2024-09-15T03:58:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.316+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799804ca10 con 0x7f799812f5d0 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450406943 0 0) 0x7f799809c910 con 0x7f79981300b0 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 >> v1:172.21.15.122:6791/0 conn(0x7f799812f5d0 legacy=0x7f799812f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 >> v1:172.21.15.26:6791/0 conn(0x7f7998099e40 legacy=0x7f799809b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 monclient: found mon.noname-a 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:58:28.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799809ea80 con 0x7f79981300b0 2024-09-15T03:58:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f79900019c0 con 0x7f79981300b0 2024-09-15T03:58:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 5 monclient: authenticate success, global_id 194652 2024-09-15T03:58:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: dump: 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7990001ce0 con 0x7f79981300b0 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 4 set_mon_vals no callback set 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 1 -- 172.21.15.26:0/1134481427 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7990002280 con 0x7f79981300b0 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798ffff700 10 monclient: dump: 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: shutdown 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- 172.21.15.26:0/1134481427 >> v1:172.21.15.26:6789/0 conn(0x7f79981300b0 legacy=0x7f7998137830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- 172.21.15.26:0/1134481427 shutdown_connections 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- 172.21.15.26:0/1134481427 >> 172.21.15.26:0/1134481427 conn(0x7f79980a2480 msgr2=0x7f79980a2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- 172.21.15.26:0/1134481427 shutdown_connections 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- 172.21.15.26:0/1134481427 wait complete. 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: monmap: 2024-09-15T03:58:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 Processor -- start 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- start start 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: init 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient: _add_conns ranks=[2,7,5,8,6,1,3,0,4] 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient(hunting): picked mon.noname-b con 0x7f7998099e40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient(hunting): picked mon.noname-i con 0x7f799812f5d0 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient(hunting): picked mon.noname-h con 0x7f79981300b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799809a4d0 con 0x7f7998099e40 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799812d1a0 con 0x7f79981300b0 2024-09-15T03:58:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f799812cec0 con 0x7f799812f5d0 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799e66c700 10 monclient(hunting): authenticate will time out at 1652.755147s 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799d66a700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f7998099e40 0x7f799812e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42920/0 (socket says 172.21.15.26:42920) 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f799d66a700 1 -- 172.21.15.26:0/114696808 learned_addr learned my addr 172.21.15.26:0/114696808 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504274070 0 0) 0x7f799809a4d0 con 0x7f7998099e40 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798d7fa700 10 monclient(hunting): my global_id is 204536 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f797c003570 con 0x7f7998099e40 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.320+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157821502 0 0) 0x7f799812d1a0 con 0x7f79981300b0 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): my global_id is 5990 2024-09-15T03:58:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799809a4d0 con 0x7f79981300b0 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351021445 0 0) 0x7f799812cec0 con 0x7f799812f5d0 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): my global_id is 234214 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799812d1a0 con 0x7f799812f5d0 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3252851469 0 0) 0x7f797c003570 con 0x7f7998099e40 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f799812cec0 con 0x7f7998099e40 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7994001ac0 con 0x7f7998099e40 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): dump: 2024-09-15T03:58:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3299336143 0 0) 0x7f799809a4d0 con 0x7f79981300b0 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f797c003570 con 0x7f79981300b0 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7990001980 con 0x7f79981300b0 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): dump: 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1837000327 0 0) 0x7f799812cec0 con 0x7f7998099e40 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6792/0 conn(0x7f799812f5d0 legacy=0x7f799813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6791/0 conn(0x7f79981300b0 legacy=0x7f7998147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 monclient: found mon.b 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799812cbe0 con 0x7f7998099e40 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 5 monclient: authenticate success, global_id 204536 2024-09-15T03:58:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 10 monclient: _renew_subs 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799812bcf0 con 0x7f7998099e40 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 10 monclient: _renew_subs 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f799812f260 con 0x7f7998099e40 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7994004ae0 con 0x7f7998099e40 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7994005010 con 0x7f7998099e40 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798f7fe700 4 set_mon_vals no callback set 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: dump: 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f79940051f0 con 0x7f7998099e40 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7978006590 con 0x7f7978002840 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f7994043ff0 con 0x7f7998099e40 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f799de6b700 10 monclient: get_auth_request con 0x7f797c037cd0 auth_method 0 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643720751 0 0) 0x7f7978006590 con 0x7f7978002840 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient(hunting): my global_id is 234223 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799812cec0 con 0x7f7978002840 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3043412594 0 0) 0x7f799812cec0 con 0x7f7978002840 2024-09-15T03:58:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7978006590 con 0x7f7978002840 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f79880021b0 con 0x7f7978002840 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.324+0000 7f798d7fa700 10 monclient: dump: 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 943823213 0 0) 0x7f7978006590 con 0x7f7978002840 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f797800a130 con 0x7f7978002840 2024-09-15T03:58:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7988004520 con 0x7f7978002840 2024-09-15T03:58:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6792/0 conn(0x7f7978002840 legacy=0x7f7978004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.328+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6792/0 conn(0x7f7978002840 legacy=0x7f7978004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f799e66c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7978009640 con 0x7f79780090b0 2024-09-15T03:58:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220761118 0 0) 0x7f7978009640 con 0x7f79780090b0 2024-09-15T03:58:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient(hunting): my global_id is 234232 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7978006590 con 0x7f79780090b0 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 465957566 0 0) 0x7f7978006590 con 0x7f79780090b0 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7978009640 con 0x7f79780090b0 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7988002cc0 con 0x7f79780090b0 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient: dump: 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.361 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2432418951 0 0) 0x7f7978009640 con 0x7f79780090b0 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f79780098f0 con 0x7f79780090b0 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7398 (unknown 0 0 4034417913) 0x7f79880041e0 con 0x7f79780090b0 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6792/0 conn(0x7f79780090b0 legacy=0x7f7978008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.340+0000 7f798d7fa700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6792/0 conn(0x7f79780090b0 legacy=0x7f7978008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:28.362 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 94, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:28.363 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.364 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.365 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:28.366 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:28.367 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.368 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:28.369 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.370 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:28.371 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.372 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:28.374 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.375 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:28.376 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.379 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:28.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.381 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:28.382 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.122:6812/20895 conn(0x7f797c037cd0 legacy=0x7f797c03a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 10 monclient: shutdown 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 >> v1:172.21.15.26:6790/0 conn(0x7f7998099e40 legacy=0x7f799812e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799de6b700 1 -- 172.21.15.26:0/114696808 reap_dead start 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799de6b700 1 -- 172.21.15.26:0/114696808 reap_dead start 2024-09-15T03:58:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 shutdown_connections 2024-09-15T03:58:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 >> 172.21.15.26:0/114696808 conn(0x7f79980a2480 msgr2=0x7f799809cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 shutdown_connections 2024-09-15T03:58:28.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.344+0000 7f799e66c700 1 -- 172.21.15.26:0/114696808 wait complete. 2024-09-15T03:58:28.385 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T03:58:28.385 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['g'] 2024-09-15T03:58:28.385 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T03:58:28.386 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T03:58:28.386 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T03:58:28.386 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T03:58:28.386 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T03:58:28.404 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T03:58:28.405 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-09-15T03:58:28.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: get_monmap_and_config 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: monmap: 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:28.512355+0000 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:28.512355+0000 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.517 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 1 Processor -- start 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 1 -- start start 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: init 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: _add_conns ranks=[6,5,1,7,3,0,4,8,2] 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient(hunting): picked mon.noname-d con 0x7f5a2809aa90 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient(hunting): picked mon.noname-h con 0x7f5a2809fa20 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient(hunting): picked mon.noname-f con 0x7f5a28139320 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2804ca10 con 0x7f5a28139320 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2809fdf0 con 0x7f5a2809fa20 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a28069110 con 0x7f5a2809aa90 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient(hunting): authenticate will time out at 1652.942027s 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2d085700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5a2809aa90 0x7f5a2806b530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37904/0 (socket says 172.21.15.26:37904) 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2d085700 1 -- 172.21.15.26:0/684163947 learned_addr learned my addr 172.21.15.26:0/684163947 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202700287 0 0) 0x7f5a28069110 con 0x7f5a2809aa90 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient(hunting): my global_id is 194784 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a14003570 con 0x7f5a2809aa90 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777803956 0 0) 0x7f5a2809fdf0 con 0x7f5a2809fa20 2024-09-15T03:58:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient(hunting): my global_id is 5999 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a28069110 con 0x7f5a2809fa20 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2744921494 0 0) 0x7f5a14003570 con 0x7f5a2809aa90 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 >> v1:172.21.15.122:6791/0 conn(0x7f5a2809fa20 legacy=0x7f5a28137c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 >> v1:172.21.15.122:6789/0 conn(0x7f5a28139320 legacy=0x7f5a2813b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 monclient: found mon.noname-d 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280685d0 con 0x7f5a2809aa90 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5a1c0017b0 con 0x7f5a2809aa90 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 5 monclient: authenticate success, global_id 194784 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: dump: 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5a1c0019d0 con 0x7f5a2809aa90 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a2f2e9700 4 set_mon_vals no callback set 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 1 -- 172.21.15.26:0/684163947 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5a1c0048f0 con 0x7f5a2809aa90 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.508+0000 7f5a1bfff700 10 monclient: dump: 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient: shutdown 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/684163947 >> v1:172.21.15.26:6792/0 conn(0x7f5a2809aa90 legacy=0x7f5a2806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/684163947 shutdown_connections 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/684163947 >> 172.21.15.26:0/684163947 conn(0x7f5a28063bb0 msgr2=0x7f5a280a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/684163947 shutdown_connections 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/684163947 wait complete. 2024-09-15T03:58:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient: build_initial_monmap 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient: monmap: 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:28.534 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 Processor -- start 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- start start 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient: init 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:28.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient: _add_conns ranks=[3,4,1,5,2,6,7,8,0] 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient(hunting): picked mon.noname-g con 0x7f5a2809aa90 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient(hunting): picked mon.noname-c con 0x7f5a2809fa20 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient(hunting): picked mon.noname-f con 0x7f5a28139320 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2806e210 con 0x7f5a28139320 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a28142990 con 0x7f5a2809aa90 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a28121330 con 0x7f5a2809fa20 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 10 monclient(hunting): authenticate will time out at 1652.946319s 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2d085700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f5a2809aa90 legacy=0x7f5a28142280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2c884700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f5a2809fa20 0x7f5a28146e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40444/0 (socket says 172.21.15.26:40444) 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2c884700 1 -- 172.21.15.26:0/1964220165 learned_addr learned my addr 172.21.15.26:0/1964220165 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042058498 0 0) 0x7f5a28121330 con 0x7f5a2809fa20 2024-09-15T03:58:28.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): my global_id is 204502 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c003570 con 0x7f5a2809fa20 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595047852 0 0) 0x7f5a2806e210 con 0x7f5a28139320 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): my global_id is 204508 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a28121330 con 0x7f5a28139320 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 227409631 0 0) 0x7f5a0c003570 con 0x7f5a2809fa20 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a2806e210 con 0x7f5a2809fa20 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5a24001bb0 con 0x7f5a2809fa20 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): dump: 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1184958185 0 0) 0x7f5a28121330 con 0x7f5a28139320 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5a0c003570 con 0x7f5a28139320 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5a20001c20 con 0x7f5a28139320 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient(hunting): dump: 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2813678898 0 0) 0x7f5a2806e210 con 0x7f5a2809fa20 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 >> v1:172.21.15.122:6790/0 conn(0x7f5a2809aa90 legacy=0x7f5a28142280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 >> v1:172.21.15.122:6789/0 conn(0x7f5a28139320 legacy=0x7f5a2814a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 monclient: found mon.c 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2806e6f0 con 0x7f5a2809fa20 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a197fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.512+0000 7f5a2f2e9700 5 monclient: authenticate success, global_id 204502 2024-09-15T03:58:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 monclient: _renew_subs 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a28122940 con 0x7f5a2809fa20 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 monclient: _renew_subs 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28138e80 con 0x7f5a2809fa20 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5a24002110 con 0x7f5a2809fa20 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5a24002640 con 0x7f5a2809fa20 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a1b7fe700 4 set_mon_vals no callback set 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 10 monclient: dump: 2024-09-15T03:58:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5a24002820 con 0x7f5a2809fa20 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a080050b0 con 0x7f5a2809fa20 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f5a24044040 con 0x7f5a2809fa20 2024-09-15T03:58:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.516+0000 7f5a2d085700 10 monclient: get_auth_request con 0x7f5a0c037b60 auth_method 0 2024-09-15T03:58:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.524+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f5a24014e60 con 0x7f5a2809fa20 2024-09-15T03:58:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.524+0000 7f5a197fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.524+0000 7f5a197fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:58:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:28.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5a080058e0 con 0x7f5a2809fa20 2024-09-15T03:58:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a197fa700 1 -- 172.21.15.26:0/1964220165 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2700 (unknown 254746657 0 849094672) 0x7f5a24014860 con 0x7f5a2809fa20 2024-09-15T03:58:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a197fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:28.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a197fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:28.761 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":94,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":5,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:58:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 >> v1:172.21.15.122:6812/20895 conn(0x7f5a0c037b60 legacy=0x7f5a0c03a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 10 monclient: shutdown 2024-09-15T03:58:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.752+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 >> v1:172.21.15.26:6791/0 conn(0x7f5a2809fa20 legacy=0x7f5a28146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.756+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 shutdown_connections 2024-09-15T03:58:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.756+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 >> 172.21.15.26:0/1964220165 conn(0x7f5a28063bb0 msgr2=0x7f5a280a29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.756+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 shutdown_connections 2024-09-15T03:58:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:28.756+0000 7f5a2f2e9700 1 -- 172.21.15.26:0/1964220165 wait complete. 2024-09-15T03:58:31.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient: get_monmap_and_config 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient: build_initial_monmap 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient: monmap: 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:31.884525+0000 2024-09-15T03:58:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:31.884525+0000 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:31.889 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 1 Processor -- start 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 1 -- start start 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient: init 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient: _add_conns ranks=[3,1,0,7,4,5,6,8,2] 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient(hunting): picked mon.noname-g con 0x7f8e700a6a80 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient(hunting): picked mon.noname-f con 0x7f8e7009fb80 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient(hunting): picked mon.noname-a con 0x7f8e70138d30 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7004ca10 con 0x7f8e70138d30 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e700a0f50 con 0x7f8e7009fb80 2024-09-15T03:58:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e70133680 con 0x7f8e700a6a80 2024-09-15T03:58:31.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:31.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6ffff700 10 monclient(hunting): authenticate will time out at 1656.314781s 2024-09-15T03:58:31.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6effd700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f8e700a6a80 legacy=0x7f8e70132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:31.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6f7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8e70138d30 0x7f8e7013b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47736/0 (socket says 172.21.15.26:47736) 2024-09-15T03:58:31.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:31.880+0000 7f8e6f7fe700 1 -- 172.21.15.26:0/3333055255 learned_addr learned my addr 172.21.15.26:0/3333055255 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:32.084+0000 7f8e6effd700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6790/0 conn(0x7f8e700a6a80 legacy=0x7f8e70132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:32.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:32.488+0000 7f8e6effd700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6790/0 conn(0x7f8e700a6a80 legacy=0x7f8e70132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:33.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:33.292+0000 7f8e6effd700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6790/0 conn(0x7f8e700a6a80 legacy=0x7f8e70132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient(hunting): tick 2024-09-15T03:58:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 monclient(hunting): continuing hunt 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6790/0 conn(0x7f8e700a6a80 legacy=0x7f8e70132f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6789/0 conn(0x7f8e7009fb80 legacy=0x7f8e7009ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6789/0 conn(0x7f8e70138d30 legacy=0x7f8e7013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient: _add_conns ranks=[8,2,5,4,7,6] 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-e con 0x7f8e50003810 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-b con 0x7f8e50007490 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-h con 0x7f8e5000b1f0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e700a6e50 con 0x7f8e50007490 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e70133680 con 0x7f8e5000b1f0 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e50011020 con 0x7f8e50003810 2024-09-15T03:58:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:34.884+0000 7f8e6cff9700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient(hunting): tick 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 monclient(hunting): continuing hunt 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6793/0 conn(0x7f8e50003810 legacy=0x7f8e50005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6791/0 conn(0x7f8e5000b1f0 legacy=0x7f8e5000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6790/0 conn(0x7f8e50007490 legacy=0x7f8e50009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6f7fe700 1 -- 172.21.15.26:0/3333055255 reap_dead start 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient: _add_conns ranks=[7,4,6] 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-i con 0x7f8e5000f070 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-c con 0x7f8e5000f510 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-d con 0x7f8e5000fb00 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6f7fe700 1 -- 172.21.15.26:0/3333055255 reap_dead start 2024-09-15T03:58:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e50012270 con 0x7f8e5000f510 2024-09-15T03:58:37.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e50010840 con 0x7f8e5000fb00 2024-09-15T03:58:37.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e50010a60 con 0x7f8e5000f070 2024-09-15T03:58:37.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:37.884+0000 7f8e6cff9700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.880+0000 7f8e6cff9700 10 monclient(hunting): tick 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.880+0000 7f8e6cff9700 1 monclient(hunting): continuing hunt 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.880+0000 7f8e6cff9700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.880+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.122:6792/0 conn(0x7f8e5000f070 legacy=0x7f8e50005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.880+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6792/0 conn(0x7f8e5000fb00 legacy=0x7f8e50009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.880+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6791/0 conn(0x7f8e5000f510 legacy=0x7f8e5000ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 10 monclient: _add_conns ranks=[2,8,4,3,6,1,7,5,0] 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-b con 0x7f8e5000a0f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-e con 0x7f8e5000a5e0 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 10 monclient(hunting): picked mon.noname-c con 0x7f8e5000f510 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e50010c80 con 0x7f8e5000a0f0 2024-09-15T03:58:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e50019670 con 0x7f8e5000f510 2024-09-15T03:58:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5001a8c0 con 0x7f8e5000a5e0 2024-09-15T03:58:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:40.884+0000 7f8e6cff9700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923948901 0 0) 0x7f8e50019670 con 0x7f8e5000f510 2024-09-15T03:58:43.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 10 monclient(hunting): my global_id is 204511 2024-09-15T03:58:43.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e5c003570 con 0x7f8e5000f510 2024-09-15T03:58:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458646609 0 0) 0x7f8e5001a8c0 con 0x7f8e5000a5e0 2024-09-15T03:58:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 10 monclient(hunting): my global_id is 135341 2024-09-15T03:58:43.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.236+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e50019670 con 0x7f8e5000a5e0 2024-09-15T03:58:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.256+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090880217 0 0) 0x7f8e50010c80 con 0x7f8e5000a0f0 2024-09-15T03:58:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.256+0000 7f8e6dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.256+0000 7f8e6dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.256+0000 7f8e6dffb700 10 monclient(hunting): my global_id is 204545 2024-09-15T03:58:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.256+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e5001a8c0 con 0x7f8e5000a0f0 2024-09-15T03:58:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2116202137 0 0) 0x7f8e5001a8c0 con 0x7f8e5000a0f0 2024-09-15T03:58:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6793/0 conn(0x7f8e5000a5e0 legacy=0x7f8e5000a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6791/0 conn(0x7f8e5000f510 legacy=0x7f8e5000ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 monclient: found mon.noname-b 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5001bb10 con 0x7f8e5000a0f0 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6ffff700 5 monclient: authenticate success, global_id 204545 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e60004400 con 0x7f8e5000a0f0 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: dump: 2024-09-15T03:58:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6ffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8e60004660 con 0x7f8e5000a0f0 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6ffff700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 1 -- 172.21.15.26:0/3333055255 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e60004b90 con 0x7f8e5000a0f0 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6ffff700 4 set_mon_vals no callback set 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.260+0000 7f8e6dffb700 10 monclient: dump: 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.278 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient: shutdown 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- 172.21.15.26:0/3333055255 >> v1:172.21.15.26:6790/0 conn(0x7f8e5000a0f0 legacy=0x7f8e5000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6f7fe700 1 -- 172.21.15.26:0/3333055255 reap_dead start 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- 172.21.15.26:0/3333055255 shutdown_connections 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- 172.21.15.26:0/3333055255 >> 172.21.15.26:0/3333055255 conn(0x7f8e7006e380 msgr2=0x7f8e700a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- 172.21.15.26:0/3333055255 shutdown_connections 2024-09-15T03:58:43.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- 172.21.15.26:0/3333055255 wait complete. 2024-09-15T03:58:43.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient: monmap: 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.284 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 Processor -- start 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- start start 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient: init 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient: _add_conns ranks=[2,4,8,1,0,6,7,3,5] 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient(hunting): picked mon.noname-b con 0x7f8e70135b10 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient(hunting): picked mon.noname-c con 0x7f8e701365f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:43.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient(hunting): picked mon.noname-e con 0x7f8e701369c0 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e70137d90 con 0x7f8e70135b10 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7011cfa0 con 0x7f8e701365f0 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7009cfc0 con 0x7f8e701369c0 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6ffff700 10 monclient(hunting): authenticate will time out at 1667.699405s 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6e7fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f8e701365f0 0x7f8e701471c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40560/0 (socket says 172.21.15.26:40560) 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.264+0000 7f8e6e7fc700 1 -- 172.21.15.26:0/1103900663 learned_addr learned my addr 172.21.15.26:0/1103900663 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417597559 0 0) 0x7f8e70137d90 con 0x7f8e70135b10 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): my global_id is 204554 2024-09-15T03:58:43.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e50010c40 con 0x7f8e70135b10 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355258733 0 0) 0x7f8e7011cfa0 con 0x7f8e701365f0 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): my global_id is 204520 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e70137d90 con 0x7f8e701365f0 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758259522 0 0) 0x7f8e7009cfc0 con 0x7f8e701369c0 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): my global_id is 135350 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e7011cfa0 con 0x7f8e701369c0 2024-09-15T03:58:43.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3179707427 0 0) 0x7f8e50010c40 con 0x7f8e70135b10 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e7009cfc0 con 0x7f8e70135b10 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e60004b10 con 0x7f8e70135b10 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.288 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.289 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3123604596 0 0) 0x7f8e70137d90 con 0x7f8e701365f0 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e50010c40 con 0x7f8e701365f0 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e580019f0 con 0x7f8e701365f0 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.293 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3834678832 0 0) 0x7f8e7011cfa0 con 0x7f8e701369c0 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e70137d90 con 0x7f8e701369c0 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e68001bd0 con 0x7f8e701369c0 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.295 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2040302428 0 0) 0x7f8e7009cfc0 con 0x7f8e70135b10 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 >> v1:172.21.15.26:6793/0 conn(0x7f8e701369c0 legacy=0x7f8e7014a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 >> v1:172.21.15.26:6791/0 conn(0x7f8e701365f0 legacy=0x7f8e701471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 monclient: found mon.b 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e701378a0 con 0x7f8e70135b10 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 5 monclient: authenticate success, global_id 204554 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 monclient: _renew_subs 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e700a1920 con 0x7f8e70135b10 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 monclient: _renew_subs 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e70137f60 con 0x7f8e70135b10 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8e60003ec0 con 0x7f8e70135b10 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e600043f0 con 0x7f8e70135b10 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6d7fa700 4 set_mon_vals no callback set 2024-09-15T03:58:43.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 10 monclient: dump: 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.298 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e540050b0 con 0x7f8e70135b10 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8e6000fc10 con 0x7f8e70135b10 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f8e60043e80 con 0x7f8e70135b10 2024-09-15T03:58:43.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.268+0000 7f8e6e7fc700 10 monclient: get_auth_request con 0x7f8e5003aeb0 auth_method 0 2024-09-15T03:58:43.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.276+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f8e600149d0 con 0x7f8e70135b10 2024-09-15T03:58:43.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.276+0000 7f8e66ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.276+0000 7f8e66ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e6ffff700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T03:58:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e6ffff700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e6ffff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8e540058e0 con 0x7f8e70135b10 2024-09-15T03:58:43.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e66ffd700 1 -- 172.21.15.26:0/1103900663 <== mon.2 v1:172.21.15.26:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2695 (unknown 254746657 0 4153451079) 0x7f8e600147f0 con 0x7f8e70135b10 2024-09-15T03:58:43.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e66ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T03:58:43.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.504+0000 7f8e66ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:43.513 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":102,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T03:58:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 >> v1:172.21.15.122:6812/20895 conn(0x7f8e5003aeb0 legacy=0x7f8e5003d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 10 monclient: shutdown 2024-09-15T03:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 >> v1:172.21.15.26:6790/0 conn(0x7f8e70135b10 legacy=0x7f8e70135ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 shutdown_connections 2024-09-15T03:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 >> 172.21.15.26:0/1103900663 conn(0x7f8e7006e380 msgr2=0x7f8e700a4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 shutdown_connections 2024-09-15T03:58:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.508+0000 7f8e6ffff700 1 -- 172.21.15.26:0/1103900663 wait complete. 2024-09-15T03:58:43.527 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-09-15T03:58:43.527 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-09-15T03:58:43.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T03:58:43.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient: get_monmap_and_config 2024-09-15T03:58:43.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient: monmap: 2024-09-15T03:58:43.637 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.637 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:43.633657+0000 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:43.633657+0000 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.639 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 1 Processor -- start 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 1 -- start start 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient: init 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient: _add_conns ranks=[6,1,3,7,8,4,0,2,5] 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient(hunting): picked mon.noname-d con 0x7f47e00a6880 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient(hunting): picked mon.noname-f con 0x7f47e009faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient(hunting): picked mon.noname-g con 0x7f47e0138d30 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e004ca10 con 0x7f47e009faa0 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e00a0580 con 0x7f47e0138d30 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e0133780 con 0x7f47e00a6880 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47df59e700 10 monclient(hunting): authenticate will time out at 1668.063010s 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47ded9d700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f47e0138d30 legacy=0x7f47e013b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47de59c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f47e00a6880 0x7f47e0133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:59136/0 (socket says 172.21.15.26:59136) 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47de59c700 1 -- 172.21.15.26:0/929807700 learned_addr learned my addr 172.21.15.26:0/929807700 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918417152 0 0) 0x7f47e0133780 con 0x7f47e00a6880 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 10 monclient(hunting): my global_id is 194793 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c8003570 con 0x7f47e00a6880 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115828590 0 0) 0x7f47e004ca10 con 0x7f47e009faa0 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 10 monclient(hunting): my global_id is 204517 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.628+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e0133780 con 0x7f47e009faa0 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3928242017 0 0) 0x7f47c8003570 con 0x7f47e00a6880 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 >> v1:172.21.15.122:6790/0 conn(0x7f47e0138d30 legacy=0x7f47e013b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 >> v1:172.21.15.122:6789/0 conn(0x7f47e009faa0 legacy=0x7f47e009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 monclient: found mon.noname-d 2024-09-15T03:58:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00a6c50 con 0x7f47e00a6880 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47d40017b0 con 0x7f47e00a6880 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 5 monclient: authenticate success, global_id 194793 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: dump: 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f47d4001a90 con 0x7f47e00a6880 2024-09-15T03:58:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 1 -- 172.21.15.26:0/929807700 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47d4004c30 con 0x7f47e00a6880 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47dd59a700 10 monclient: dump: 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 4 set_mon_vals no callback set 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 10 monclient: shutdown 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 -- 172.21.15.26:0/929807700 >> v1:172.21.15.26:6792/0 conn(0x7f47e00a6880 legacy=0x7f47e0133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 -- 172.21.15.26:0/929807700 shutdown_connections 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 -- 172.21.15.26:0/929807700 >> 172.21.15.26:0/929807700 conn(0x7f47e006e2a0 msgr2=0x7f47e00a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 -- 172.21.15.26:0/929807700 shutdown_connections 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 -- 172.21.15.26:0/929807700 wait complete. 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 10 monclient: monmap: 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.650 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 Processor -- start 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 1 -- start start 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.632+0000 7f47df59e700 10 monclient: init 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient: _add_conns ranks=[0,3,1,5,6,2,8,4,7] 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient(hunting): picked mon.noname-a con 0x7f47e009faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient(hunting): picked mon.noname-g con 0x7f47e00a6880 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:43.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient(hunting): picked mon.noname-f con 0x7f47e0138d30 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e009cf10 con 0x7f47e009faa0 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e011caa0 con 0x7f47e0138d30 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e0137430 con 0x7f47e00a6880 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient(hunting): authenticate will time out at 1668.068486s 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ddd9b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f47e00a6880 legacy=0x7f47e0135a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47de59c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f47e009faa0 0x7f47e0135300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55632/0 (socket says 172.21.15.26:55632) 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47de59c700 1 -- 172.21.15.26:0/2634344298 learned_addr learned my addr 172.21.15.26:0/2634344298 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271427501 0 0) 0x7f47e009cf10 con 0x7f47e009faa0 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): my global_id is 194661 2024-09-15T03:58:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c0003570 con 0x7f47e009faa0 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887229935 0 0) 0x7f47e011caa0 con 0x7f47e0138d30 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): my global_id is 204526 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e009cf10 con 0x7f47e0138d30 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4195602771 0 0) 0x7f47c0003570 con 0x7f47e009faa0 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47e011caa0 con 0x7f47e009faa0 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47d4004b70 con 0x7f47e009faa0 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): dump: 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.658 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3229480587 0 0) 0x7f47e011caa0 con 0x7f47e009faa0 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.122:6790/0 conn(0x7f47e00a6880 legacy=0x7f47e0135a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.122:6789/0 conn(0x7f47e0138d30 legacy=0x7f47e014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 monclient: found mon.a 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e01389e0 con 0x7f47e009faa0 2024-09-15T03:58:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 5 monclient: authenticate success, global_id 194661 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient: _renew_subs 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e0136120 con 0x7f47e009faa0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient: _renew_subs 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e0137af0 con 0x7f47e009faa0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f47d4004f00 con 0x7f47e009faa0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47d40053b0 con 0x7f47e009faa0 2024-09-15T03:58:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47dcd99700 4 set_mon_vals no callback set 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 10 monclient: dump: 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f47c4006590 con 0x7f47c4002840 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.636+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f47d400fd70 con 0x7f47e009faa0 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061064321 0 0) 0x7f47c4006590 con 0x7f47c4002840 2024-09-15T03:58:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient(hunting): my global_id is 194670 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e011caa0 con 0x7f47c4002840 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f47d40441d0 con 0x7f47e009faa0 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ded9d700 10 monclient: get_auth_request con 0x7f47c00374a0 auth_method 0 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2271681870 0 0) 0x7f47e011caa0 con 0x7f47c4002840 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47c4006590 con 0x7f47c4002840 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47d8003080 con 0x7f47c4002840 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient: dump: 2024-09-15T03:58:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3735811068 0 0) 0x7f47c4006590 con 0x7f47c4002840 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47c400a130 con 0x7f47c4002840 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f47d80068d0 con 0x7f47c4002840 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.26:6789/0 conn(0x7f47c4002840 legacy=0x7f47c4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.640+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.26:6789/0 conn(0x7f47c4002840 legacy=0x7f47c4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f47c4009640 con 0x7f47c40090b0 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658219858 0 0) 0x7f47c4009640 con 0x7f47c40090b0 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient(hunting): my global_id is 194679 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c4006590 con 0x7f47c40090b0 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 629222390 0 0) 0x7f47c4006590 con 0x7f47c40090b0 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47c4009640 con 0x7f47c40090b0 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f47d8005120 con 0x7f47c40090b0 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient: dump: 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.669 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 189266 0 0) 0x7f47c4009640 con 0x7f47c40090b0 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f47c40098f0 con 0x7f47c40090b0 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 1032271968) 0x7f47d8006620 con 0x7f47c40090b0 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.26:6789/0 conn(0x7f47c40090b0 legacy=0x7f47c4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ce7fc700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.26:6789/0 conn(0x7f47c40090b0 legacy=0x7f47c4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:43.671 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:43.672 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:43.673 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:43.674 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:43.675 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.676 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:43.677 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.679 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:43.681 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:43.682 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.683 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:43.685 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:43.686 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:43.687 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.689 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:43.690 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.122:6812/20895 conn(0x7f47c00374a0 legacy=0x7f47c0039950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 10 monclient: shutdown 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 >> v1:172.21.15.26:6789/0 conn(0x7f47e009faa0 legacy=0x7f47e0135300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ded9d700 1 -- 172.21.15.26:0/2634344298 reap_dead start 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47ded9d700 1 -- 172.21.15.26:0/2634344298 reap_dead start 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 shutdown_connections 2024-09-15T03:58:43.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 >> 172.21.15.26:0/2634344298 conn(0x7f47e006e2a0 msgr2=0x7f47e00a4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 shutdown_connections 2024-09-15T03:58:43.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.652+0000 7f47df59e700 1 -- 172.21.15.26:0/2634344298 wait complete. 2024-09-15T03:58:43.693 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:43.693 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-09-15T03:58:43.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient: get_monmap_and_config 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient: monmap: 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:43.800437+0000 2024-09-15T03:58:43.804 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:43.800437+0000 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.805 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 1 Processor -- start 2024-09-15T03:58:43.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 1 -- start start 2024-09-15T03:58:43.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient: init 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient: _add_conns ranks=[5,4,8,6,0,7,1,3,2] 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient(hunting): picked mon.noname-h con 0x7f8f000a4b20 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient(hunting): picked mon.noname-c con 0x7f8f00067f20 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient(hunting): picked mon.noname-e con 0x7f8f00139410 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0004ca10 con 0x7f8f00067f20 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f00068a80 con 0x7f8f000a4b20 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0009dae0 con 0x7f8f00139410 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 10 monclient(hunting): authenticate will time out at 1668.230487s 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8eff7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8f00139410 0x7f8f0013b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50938/0 (socket says 172.21.15.26:50938) 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8eff7fe700 1 -- 172.21.15.26:0/1110218765 learned_addr learned my addr 172.21.15.26:0/1110218765 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686664212 0 0) 0x7f8f0009dae0 con 0x7f8f00139410 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): my global_id is 135359 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee8003570 con 0x7f8f00139410 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034628701 0 0) 0x7f8f0004ca10 con 0x7f8f00067f20 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): my global_id is 204529 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f0009dae0 con 0x7f8f00067f20 2024-09-15T03:58:43.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305794000 0 0) 0x7f8f00068a80 con 0x7f8f000a4b20 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient(hunting): my global_id is 6008 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f0004ca10 con 0x7f8f000a4b20 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3877060351 0 0) 0x7f8f0009dae0 con 0x7f8f00067f20 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 >> v1:172.21.15.26:6793/0 conn(0x7f8f00139410 legacy=0x7f8f0013b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 >> v1:172.21.15.122:6791/0 conn(0x7f8f000a4b20 legacy=0x7f8f0009aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 monclient: found mon.noname-c 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f000a0670 con 0x7f8f00067f20 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 5 monclient: authenticate success, global_id 204529 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8f052d3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8ef4001790 con 0x7f8f00067f20 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.796+0000 7f8efdffb700 10 monclient: dump: 2024-09-15T03:58:43.812 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.813 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8ef40019c0 con 0x7f8f00067f20 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efdffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 4 set_mon_vals no callback set 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efdffb700 1 -- 172.21.15.26:0/1110218765 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8ef4001ef0 con 0x7f8f00067f20 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efdffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efdffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efdffb700 10 monclient: dump: 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.814 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: shutdown 2024-09-15T03:58:43.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- 172.21.15.26:0/1110218765 >> v1:172.21.15.26:6791/0 conn(0x7f8f00067f20 legacy=0x7f8f00137cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- 172.21.15.26:0/1110218765 shutdown_connections 2024-09-15T03:58:43.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- 172.21.15.26:0/1110218765 >> 172.21.15.26:0/1110218765 conn(0x7f8f00063a10 msgr2=0x7f8f000a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- 172.21.15.26:0/1110218765 shutdown_connections 2024-09-15T03:58:43.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- 172.21.15.26:0/1110218765 wait complete. 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: monmap: 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:43.820 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 Processor -- start 2024-09-15T03:58:43.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- start start 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: init 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient: _add_conns ranks=[8,3,7,6,2,0,1,5,4] 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient(hunting): picked mon.noname-e con 0x7f8f00067f20 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient(hunting): picked mon.noname-g con 0x7f8f000a4b20 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient(hunting): picked mon.noname-i con 0x7f8f00139410 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0006e690 con 0x7f8f000a4b20 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f00138340 con 0x7f8f00139410 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f00138a00 con 0x7f8f00067f20 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8f052d3700 10 monclient(hunting): authenticate will time out at 1668.235455s 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efe7fc700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f8f000a4b20 legacy=0x7f8f0006d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:43.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efeffd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8f00067f20 0x7f8f001419d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50940/0 (socket says 172.21.15.26:50940) 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8efeffd700 1 -- 172.21.15.26:0/2364733724 learned_addr learned my addr 172.21.15.26:0/2364733724 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500695094 0 0) 0x7f8f00138a00 con 0x7f8f00067f20 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8ee6ffd700 10 monclient(hunting): my global_id is 135368 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.800+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eec003570 con 0x7f8f00067f20 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725450219 0 0) 0x7f8f00138340 con 0x7f8f00139410 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): my global_id is 234241 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f00138a00 con 0x7f8f00139410 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1268621937 0 0) 0x7f8eec003570 con 0x7f8f00067f20 2024-09-15T03:58:43.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f00138340 con 0x7f8f00067f20 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8ef0001ea0 con 0x7f8f00067f20 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): dump: 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.824 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1917281468 0 0) 0x7f8f00138340 con 0x7f8f00067f20 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.122:6792/0 conn(0x7f8f00139410 legacy=0x7f8f00149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.122:6790/0 conn(0x7f8f000a4b20 legacy=0x7f8f0006d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 monclient: found mon.e 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0006b890 con 0x7f8f00067f20 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 5 monclient: authenticate success, global_id 135368 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 10 monclient: _renew_subs 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0006bb70 con 0x7f8f00067f20 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 10 monclient: _renew_subs 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f00142760 con 0x7f8f00067f20 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8ef0002480 con 0x7f8f00067f20 2024-09-15T03:58:43.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8ef00029b0 con 0x7f8f00067f20 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8efcff9700 4 set_mon_vals no callback set 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: dump: 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.827 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8ee0006590 con 0x7f8ee0002840 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8ef0002b10 con 0x7f8f00067f20 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f8ef0044280 con 0x7f8f00067f20 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212323036 0 0) 0x7f8ee0006590 con 0x7f8ee0002840 2024-09-15T03:58:43.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient(hunting): my global_id is 204563 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f00138340 con 0x7f8ee0002840 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8eff7fe700 10 monclient: get_auth_request con 0x7f8eec0374d0 auth_method 0 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3265131685 0 0) 0x7f8f00138340 con 0x7f8ee0002840 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ee0006590 con 0x7f8ee0002840 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8ef4001e70 con 0x7f8ee0002840 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: dump: 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.829 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 509652941 0 0) 0x7f8ee0006590 con 0x7f8ee0002840 2024-09-15T03:58:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ee000a130 con 0x7f8ee0002840 2024-09-15T03:58:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8ef4004aa0 con 0x7f8ee0002840 2024-09-15T03:58:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.26:6790/0 conn(0x7f8ee0002840 legacy=0x7f8ee0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.804+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.26:6790/0 conn(0x7f8ee0002840 legacy=0x7f8ee0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8f052d3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8ee0009640 con 0x7f8ee00090b0 2024-09-15T03:58:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592189515 0 0) 0x7f8ee0009640 con 0x7f8ee00090b0 2024-09-15T03:58:43.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 10 monclient(hunting): my global_id is 204572 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee0006590 con 0x7f8ee00090b0 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3067272220 0 0) 0x7f8ee0006590 con 0x7f8ee00090b0 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ee0009640 con 0x7f8ee00090b0 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8ef4005280 con 0x7f8ee00090b0 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.812+0000 7f8ee6ffd700 10 monclient: dump: 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3655473171 0 0) 0x7f8ee0009640 con 0x7f8ee00090b0 2024-09-15T03:58:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8ee00098f0 con 0x7f8ee00090b0 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7388 (unknown 0 0 3445108325) 0x7f8ef40075d0 con 0x7f8ee00090b0 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.26:6790/0 conn(0x7f8ee00090b0 legacy=0x7f8ee0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8ee6ffd700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.26:6790/0 conn(0x7f8ee00090b0 legacy=0x7f8ee0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:43.835 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:43.836 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:43.837 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:43.838 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:43.839 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:43.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:43.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.844 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:43.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:43.852 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:43.853 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:43.854 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:43.855 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:43.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.122:6812/20895 conn(0x7f8eec0374d0 legacy=0x7f8eec039980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 10 monclient: shutdown 2024-09-15T03:58:43.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 >> v1:172.21.15.26:6793/0 conn(0x7f8f00067f20 legacy=0x7f8f001419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8eff7fe700 1 -- 172.21.15.26:0/2364733724 reap_dead start 2024-09-15T03:58:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8eff7fe700 1 -- 172.21.15.26:0/2364733724 reap_dead start 2024-09-15T03:58:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 shutdown_connections 2024-09-15T03:58:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 >> 172.21.15.26:0/2364733724 conn(0x7f8f00063a10 msgr2=0x7f8f0009c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 shutdown_connections 2024-09-15T03:58:43.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.816+0000 7f8f052d3700 1 -- 172.21.15.26:0/2364733724 wait complete. 2024-09-15T03:58:43.857 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:43.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-09-15T03:58:43.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T03:58:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.964+0000 7f20e7bae700 10 monclient: get_monmap_and_config 2024-09-15T03:58:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.964+0000 7f20e7bae700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.964+0000 7f20e7bae700 10 monclient: monmap: 2024-09-15T03:58:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:43.970692+0000 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:43.970692+0000 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.975 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.964+0000 7f20e7bae700 1 Processor -- start 2024-09-15T03:58:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.964+0000 7f20e7bae700 1 -- start start 2024-09-15T03:58:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.964+0000 7f20e7bae700 10 monclient: init 2024-09-15T03:58:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient: _add_conns ranks=[2,0,1,4,5,6,7,8,3] 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient(hunting): picked mon.noname-b con 0x7f20e00993f0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient(hunting): picked mon.noname-a con 0x7f20e01333d0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient(hunting): picked mon.noname-f con 0x7f20e01337a0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e004ca10 con 0x7f20e01333d0 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00a11b0 con 0x7f20e01337a0 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00a2640 con 0x7f20e00993f0 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient(hunting): authenticate will time out at 1668.400431s 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e5149700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20e01333d0 0x7f20e0137fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55674/0 (socket says 172.21.15.26:55674) 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e594a700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f20e00993f0 0x7f20e00997c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:38920/0 (socket says 172.21.15.26:38920) 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e5149700 1 -- 172.21.15.26:0/1862980334 learned_addr learned my addr 172.21.15.26:0/1862980334 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913214907 0 0) 0x7f20e004ca10 con 0x7f20e01333d0 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): my global_id is 194688 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc003570 con 0x7f20e01333d0 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85903812 0 0) 0x7f20e00a11b0 con 0x7f20e01337a0 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): my global_id is 204535 2024-09-15T03:58:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e004ca10 con 0x7f20e01337a0 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948938240 0 0) 0x7f20e00a2640 con 0x7f20e00993f0 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient(hunting): my global_id is 204581 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e00a11b0 con 0x7f20e00993f0 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2647950516 0 0) 0x7f20cc003570 con 0x7f20e01333d0 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 >> v1:172.21.15.26:6790/0 conn(0x7f20e00993f0 legacy=0x7f20e00997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 >> v1:172.21.15.122:6789/0 conn(0x7f20e01337a0 legacy=0x7f20e013b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: _finish_hunting 0 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 monclient: found mon.noname-a 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:58:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0132e70 con 0x7f20e01333d0 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: _finish_auth 0 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20dc001b30 con 0x7f20e01333d0 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: dump: 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 5 monclient: authenticate success, global_id 194688 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f20dc001e10 con 0x7f20e01333d0 2024-09-15T03:58:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 4 set_mon_vals no callback set 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 1 -- 172.21.15.26:0/1862980334 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20dc004fb0 con 0x7f20e01333d0 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e4948700 10 monclient: dump: 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 10 monclient: shutdown 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- 172.21.15.26:0/1862980334 >> v1:172.21.15.26:6789/0 conn(0x7f20e01333d0 legacy=0x7f20e0137fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- 172.21.15.26:0/1862980334 shutdown_connections 2024-09-15T03:58:43.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- 172.21.15.26:0/1862980334 >> 172.21.15.26:0/1862980334 conn(0x7f20e00a6230 msgr2=0x7f20e00a6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- 172.21.15.26:0/1862980334 shutdown_connections 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.968+0000 7f20e7bae700 1 -- 172.21.15.26:0/1862980334 wait complete. 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: build_initial_monmap 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: monmap: 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:43.988 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 Processor -- start 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 -- start start 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: init 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: _add_conns ranks=[4,5,8,2,6,7,1,3,0] 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient(hunting): picked mon.noname-c con 0x7f20e00993f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient(hunting): picked mon.noname-h con 0x7f20e01333d0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient(hunting): picked mon.noname-e con 0x7f20e01337a0 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00a1df0 con 0x7f20e00993f0 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00a1b20 con 0x7f20e01333d0 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e01372a0 con 0x7f20e01337a0 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient(hunting): authenticate will time out at 1668.405967s 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e614b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f20e01337a0 0x7f20e014aa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50944/0 (socket says 172.21.15.26:50944) 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e614b700 1 -- 172.21.15.26:0/3730295074 learned_addr learned my addr 172.21.15.26:0/3730295074 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471617096 0 0) 0x7f20e00a1df0 con 0x7f20e00993f0 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): my global_id is 204538 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c4003570 con 0x7f20e00993f0 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305339733 0 0) 0x7f20e01372a0 con 0x7f20e01337a0 2024-09-15T03:58:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): my global_id is 135377 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e00a1df0 con 0x7f20e01337a0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172719445 0 0) 0x7f20e00a1b20 con 0x7f20e01333d0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): my global_id is 6017 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e01372a0 con 0x7f20e01333d0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 509474971 0 0) 0x7f20c4003570 con 0x7f20e00993f0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20e00a1b20 con 0x7f20e00993f0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1445410139 0 0) 0x7f20e00a1df0 con 0x7f20e01337a0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20c4003570 con 0x7f20e01337a0 2024-09-15T03:58:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20d0001a60 con 0x7f20e00993f0 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): dump: 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2941358479 0 0) 0x7f20e01372a0 con 0x7f20e01333d0 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20e00a1df0 con 0x7f20e01333d0 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20d8001a00 con 0x7f20e01337a0 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): dump: 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:43.999 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20dc004ef0 con 0x7f20e01333d0 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient(hunting): dump: 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.000 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3954068007 0 0) 0x7f20e00a1b20 con 0x7f20e00993f0 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.26:6793/0 conn(0x7f20e01337a0 legacy=0x7f20e014aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.122:6791/0 conn(0x7f20e01333d0 legacy=0x7f20e0147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 monclient: found mon.c 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e0135800 con 0x7f20e00993f0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 5 monclient: authenticate success, global_id 204538 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: _renew_subs 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e0135cf0 con 0x7f20e00993f0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f20d0001c00 con 0x7f20e00993f0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: _renew_subs 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e01363b0 con 0x7f20e00993f0 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20d0004ea0 con 0x7f20e00993f0 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.972+0000 7f20d5ffb700 10 monclient: dump: 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.003 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d7fff700 4 set_mon_vals no callback set 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20e7bae700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f20d0005000 con 0x7f20e00993f0 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f20e01350d0 con 0x7f20e0134d00 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f20d0043f50 con 0x7f20e00993f0 2024-09-15T03:58:44.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20e5149700 10 monclient: get_auth_request con 0x7f20c4038080 auth_method 0 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230485032 0 0) 0x7f20e01350d0 con 0x7f20e0134d00 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient(hunting): my global_id is 204547 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e00a1b20 con 0x7f20e0134d00 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1498185055 0 0) 0x7f20e00a1b20 con 0x7f20e0134d00 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20e01350d0 con 0x7f20e0134d00 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20d8002190 con 0x7f20e0134d00 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient: dump: 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.005 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.006 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4018043133 0 0) 0x7f20e01350d0 con 0x7f20e0134d00 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20e004bbf0 con 0x7f20e0134d00 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f20d8002740 con 0x7f20e0134d00 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.26:6791/0 conn(0x7f20e0134d00 legacy=0x7f20e00a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.976+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.26:6791/0 conn(0x7f20e0134d00 legacy=0x7f20e00a4030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20e7bae700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f20e006f230 con 0x7f20e01302d0 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888450847 0 0) 0x7f20e006f230 con 0x7f20e01302d0 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20d5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20d5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20d5ffb700 10 monclient(hunting): my global_id is 204556 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e01350d0 con 0x7f20e01302d0 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.988+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1449557769 0 0) 0x7f20e01350d0 con 0x7f20e01302d0 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20e006f230 con 0x7f20e01302d0 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f20d8002c60 con 0x7f20e01302d0 2024-09-15T03:58:44.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 10 monclient: dump: 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.010 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1487468421 0 0) 0x7f20e006f230 con 0x7f20e01302d0 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f20e0099eb0 con 0x7f20e01302d0 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7388 (unknown 0 0 3713301024) 0x7f20d8007cf0 con 0x7f20e01302d0 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.26:6791/0 conn(0x7f20e01302d0 legacy=0x7f20e002ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20d5ffb700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.26:6791/0 conn(0x7f20e01302d0 legacy=0x7f20e002ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:44.012 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.013 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.014 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.016 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.019 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:44.022 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.027 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.029 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:44.030 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.032 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:44.032 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.122:6812/20895 conn(0x7f20c4038080 legacy=0x7f20c403a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 10 monclient: shutdown 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 >> v1:172.21.15.26:6791/0 conn(0x7f20e00993f0 legacy=0x7f20e012fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e614b700 1 -- 172.21.15.26:0/3730295074 reap_dead start 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 shutdown_connections 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 >> 172.21.15.26:0/3730295074 conn(0x7f20e00a6230 msgr2=0x7f20e009f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 shutdown_connections 2024-09-15T03:58:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:43.992+0000 7f20e7bae700 1 -- 172.21.15.26:0/3730295074 wait complete. 2024-09-15T03:58:44.034 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:44.034 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-09-15T03:58:44.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient: get_monmap_and_config 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient: monmap: 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:44.149462+0000 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:44.149462+0000 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.153 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.154 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 1 Processor -- start 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 1 -- start start 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient: init 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient: _add_conns ranks=[8,7,5,0,3,1,6,2,4] 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient(hunting): picked mon.noname-e con 0x7f8f780a6880 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient(hunting): picked mon.noname-i con 0x7f8f7809faa0 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient(hunting): picked mon.noname-h con 0x7f8f78138d30 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7804ca10 con 0x7f8f78138d30 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f780a0580 con 0x7f8f7809faa0 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f78133780 con 0x7f8f780a6880 2024-09-15T03:58:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f7ce99700 10 monclient(hunting): authenticate will time out at 1668.579125s 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f777fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8f780a6880 0x7f8f78133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50960/0 (socket says 172.21.15.26:50960) 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f777fe700 1 -- 172.21.15.26:0/1951831683 learned_addr learned my addr 172.21.15.26:0/1951831683 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838513004 0 0) 0x7f8f78133780 con 0x7f8f780a6880 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 10 monclient(hunting): my global_id is 135386 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f64003570 con 0x7f8f780a6880 2024-09-15T03:58:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960940193 0 0) 0x7f8f60001490 con 0x7f8f7809faa0 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 10 monclient(hunting): my global_id is 234250 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.144+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f78133780 con 0x7f8f7809faa0 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798908310 0 0) 0x7f8f7804ca10 con 0x7f8f78138d30 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient(hunting): my global_id is 6026 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f60001490 con 0x7f8f78138d30 2024-09-15T03:58:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040441577 0 0) 0x7f8f64003570 con 0x7f8f780a6880 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 >> v1:172.21.15.122:6792/0 conn(0x7f8f7809faa0 legacy=0x7f8f7809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 >> v1:172.21.15.122:6791/0 conn(0x7f8f78138d30 legacy=0x7f8f7813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 monclient: found mon.noname-e 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780a6c50 con 0x7f8f780a6880 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8f680017b0 con 0x7f8f780a6880 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: dump: 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 5 monclient: authenticate success, global_id 135386 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8f68001a90 con 0x7f8f780a6880 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 4 set_mon_vals no callback set 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 1 -- 172.21.15.26:0/1951831683 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8f68004c30 con 0x7f8f780a6880 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f767fc700 10 monclient: dump: 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.168 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: shutdown 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- 172.21.15.26:0/1951831683 >> v1:172.21.15.26:6793/0 conn(0x7f8f780a6880 legacy=0x7f8f78133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- 172.21.15.26:0/1951831683 shutdown_connections 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- 172.21.15.26:0/1951831683 >> 172.21.15.26:0/1951831683 conn(0x7f8f7806e2a0 msgr2=0x7f8f780a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- 172.21.15.26:0/1951831683 shutdown_connections 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- 172.21.15.26:0/1951831683 wait complete. 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: monmap: 2024-09-15T03:58:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.171 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 Processor -- start 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- start start 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: init 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient: _add_conns ranks=[6,3,5,4,7,1,2,8,0] 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient(hunting): picked mon.noname-d con 0x7f8f7809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient(hunting): picked mon.noname-g con 0x7f8f780a6880 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient(hunting): picked mon.noname-h con 0x7f8f78138d30 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:44.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7809baf0 con 0x7f8f780a6880 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f780a18c0 con 0x7f8f78138d30 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7809cf10 con 0x7f8f7809faa0 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f7ce99700 10 monclient(hunting): authenticate will time out at 1668.582825s 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f76ffd700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f8f780a6880 legacy=0x7f8f78099ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f777fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f8f7809faa0 0x7f8f78099590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:59144/0 (socket says 172.21.15.26:59144) 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f777fe700 1 -- 172.21.15.26:0/2464528991 learned_addr learned my addr 172.21.15.26:0/2464528991 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857617360 0 0) 0x7f8f7809cf10 con 0x7f8f7809faa0 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): my global_id is 194802 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f58003570 con 0x7f8f7809faa0 2024-09-15T03:58:44.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160322649 0 0) 0x7f8f780a18c0 con 0x7f8f78138d30 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): my global_id is 6035 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7809cf10 con 0x7f8f78138d30 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3918578490 0 0) 0x7f8f58003570 con 0x7f8f7809faa0 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f780a18c0 con 0x7f8f7809faa0 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8f68004b70 con 0x7f8f7809faa0 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.148+0000 7f8f6f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient(hunting): dump: 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.174 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3802540625 0 0) 0x7f8f780a18c0 con 0x7f8f7809faa0 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.122:6791/0 conn(0x7f8f78138d30 legacy=0x7f8f7809a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.122:6790/0 conn(0x7f8f780a6880 legacy=0x7f8f78099ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 monclient: found mon.d 2024-09-15T03:58:44.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7811d2a0 con 0x7f8f7809faa0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 5 monclient: authenticate success, global_id 194802 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 10 monclient: _renew_subs 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7809b600 con 0x7f8f7809faa0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 10 monclient: _renew_subs 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f78131bc0 con 0x7f8f7809faa0 2024-09-15T03:58:44.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8f68004f00 con 0x7f8f7809faa0 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8f680011d0 con 0x7f8f7809faa0 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f75ffb700 4 set_mon_vals no callback set 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: dump: 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.178 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8f6800fc10 con 0x7f8f7809faa0 2024-09-15T03:58:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8f5c006590 con 0x7f8f5c002840 2024-09-15T03:58:44.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(635..635 src has 435..635) v4 ==== 3927+0+0 (unknown 1531667775 0 0) 0x7f8f680440d0 con 0x7f8f7809faa0 2024-09-15T03:58:44.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874154735 0 0) 0x7f8f5c006590 con 0x7f8f5c002840 2024-09-15T03:58:44.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient(hunting): my global_id is 194811 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f780a18c0 con 0x7f8f5c002840 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f77fff700 10 monclient: get_auth_request con 0x7f8f58037460 auth_method 0 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4113606958 0 0) 0x7f8f780a18c0 con 0x7f8f5c002840 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f5c006590 con 0x7f8f5c002840 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8f60004770 con 0x7f8f5c002840 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: dump: 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.181 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.182 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.183 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2891491402 0 0) 0x7f8f5c006590 con 0x7f8f5c002840 2024-09-15T03:58:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f5c00a130 con 0x7f8f5c002840 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8f60006ae0 con 0x7f8f5c002840 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.26:6792/0 conn(0x7f8f5c002840 legacy=0x7f8f5c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.152+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.26:6792/0 conn(0x7f8f5c002840 legacy=0x7f8f5c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8f5c007520 con 0x7f8f5c0068d0 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877197817 0 0) 0x7f8f5c007520 con 0x7f8f5c0068d0 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient(hunting): my global_id is 194820 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f5c006590 con 0x7f8f5c0068d0 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1371352271 0 0) 0x7f8f5c006590 con 0x7f8f5c0068d0 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f5c007520 con 0x7f8f5c0068d0 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8f600052c0 con 0x7f8f5c0068d0 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient: dump: 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.186 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3912792744 0 0) 0x7f8f5c007520 con 0x7f8f5c0068d0 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8f5c008760 con 0x7f8f5c0068d0 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 1817884769) 0x7f8f60006800 con 0x7f8f5c0068d0 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:44.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.26:6792/0 conn(0x7f8f5c0068d0 legacy=0x7f8f5c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f6f7fe700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.26:6792/0 conn(0x7f8f5c0068d0 legacy=0x7f8f5c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:44.188 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.189 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.190 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:44.191 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.192 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:44.193 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.194 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:44.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.200 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.202 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.203 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.205 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.206 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.207 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T03:58:44.207 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.207 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.207 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:44.207 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.122:6812/20895 conn(0x7f8f58037460 legacy=0x7f8f58039910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 10 monclient: shutdown 2024-09-15T03:58:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 >> v1:172.21.15.26:6792/0 conn(0x7f8f7809faa0 legacy=0x7f8f78099590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f77fff700 1 -- 172.21.15.26:0/2464528991 reap_dead start 2024-09-15T03:58:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f77fff700 1 -- 172.21.15.26:0/2464528991 reap_dead start 2024-09-15T03:58:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 shutdown_connections 2024-09-15T03:58:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 >> 172.21.15.26:0/2464528991 conn(0x7f8f7806e2a0 msgr2=0x7f8f78067880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 shutdown_connections 2024-09-15T03:58:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.164+0000 7f8f7ce99700 1 -- 172.21.15.26:0/2464528991 wait complete. 2024-09-15T03:58:44.209 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:44.209 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon e 2024-09-15T03:58:44.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T03:58:44.260 INFO:tasks.rados.rados.0.smithi122.stdout:2818: done (0 left) 2024-09-15T03:58:44.260 INFO:tasks.rados.rados.0.smithi122.stdout:2819: write (excl) oid 27 current snap is 316 2024-09-15T03:58:44.260 INFO:tasks.rados.rados.0.smithi122.stdout:2819: seq_num 1008 ranges {541456=409588,1569261=562350,2814612=126844} 2024-09-15T03:58:44.268 INFO:tasks.rados.rados.0.smithi122.stdout:2819: writing smithi12223906-27 from 541456 to 951044 tid 1 2024-09-15T03:58:44.275 INFO:tasks.rados.rados.0.smithi122.stdout:2819: writing smithi12223906-27 from 1569261 to 2131611 tid 2 2024-09-15T03:58:44.280 INFO:tasks.rados.rados.0.smithi122.stdout:2819: writing smithi12223906-27 from 2814612 to 2941456 tid 3 2024-09-15T03:58:44.280 INFO:tasks.rados.rados.0.smithi122.stdout:2820: read oid 8 snap 314 2024-09-15T03:58:44.280 INFO:tasks.rados.rados.0.smithi122.stdout:2820: expect (ObjNum 958 snap 304 seq_num 958) 2024-09-15T03:58:44.280 INFO:tasks.rados.rados.0.smithi122.stdout:2821: rollback oid 50 current snap is 316 2024-09-15T03:58:44.280 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 287 2024-09-15T03:58:44.281 INFO:tasks.rados.rados.0.smithi122.stdout:2822: copy_from oid 15 from oid 22 current snap is 316 2024-09-15T03:58:44.281 INFO:tasks.rados.rados.0.smithi122.stdout:2823: copy_from oid 48 from oid 30 current snap is 316 2024-09-15T03:58:44.281 INFO:tasks.rados.rados.0.smithi122.stdout:2824: read oid 14 snap -1 2024-09-15T03:58:44.281 INFO:tasks.rados.rados.0.smithi122.stdout:2824: expect (ObjNum 977 snap 307 seq_num 977) 2024-09-15T03:58:44.281 INFO:tasks.rados.rados.0.smithi122.stdout:2825: write oid 25 current snap is 316 2024-09-15T03:58:44.281 INFO:tasks.rados.rados.0.smithi122.stdout:2825: seq_num 1009 ranges {558263=484837,1627743=706008} 2024-09-15T03:58:44.286 INFO:tasks.rados.rados.0.smithi122.stdout:2825: writing smithi12223906-25 from 558263 to 1043100 tid 1 2024-09-15T03:58:44.294 INFO:tasks.rados.rados.0.smithi122.stdout:2825: writing smithi12223906-25 from 1627743 to 2333751 tid 2 2024-09-15T03:58:44.294 INFO:tasks.rados.rados.0.smithi122.stdout:2826: write oid 40 current snap is 316 2024-09-15T03:58:44.294 INFO:tasks.rados.rados.0.smithi122.stdout:2826: seq_num 1010 ranges {575070=560086,1686225=566019,2869458=105612} 2024-09-15T03:58:44.300 INFO:tasks.rados.rados.0.smithi122.stdout:2826: writing smithi12223906-40 from 575070 to 1135156 tid 1 2024-09-15T03:58:44.307 INFO:tasks.rados.rados.0.smithi122.stdout:2826: writing smithi12223906-40 from 1686225 to 2252244 tid 2 2024-09-15T03:58:44.312 INFO:tasks.rados.rados.0.smithi122.stdout:2826: writing smithi12223906-40 from 2869458 to 2975070 tid 3 2024-09-15T03:58:44.312 INFO:tasks.rados.rados.0.smithi122.stdout:2827: read oid 24 snap 303 2024-09-15T03:58:44.312 INFO:tasks.rados.rados.0.smithi122.stdout:2827: expect deleted 2024-09-15T03:58:44.312 INFO:tasks.rados.rados.0.smithi122.stdout:2828: copy_from oid 26 from oid 46 current snap is 316 2024-09-15T03:58:44.312 INFO:tasks.rados.rados.0.smithi122.stdout:2820: expect (ObjNum 958 snap 304 seq_num 958) 2024-09-15T03:58:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient: get_monmap_and_config 2024-09-15T03:58:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient: monmap: 2024-09-15T03:58:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.338 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:44.334723+0000 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:44.334723+0000 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.339 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 1 Processor -- start 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 1 -- start start 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient: init 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient: _add_conns ranks=[3,0,4,6,1,7,2,8,5] 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient(hunting): picked mon.noname-g con 0x7ff340132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient(hunting): picked mon.noname-a con 0x7ff34009faa0 addr v1:172.21.15.26:6789/0 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.328+0000 7ff3452cd700 10 monclient(hunting): picked mon.noname-c con 0x7ff340138d40 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34004ca10 con 0x7ff34009faa0 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34009d5d0 con 0x7ff340132b40 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff340133620 con 0x7ff340138d40 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient(hunting): authenticate will time out at 1668.764069s 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33ffff700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7ff340132b40 legacy=0x7ff340132f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff337fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff34009faa0 0x7ff34009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55688/0 (socket says 172.21.15.26:55688) 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff337fff700 1 -- 172.21.15.26:0/1138467351 learned_addr learned my addr 172.21.15.26:0/1138467351 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071058407 0 0) 0x7ff34004ca10 con 0x7ff34009faa0 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient(hunting): my global_id is 194697 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff32c003570 con 0x7ff34009faa0 2024-09-15T03:58:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747132004 0 0) 0x7ff340133620 con 0x7ff340138d40 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient(hunting): my global_id is 204565 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff34004ca10 con 0x7ff340138d40 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413226918 0 0) 0x7ff32c003570 con 0x7ff34009faa0 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 >> v1:172.21.15.26:6791/0 conn(0x7ff340138d40 legacy=0x7ff34013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 >> v1:172.21.15.122:6790/0 conn(0x7ff340132b40 legacy=0x7ff340132f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 monclient: found mon.noname-a 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T03:58:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400a0cc0 con 0x7ff34009faa0 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 5 monclient: authenticate success, global_id 194697 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff3280017b0 con 0x7ff34009faa0 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: dump: 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff328001a20 con 0x7ff34009faa0 2024-09-15T03:58:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 1 -- 172.21.15.26:0/1138467351 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff328004c30 con 0x7ff34009faa0 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 4 set_mon_vals no callback set 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff33f7fe700 10 monclient: dump: 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.352 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient: shutdown 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- 172.21.15.26:0/1138467351 >> v1:172.21.15.26:6789/0 conn(0x7ff34009faa0 legacy=0x7ff34009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- 172.21.15.26:0/1138467351 shutdown_connections 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- 172.21.15.26:0/1138467351 >> 172.21.15.26:0/1138467351 conn(0x7ff34006e370 msgr2=0x7ff3400a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- 172.21.15.26:0/1138467351 shutdown_connections 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- 172.21.15.26:0/1138467351 wait complete. 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient: monmap: 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 Processor -- start 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 1 -- start start 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.332+0000 7ff3452cd700 10 monclient: init 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient: _add_conns ranks=[3,7,2,0,8,6,1,5,4] 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient(hunting): picked mon.noname-g con 0x7ff34009faa0 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient(hunting): picked mon.noname-i con 0x7ff340132b40 addr v1:172.21.15.122:6792/0 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient(hunting): picked mon.noname-b con 0x7ff340138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34009c610 con 0x7ff340138d40 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34009ccc0 con 0x7ff34009faa0 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff340098880 con 0x7ff340132b40 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient(hunting): authenticate will time out at 1668.768239s 2024-09-15T03:58:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33ffff700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7ff34009faa0 legacy=0x7ff340097a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff344acc700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7ff340138d40 0x7ff34014a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:38926/0 (socket says 172.21.15.26:38926) 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff344acc700 1 -- 172.21.15.26:0/4173533867 learned_addr learned my addr 172.21.15.26:0/4173533867 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517559079 0 0) 0x7ff34009c610 con 0x7ff340138d40 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): my global_id is 204590 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff320003570 con 0x7ff340138d40 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220026423 0 0) 0x7ff340098880 con 0x7ff340132b40 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): my global_id is 234259 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff34009c610 con 0x7ff340132b40 2024-09-15T03:58:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3896583879 0 0) 0x7ff320003570 con 0x7ff340138d40 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff340098880 con 0x7ff340138d40 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff338002d90 con 0x7ff340138d40 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): dump: 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 433435287 0 0) 0x7ff340098880 con 0x7ff340138d40 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.122:6792/0 conn(0x7ff340132b40 legacy=0x7ff340098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.122:6790/0 conn(0x7ff34009faa0 legacy=0x7ff340097a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 monclient: found mon.b 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34009a740 con 0x7ff340138d40 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 5 monclient: authenticate success, global_id 204590 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient: _renew_subs 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff340098c10 con 0x7ff340138d40 2024-09-15T03:58:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient: _renew_subs 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34009b9b0 con 0x7ff340138d40 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff338001900 con 0x7ff340138d40 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff338001e30 con 0x7ff340138d40 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33effd700 4 set_mon_vals no callback set 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient: dump: 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff324006590 con 0x7ff324002840 2024-09-15T03:58:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff338002010 con 0x7ff340138d40 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(636..636 src has 435..636) v4 ==== 3971+0+0 (unknown 3448633361 0 0) 0x7ff338044320 con 0x7ff340138d40 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325602768 0 0) 0x7ff324006590 con 0x7ff324002840 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 10 monclient(hunting): my global_id is 135395 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff340098880 con 0x7ff324002840 2024-09-15T03:58:44.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.336+0000 7ff337fff700 10 monclient: get_auth_request con 0x7ff320037520 auth_method 0 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4044477903 0 0) 0x7ff340098880 con 0x7ff324002840 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff324006590 con 0x7ff324002840 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff3300040d0 con 0x7ff324002840 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 10 monclient: dump: 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1054729316 0 0) 0x7ff324006590 con 0x7ff324002840 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff32400a130 con 0x7ff324002840 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff3300064c0 con 0x7ff324002840 2024-09-15T03:58:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.26:6793/0 conn(0x7ff324002840 legacy=0x7ff324004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.340+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.26:6793/0 conn(0x7ff324002840 legacy=0x7ff324004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff3452cd700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff324009640 con 0x7ff3240090b0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628248450 0 0) 0x7ff324009640 con 0x7ff3240090b0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff33cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff33cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff33cff9700 10 monclient(hunting): my global_id is 135404 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.348+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff324006590 con 0x7ff3240090b0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3453924115 0 0) 0x7ff324006590 con 0x7ff3240090b0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff324009640 con 0x7ff3240090b0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff330004c30 con 0x7ff3240090b0 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 10 monclient: dump: 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 754484700 0 0) 0x7ff324009640 con 0x7ff3240090b0 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3240098f0 con 0x7ff3240090b0 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7558 (unknown 0 0 2304738604) 0x7ff330006200 con 0x7ff3240090b0 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.26:6793/0 conn(0x7ff3240090b0 legacy=0x7ff324008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff33cff9700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.26:6793/0 conn(0x7ff3240090b0 legacy=0x7ff324008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:44.370 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.371 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.372 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.373 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:44.375 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.376 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.378 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:44.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.380 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.381 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.382 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.383 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.384 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.385 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.387 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.388 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:44.389 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:44.390 INFO:tasks.rados.rados.0.smithi122.stdout:2819: finishing write tid 1 to smithi12223906-27 2024-09-15T03:58:44.390 INFO:tasks.rados.rados.0.smithi122.stdout:2823: finishing copy_from racing read to smithi12223906-48 2024-09-15T03:58:44.390 INFO:tasks.rados.rados.0.smithi122.stdout:2819: finishing write tid 2 to smithi12223906-27 2024-09-15T03:58:44.390 INFO:tasks.rados.rados.0.smithi122.stdout:2824: expect (ObjNum 977 snap 307 seq_num 977) 2024-09-15T03:58:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.122:6812/20895 conn(0x7ff320037520 legacy=0x7ff3200399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 10 monclient: shutdown 2024-09-15T03:58:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 >> v1:172.21.15.26:6790/0 conn(0x7ff340138d40 legacy=0x7ff34014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff344acc700 1 -- 172.21.15.26:0/4173533867 reap_dead start 2024-09-15T03:58:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff344acc700 1 -- 172.21.15.26:0/4173533867 reap_dead start 2024-09-15T03:58:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 shutdown_connections 2024-09-15T03:58:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 >> 172.21.15.26:0/4173533867 conn(0x7ff34006e370 msgr2=0x7ff3400a2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 shutdown_connections 2024-09-15T03:58:44.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.352+0000 7ff3452cd700 1 -- 172.21.15.26:0/4173533867 wait complete. 2024-09-15T03:58:44.392 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:44.392 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-09-15T03:58:44.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T03:58:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.496+0000 7f2ea496b700 10 monclient: get_monmap_and_config 2024-09-15T03:58:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.496+0000 7f2ea496b700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.496+0000 7f2ea496b700 10 monclient: monmap: 2024-09-15T03:58:44.505 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:44.501867+0000 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:44.501867+0000 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.506 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.507 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.496+0000 7f2ea496b700 1 Processor -- start 2024-09-15T03:58:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.496+0000 7f2ea496b700 1 -- start start 2024-09-15T03:58:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.496+0000 7f2ea496b700 10 monclient: init 2024-09-15T03:58:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient: _add_conns ranks=[5,2,3,0,7,4,6,1,8] 2024-09-15T03:58:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient(hunting): picked mon.noname-h con 0x7f2ea00a01b0 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient(hunting): picked mon.noname-b con 0x7f2ea0099aa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:44.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient(hunting): picked mon.noname-g con 0x7f2ea009d0d0 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea004ca10 con 0x7f2ea0099aa0 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea009f8f0 con 0x7f2ea009d0d0 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea0098af0 con 0x7f2ea00a01b0 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient(hunting): authenticate will time out at 1668.933149s 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9ffff700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f2ea009d0d0 legacy=0x7f2ea0138160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9effd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f2ea0099aa0 0x7f2ea0134a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:38936/0 (socket says 172.21.15.26:38936) 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9effd700 1 -- 172.21.15.26:0/2164783387 learned_addr learned my addr 172.21.15.26:0/2164783387 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556032728 0 0) 0x7f2ea004ca10 con 0x7f2ea0099aa0 2024-09-15T03:58:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient(hunting): my global_id is 204599 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e88003570 con 0x7f2ea0099aa0 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381630068 0 0) 0x7f2ea0098af0 con 0x7f2ea00a01b0 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient(hunting): my global_id is 6044 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea004ca10 con 0x7f2ea00a01b0 2024-09-15T03:58:44.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2766127053 0 0) 0x7f2e88003570 con 0x7f2ea0099aa0 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 >> v1:172.21.15.122:6791/0 conn(0x7f2ea00a01b0 legacy=0x7f2ea009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 >> v1:172.21.15.122:6790/0 conn(0x7f2ea009d0d0 legacy=0x7f2ea0138160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 monclient: found mon.noname-b 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea00a0580 con 0x7f2ea0099aa0 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 5 monclient: authenticate success, global_id 204599 2024-09-15T03:58:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e940017b0 con 0x7f2ea0099aa0 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: dump: 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.516 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2e94001a20 con 0x7f2ea0099aa0 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 4 set_mon_vals no callback set 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 1 -- 172.21.15.26:0/2164783387 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e94004c30 con 0x7f2ea0099aa0 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2e9e7fc700 10 monclient: dump: 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.517 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.518 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 10 monclient: shutdown 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- 172.21.15.26:0/2164783387 >> v1:172.21.15.26:6790/0 conn(0x7f2ea0099aa0 legacy=0x7f2ea0134a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- 172.21.15.26:0/2164783387 shutdown_connections 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- 172.21.15.26:0/2164783387 >> 172.21.15.26:0/2164783387 conn(0x7f2ea00a3b70 msgr2=0x7f2ea009faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- 172.21.15.26:0/2164783387 shutdown_connections 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.500+0000 7f2ea496b700 1 -- 172.21.15.26:0/2164783387 wait complete. 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: monmap: 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:44.519 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.520 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.523 INFO:tasks.rados.rados.0.smithi122.stdout:2823: finishing copy_from to smithi12223906-48 2024-09-15T03:58:44.523 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 842 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2819: finishing write tid 3 to smithi12223906-27 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2819: finishing write tid 4 to smithi12223906-27 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2820: done (9 left) 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2823: done (8 left) 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2824: done (7 left) 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2829: read oid 28 snap 297 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2829: expect deleted 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2830: rollback oid 35 current snap is 316 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 231 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:2819: finishing write tid 5 to smithi12223906-27 2024-09-15T03:58:44.524 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 2155 (ObjNum 1008 snap 316 seq_num 1008) dirty exists 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2819: left oid 27 (ObjNum 1008 snap 316 seq_num 1008) 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2825: finishing write tid 1 to smithi12223906-25 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2825: finishing write tid 2 to smithi12223906-25 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2825: finishing write tid 3 to smithi12223906-25 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2825: finishing write tid 4 to smithi12223906-25 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2246 (ObjNum 1009 snap 316 seq_num 1009) dirty exists 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2825: left oid 25 (ObjNum 1009 snap 316 seq_num 1009) 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2821: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2821: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2821: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2159 (ObjNum 855 snap 277 seq_num 855) dirty exists 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2826: finishing write tid 1 to smithi12223906-40 2024-09-15T03:58:44.525 INFO:tasks.rados.rados.0.smithi122.stdout:2822: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:2826: finishing write tid 2 to smithi12223906-40 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:2822: finishing copy_from to smithi12223906-15 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1875 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:2826: finishing write tid 3 to smithi12223906-40 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:2826: finishing write tid 4 to smithi12223906-40 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:2826: finishing write tid 5 to smithi12223906-40 2024-09-15T03:58:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 846 (ObjNum 1010 snap 316 seq_num 1010) dirty exists 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:2826: left oid 40 (ObjNum 1010 snap 316 seq_num 1010) 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:2828: finishing copy_from racing read to smithi12223906-26 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:2828: finishing copy_from to smithi12223906-26 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1581 (ObjNum 984 snap 311 seq_num 984) dirty exists 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:2819: done (8 left) 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:2821: done (7 left) 2024-09-15T03:58:44.528 INFO:tasks.rados.rados.0.smithi122.stdout:2822: done (6 left) 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2825: done (5 left) 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2826: done (4 left) 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2827: done (3 left) 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2828: done (2 left) 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2831: copy_from oid 19 from oid 41 current snap is 316 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2832: write oid 1 current snap is 316 2024-09-15T03:58:44.529 INFO:tasks.rados.rados.0.smithi122.stdout:2832: seq_num 1011 ranges {591877=635335,1628354=709677} 2024-09-15T03:58:44.530 INFO:tasks.rados.rados.0.smithi122.stdout:2832: writing smithi12223906-1 from 591877 to 1227212 tid 1 2024-09-15T03:58:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 Processor -- start 2024-09-15T03:58:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 -- start start 2024-09-15T03:58:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: init 2024-09-15T03:58:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: _add_conns ranks=[1,8,2,4,7,6,5,3,0] 2024-09-15T03:58:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient(hunting): picked mon.noname-f con 0x7f2ea0099aa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient(hunting): picked mon.noname-e con 0x7f2ea009d0d0 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient(hunting): picked mon.noname-b con 0x7f2ea00a01b0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea0097c90 con 0x7f2ea0099aa0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea006e680 con 0x7f2ea00a01b0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea011ad50 con 0x7f2ea009d0d0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient(hunting): authenticate will time out at 1668.937812s 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e9ffff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f2ea00a01b0 0x7f2ea0149d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:38946/0 (socket says 172.21.15.26:38946) 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e9effd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f2ea009d0d0 0x7f2ea011a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50994/0 (socket says 172.21.15.26:50994) 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e9ffff700 1 -- 172.21.15.26:0/2373784750 learned_addr learned my addr 172.21.15.26:0/2373784750 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950648744 0 0) 0x7f2ea006e680 con 0x7f2ea00a01b0 2024-09-15T03:58:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): my global_id is 204608 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e8c003570 con 0x7f2ea00a01b0 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016245646 0 0) 0x7f2ea011ad50 con 0x7f2ea009d0d0 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): my global_id is 135413 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea006e680 con 0x7f2ea009d0d0 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580039317 0 0) 0x7f2ea0097c90 con 0x7f2ea0099aa0 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): my global_id is 204544 2024-09-15T03:58:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea011ad50 con 0x7f2ea0099aa0 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3196345896 0 0) 0x7f2ea006e680 con 0x7f2ea009d0d0 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ea0097c90 con 0x7f2ea009d0d0 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e94001830 con 0x7f2ea009d0d0 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): dump: 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2125804043 0 0) 0x7f2e8c003570 con 0x7f2ea00a01b0 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ea006e680 con 0x7f2ea00a01b0 2024-09-15T03:58:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e98004570 con 0x7f2ea00a01b0 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient(hunting): dump: 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1472583029 0 0) 0x7f2ea0097c90 con 0x7f2ea009d0d0 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.26:6790/0 conn(0x7f2ea00a01b0 legacy=0x7f2ea0149d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.122:6789/0 conn(0x7f2ea0099aa0 legacy=0x7f2ea0119f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 monclient: found mon.e 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea011b450 con 0x7f2ea009d0d0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 5 monclient: authenticate success, global_id 135413 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: _renew_subs 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea0071cd0 con 0x7f2ea009d0d0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: _renew_subs 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea00717c0 con 0x7f2ea009d0d0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2e94004f00 con 0x7f2ea009d0d0 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e940053b0 con 0x7f2ea009d0d0 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e9dffb700 4 set_mon_vals no callback set 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.504+0000 7f2e877fe700 10 monclient: dump: 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2ea496b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2e80006590 con 0x7f2e80002840 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2e9400fd70 con 0x7f2ea009d0d0 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(636..636 src has 435..636) v4 ==== 3971+0+0 (unknown 3448633361 0 0) 0x7f2e94044260 con 0x7f2ea009d0d0 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961918114 0 0) 0x7f2e80006590 con 0x7f2e80002840 2024-09-15T03:58:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient(hunting): my global_id is 204553 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea0097c90 con 0x7f2e80002840 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1402060926 0 0) 0x7f2ea0097c90 con 0x7f2e80002840 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e80006590 con 0x7f2e80002840 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e900018d0 con 0x7f2e80002840 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient: dump: 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.545 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 489709024 0 0) 0x7f2e80006590 con 0x7f2e80002840 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e8000a130 con 0x7f2e80002840 2024-09-15T03:58:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2e90001c70 con 0x7f2e80002840 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.122:6789/0 conn(0x7f2e80002840 legacy=0x7f2e80004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.122:6789/0 conn(0x7f2e80002840 legacy=0x7f2e80004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.508+0000 7f2e9ffff700 10 monclient: get_auth_request con 0x7f2e8c037d70 auth_method 0 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2ea496b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2e80009640 con 0x7f2e800090b0 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930842513 0 0) 0x7f2e80009640 con 0x7f2e800090b0 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient(hunting): my global_id is 204562 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e80006590 con 0x7f2e800090b0 2024-09-15T03:58:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4090929352 0 0) 0x7f2e80006590 con 0x7f2e800090b0 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e80009640 con 0x7f2e800090b0 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2e90002430 con 0x7f2e800090b0 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient: dump: 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2523757172 0 0) 0x7f2e80009640 con 0x7f2e800090b0 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2e800098f0 con 0x7f2e800090b0 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7388 (unknown 0 0 596081415) 0x7f2e900028d0 con 0x7f2e800090b0 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.122:6789/0 conn(0x7f2e800090b0 legacy=0x7f2e80008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2e877fe700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.122:6789/0 conn(0x7f2e800090b0 legacy=0x7f2e80008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:44.550 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.551 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.552 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.553 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:44.554 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.555 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.556 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:44.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.559 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.562 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.564 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:44.565 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.566 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.567 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.568 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.569 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.569 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:44.569 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:44.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.520+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.122:6812/20895 conn(0x7f2e8c037d70 legacy=0x7f2e8c03a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 10 monclient: shutdown 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 >> v1:172.21.15.26:6793/0 conn(0x7f2ea009d0d0 legacy=0x7f2ea011a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2e9ffff700 1 -- 172.21.15.26:0/2373784750 reap_dead start 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2e9ffff700 1 -- 172.21.15.26:0/2373784750 reap_dead start 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 shutdown_connections 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 >> 172.21.15.26:0/2373784750 conn(0x7f2ea00a3b70 msgr2=0x7f2ea0131000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 shutdown_connections 2024-09-15T03:58:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.524+0000 7f2ea496b700 1 -- 172.21.15.26:0/2373784750 wait complete. 2024-09-15T03:58:44.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:44.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-09-15T03:58:44.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T03:58:44.574 INFO:tasks.rados.rados.0.smithi122.stdout:2832: writing smithi12223906-1 from 1628354 to 2338031 tid 2 2024-09-15T03:58:44.574 INFO:tasks.rados.rados.0.smithi122.stdout:2833: write (excl) oid 40 current snap is 316 2024-09-15T03:58:44.574 INFO:tasks.rados.rados.0.smithi122.stdout:2833: seq_num 1012 ranges {608684=426937,1803189=569688,2924304=84380} 2024-09-15T03:58:44.574 INFO:tasks.rados.rados.0.smithi122.stdout:2833: writing smithi12223906-40 from 608684 to 1035621 tid 1 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2833: writing smithi12223906-40 from 1803189 to 2372877 tid 2 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2833: writing smithi12223906-40 from 2924304 to 3008684 tid 3 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2834: read oid 43 snap -1 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2834: expect deleted 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2835: read oid 6 snap -1 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2835: expect (ObjNum 948 snap 296 seq_num 948) 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2836: snap_create 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2831: finishing copy_from to smithi12223906-19 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2831: got expected ENOENT (src dne) 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2831: finishing copy_from racing read to smithi12223906-19 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2830: finishing rollback tid 0 to smithi12223906-35 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2830: finishing rollback tid 1 to smithi12223906-35 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1836 (ObjNum 97 snap 0 seq_num 1843145152) dirty dne 2024-09-15T03:58:44.575 INFO:tasks.rados.rados.0.smithi122.stdout:2832: finishing write tid 1 to smithi12223906-1 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2832: finishing write tid 2 to smithi12223906-1 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2832: finishing write tid 3 to smithi12223906-1 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2832: finishing write tid 4 to smithi12223906-1 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1878 (ObjNum 1011 snap 316 seq_num 1011) dirty exists 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2832: left oid 1 (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2833: finishing write tid 1 to smithi12223906-40 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2833: finishing write tid 2 to smithi12223906-40 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2833: finishing write tid 3 to smithi12223906-40 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2833: finishing write tid 4 to smithi12223906-40 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2833: finishing write tid 5 to smithi12223906-40 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 850 (ObjNum 1012 snap 316 seq_num 1012) dirty exists 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2833: left oid 40 (ObjNum 1012 snap 316 seq_num 1012) 2024-09-15T03:58:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:2835: expect (ObjNum 948 snap 296 seq_num 948) 2024-09-15T03:58:44.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.680+0000 7fd408815700 10 monclient: get_monmap_and_config 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.680+0000 7fd408815700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.680+0000 7fd408815700 10 monclient: monmap: 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:44.687116+0000 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:44.687116+0000 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.692 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 Processor -- start 2024-09-15T03:58:44.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- start start 2024-09-15T03:58:44.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient: init 2024-09-15T03:58:44.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient: _add_conns ranks=[3,1,4,7,2,8,0,6,5] 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient(hunting): picked mon.noname-g con 0x7fd40412fd20 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient(hunting): picked mon.noname-f con 0x7fd4041300f0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient(hunting): picked mon.noname-c con 0x7fd404130520 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40404ca10 con 0x7fd4041300f0 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4040a2830 con 0x7fd40412fd20 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd404097b70 con 0x7fd404130520 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient(hunting): authenticate will time out at 1669.117139s 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd40259c700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fd40412fd20 legacy=0x7fd404133270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd402d9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd404130520 0x7fd40413b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40618/0 (socket says 172.21.15.26:40618) 2024-09-15T03:58:44.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd402d9d700 1 -- 172.21.15.26:0/4187117691 learned_addr learned my addr 172.21.15.26:0/4187117691 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856744979 0 0) 0x7fd404097b70 con 0x7fd404130520 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient(hunting): my global_id is 204574 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f0003570 con 0x7fd404130520 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784547277 0 0) 0x7fd40404ca10 con 0x7fd4041300f0 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient(hunting): my global_id is 204571 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd404097b70 con 0x7fd4041300f0 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1711055989 0 0) 0x7fd404097b70 con 0x7fd4041300f0 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 >> v1:172.21.15.26:6791/0 conn(0x7fd404130520 legacy=0x7fd40413b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 >> v1:172.21.15.122:6790/0 conn(0x7fd40412fd20 legacy=0x7fd404133270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 monclient: found mon.noname-f 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404133980 con 0x7fd4041300f0 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd3ec001790 con 0x7fd4041300f0 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 5 monclient: authenticate success, global_id 204571 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: dump: 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.698 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd3ec001a40 con 0x7fd4041300f0 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 1 -- 172.21.15.26:0/4187117691 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd3ec002240 con 0x7fd4041300f0 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 4 set_mon_vals no callback set 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd401d9b700 10 monclient: dump: 2024-09-15T03:58:44.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 10 monclient: shutdown 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- 172.21.15.26:0/4187117691 >> v1:172.21.15.122:6789/0 conn(0x7fd4041300f0 legacy=0x7fd404137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- 172.21.15.26:0/4187117691 shutdown_connections 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.684+0000 7fd408815700 1 -- 172.21.15.26:0/4187117691 >> 172.21.15.26:0/4187117691 conn(0x7fd40406e410 msgr2=0x7fd4040a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 -- 172.21.15.26:0/4187117691 shutdown_connections 2024-09-15T03:58:44.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 -- 172.21.15.26:0/4187117691 wait complete. 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient: monmap: 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.707 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 Processor -- start 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 -- start start 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient: init 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient: _add_conns ranks=[6,4,5,2,1,7,3,0,8] 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient(hunting): picked mon.noname-d con 0x7fd40412fd20 addr v1:172.21.15.26:6792/0 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient(hunting): picked mon.noname-c con 0x7fd4041300f0 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:44.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient(hunting): picked mon.noname-h con 0x7fd404130520 addr v1:172.21.15.122:6791/0 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4040a1b50 con 0x7fd4041300f0 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4040a1880 con 0x7fd404130520 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40411d6b0 con 0x7fd40412fd20 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd408815700 10 monclient(hunting): authenticate will time out at 1669.123042s 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd40259c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd40412fd20 0x7fd40411c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:59178/0 (socket says 172.21.15.26:59178) 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd40259c700 1 -- 172.21.15.26:0/1952738487 learned_addr learned my addr 172.21.15.26:0/1952738487 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096433465 0 0) 0x7fd40411d6b0 con 0x7fd40412fd20 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): my global_id is 194829 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e4003570 con 0x7fd40412fd20 2024-09-15T03:58:44.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656472035 0 0) 0x7fd4040a1b50 con 0x7fd4041300f0 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): my global_id is 204583 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40411d6b0 con 0x7fd4041300f0 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659936630 0 0) 0x7fd4040a1880 con 0x7fd404130520 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 10 monclient(hunting): my global_id is 6053 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.688+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4040a1b50 con 0x7fd404130520 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2415988558 0 0) 0x7fd3e4003570 con 0x7fd40412fd20 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd4040a1880 con 0x7fd40412fd20 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd3f4004380 con 0x7fd40412fd20 2024-09-15T03:58:44.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): dump: 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.711 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2043410590 0 0) 0x7fd4040a1880 con 0x7fd40412fd20 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.122:6791/0 conn(0x7fd404130520 legacy=0x7fd404149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.26:6791/0 conn(0x7fd4041300f0 legacy=0x7fd40411cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 monclient: found mon.d 2024-09-15T03:58:44.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40409cad0 con 0x7fd40412fd20 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 5 monclient: authenticate success, global_id 194829 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 10 monclient: _renew_subs 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40409c5c0 con 0x7fd40412fd20 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 10 monclient: _renew_subs 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40411fb40 con 0x7fd40412fd20 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd3f40045a0 con 0x7fd40412fd20 2024-09-15T03:58:44.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd3f4004a50 con 0x7fd40412fd20 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd40159a700 4 set_mon_vals no callback set 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: dump: 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd3f4004c30 con 0x7fd40412fd20 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd3e8006590 con 0x7fd3e8002840 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(636..636 src has 435..636) v4 ==== 3971+0+0 (unknown 3448633361 0 0) 0x7fd3f4043fa0 con 0x7fd40412fd20 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd402d9d700 10 monclient: get_auth_request con 0x7fd3e40377c0 auth_method 0 2024-09-15T03:58:44.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149160229 0 0) 0x7fd3e8006590 con 0x7fd3e8002840 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient(hunting): my global_id is 6062 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4040a1880 con 0x7fd3e8002840 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3074069230 0 0) 0x7fd4040a1880 con 0x7fd3e8002840 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd3e8006590 con 0x7fd3e8002840 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd3ec002510 con 0x7fd3e8002840 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 10 monclient: dump: 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.716 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.717 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3557465251 0 0) 0x7fd3e8006590 con 0x7fd3e8002840 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.692+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3e800a130 con 0x7fd3e8002840 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.696+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd3ec0046f0 con 0x7fd3e8002840 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.696+0000 7fd3faffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.696+0000 7fd3faffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.696+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.122:6791/0 conn(0x7fd3e8002840 legacy=0x7fd3e8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.696+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.122:6791/0 conn(0x7fd3e8002840 legacy=0x7fd3e8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd408815700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:44.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd3e8009640 con 0x7fd3e80090b0 2024-09-15T03:58:44.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922666278 0 0) 0x7fd3e8009640 con 0x7fd3e80090b0 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 10 monclient(hunting): my global_id is 6071 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e8006590 con 0x7fd3e80090b0 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 861843996 0 0) 0x7fd3e8006590 con 0x7fd3e80090b0 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.704+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd3e8009640 con 0x7fd3e80090b0 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd3ec002f60 con 0x7fd3e80090b0 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 10 monclient: dump: 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.721 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3034509833 0 0) 0x7fd3e8009640 con 0x7fd3e80090b0 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd3e80098f0 con 0x7fd3e80090b0 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7730 (unknown 0 0 1136993371) 0x7fd3ec0049a0 con 0x7fd3e80090b0 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.122:6791/0 conn(0x7fd3e80090b0 legacy=0x7fd3e8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd3faffd700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.122:6791/0 conn(0x7fd3e80090b0 legacy=0x7fd3e8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:44.723 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.724 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.725 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.728 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.731 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.735 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.738 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.741 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.742 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.743 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T03:58:44.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:44.745 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.122:6812/20895 conn(0x7fd3e40377c0 legacy=0x7fd3e4039c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 10 monclient: shutdown 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 >> v1:172.21.15.26:6792/0 conn(0x7fd40412fd20 legacy=0x7fd40411c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd402d9d700 1 -- 172.21.15.26:0/1952738487 reap_dead start 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd402d9d700 1 -- 172.21.15.26:0/1952738487 reap_dead start 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 shutdown_connections 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 >> 172.21.15.26:0/1952738487 conn(0x7fd40406e410 msgr2=0x7fd4040a46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 shutdown_connections 2024-09-15T03:58:44.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.708+0000 7fd408815700 1 -- 172.21.15.26:0/1952738487 wait complete. 2024-09-15T03:58:44.748 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:44.748 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-09-15T03:58:44.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.856+0000 7fb89a6ee700 10 monclient: get_monmap_and_config 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.856+0000 7fb89a6ee700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.856+0000 7fb89a6ee700 10 monclient: monmap: 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:58:44.863490+0000 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:58:44.863490+0000 2024-09-15T03:58:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.868 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 Processor -- start 2024-09-15T03:58:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- start start 2024-09-15T03:58:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient: init 2024-09-15T03:58:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient: _add_conns ranks=[3,1,2,5,4,0,6,8,7] 2024-09-15T03:58:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient(hunting): picked mon.noname-g con 0x7fb894132b40 addr v1:172.21.15.122:6790/0 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient(hunting): picked mon.noname-f con 0x7fb89409faa0 addr v1:172.21.15.122:6789/0 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient(hunting): picked mon.noname-b con 0x7fb894138d40 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89404ca10 con 0x7fb89409faa0 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89409d5d0 con 0x7fb894138d40 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb894133620 con 0x7fb894132b40 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient(hunting): authenticate will time out at 1669.293025s 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb8996ec700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fb894132b40 legacy=0x7fb894132f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb899eed700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb894138d40 0x7fb89413b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:38958/0 (socket says 172.21.15.26:38958) 2024-09-15T03:58:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb899eed700 1 -- 172.21.15.26:0/176690962 learned_addr learned my addr 172.21.15.26:0/176690962 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb898eeb700 1 --1- 172.21.15.26:0/176690962 >> v1:172.21.15.122:6789/0 conn(0x7fb89409faa0 0x7fb89409fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:52472/0 (socket says 172.21.15.26:52472) 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750927394 0 0) 0x7fb89404ca10 con 0x7fb89409faa0 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient(hunting): my global_id is 204580 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb880003570 con 0x7fb89409faa0 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156848188 0 0) 0x7fb89409d5d0 con 0x7fb894138d40 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient(hunting): my global_id is 204617 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb89404ca10 con 0x7fb894138d40 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 764877842 0 0) 0x7fb89404ca10 con 0x7fb894138d40 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 >> v1:172.21.15.122:6790/0 conn(0x7fb894132b40 legacy=0x7fb894132f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T03:58:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 >> v1:172.21.15.122:6789/0 conn(0x7fb89409faa0 legacy=0x7fb89409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 monclient: found mon.noname-b 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940a0cc0 con 0x7fb894138d40 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 5 monclient: authenticate success, global_id 204617 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb88c001b10 con 0x7fb894138d40 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T03:58:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: dump: 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb88c001dc0 con 0x7fb894138d40 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient: get_monmap_and_config success 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 4 set_mon_vals no callback set 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 1 -- 172.21.15.26:0/176690962 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb88c004fb0 con 0x7fb894138d40 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb88bfff700 10 monclient: dump: 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 10 monclient: shutdown 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- 172.21.15.26:0/176690962 >> v1:172.21.15.26:6790/0 conn(0x7fb894138d40 legacy=0x7fb89413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- 172.21.15.26:0/176690962 shutdown_connections 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.860+0000 7fb89a6ee700 1 -- 172.21.15.26:0/176690962 >> 172.21.15.26:0/176690962 conn(0x7fb89406e370 msgr2=0x7fb8940a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 -- 172.21.15.26:0/176690962 shutdown_connections 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 -- 172.21.15.26:0/176690962 wait complete. 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient: build_initial_monmap 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient: monmap: 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T03:58:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T03:58:44.884 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 Processor -- start 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 -- start start 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient: init 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient: _reopen_session rank -1 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient: _add_conns ranks=[2,4,8,5,1,6,3,0,7] 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient(hunting): picked mon.noname-b con 0x7fb89409faa0 addr v1:172.21.15.26:6790/0 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient(hunting): picked mon.noname-c con 0x7fb894132b40 addr v1:172.21.15.26:6791/0 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient(hunting): picked mon.noname-e con 0x7fb894138d40 addr v1:172.21.15.26:6793/0 2024-09-15T03:58:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89409cfa0 con 0x7fb89409faa0 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb894137be0 con 0x7fb894132b40 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb894135dd0 con 0x7fb894138d40 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient(hunting): _renew_subs 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb89a6ee700 10 monclient(hunting): authenticate will time out at 1669.298375s 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb899eed700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fb894138d40 0x7fb89414a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:51010/0 (socket says 172.21.15.26:51010) 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb899eed700 1 -- 172.21.15.26:0/2666524546 learned_addr learned my addr 172.21.15.26:0/2666524546 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948833762 0 0) 0x7fb894135dd0 con 0x7fb894138d40 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): my global_id is 135422 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb878003570 con 0x7fb894138d40 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199968520 0 0) 0x7fb89409cfa0 con 0x7fb89409faa0 2024-09-15T03:58:44.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): my global_id is 204626 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb894135dd0 con 0x7fb89409faa0 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955895061 0 0) 0x7fb894137be0 con 0x7fb894132b40 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): my global_id is 204592 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb89409cfa0 con 0x7fb894132b40 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 58613308 0 0) 0x7fb878003570 con 0x7fb894138d40 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb894137be0 con 0x7fb894138d40 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb88c004fe0 con 0x7fb894138d40 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T03:58:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): dump: 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3914000619 0 0) 0x7fb89409cfa0 con 0x7fb894132b40 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb878003570 con 0x7fb894132b40 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb884001a00 con 0x7fb894132b40 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): dump: 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 915925045 0 0) 0x7fb894135dd0 con 0x7fb89409faa0 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb89409cfa0 con 0x7fb89409faa0 2024-09-15T03:58:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb8900042f0 con 0x7fb89409faa0 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.864+0000 7fb8897fa700 10 monclient(hunting): dump: 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 43127031 0 0) 0x7fb878003570 con 0x7fb894132b40 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.26:6793/0 conn(0x7fb894138d40 legacy=0x7fb89414a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.26:6790/0 conn(0x7fb89409faa0 legacy=0x7fb894134370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: _finish_hunting 0 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 monclient: found mon.c 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb894137900 con 0x7fb894132b40 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: _finish_auth 0 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 5 monclient: authenticate success, global_id 204592 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 10 monclient: _renew_subs 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb894137ec0 con 0x7fb894132b40 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 10 monclient: _renew_subs 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb894135330 con 0x7fb894132b40 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb8840048f0 con 0x7fb894132b40 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb884004da0 con 0x7fb894132b40 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb88b7fe700 4 set_mon_vals no callback set 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: dump: 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.895 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb884005010 con 0x7fb894132b40 2024-09-15T03:58:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(636..636 src has 435..636) v4 ==== 3971+0+0 (unknown 3448633361 0 0) 0x7fb884043de0 con 0x7fb894132b40 2024-09-15T03:58:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb874006590 con 0x7fb874002840 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8996ec700 10 monclient: get_auth_request con 0x7fb878037fa0 auth_method 0 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483540222 0 0) 0x7fb874006590 con 0x7fb874002840 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient(hunting): my global_id is 234268 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb878003570 con 0x7fb874002840 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3036751016 0 0) 0x7fb878003570 con 0x7fb874002840 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb874006590 con 0x7fb874002840 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb88c001810 con 0x7fb874002840 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 10 monclient: dump: 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.899 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1916533362 0 0) 0x7fb874006590 con 0x7fb874002840 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.868+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb87400a130 con 0x7fb874002840 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.872+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb88c007900 con 0x7fb874002840 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.872+0000 7fb8897fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.872+0000 7fb8897fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.872+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.122:6792/0 conn(0x7fb874002840 legacy=0x7fb874004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.872+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.122:6792/0 conn(0x7fb874002840 legacy=0x7fb874004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb874009640 con 0x7fb8740090b0 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106692022 0 0) 0x7fb874009640 con 0x7fb8740090b0 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T03:58:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient(hunting): my global_id is 234277 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb874006590 con 0x7fb8740090b0 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4197705608 0 0) 0x7fb874006590 con 0x7fb8740090b0 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb874009640 con 0x7fb8740090b0 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb88c005f50 con 0x7fb8740090b0 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient: dump: 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T03:58:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 811361961 0 0) 0x7fb874009640 con 0x7fb8740090b0 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb8740098f0 con 0x7fb8740090b0 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7388 (unknown 0 0 4171180120) 0x7fb88c007bb0 con 0x7fb8740090b0 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T03:58:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.122:6792/0 conn(0x7fb8740090b0 legacy=0x7fb874008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb8897fa700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.122:6792/0 conn(0x7fb8740090b0 legacy=0x7fb874008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 102, 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T03:58:44.903 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.904 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.905 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.906 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.906 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.906 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.906 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.906 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T03:58:44.906 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T03:58:44.907 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.907 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.907 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.907 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.907 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.907 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T03:58:44.908 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T03:58:44.909 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T03:58:44.909 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T03:58:44.909 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T03:58:44.909 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T03:58:44.909 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.909 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T03:58:44.910 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T03:58:44.911 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T03:58:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T03:58:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T03:58:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T03:58:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.914 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.915 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.916 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.917 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.918 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T03:58:44.919 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.920 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.921 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.922 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T03:58:44.923 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T03:58:44.924 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.925 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T03:58:44.927 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T03:58:44.928 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T03:58:44.929 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.122:6812/20895 conn(0x7fb878037fa0 legacy=0x7fb87803a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 10 monclient: shutdown 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb899eed700 1 -- 172.21.15.26:0/2666524546 reap_dead start 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 >> v1:172.21.15.26:6791/0 conn(0x7fb894132b40 legacy=0x7fb894134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 shutdown_connections 2024-09-15T03:58:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.880+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 >> 172.21.15.26:0/2666524546 conn(0x7fb89406e370 msgr2=0x7fb8940a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T03:58:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.884+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 shutdown_connections 2024-09-15T03:58:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T03:58:44.884+0000 7fb89a6ee700 1 -- 172.21.15.26:0/2666524546 wait complete. 2024-09-15T03:58:44.931 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-09-15T03:58:44.931 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T03:58:45.272 INFO:tasks.rados.rados.0.smithi122.stdout:2829: done (7 left) 2024-09-15T03:58:45.272 INFO:tasks.rados.rados.0.smithi122.stdout:2830: done (6 left) 2024-09-15T03:58:45.272 INFO:tasks.rados.rados.0.smithi122.stdout:2831: done (5 left) 2024-09-15T03:58:45.273 INFO:tasks.rados.rados.0.smithi122.stdout:2832: done (4 left) 2024-09-15T03:58:45.273 INFO:tasks.rados.rados.0.smithi122.stdout:2833: done (3 left) 2024-09-15T03:58:45.273 INFO:tasks.rados.rados.0.smithi122.stdout:2834: done (2 left) 2024-09-15T03:58:45.273 INFO:tasks.rados.rados.0.smithi122.stdout:2835: done (1 left) 2024-09-15T03:58:45.273 INFO:tasks.rados.rados.0.smithi122.stdout:2836: done (0 left) 2024-09-15T03:58:45.273 INFO:tasks.rados.rados.0.smithi122.stdout:2837: snap_create 2024-09-15T03:58:46.067 INFO:tasks.rados.rados.0.smithi122.stdout:2837: done (0 left) 2024-09-15T03:58:46.067 INFO:tasks.rados.rados.0.smithi122.stdout:2838: write oid 49 current snap is 318 2024-09-15T03:58:46.067 INFO:tasks.rados.rados.0.smithi122.stdout:2838: seq_num 1013 ranges {625491=502186,1861671=713346} 2024-09-15T03:58:46.074 INFO:tasks.rados.rados.0.smithi122.stdout:2838: writing smithi12223906-49 from 625491 to 1127677 tid 1 2024-09-15T03:58:46.084 INFO:tasks.rados.rados.0.smithi122.stdout:2838: writing smithi12223906-49 from 1861671 to 2575017 tid 2 2024-09-15T03:58:46.084 INFO:tasks.rados.rados.0.smithi122.stdout:2839: snap_remove snap 286 2024-09-15T03:58:47.082 INFO:tasks.rados.rados.0.smithi122.stdout:2839: done (1 left) 2024-09-15T03:58:47.082 INFO:tasks.rados.rados.0.smithi122.stdout:2840: write (excl) oid 10 current snap is 318 2024-09-15T03:58:47.082 INFO:tasks.rados.rados.0.smithi122.stdout:2840: seq_num 1014 ranges {642298=577435,1920153=573357,2979150=63148} 2024-09-15T03:58:47.091 INFO:tasks.rados.rados.0.smithi122.stdout:2840: writing smithi12223906-10 from 642298 to 1219733 tid 1 2024-09-15T03:58:47.099 INFO:tasks.rados.rados.0.smithi122.stdout:2840: writing smithi12223906-10 from 1920153 to 2493510 tid 2 2024-09-15T03:58:47.103 INFO:tasks.rados.rados.0.smithi122.stdout:2840: writing smithi12223906-10 from 2979150 to 3042298 tid 3 2024-09-15T03:58:47.103 INFO:tasks.rados.rados.0.smithi122.stdout:2841: rollback oid 48 current snap is 318 2024-09-15T03:58:47.103 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 48 to 301 2024-09-15T03:58:47.103 INFO:tasks.rados.rados.0.smithi122.stdout:2838: finishing write tid 1 to smithi12223906-49 2024-09-15T03:58:47.103 INFO:tasks.rados.rados.0.smithi122.stdout:2838: finishing write tid 2 to smithi12223906-49 2024-09-15T03:58:47.103 INFO:tasks.rados.rados.0.smithi122.stdout:2838: finishing write tid 3 to smithi12223906-49 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:2838: finishing write tid 4 to smithi12223906-49 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1062 (ObjNum 1013 snap 318 seq_num 1013) dirty exists 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:2838: left oid 49 (ObjNum 1013 snap 318 seq_num 1013) 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:2840: finishing write tid 1 to smithi12223906-10 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:2838: done (2 left) 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:2842: write oid 50 current snap is 318 2024-09-15T03:58:47.104 INFO:tasks.rados.rados.0.smithi122.stdout:2842: seq_num 1015 ranges {659105=652684,1862282=717015} 2024-09-15T03:58:47.111 INFO:tasks.rados.rados.0.smithi122.stdout:2842: writing smithi12223906-50 from 659105 to 1311789 tid 1 2024-09-15T03:58:47.118 INFO:tasks.rados.rados.0.smithi122.stdout:2842: writing smithi12223906-50 from 1862282 to 2579297 tid 2 2024-09-15T03:58:47.118 INFO:tasks.rados.rados.0.smithi122.stdout:2843: snap_remove snap 305 2024-09-15T03:58:48.089 INFO:tasks.rados.rados.0.smithi122.stdout:2843: done (3 left) 2024-09-15T03:58:48.089 INFO:tasks.rados.rados.0.smithi122.stdout:2840: finishing write tid 2 to smithi12223906-10 2024-09-15T03:58:48.089 INFO:tasks.rados.rados.0.smithi122.stdout:2840: finishing write tid 3 to smithi12223906-10 2024-09-15T03:58:48.089 INFO:tasks.rados.rados.0.smithi122.stdout:2840: finishing write tid 4 to smithi12223906-10 2024-09-15T03:58:48.089 INFO:tasks.rados.rados.0.smithi122.stdout:2840: finishing write tid 5 to smithi12223906-10 2024-09-15T03:58:48.089 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1099 (ObjNum 1014 snap 318 seq_num 1014) dirty exists 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2840: left oid 10 (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2841: finishing rollback tid 0 to smithi12223906-48 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2841: finishing rollback tid 1 to smithi12223906-48 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2841: finishing rollback tid 2 to smithi12223906-48 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1102 (ObjNum 942 snap 295 seq_num 942) dirty exists 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2842: finishing write tid 1 to smithi12223906-50 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2842: finishing write tid 2 to smithi12223906-50 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2842: finishing write tid 3 to smithi12223906-50 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2842: finishing write tid 4 to smithi12223906-50 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2177 (ObjNum 1015 snap 318 seq_num 1015) dirty exists 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2842: left oid 50 (ObjNum 1015 snap 318 seq_num 1015) 2024-09-15T03:58:48.090 INFO:tasks.rados.rados.0.smithi122.stdout:2844: read oid 35 snap 301 2024-09-15T03:58:48.091 INFO:tasks.rados.rados.0.smithi122.stdout:2844: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:58:48.091 INFO:tasks.rados.rados.0.smithi122.stdout:2840: done (3 left) 2024-09-15T03:58:48.091 INFO:tasks.rados.rados.0.smithi122.stdout:2841: done (2 left) 2024-09-15T03:58:48.091 INFO:tasks.rados.rados.0.smithi122.stdout:2842: done (1 left) 2024-09-15T03:58:48.091 INFO:tasks.rados.rados.0.smithi122.stdout:2845: write oid 21 current snap is 318 2024-09-15T03:58:48.091 INFO:tasks.rados.rados.0.smithi122.stdout:2845: seq_num 1016 ranges {675912=727933,1920764=460673,2801290=274622} 2024-09-15T03:58:48.099 INFO:tasks.rados.rados.0.smithi122.stdout:2845: writing smithi12223906-21 from 675912 to 1403845 tid 1 2024-09-15T03:58:48.105 INFO:tasks.rados.rados.0.smithi122.stdout:2845: writing smithi12223906-21 from 1920764 to 2381437 tid 2 2024-09-15T03:58:48.109 INFO:tasks.rados.rados.0.smithi122.stdout:2845: writing smithi12223906-21 from 2801290 to 3075912 tid 3 2024-09-15T03:58:48.109 INFO:tasks.rados.rados.0.smithi122.stdout:2846: rollback oid 12 current snap is 318 2024-09-15T03:58:48.109 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 303 2024-09-15T03:58:48.110 INFO:tasks.rados.rados.0.smithi122.stdout:2844: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:58:48.194 INFO:tasks.rados.rados.0.smithi122.stdout:2845: finishing write tid 1 to smithi12223906-21 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2845: finishing write tid 2 to smithi12223906-21 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2846: finishing rollback tid 1 to smithi12223906-12 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 760 (ObjNum 129 snap 0 seq_num 1842393952) dirty dne 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2845: finishing write tid 3 to smithi12223906-21 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2845: finishing write tid 4 to smithi12223906-21 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2845: finishing write tid 5 to smithi12223906-21 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1108 (ObjNum 1016 snap 318 seq_num 1016) dirty exists 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2845: left oid 21 (ObjNum 1016 snap 318 seq_num 1016) 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2844: done (2 left) 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2845: done (1 left) 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2846: done (0 left) 2024-09-15T03:58:48.195 INFO:tasks.rados.rados.0.smithi122.stdout:2847: snap_create 2024-09-15T03:58:49.100 INFO:tasks.rados.rados.0.smithi122.stdout:2847: done (0 left) 2024-09-15T03:58:49.100 INFO:tasks.rados.rados.0.smithi122.stdout:2848: write (excl) oid 16 current snap is 319 2024-09-15T03:58:49.100 INFO:tasks.rados.rados.0.smithi122.stdout:2848: seq_num 1017 ranges {692719=403182,1579246=720684,2828713=264006} 2024-09-15T03:58:49.107 INFO:tasks.rados.rados.0.smithi122.stdout:2848: writing smithi12223906-16 from 692719 to 1095901 tid 1 2024-09-15T03:58:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:2848: writing smithi12223906-16 from 1579246 to 2299930 tid 2 2024-09-15T03:58:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2848: writing smithi12223906-16 from 2828713 to 3092719 tid 3 2024-09-15T03:58:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2849: write (excl) oid 35 current snap is 319 2024-09-15T03:58:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2848: finishing write tid 1 to smithi12223906-16 2024-09-15T03:58:49.120 INFO:tasks.rados.rados.0.smithi122.stdout:2849: seq_num 1018 ranges {709526=478431,1637728=464342,2856136=253390} 2024-09-15T03:58:49.127 INFO:tasks.rados.rados.0.smithi122.stdout:2849: writing smithi12223906-35 from 709526 to 1187957 tid 1 2024-09-15T03:58:49.132 INFO:tasks.rados.rados.0.smithi122.stdout:2849: writing smithi12223906-35 from 1637728 to 2102070 tid 2 2024-09-15T03:58:49.138 INFO:tasks.rados.rados.0.smithi122.stdout:2849: writing smithi12223906-35 from 2856136 to 3109526 tid 3 2024-09-15T03:58:49.138 INFO:tasks.rados.rados.0.smithi122.stdout:2850: write (excl) oid 47 current snap is 319 2024-09-15T03:58:49.138 INFO:tasks.rados.rados.0.smithi122.stdout:2850: seq_num 1019 ranges {726333=670033,2096210=724353} 2024-09-15T03:58:49.146 INFO:tasks.rados.rados.0.smithi122.stdout:2850: writing smithi12223906-47 from 726333 to 1396366 tid 1 2024-09-15T03:58:49.154 INFO:tasks.rados.rados.0.smithi122.stdout:2850: writing smithi12223906-47 from 2096210 to 2820563 tid 2 2024-09-15T03:58:49.154 INFO:tasks.rados.rados.0.smithi122.stdout:2851: write oid 49 current snap is 319 2024-09-15T03:58:49.154 INFO:tasks.rados.rados.0.smithi122.stdout:2851: seq_num 1020 ranges {743140=745282,2154692=468011} 2024-09-15T03:58:49.162 INFO:tasks.rados.rados.0.smithi122.stdout:2851: writing smithi12223906-49 from 743140 to 1488422 tid 1 2024-09-15T03:58:49.169 INFO:tasks.rados.rados.0.smithi122.stdout:2851: writing smithi12223906-49 from 2154692 to 2622703 tid 2 2024-09-15T03:58:49.169 INFO:tasks.rados.rados.0.smithi122.stdout:2852: snap_remove snap 311 2024-09-15T03:58:50.124 INFO:tasks.rados.rados.0.smithi122.stdout:2852: done (4 left) 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2853: delete oid 9 current snap is 319 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2848: finishing write tid 2 to smithi12223906-16 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2848: finishing write tid 3 to smithi12223906-16 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2848: finishing write tid 4 to smithi12223906-16 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2848: finishing write tid 5 to smithi12223906-16 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1882 (ObjNum 1017 snap 319 seq_num 1017) dirty exists 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2848: left oid 16 (ObjNum 1017 snap 319 seq_num 1017) 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2849: finishing write tid 1 to smithi12223906-35 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2849: finishing write tid 2 to smithi12223906-35 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2849: finishing write tid 3 to smithi12223906-35 2024-09-15T03:58:50.125 INFO:tasks.rados.rados.0.smithi122.stdout:2849: finishing write tid 4 to smithi12223906-35 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2849: finishing write tid 5 to smithi12223906-35 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1840 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2849: left oid 35 (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2850: finishing write tid 1 to smithi12223906-47 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2850: finishing write tid 2 to smithi12223906-47 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2850: finishing write tid 3 to smithi12223906-47 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2850: finishing write tid 4 to smithi12223906-47 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 894 (ObjNum 1019 snap 319 seq_num 1019) dirty exists 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2850: left oid 47 (ObjNum 1019 snap 319 seq_num 1019) 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2851: finishing write tid 1 to smithi12223906-49 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2851: finishing write tid 2 to smithi12223906-49 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2851: finishing write tid 3 to smithi12223906-49 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:2851: finishing write tid 4 to smithi12223906-49 2024-09-15T03:58:50.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1065 (ObjNum 1020 snap 319 seq_num 1020) dirty exists 2024-09-15T03:58:50.127 INFO:tasks.rados.rados.0.smithi122.stdout:2851: left oid 49 (ObjNum 1020 snap 319 seq_num 1020) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2848: done (4 left) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2849: done (3 left) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2850: done (2 left) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2851: done (1 left) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2853: done (0 left) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2854: read oid 20 snap 312 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2854: expect (ObjNum 933 snap 293 seq_num 933) 2024-09-15T03:58:50.132 INFO:tasks.rados.rados.0.smithi122.stdout:2855: snap_create 2024-09-15T03:58:50.139 INFO:tasks.rados.rados.0.smithi122.stdout:2854: expect (ObjNum 933 snap 293 seq_num 933) 2024-09-15T03:58:51.068 INFO:tasks.rados.rados.0.smithi122.stdout:2854: done (1 left) 2024-09-15T03:58:51.068 INFO:tasks.rados.rados.0.smithi122.stdout:2855: done (0 left) 2024-09-15T03:58:51.068 INFO:tasks.rados.rados.0.smithi122.stdout:2856: write oid 7 current snap is 320 2024-09-15T03:58:51.069 INFO:tasks.rados.rados.0.smithi122.stdout:2856: seq_num 1022 ranges {776754=495780,1871656=471680,2965828=210926} 2024-09-15T03:58:51.075 INFO:tasks.rados.rados.0.smithi122.stdout:2856: writing smithi12223906-7 from 776754 to 1272534 tid 1 2024-09-15T03:58:51.082 INFO:tasks.rados.rados.0.smithi122.stdout:2856: writing smithi12223906-7 from 1871656 to 2343336 tid 2 2024-09-15T03:58:51.087 INFO:tasks.rados.rados.0.smithi122.stdout:2856: writing smithi12223906-7 from 2965828 to 3176754 tid 3 2024-09-15T03:58:51.087 INFO:tasks.rados.rados.0.smithi122.stdout:2857: write (excl) oid 31 current snap is 320 2024-09-15T03:58:51.087 INFO:tasks.rados.rados.0.smithi122.stdout:2856: finishing write tid 1 to smithi12223906-7 2024-09-15T03:58:51.087 INFO:tasks.rados.rados.0.smithi122.stdout:2857: seq_num 1023 ranges {793561=571029,1813785=731691,2993251=200310} 2024-09-15T03:58:51.095 INFO:tasks.rados.rados.0.smithi122.stdout:2857: writing smithi12223906-31 from 793561 to 1364590 tid 1 2024-09-15T03:58:51.102 INFO:tasks.rados.rados.0.smithi122.stdout:2857: writing smithi12223906-31 from 1813785 to 2545476 tid 2 2024-09-15T03:58:51.106 INFO:tasks.rados.rados.0.smithi122.stdout:2857: writing smithi12223906-31 from 2993251 to 3193561 tid 3 2024-09-15T03:58:51.106 INFO:tasks.rados.rados.0.smithi122.stdout:2858: delete oid 47 current snap is 320 2024-09-15T03:58:51.106 INFO:tasks.rados.rados.0.smithi122.stdout:2856: finishing write tid 2 to smithi12223906-7 2024-09-15T03:58:51.107 INFO:tasks.rados.rados.0.smithi122.stdout:2856: finishing write tid 3 to smithi12223906-7 2024-09-15T03:58:51.107 INFO:tasks.rados.rados.0.smithi122.stdout:2856: finishing write tid 4 to smithi12223906-7 2024-09-15T03:58:51.107 INFO:tasks.rados.rados.0.smithi122.stdout:2856: finishing write tid 5 to smithi12223906-7 2024-09-15T03:58:51.107 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1590 (ObjNum 1022 snap 320 seq_num 1022) dirty exists 2024-09-15T03:58:51.107 INFO:tasks.rados.rados.0.smithi122.stdout:2856: left oid 7 (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:58:51.107 INFO:tasks.rados.rados.0.smithi122.stdout:2857: finishing write tid 1 to smithi12223906-31 2024-09-15T03:58:51.110 INFO:tasks.rados.rados.0.smithi122.stdout:2857: finishing write tid 2 to smithi12223906-31 2024-09-15T03:58:51.111 INFO:tasks.rados.rados.0.smithi122.stdout:2857: finishing write tid 3 to smithi12223906-31 2024-09-15T03:58:51.114 INFO:tasks.rados.rados.0.smithi122.stdout:2857: finishing write tid 4 to smithi12223906-31 2024-09-15T03:58:51.114 INFO:tasks.rados.rados.0.smithi122.stdout:2857: finishing write tid 5 to smithi12223906-31 2024-09-15T03:58:51.114 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1892 (ObjNum 1023 snap 320 seq_num 1023) dirty exists 2024-09-15T03:58:51.114 INFO:tasks.rados.rados.0.smithi122.stdout:2857: left oid 31 (ObjNum 1023 snap 320 seq_num 1023) 2024-09-15T03:58:51.118 INFO:tasks.rados.rados.0.smithi122.stdout:2856: done (2 left) 2024-09-15T03:58:51.118 INFO:tasks.rados.rados.0.smithi122.stdout:2857: done (1 left) 2024-09-15T03:58:51.118 INFO:tasks.rados.rados.0.smithi122.stdout:2858: done (0 left) 2024-09-15T03:58:51.118 INFO:tasks.rados.rados.0.smithi122.stdout:2859: write (excl) oid 36 current snap is 320 2024-09-15T03:58:51.118 INFO:tasks.rados.rados.0.smithi122.stdout:2859: seq_num 1025 ranges {427175=721527,1930749=735360} 2024-09-15T03:58:51.127 INFO:tasks.rados.rados.0.smithi122.stdout:2859: writing smithi12223906-36 from 427175 to 1148702 tid 1 2024-09-15T03:58:51.137 INFO:tasks.rados.rados.0.smithi122.stdout:2859: writing smithi12223906-36 from 1930749 to 2666109 tid 2 2024-09-15T03:58:51.137 INFO:tasks.rados.rados.0.smithi122.stdout:2860: read oid 18 snap -1 2024-09-15T03:58:51.137 INFO:tasks.rados.rados.0.smithi122.stdout:2860: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:58:51.137 INFO:tasks.rados.rados.0.smithi122.stdout:2861: copy_from oid 11 from oid 43 current snap is 320 2024-09-15T03:58:51.137 INFO:tasks.rados.rados.0.smithi122.stdout:2859: finishing write tid 1 to smithi12223906-36 2024-09-15T03:58:51.137 INFO:tasks.rados.rados.0.smithi122.stdout:2862: delete oid 17 current snap is 320 2024-09-15T03:58:51.140 INFO:tasks.rados.rados.0.smithi122.stdout:2862: done (3 left) 2024-09-15T03:58:51.140 INFO:tasks.rados.rados.0.smithi122.stdout:2863: snap_create 2024-09-15T03:58:51.145 INFO:tasks.rados.rados.0.smithi122.stdout:2860: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:58:51.313 INFO:tasks.rados.rados.0.smithi122.stdout:2859: finishing write tid 2 to smithi12223906-36 2024-09-15T03:58:51.313 INFO:tasks.rados.rados.0.smithi122.stdout:2861: finishing copy_from to smithi12223906-11 2024-09-15T03:58:51.313 INFO:tasks.rados.rados.0.smithi122.stdout:2861: got expected ENOENT (src dne) 2024-09-15T03:58:51.313 INFO:tasks.rados.rados.0.smithi122.stdout:2859: finishing write tid 3 to smithi12223906-36 2024-09-15T03:58:51.314 INFO:tasks.rados.rados.0.smithi122.stdout:2861: finishing copy_from racing read to smithi12223906-11 2024-09-15T03:58:51.314 INFO:tasks.rados.rados.0.smithi122.stdout:2859: finishing write tid 4 to smithi12223906-36 2024-09-15T03:58:51.314 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 2278 (ObjNum 1025 snap 320 seq_num 1025) dirty exists 2024-09-15T03:58:51.314 INFO:tasks.rados.rados.0.smithi122.stdout:2859: left oid 36 (ObjNum 1025 snap 320 seq_num 1025) 2024-09-15T03:58:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:2859: done (3 left) 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2860: done (2 left) 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2861: done (1 left) 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2863: done (0 left) 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2864: copy_from oid 20 from oid 29 current snap is 321 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2865: read oid 7 snap -1 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2865: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:58:52.119 INFO:tasks.rados.rados.0.smithi122.stdout:2866: delete oid 23 current snap is 321 2024-09-15T03:58:52.127 INFO:tasks.rados.rados.0.smithi122.stdout:2866: done (2 left) 2024-09-15T03:58:52.127 INFO:tasks.rados.rados.0.smithi122.stdout:2867: read oid 10 snap 301 2024-09-15T03:58:52.127 INFO:tasks.rados.rados.0.smithi122.stdout:2867: expect (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:58:52.127 INFO:tasks.rados.rados.0.smithi122.stdout:2868: delete oid 12 current snap is 321 2024-09-15T03:58:52.127 INFO:tasks.rados.rados.0.smithi122.stdout:2865: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:58:52.234 INFO:tasks.rados.rados.0.smithi122.stdout:2864: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:58:52.234 INFO:tasks.rados.rados.0.smithi122.stdout:2864: finishing copy_from to smithi12223906-20 2024-09-15T03:58:52.234 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1573 (ObjNum 900 snap 286 seq_num 900) dirty exists 2024-09-15T03:58:52.235 INFO:tasks.rados.rados.0.smithi122.stdout:2867: expect (ObjNum 937 snap 295 seq_num 937) 2024-09-15T03:58:52.530 INFO:tasks.rados.rados.0.smithi122.stdout:2864: done (3 left) 2024-09-15T03:58:52.530 INFO:tasks.rados.rados.0.smithi122.stdout:2865: done (2 left) 2024-09-15T03:58:52.530 INFO:tasks.rados.rados.0.smithi122.stdout:2867: done (1 left) 2024-09-15T03:58:52.530 INFO:tasks.rados.rados.0.smithi122.stdout:2868: done (0 left) 2024-09-15T03:58:52.530 INFO:tasks.rados.rados.0.smithi122.stdout:2869: write (excl) oid 15 current snap is 321 2024-09-15T03:58:52.530 INFO:tasks.rados.rados.0.smithi122.stdout:2869: seq_num 1029 ranges {494403=738876,1764677=626345,3041436=252967} 2024-09-15T03:58:52.545 INFO:tasks.rados.rados.0.smithi122.stdout:2869: writing smithi12223906-15 from 494403 to 1233279 tid 1 2024-09-15T03:58:52.551 INFO:tasks.rados.rados.0.smithi122.stdout:2869: writing smithi12223906-15 from 1764677 to 2391022 tid 2 2024-09-15T03:58:52.557 INFO:tasks.rados.rados.0.smithi122.stdout:2869: writing smithi12223906-15 from 3041436 to 3294403 tid 3 2024-09-15T03:58:52.557 INFO:tasks.rados.rados.0.smithi122.stdout:2870: snap_remove snap 307 2024-09-15T03:58:53.136 INFO:tasks.rados.rados.0.smithi122.stdout:2869: finishing write tid 1 to smithi12223906-15 2024-09-15T03:58:53.136 INFO:tasks.rados.rados.0.smithi122.stdout:2869: finishing write tid 2 to smithi12223906-15 2024-09-15T03:58:53.136 INFO:tasks.rados.rados.0.smithi122.stdout:2869: finishing write tid 3 to smithi12223906-15 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2869: finishing write tid 4 to smithi12223906-15 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2869: finishing write tid 5 to smithi12223906-15 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1886 (ObjNum 1029 snap 321 seq_num 1029) dirty exists 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2869: left oid 15 (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2869: done (1 left) 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2870: done (0 left) 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2871: write oid 33 current snap is 321 2024-09-15T03:58:53.137 INFO:tasks.rados.rados.0.smithi122.stdout:2871: seq_num 1030 ranges {511210=414125,1423159=486356,2668859=589585} 2024-09-15T03:58:53.144 INFO:tasks.rados.rados.0.smithi122.stdout:2871: writing smithi12223906-33 from 511210 to 925335 tid 1 2024-09-15T03:58:53.150 INFO:tasks.rados.rados.0.smithi122.stdout:2871: writing smithi12223906-33 from 1423159 to 1909515 tid 2 2024-09-15T03:58:53.158 INFO:tasks.rados.rados.0.smithi122.stdout:2871: writing smithi12223906-33 from 2668859 to 3258444 tid 3 2024-09-15T03:58:53.158 INFO:tasks.rados.rados.0.smithi122.stdout:2872: snap_create 2024-09-15T03:58:53.158 INFO:tasks.rados.rados.0.smithi122.stdout:2871: finishing write tid 1 to smithi12223906-33 2024-09-15T03:58:53.158 INFO:tasks.rados.rados.0.smithi122.stdout:2871: finishing write tid 2 to smithi12223906-33 2024-09-15T03:58:53.162 INFO:tasks.rados.rados.0.smithi122.stdout:2871: finishing write tid 3 to smithi12223906-33 2024-09-15T03:58:53.163 INFO:tasks.rados.rados.0.smithi122.stdout:2871: finishing write tid 4 to smithi12223906-33 2024-09-15T03:58:53.163 INFO:tasks.rados.rados.0.smithi122.stdout:2871: finishing write tid 5 to smithi12223906-33 2024-09-15T03:58:53.163 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2207 (ObjNum 1030 snap 321 seq_num 1030) dirty exists 2024-09-15T03:58:53.163 INFO:tasks.rados.rados.0.smithi122.stdout:2871: left oid 33 (ObjNum 1030 snap 321 seq_num 1030) 2024-09-15T03:58:54.155 INFO:tasks.rados.rados.0.smithi122.stdout:2871: done (1 left) 2024-09-15T03:58:54.155 INFO:tasks.rados.rados.0.smithi122.stdout:2872: done (0 left) 2024-09-15T03:58:54.155 INFO:tasks.rados.rados.0.smithi122.stdout:2873: snap_create 2024-09-15T03:58:55.167 INFO:tasks.rados.rados.0.smithi122.stdout:2873: done (0 left) 2024-09-15T03:58:55.167 INFO:tasks.rados.rados.0.smithi122.stdout:2874: read oid 10 snap -1 2024-09-15T03:58:55.167 INFO:tasks.rados.rados.0.smithi122.stdout:2874: expect (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T03:58:55.168 INFO:tasks.rados.rados.0.smithi122.stdout:2875: read oid 26 snap -1 2024-09-15T03:58:55.168 INFO:tasks.rados.rados.0.smithi122.stdout:2875: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:55.168 INFO:tasks.rados.rados.0.smithi122.stdout:2876: snap_create 2024-09-15T03:58:55.179 INFO:tasks.rados.rados.0.smithi122.stdout:2875: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:55.370 INFO:tasks.rados.rados.0.smithi122.stdout:2874: expect (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T03:58:56.072 INFO:tasks.rados.rados.0.smithi122.stdout:2874: done (2 left) 2024-09-15T03:58:56.213 INFO:tasks.rados.rados.0.smithi122.stdout:2875: done (1 left) 2024-09-15T03:58:56.213 INFO:tasks.rados.rados.0.smithi122.stdout:2876: done (0 left) 2024-09-15T03:58:56.213 INFO:tasks.rados.rados.0.smithi122.stdout:2877: read oid 10 snap -1 2024-09-15T03:58:56.213 INFO:tasks.rados.rados.0.smithi122.stdout:2877: expect (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T03:58:56.213 INFO:tasks.rados.rados.0.smithi122.stdout:2878: snap_remove snap 298 2024-09-15T03:58:57.075 INFO:tasks.rados.rados.0.smithi122.stdout:2878: done (1 left) 2024-09-15T03:58:57.075 INFO:tasks.rados.rados.0.smithi122.stdout:2879: read oid 4 snap -1 2024-09-15T03:58:57.076 INFO:tasks.rados.rados.0.smithi122.stdout:2879: expect (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:58:57.076 INFO:tasks.rados.rados.0.smithi122.stdout:2880: write (excl) oid 5 current snap is 324 2024-09-15T03:58:57.076 INFO:tasks.rados.rados.0.smithi122.stdout:2880: seq_num 1031 ranges {528017=489374,1481641=630014,2696282=400857} 2024-09-15T03:58:57.082 INFO:tasks.rados.rados.0.smithi122.stdout:2880: writing smithi12223906-5 from 528017 to 1017391 tid 1 2024-09-15T03:58:57.090 INFO:tasks.rados.rados.0.smithi122.stdout:2880: writing smithi12223906-5 from 1481641 to 2111655 tid 2 2024-09-15T03:58:57.096 INFO:tasks.rados.rados.0.smithi122.stdout:2880: writing smithi12223906-5 from 2696282 to 3097139 tid 3 2024-09-15T03:58:57.096 INFO:tasks.rados.rados.0.smithi122.stdout:2881: read oid 7 snap -1 2024-09-15T03:58:57.096 INFO:tasks.rados.rados.0.smithi122.stdout:2881: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:58:57.096 INFO:tasks.rados.rados.0.smithi122.stdout:2882: snap_create 2024-09-15T03:58:57.096 INFO:tasks.rados.rados.0.smithi122.stdout:2877: expect (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T03:58:57.322 INFO:tasks.rados.rados.0.smithi122.stdout:2880: finishing write tid 1 to smithi12223906-5 2024-09-15T03:58:57.322 INFO:tasks.rados.rados.0.smithi122.stdout:2879: expect (ObjNum 926 snap 292 seq_num 926) 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:2880: finishing write tid 2 to smithi12223906-5 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:2880: finishing write tid 3 to smithi12223906-5 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:2880: finishing write tid 4 to smithi12223906-5 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:2880: finishing write tid 5 to smithi12223906-5 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 899 (ObjNum 1031 snap 324 seq_num 1031) dirty exists 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:2880: left oid 5 (ObjNum 1031 snap 324 seq_num 1031) 2024-09-15T03:58:57.572 INFO:tasks.rados.rados.0.smithi122.stdout:2881: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:58:58.089 INFO:tasks.rados.rados.0.smithi122.stdout:2877: done (4 left) 2024-09-15T03:58:58.089 INFO:tasks.rados.rados.0.smithi122.stdout:2879: done (3 left) 2024-09-15T03:58:58.089 INFO:tasks.rados.rados.0.smithi122.stdout:2880: done (2 left) 2024-09-15T03:58:58.089 INFO:tasks.rados.rados.0.smithi122.stdout:2881: done (1 left) 2024-09-15T03:58:58.089 INFO:tasks.rados.rados.0.smithi122.stdout:2882: done (0 left) 2024-09-15T03:58:58.089 INFO:tasks.rados.rados.0.smithi122.stdout:2883: delete oid 20 current snap is 325 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:2883: done (0 left) 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:2884: rollback oid 20 current snap is 325 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 302 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:2885: read oid 33 snap 313 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:2885: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:2886: write oid 30 current snap is 325 2024-09-15T03:58:58.093 INFO:tasks.rados.rados.0.smithi122.stdout:2886: seq_num 1033 ranges {561631=639872,1882252=633683,3034775=326856} 2024-09-15T03:58:58.103 INFO:tasks.rados.rados.0.smithi122.stdout:2886: writing smithi12223906-30 from 561631 to 1201503 tid 1 2024-09-15T03:58:58.111 INFO:tasks.rados.rados.0.smithi122.stdout:2886: writing smithi12223906-30 from 1882252 to 2515935 tid 2 2024-09-15T03:58:58.116 INFO:tasks.rados.rados.0.smithi122.stdout:2886: writing smithi12223906-30 from 3034775 to 3361631 tid 3 2024-09-15T03:58:58.116 INFO:tasks.rados.rados.0.smithi122.stdout:2887: write oid 43 current snap is 325 2024-09-15T03:58:58.117 INFO:tasks.rados.rados.0.smithi122.stdout:2884: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:58:58.117 INFO:tasks.rados.rados.0.smithi122.stdout:2887: seq_num 1034 ranges {578438=431474,1657087=493694,2778551=599887} 2024-09-15T03:58:58.122 INFO:tasks.rados.rados.0.smithi122.stdout:2887: writing smithi12223906-43 from 578438 to 1009912 tid 1 2024-09-15T03:58:58.128 INFO:tasks.rados.rados.0.smithi122.stdout:2887: writing smithi12223906-43 from 1657087 to 2150781 tid 2 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2887: writing smithi12223906-43 from 2778551 to 3378438 tid 3 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2888: read oid 26 snap 306 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2888: expect (ObjNum 963 snap 305 seq_num 963) 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2889: read oid 46 snap -1 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2889: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2890: rollback oid 17 current snap is 325 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 313 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:2884: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:58:58.136 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1576 (ObjNum 908 snap 288 seq_num 908) dirty exists 2024-09-15T03:58:58.137 INFO:tasks.rados.rados.0.smithi122.stdout:2884: done (6 left) 2024-09-15T03:58:58.137 INFO:tasks.rados.rados.0.smithi122.stdout:2891: snap_create 2024-09-15T03:58:58.137 INFO:tasks.rados.rados.0.smithi122.stdout:2885: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:58:58.263 INFO:tasks.rados.rados.0.smithi122.stdout:2886: finishing write tid 1 to smithi12223906-30 2024-09-15T03:58:58.263 INFO:tasks.rados.rados.0.smithi122.stdout:2886: finishing write tid 2 to smithi12223906-30 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2886: finishing write tid 3 to smithi12223906-30 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2886: finishing write tid 4 to smithi12223906-30 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2886: finishing write tid 5 to smithi12223906-30 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 854 (ObjNum 1033 snap 325 seq_num 1033) dirty exists 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2886: left oid 30 (ObjNum 1033 snap 325 seq_num 1033) 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2887: finishing write tid 1 to smithi12223906-43 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2887: finishing write tid 2 to smithi12223906-43 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2890: finishing rollback tid 1 to smithi12223906-17 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1056 (ObjNum 65 snap 0 seq_num 1842331040) dirty dne 2024-09-15T03:58:58.264 INFO:tasks.rados.rados.0.smithi122.stdout:2889: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T03:58:58.451 INFO:tasks.rados.rados.0.smithi122.stdout:2887: finishing write tid 3 to smithi12223906-43 2024-09-15T03:58:58.451 INFO:tasks.rados.rados.0.smithi122.stdout:2888: expect (ObjNum 963 snap 305 seq_num 963) 2024-09-15T03:58:58.600 INFO:tasks.rados.rados.0.smithi122.stdout:2887: finishing write tid 4 to smithi12223906-43 2024-09-15T03:58:58.601 INFO:tasks.rados.rados.0.smithi122.stdout:2887: finishing write tid 5 to smithi12223906-43 2024-09-15T03:58:58.601 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 2300 (ObjNum 1034 snap 325 seq_num 1034) dirty exists 2024-09-15T03:58:58.601 INFO:tasks.rados.rados.0.smithi122.stdout:2887: left oid 43 (ObjNum 1034 snap 325 seq_num 1034) 2024-09-15T03:58:59.092 INFO:tasks.rados.rados.0.smithi122.stdout:2885: done (6 left) 2024-09-15T03:58:59.092 INFO:tasks.rados.rados.0.smithi122.stdout:2886: done (5 left) 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2887: done (4 left) 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2888: done (3 left) 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2889: done (2 left) 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2890: done (1 left) 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2891: done (0 left) 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2892: write oid 9 current snap is 326 2024-09-15T03:58:59.093 INFO:tasks.rados.rados.0.smithi122.stdout:2892: seq_num 1035 ranges {595245=506723,1715569=637352,2805974=562298} 2024-09-15T03:58:59.101 INFO:tasks.rados.rados.0.smithi122.stdout:2892: writing smithi12223906-9 from 595245 to 1101968 tid 1 2024-09-15T03:58:59.108 INFO:tasks.rados.rados.0.smithi122.stdout:2892: writing smithi12223906-9 from 1715569 to 2352921 tid 2 2024-09-15T03:58:59.115 INFO:tasks.rados.rados.0.smithi122.stdout:2892: writing smithi12223906-9 from 2805974 to 3368272 tid 3 2024-09-15T03:58:59.115 INFO:tasks.rados.rados.0.smithi122.stdout:2893: read oid 3 snap -1 2024-09-15T03:58:59.116 INFO:tasks.rados.rados.0.smithi122.stdout:2893: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T03:58:59.116 INFO:tasks.rados.rados.0.smithi122.stdout:2894: snap_create 2024-09-15T03:58:59.116 INFO:tasks.rados.rados.0.smithi122.stdout:2892: finishing write tid 1 to smithi12223906-9 2024-09-15T03:58:59.116 INFO:tasks.rados.rados.0.smithi122.stdout:2892: finishing write tid 2 to smithi12223906-9 2024-09-15T03:58:59.118 INFO:tasks.rados.rados.0.smithi122.stdout:2892: finishing write tid 3 to smithi12223906-9 2024-09-15T03:58:59.119 INFO:tasks.rados.rados.0.smithi122.stdout:2892: finishing write tid 4 to smithi12223906-9 2024-09-15T03:58:59.120 INFO:tasks.rados.rados.0.smithi122.stdout:2892: finishing write tid 5 to smithi12223906-9 2024-09-15T03:58:59.120 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 1339 (ObjNum 1035 snap 326 seq_num 1035) dirty exists 2024-09-15T03:58:59.120 INFO:tasks.rados.rados.0.smithi122.stdout:2892: left oid 9 (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T03:58:59.125 INFO:tasks.rados.rados.0.smithi122.stdout:2893: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T03:59:00.102 INFO:tasks.rados.rados.0.smithi122.stdout:2892: done (2 left) 2024-09-15T03:59:00.102 INFO:tasks.rados.rados.0.smithi122.stdout:2893: done (1 left) 2024-09-15T03:59:00.102 INFO:tasks.rados.rados.0.smithi122.stdout:2894: done (0 left) 2024-09-15T03:59:00.102 INFO:tasks.rados.rados.0.smithi122.stdout:2895: write (excl) oid 14 current snap is 327 2024-09-15T03:59:00.102 INFO:tasks.rados.rados.0.smithi122.stdout:2895: seq_num 1036 ranges {612052=581972,1657698=497363,2717044=695008} 2024-09-15T03:59:00.111 INFO:tasks.rados.rados.0.smithi122.stdout:2895: writing smithi12223906-14 from 612052 to 1194024 tid 1 2024-09-15T03:59:00.118 INFO:tasks.rados.rados.0.smithi122.stdout:2895: writing smithi12223906-14 from 1657698 to 2155061 tid 2 2024-09-15T03:59:00.125 INFO:tasks.rados.rados.0.smithi122.stdout:2895: writing smithi12223906-14 from 2717044 to 3412052 tid 3 2024-09-15T03:59:00.125 INFO:tasks.rados.rados.0.smithi122.stdout:2896: read oid 33 snap -1 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2896: expect (ObjNum 1030 snap 321 seq_num 1030) 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2897: read oid 13 snap -1 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2897: expect deleted 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2898: rollback oid 50 current snap is 327 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 251 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2895: finishing write tid 1 to smithi12223906-14 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2899: write oid 29 current snap is 327 2024-09-15T03:59:00.126 INFO:tasks.rados.rados.0.smithi122.stdout:2899: seq_num 1037 ranges {628859=657221,1716180=641021,3144467=284392} 2024-09-15T03:59:00.132 INFO:tasks.rados.rados.0.smithi122.stdout:2899: writing smithi12223906-29 from 628859 to 1286080 tid 1 2024-09-15T03:59:00.138 INFO:tasks.rados.rados.0.smithi122.stdout:2899: writing smithi12223906-29 from 1716180 to 2357201 tid 2 2024-09-15T03:59:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:2899: writing smithi12223906-29 from 3144467 to 3428859 tid 3 2024-09-15T03:59:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:2900: write (excl) oid 5 current snap is 327 2024-09-15T03:59:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:2900: seq_num 1038 ranges {645666=732470,2174662=501032,3288243=157423} 2024-09-15T03:59:00.152 INFO:tasks.rados.rados.0.smithi122.stdout:2900: writing smithi12223906-5 from 645666 to 1378136 tid 1 2024-09-15T03:59:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:2900: writing smithi12223906-5 from 2174662 to 2675694 tid 2 2024-09-15T03:59:00.164 INFO:tasks.rados.rados.0.smithi122.stdout:2900: writing smithi12223906-5 from 3288243 to 3445666 tid 3 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2901: read oid 45 snap -1 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2901: expect (ObjNum 962 snap 305 seq_num 962) 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2902: snap_create 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2895: finishing write tid 2 to smithi12223906-14 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2895: finishing write tid 3 to smithi12223906-14 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2895: finishing write tid 4 to smithi12223906-14 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2895: finishing write tid 5 to smithi12223906-14 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 2305 (ObjNum 1036 snap 327 seq_num 1036) dirty exists 2024-09-15T03:59:00.165 INFO:tasks.rados.rados.0.smithi122.stdout:2895: left oid 14 (ObjNum 1036 snap 327 seq_num 1036) 2024-09-15T03:59:00.166 INFO:tasks.rados.rados.0.smithi122.stdout:2896: expect (ObjNum 1030 snap 321 seq_num 1030) 2024-09-15T03:59:00.320 INFO:tasks.rados.rados.0.smithi122.stdout:2898: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:59:00.320 INFO:tasks.rados.rados.0.smithi122.stdout:2898: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:59:00.320 INFO:tasks.rados.rados.0.smithi122.stdout:2898: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2221 (ObjNum 715 snap 217 seq_num 715) dirty exists 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:2899: finishing write tid 1 to smithi12223906-29 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:2899: finishing write tid 2 to smithi12223906-29 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:2899: finishing write tid 3 to smithi12223906-29 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:2899: finishing write tid 4 to smithi12223906-29 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:2899: finishing write tid 5 to smithi12223906-29 2024-09-15T03:59:00.321 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 1594 (ObjNum 1037 snap 327 seq_num 1037) dirty exists 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2899: left oid 29 (ObjNum 1037 snap 327 seq_num 1037) 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2900: finishing write tid 1 to smithi12223906-5 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2900: finishing write tid 2 to smithi12223906-5 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2900: finishing write tid 3 to smithi12223906-5 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2900: finishing write tid 4 to smithi12223906-5 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2900: finishing write tid 5 to smithi12223906-5 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 903 (ObjNum 1038 snap 327 seq_num 1038) dirty exists 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2900: left oid 5 (ObjNum 1038 snap 327 seq_num 1038) 2024-09-15T03:59:00.322 INFO:tasks.rados.rados.0.smithi122.stdout:2901: expect (ObjNum 962 snap 305 seq_num 962) 2024-09-15T03:59:01.074 INFO:tasks.rados.rados.0.smithi122.stdout:2895: done (7 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2896: done (6 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2897: done (5 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2898: done (4 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2899: done (3 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2900: done (2 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2901: done (1 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2902: done (0 left) 2024-09-15T03:59:01.075 INFO:tasks.rados.rados.0.smithi122.stdout:2903: snap_create 2024-09-15T03:59:02.075 INFO:tasks.rados.rados.0.smithi122.stdout:2903: done (0 left) 2024-09-15T03:59:02.076 INFO:tasks.rados.rados.0.smithi122.stdout:2904: read oid 16 snap 320 2024-09-15T03:59:02.076 INFO:tasks.rados.rados.0.smithi122.stdout:2904: expect (ObjNum 1017 snap 319 seq_num 1017) 2024-09-15T03:59:02.076 INFO:tasks.rados.rados.0.smithi122.stdout:2905: snap_create 2024-09-15T03:59:02.083 INFO:tasks.rados.rados.0.smithi122.stdout:2904: expect (ObjNum 1017 snap 319 seq_num 1017) 2024-09-15T03:59:03.077 INFO:tasks.rados.rados.0.smithi122.stdout:2904: done (1 left) 2024-09-15T03:59:03.078 INFO:tasks.rados.rados.0.smithi122.stdout:2905: done (0 left) 2024-09-15T03:59:03.078 INFO:tasks.rados.rados.0.smithi122.stdout:2906: write oid 28 current snap is 330 2024-09-15T03:59:03.078 INFO:tasks.rados.rados.0.smithi122.stdout:2906: seq_num 1039 ranges {662473=407719,1833144=644690,3199313=263160} 2024-09-15T03:59:03.087 INFO:tasks.rados.rados.0.smithi122.stdout:2906: writing smithi12223906-28 from 662473 to 1070192 tid 1 2024-09-15T03:59:03.096 INFO:tasks.rados.rados.0.smithi122.stdout:2906: writing smithi12223906-28 from 1833144 to 2477834 tid 2 2024-09-15T03:59:03.101 INFO:tasks.rados.rados.0.smithi122.stdout:2906: writing smithi12223906-28 from 3199313 to 3462473 tid 3 2024-09-15T03:59:03.101 INFO:tasks.rados.rados.0.smithi122.stdout:2907: snap_remove snap 308 2024-09-15T03:59:04.103 INFO:tasks.rados.rados.0.smithi122.stdout:2907: done (1 left) 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2908: read oid 2 snap -1 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2908: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2909: copy_from oid 43 from oid 22 current snap is 330 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2906: finishing write tid 1 to smithi12223906-28 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2906: finishing write tid 2 to smithi12223906-28 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2906: finishing write tid 3 to smithi12223906-28 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2906: finishing write tid 4 to smithi12223906-28 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:2906: finishing write tid 5 to smithi12223906-28 2024-09-15T03:59:04.104 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 1598 (ObjNum 1039 snap 330 seq_num 1039) dirty exists 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2906: left oid 28 (ObjNum 1039 snap 330 seq_num 1039) 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2906: done (2 left) 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2910: rollback oid 47 current snap is 330 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 47 to 231 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2911: read oid 33 snap 329 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2911: expect (ObjNum 1030 snap 321 seq_num 1030) 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2912: write (excl) oid 21 current snap is 330 2024-09-15T03:59:04.105 INFO:tasks.rados.rados.0.smithi122.stdout:2912: seq_num 1040 ranges {679280=482968,1775273=788348,3110383=368897} 2024-09-15T03:59:04.113 INFO:tasks.rados.rados.0.smithi122.stdout:2912: writing smithi12223906-21 from 679280 to 1162248 tid 1 2024-09-15T03:59:04.121 INFO:tasks.rados.rados.0.smithi122.stdout:2912: writing smithi12223906-21 from 1775273 to 2563621 tid 2 2024-09-15T03:59:04.127 INFO:tasks.rados.rados.0.smithi122.stdout:2912: writing smithi12223906-21 from 3110383 to 3479280 tid 3 2024-09-15T03:59:04.127 INFO:tasks.rados.rados.0.smithi122.stdout:2913: snap_remove snap 318 2024-09-15T03:59:05.118 INFO:tasks.rados.rados.0.smithi122.stdout:2913: done (5 left) 2024-09-15T03:59:05.118 INFO:tasks.rados.rados.0.smithi122.stdout:2914: write oid 39 current snap is 330 2024-09-15T03:59:05.118 INFO:tasks.rados.rados.0.smithi122.stdout:2910: finishing rollback tid 1 to smithi12223906-47 2024-09-15T03:59:05.118 INFO:tasks.rados.rados.0.smithi122.stdout:2914: seq_num 1041 ranges {696087=558217,1833755=648359,3137806=358281} 2024-09-15T03:59:05.125 INFO:tasks.rados.rados.0.smithi122.stdout:2914: writing smithi12223906-39 from 696087 to 1254304 tid 1 2024-09-15T03:59:05.132 INFO:tasks.rados.rados.0.smithi122.stdout:2914: writing smithi12223906-39 from 1833755 to 2482114 tid 2 2024-09-15T03:59:05.138 INFO:tasks.rados.rados.0.smithi122.stdout:2914: writing smithi12223906-39 from 3137806 to 3496087 tid 3 2024-09-15T03:59:05.138 INFO:tasks.rados.rados.0.smithi122.stdout:2915: snap_create 2024-09-15T03:59:05.138 INFO:tasks.rados.rados.0.smithi122.stdout:2908: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:59:05.301 INFO:tasks.rados.rados.0.smithi122.stdout:2910: finishing rollback tid 2 to smithi12223906-47 2024-09-15T03:59:05.302 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 905 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:59:05.302 INFO:tasks.rados.rados.0.smithi122.stdout:2911: expect (ObjNum 1030 snap 321 seq_num 1030) 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2912: finishing write tid 1 to smithi12223906-21 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2909: finishing copy_from racing read to smithi12223906-43 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2909: finishing copy_from to smithi12223906-43 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 1875 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2912: finishing write tid 2 to smithi12223906-21 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2912: finishing write tid 3 to smithi12223906-21 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2912: finishing write tid 4 to smithi12223906-21 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:2912: finishing write tid 5 to smithi12223906-21 2024-09-15T03:59:05.453 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1135 (ObjNum 1040 snap 330 seq_num 1040) dirty exists 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2912: left oid 21 (ObjNum 1040 snap 330 seq_num 1040) 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2914: finishing write tid 1 to smithi12223906-39 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2914: finishing write tid 2 to smithi12223906-39 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2914: finishing write tid 3 to smithi12223906-39 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2914: finishing write tid 4 to smithi12223906-39 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2914: finishing write tid 5 to smithi12223906-39 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2323 (ObjNum 1041 snap 330 seq_num 1041) dirty exists 2024-09-15T03:59:05.454 INFO:tasks.rados.rados.0.smithi122.stdout:2914: left oid 39 (ObjNum 1041 snap 330 seq_num 1041) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2908: done (6 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2909: done (5 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2910: done (4 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2911: done (3 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2912: done (2 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2914: done (1 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2915: done (0 left) 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2916: write (excl) oid 40 current snap is 331 2024-09-15T03:59:06.080 INFO:tasks.rados.rados.0.smithi122.stdout:2916: seq_num 1042 ranges {712894=749819,2008590=792017,3281582=231312} 2024-09-15T03:59:06.090 INFO:tasks.rados.rados.0.smithi122.stdout:2916: writing smithi12223906-40 from 712894 to 1462713 tid 1 2024-09-15T03:59:06.098 INFO:tasks.rados.rados.0.smithi122.stdout:2916: writing smithi12223906-40 from 2008590 to 2800607 tid 2 2024-09-15T03:59:06.102 INFO:tasks.rados.rados.0.smithi122.stdout:2916: writing smithi12223906-40 from 3281582 to 3512894 tid 3 2024-09-15T03:59:06.102 INFO:tasks.rados.rados.0.smithi122.stdout:2917: copy_from oid 38 from oid 22 current snap is 331 2024-09-15T03:59:06.102 INFO:tasks.rados.rados.0.smithi122.stdout:2918: rollback oid 50 current snap is 331 2024-09-15T03:59:06.102 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 312 2024-09-15T03:59:06.103 INFO:tasks.rados.rados.0.smithi122.stdout:2919: snap_create 2024-09-15T03:59:06.109 INFO:tasks.rados.rados.0.smithi122.stdout:2916: finishing write tid 1 to smithi12223906-40 2024-09-15T03:59:06.110 INFO:tasks.rados.rados.0.smithi122.stdout:2918: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:59:06.111 INFO:tasks.rados.rados.0.smithi122.stdout:2918: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:59:06.112 INFO:tasks.rados.rados.0.smithi122.stdout:2918: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:59:06.112 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2239 (ObjNum 990 snap 311 seq_num 990) dirty exists 2024-09-15T03:59:06.114 INFO:tasks.rados.rados.0.smithi122.stdout:2916: finishing write tid 2 to smithi12223906-40 2024-09-15T03:59:06.114 INFO:tasks.rados.rados.0.smithi122.stdout:2916: finishing write tid 3 to smithi12223906-40 2024-09-15T03:59:06.115 INFO:tasks.rados.rados.0.smithi122.stdout:2916: finishing write tid 4 to smithi12223906-40 2024-09-15T03:59:06.115 INFO:tasks.rados.rados.0.smithi122.stdout:2916: finishing write tid 5 to smithi12223906-40 2024-09-15T03:59:06.115 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 858 (ObjNum 1042 snap 331 seq_num 1042) dirty exists 2024-09-15T03:59:06.115 INFO:tasks.rados.rados.0.smithi122.stdout:2916: left oid 40 (ObjNum 1042 snap 331 seq_num 1042) 2024-09-15T03:59:06.121 INFO:tasks.rados.rados.0.smithi122.stdout:2917: finishing copy_from racing read to smithi12223906-38 2024-09-15T03:59:06.125 INFO:tasks.rados.rados.0.smithi122.stdout:2917: finishing copy_from to smithi12223906-38 2024-09-15T03:59:06.125 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1875 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:59:07.094 INFO:tasks.rados.rados.0.smithi122.stdout:2916: done (3 left) 2024-09-15T03:59:07.094 INFO:tasks.rados.rados.0.smithi122.stdout:2917: done (2 left) 2024-09-15T03:59:07.094 INFO:tasks.rados.rados.0.smithi122.stdout:2918: done (1 left) 2024-09-15T03:59:07.094 INFO:tasks.rados.rados.0.smithi122.stdout:2919: done (0 left) 2024-09-15T03:59:07.094 INFO:tasks.rados.rados.0.smithi122.stdout:2920: write oid 45 current snap is 332 2024-09-15T03:59:07.094 INFO:tasks.rados.rados.0.smithi122.stdout:2920: seq_num 1043 ranges {729701=425068,1667072=652028,2909005=485180} 2024-09-15T03:59:07.103 INFO:tasks.rados.rados.0.smithi122.stdout:2920: writing smithi12223906-45 from 729701 to 1154769 tid 1 2024-09-15T03:59:07.110 INFO:tasks.rados.rados.0.smithi122.stdout:2920: writing smithi12223906-45 from 1667072 to 2319100 tid 2 2024-09-15T03:59:07.116 INFO:tasks.rados.rados.0.smithi122.stdout:2920: writing smithi12223906-45 from 2909005 to 3394185 tid 3 2024-09-15T03:59:07.116 INFO:tasks.rados.rados.0.smithi122.stdout:2921: rollback oid 38 current snap is 332 2024-09-15T03:59:07.116 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 326 2024-09-15T03:59:07.116 INFO:tasks.rados.rados.0.smithi122.stdout:2920: finishing write tid 1 to smithi12223906-45 2024-09-15T03:59:07.117 INFO:tasks.rados.rados.0.smithi122.stdout:2920: finishing write tid 2 to smithi12223906-45 2024-09-15T03:59:07.117 INFO:tasks.rados.rados.0.smithi122.stdout:2922: delete oid 37 current snap is 332 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:2922: done (2 left) 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:2923: read oid 7 snap -1 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:2923: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:2924: rollback oid 43 current snap is 332 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 43 to 302 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:2925: write (excl) oid 27 current snap is 332 2024-09-15T03:59:07.123 INFO:tasks.rados.rados.0.smithi122.stdout:2925: seq_num 1045 ranges {763315=575566,2067683=655697,3247498=315817} 2024-09-15T03:59:07.131 INFO:tasks.rados.rados.0.smithi122.stdout:2925: writing smithi12223906-27 from 763315 to 1338881 tid 1 2024-09-15T03:59:07.139 INFO:tasks.rados.rados.0.smithi122.stdout:2925: writing smithi12223906-27 from 2067683 to 2723380 tid 2 2024-09-15T03:59:07.144 INFO:tasks.rados.rados.0.smithi122.stdout:2925: writing smithi12223906-27 from 3247498 to 3563315 tid 3 2024-09-15T03:59:07.144 INFO:tasks.rados.rados.0.smithi122.stdout:2926: read oid 30 snap -1 2024-09-15T03:59:07.144 INFO:tasks.rados.rados.0.smithi122.stdout:2926: expect (ObjNum 1033 snap 325 seq_num 1033) 2024-09-15T03:59:07.145 INFO:tasks.rados.rados.0.smithi122.stdout:2927: snap_remove snap 251 2024-09-15T03:59:08.102 INFO:tasks.rados.rados.0.smithi122.stdout:2927: done (6 left) 2024-09-15T03:59:08.106 INFO:tasks.rados.rados.0.smithi122.stdout:2920: finishing write tid 3 to smithi12223906-45 2024-09-15T03:59:08.106 INFO:tasks.rados.rados.0.smithi122.stdout:2921: finishing rollback tid 0 to smithi12223906-38 2024-09-15T03:59:08.106 INFO:tasks.rados.rados.0.smithi122.stdout:2921: finishing rollback tid 1 to smithi12223906-38 2024-09-15T03:59:08.106 INFO:tasks.rados.rados.0.smithi122.stdout:2920: finishing write tid 4 to smithi12223906-45 2024-09-15T03:59:08.106 INFO:tasks.rados.rados.0.smithi122.stdout:2920: finishing write tid 5 to smithi12223906-45 2024-09-15T03:59:08.106 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 1588 (ObjNum 1043 snap 332 seq_num 1043) dirty exists 2024-09-15T03:59:08.107 INFO:tasks.rados.rados.0.smithi122.stdout:2920: left oid 45 (ObjNum 1043 snap 332 seq_num 1043) 2024-09-15T03:59:08.107 INFO:tasks.rados.rados.0.smithi122.stdout:2921: finishing rollback tid 2 to smithi12223906-38 2024-09-15T03:59:08.107 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1878 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T03:59:08.107 INFO:tasks.rados.rados.0.smithi122.stdout:2923: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:59:08.211 INFO:tasks.rados.rados.0.smithi122.stdout:2924: finishing rollback tid 0 to smithi12223906-43 2024-09-15T03:59:08.211 INFO:tasks.rados.rados.0.smithi122.stdout:2924: finishing rollback tid 1 to smithi12223906-43 2024-09-15T03:59:08.211 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 2326 (ObjNum 33 snap 0 seq_num 1817658064) dirty dne 2024-09-15T03:59:08.211 INFO:tasks.rados.rados.0.smithi122.stdout:2925: finishing write tid 1 to smithi12223906-27 2024-09-15T03:59:08.211 INFO:tasks.rados.rados.0.smithi122.stdout:2925: finishing write tid 2 to smithi12223906-27 2024-09-15T03:59:08.211 INFO:tasks.rados.rados.0.smithi122.stdout:2925: finishing write tid 3 to smithi12223906-27 2024-09-15T03:59:08.212 INFO:tasks.rados.rados.0.smithi122.stdout:2925: finishing write tid 4 to smithi12223906-27 2024-09-15T03:59:08.212 INFO:tasks.rados.rados.0.smithi122.stdout:2925: finishing write tid 5 to smithi12223906-27 2024-09-15T03:59:08.212 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 2244 (ObjNum 1045 snap 332 seq_num 1045) dirty exists 2024-09-15T03:59:08.212 INFO:tasks.rados.rados.0.smithi122.stdout:2925: left oid 27 (ObjNum 1045 snap 332 seq_num 1045) 2024-09-15T03:59:08.212 INFO:tasks.rados.rados.0.smithi122.stdout:2926: expect (ObjNum 1033 snap 325 seq_num 1033) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2928: read oid 35 snap -1 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2928: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2920: done (6 left) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2921: done (5 left) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2923: done (4 left) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2924: done (3 left) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2925: done (2 left) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2926: done (1 left) 2024-09-15T03:59:08.399 INFO:tasks.rados.rados.0.smithi122.stdout:2929: read oid 43 snap -1 2024-09-15T03:59:08.400 INFO:tasks.rados.rados.0.smithi122.stdout:2929: expect deleted 2024-09-15T03:59:08.400 INFO:tasks.rados.rados.0.smithi122.stdout:2930: rollback oid 50 current snap is 332 2024-09-15T03:59:08.400 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 287 2024-09-15T03:59:08.400 INFO:tasks.rados.rados.0.smithi122.stdout:2931: snap_remove snap 320 2024-09-15T03:59:09.115 INFO:tasks.rados.rados.0.smithi122.stdout:2930: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:59:09.116 INFO:tasks.rados.rados.0.smithi122.stdout:2930: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:59:09.116 INFO:tasks.rados.rados.0.smithi122.stdout:2930: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:59:09.116 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2260 (ObjNum 855 snap 277 seq_num 855) dirty exists 2024-09-15T03:59:09.116 INFO:tasks.rados.rados.0.smithi122.stdout:2928: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T03:59:09.222 INFO:tasks.rados.rados.0.smithi122.stdout:2928: done (3 left) 2024-09-15T03:59:09.222 INFO:tasks.rados.rados.0.smithi122.stdout:2929: done (2 left) 2024-09-15T03:59:09.223 INFO:tasks.rados.rados.0.smithi122.stdout:2930: done (1 left) 2024-09-15T03:59:09.223 INFO:tasks.rados.rados.0.smithi122.stdout:2931: done (0 left) 2024-09-15T03:59:09.223 INFO:tasks.rados.rados.0.smithi122.stdout:2932: snap_create 2024-09-15T03:59:10.127 INFO:tasks.rados.rados.0.smithi122.stdout:2932: done (0 left) 2024-09-15T03:59:10.127 INFO:tasks.rados.rados.0.smithi122.stdout:2933: rollback oid 20 current snap is 333 2024-09-15T03:59:10.127 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 317 2024-09-15T03:59:10.127 INFO:tasks.rados.rados.0.smithi122.stdout:2934: snap_create 2024-09-15T03:59:10.133 INFO:tasks.rados.rados.0.smithi122.stdout:2933: finishing rollback tid 0 to smithi12223906-20 2024-09-15T03:59:10.135 INFO:tasks.rados.rados.0.smithi122.stdout:2933: finishing rollback tid 1 to smithi12223906-20 2024-09-15T03:59:10.138 INFO:tasks.rados.rados.0.smithi122.stdout:2933: finishing rollback tid 2 to smithi12223906-20 2024-09-15T03:59:10.138 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1579 (ObjNum 933 snap 293 seq_num 933) dirty exists 2024-09-15T03:59:11.080 INFO:tasks.rados.rados.0.smithi122.stdout:2933: done (1 left) 2024-09-15T03:59:11.080 INFO:tasks.rados.rados.0.smithi122.stdout:2934: done (0 left) 2024-09-15T03:59:11.080 INFO:tasks.rados.rados.0.smithi122.stdout:2935: snap_create 2024-09-15T03:59:12.080 INFO:tasks.rados.rados.0.smithi122.stdout:2935: done (0 left) 2024-09-15T03:59:12.081 INFO:tasks.rados.rados.0.smithi122.stdout:2936: write (excl) oid 48 current snap is 335 2024-09-15T03:59:12.081 INFO:tasks.rados.rados.0.smithi122.stdout:2936: seq_num 1046 ranges {780122=650815,2126165=799355} 2024-09-15T03:59:12.092 INFO:tasks.rados.rados.0.smithi122.stdout:2936: writing smithi12223906-48 from 780122 to 1430937 tid 1 2024-09-15T03:59:12.100 INFO:tasks.rados.rados.0.smithi122.stdout:2936: writing smithi12223906-48 from 2126165 to 2925520 tid 2 2024-09-15T03:59:12.101 INFO:tasks.rados.rados.0.smithi122.stdout:2937: snap_remove snap 281 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2937: done (1 left) 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2938: delete oid 36 current snap is 335 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2936: finishing write tid 1 to smithi12223906-48 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2936: finishing write tid 2 to smithi12223906-48 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2936: finishing write tid 3 to smithi12223906-48 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2936: finishing write tid 4 to smithi12223906-48 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1148 (ObjNum 1046 snap 335 seq_num 1046) dirty exists 2024-09-15T03:59:13.094 INFO:tasks.rados.rados.0.smithi122.stdout:2936: left oid 48 (ObjNum 1046 snap 335 seq_num 1046) 2024-09-15T03:59:13.107 INFO:tasks.rados.rados.0.smithi122.stdout:2936: done (1 left) 2024-09-15T03:59:13.107 INFO:tasks.rados.rados.0.smithi122.stdout:2938: done (0 left) 2024-09-15T03:59:13.107 INFO:tasks.rados.rados.0.smithi122.stdout:2939: copy_from oid 30 from oid 42 current snap is 335 2024-09-15T03:59:13.107 INFO:tasks.rados.rados.0.smithi122.stdout:2940: write (excl) oid 31 current snap is 335 2024-09-15T03:59:13.107 INFO:tasks.rados.rados.0.smithi122.stdout:2940: seq_num 1048 ranges {413736=401313,1326776=403024,2413414=457893,3438716=175020} 2024-09-15T03:59:13.112 INFO:tasks.rados.rados.0.smithi122.stdout:2940: writing smithi12223906-31 from 413736 to 815049 tid 1 2024-09-15T03:59:13.119 INFO:tasks.rados.rados.0.smithi122.stdout:2940: writing smithi12223906-31 from 1326776 to 1729800 tid 2 2024-09-15T03:59:13.125 INFO:tasks.rados.rados.0.smithi122.stdout:2940: writing smithi12223906-31 from 2413414 to 2871307 tid 3 2024-09-15T03:59:13.130 INFO:tasks.rados.rados.0.smithi122.stdout:2940: writing smithi12223906-31 from 3438716 to 3613736 tid 4 2024-09-15T03:59:13.130 INFO:tasks.rados.rados.0.smithi122.stdout:2941: snap_remove snap 313 2024-09-15T03:59:14.103 INFO:tasks.rados.rados.0.smithi122.stdout:2940: finishing write tid 1 to smithi12223906-31 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2939: finishing copy_from racing read to smithi12223906-30 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2939: finishing copy_from to smithi12223906-30 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 2212 (ObjNum 613 snap 183 seq_num 613) dirty exists 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: finishing write tid 2 to smithi12223906-31 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: finishing write tid 3 to smithi12223906-31 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: finishing write tid 4 to smithi12223906-31 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: finishing write tid 5 to smithi12223906-31 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: finishing write tid 6 to smithi12223906-31 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1897 (ObjNum 1048 snap 335 seq_num 1048) dirty exists 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: left oid 31 (ObjNum 1048 snap 335 seq_num 1048) 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2939: done (2 left) 2024-09-15T03:59:14.104 INFO:tasks.rados.rados.0.smithi122.stdout:2940: done (1 left) 2024-09-15T03:59:14.105 INFO:tasks.rados.rados.0.smithi122.stdout:2941: done (0 left) 2024-09-15T03:59:14.105 INFO:tasks.rados.rados.0.smithi122.stdout:2942: delete oid 16 current snap is 335 2024-09-15T03:59:14.112 INFO:tasks.rados.rados.0.smithi122.stdout:2942: done (0 left) 2024-09-15T03:59:14.112 INFO:tasks.rados.rados.0.smithi122.stdout:2943: read oid 24 snap -1 2024-09-15T03:59:14.113 INFO:tasks.rados.rados.0.smithi122.stdout:2943: expect deleted 2024-09-15T03:59:14.113 INFO:tasks.rados.rados.0.smithi122.stdout:2944: write oid 41 current snap is 335 2024-09-15T03:59:14.113 INFO:tasks.rados.rados.0.smithi122.stdout:2944: seq_num 1050 ranges {447350=668164,1560093=406693,2584613=480437} 2024-09-15T03:59:14.119 INFO:tasks.rados.rados.0.smithi122.stdout:2944: writing smithi12223906-41 from 447350 to 1115514 tid 1 2024-09-15T03:59:14.123 INFO:tasks.rados.rados.0.smithi122.stdout:2944: writing smithi12223906-41 from 1560093 to 1966786 tid 2 2024-09-15T03:59:14.130 INFO:tasks.rados.rados.0.smithi122.stdout:2944: writing smithi12223906-41 from 2584613 to 3065050 tid 3 2024-09-15T03:59:14.130 INFO:tasks.rados.rados.0.smithi122.stdout:2945: write (excl) oid 4 current snap is 335 2024-09-15T03:59:14.130 INFO:tasks.rados.rados.0.smithi122.stdout:2945: seq_num 1051 ranges {464157=743413,1618575=666704,3012036=408062} 2024-09-15T03:59:14.137 INFO:tasks.rados.rados.0.smithi122.stdout:2945: writing smithi12223906-4 from 464157 to 1207570 tid 1 2024-09-15T03:59:14.143 INFO:tasks.rados.rados.0.smithi122.stdout:2945: writing smithi12223906-4 from 1618575 to 2285279 tid 2 2024-09-15T03:59:14.149 INFO:tasks.rados.rados.0.smithi122.stdout:2945: writing smithi12223906-4 from 3012036 to 3420098 tid 3 2024-09-15T03:59:14.149 INFO:tasks.rados.rados.0.smithi122.stdout:2946: snap_create 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2944: finishing write tid 1 to smithi12223906-41 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2944: finishing write tid 2 to smithi12223906-41 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2944: finishing write tid 3 to smithi12223906-41 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2944: finishing write tid 4 to smithi12223906-41 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2944: finishing write tid 5 to smithi12223906-41 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2216 (ObjNum 1050 snap 335 seq_num 1050) dirty exists 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2944: left oid 41 (ObjNum 1050 snap 335 seq_num 1050) 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2945: finishing write tid 1 to smithi12223906-4 2024-09-15T03:59:14.150 INFO:tasks.rados.rados.0.smithi122.stdout:2945: finishing write tid 2 to smithi12223906-4 2024-09-15T03:59:14.155 INFO:tasks.rados.rados.0.smithi122.stdout:2945: finishing write tid 3 to smithi12223906-4 2024-09-15T03:59:14.155 INFO:tasks.rados.rados.0.smithi122.stdout:2945: finishing write tid 4 to smithi12223906-4 2024-09-15T03:59:14.155 INFO:tasks.rados.rados.0.smithi122.stdout:2945: finishing write tid 5 to smithi12223906-4 2024-09-15T03:59:14.156 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1901 (ObjNum 1051 snap 335 seq_num 1051) dirty exists 2024-09-15T03:59:14.156 INFO:tasks.rados.rados.0.smithi122.stdout:2945: left oid 4 (ObjNum 1051 snap 335 seq_num 1051) 2024-09-15T03:59:15.119 INFO:tasks.rados.rados.0.smithi122.stdout:2943: done (3 left) 2024-09-15T03:59:15.119 INFO:tasks.rados.rados.0.smithi122.stdout:2944: done (2 left) 2024-09-15T03:59:15.119 INFO:tasks.rados.rados.0.smithi122.stdout:2945: done (1 left) 2024-09-15T03:59:15.119 INFO:tasks.rados.rados.0.smithi122.stdout:2946: done (0 left) 2024-09-15T03:59:15.119 INFO:tasks.rados.rados.0.smithi122.stdout:2947: write (excl) oid 33 current snap is 336 2024-09-15T03:59:15.119 INFO:tasks.rados.rados.0.smithi122.stdout:2947: seq_num 1052 ranges {480964=418662,1560704=410362,2523106=619334} 2024-09-15T03:59:15.125 INFO:tasks.rados.rados.0.smithi122.stdout:2947: writing smithi12223906-33 from 480964 to 899626 tid 1 2024-09-15T03:59:15.133 INFO:tasks.rados.rados.0.smithi122.stdout:2947: writing smithi12223906-33 from 1560704 to 1971066 tid 2 2024-09-15T03:59:15.140 INFO:tasks.rados.rados.0.smithi122.stdout:2947: writing smithi12223906-33 from 2523106 to 3142440 tid 3 2024-09-15T03:59:15.140 INFO:tasks.rados.rados.0.smithi122.stdout:2948: read oid 47 snap -1 2024-09-15T03:59:15.140 INFO:tasks.rados.rados.0.smithi122.stdout:2948: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:59:15.140 INFO:tasks.rados.rados.0.smithi122.stdout:2949: snap_create 2024-09-15T03:59:15.140 INFO:tasks.rados.rados.0.smithi122.stdout:2947: finishing write tid 1 to smithi12223906-33 2024-09-15T03:59:15.141 INFO:tasks.rados.rados.0.smithi122.stdout:2947: finishing write tid 2 to smithi12223906-33 2024-09-15T03:59:15.147 INFO:tasks.rados.rados.0.smithi122.stdout:2947: finishing write tid 3 to smithi12223906-33 2024-09-15T03:59:15.147 INFO:tasks.rados.rados.0.smithi122.stdout:2947: finishing write tid 4 to smithi12223906-33 2024-09-15T03:59:15.148 INFO:tasks.rados.rados.0.smithi122.stdout:2947: finishing write tid 5 to smithi12223906-33 2024-09-15T03:59:15.148 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2291 (ObjNum 1052 snap 336 seq_num 1052) dirty exists 2024-09-15T03:59:15.148 INFO:tasks.rados.rados.0.smithi122.stdout:2947: left oid 33 (ObjNum 1052 snap 336 seq_num 1052) 2024-09-15T03:59:15.149 INFO:tasks.rados.rados.0.smithi122.stdout:2948: expect (ObjNum 613 snap 183 seq_num 613) 2024-09-15T03:59:16.084 INFO:tasks.rados.rados.0.smithi122.stdout:2947: done (2 left) 2024-09-15T03:59:16.084 INFO:tasks.rados.rados.0.smithi122.stdout:2948: done (1 left) 2024-09-15T03:59:16.084 INFO:tasks.rados.rados.0.smithi122.stdout:2949: done (0 left) 2024-09-15T03:59:16.084 INFO:tasks.rados.rados.0.smithi122.stdout:2950: snap_create 2024-09-15T03:59:17.084 INFO:tasks.rados.rados.0.smithi122.stdout:2950: done (0 left) 2024-09-15T03:59:17.084 INFO:tasks.rados.rados.0.smithi122.stdout:2951: snap_create 2024-09-15T03:59:18.092 INFO:tasks.rados.rados.0.smithi122.stdout:2951: done (0 left) 2024-09-15T03:59:18.093 INFO:tasks.rados.rados.0.smithi122.stdout:2952: write (excl) oid 46 current snap is 339 2024-09-15T03:59:18.093 INFO:tasks.rados.rados.0.smithi122.stdout:2952: seq_num 1053 ranges {497771=493911,1619186=554020,2950529=430606} 2024-09-15T03:59:18.100 INFO:tasks.rados.rados.0.smithi122.stdout:2952: writing smithi12223906-46 from 497771 to 991682 tid 1 2024-09-15T03:59:18.107 INFO:tasks.rados.rados.0.smithi122.stdout:2952: writing smithi12223906-46 from 1619186 to 2173206 tid 2 2024-09-15T03:59:18.114 INFO:tasks.rados.rados.0.smithi122.stdout:2952: writing smithi12223906-46 from 2950529 to 3381135 tid 3 2024-09-15T03:59:18.114 INFO:tasks.rados.rados.0.smithi122.stdout:2953: write oid 42 current snap is 339 2024-09-15T03:59:18.114 INFO:tasks.rados.rados.0.smithi122.stdout:2953: seq_num 1054 ranges {514578=569160,1677668=414031,2577952=641878} 2024-09-15T03:59:18.119 INFO:tasks.rados.rados.0.smithi122.stdout:2953: writing smithi12223906-42 from 514578 to 1083738 tid 1 2024-09-15T03:59:18.125 INFO:tasks.rados.rados.0.smithi122.stdout:2953: writing smithi12223906-42 from 1677668 to 2091699 tid 2 2024-09-15T03:59:18.132 INFO:tasks.rados.rados.0.smithi122.stdout:2953: writing smithi12223906-42 from 2577952 to 3219830 tid 3 2024-09-15T03:59:18.132 INFO:tasks.rados.rados.0.smithi122.stdout:2954: read oid 40 snap 331 2024-09-15T03:59:18.132 INFO:tasks.rados.rados.0.smithi122.stdout:2954: expect (ObjNum 1042 snap 331 seq_num 1042) 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2955: read oid 23 snap 327 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2955: expect deleted 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2956: read oid 24 snap 336 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2956: expect deleted 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2957: snap_create 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2952: finishing write tid 1 to smithi12223906-46 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2952: finishing write tid 2 to smithi12223906-46 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2952: finishing write tid 3 to smithi12223906-46 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2952: finishing write tid 4 to smithi12223906-46 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2952: finishing write tid 5 to smithi12223906-46 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 1592 (ObjNum 1053 snap 339 seq_num 1053) dirty exists 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2952: left oid 46 (ObjNum 1053 snap 339 seq_num 1053) 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2953: finishing write tid 1 to smithi12223906-42 2024-09-15T03:59:18.133 INFO:tasks.rados.rados.0.smithi122.stdout:2953: finishing write tid 2 to smithi12223906-42 2024-09-15T03:59:18.139 INFO:tasks.rados.rados.0.smithi122.stdout:2953: finishing write tid 3 to smithi12223906-42 2024-09-15T03:59:18.139 INFO:tasks.rados.rados.0.smithi122.stdout:2953: finishing write tid 4 to smithi12223906-42 2024-09-15T03:59:18.139 INFO:tasks.rados.rados.0.smithi122.stdout:2953: finishing write tid 5 to smithi12223906-42 2024-09-15T03:59:18.139 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 2369 (ObjNum 1054 snap 339 seq_num 1054) dirty exists 2024-09-15T03:59:18.139 INFO:tasks.rados.rados.0.smithi122.stdout:2953: left oid 42 (ObjNum 1054 snap 339 seq_num 1054) 2024-09-15T03:59:18.147 INFO:tasks.rados.rados.0.smithi122.stdout:2954: expect (ObjNum 1042 snap 331 seq_num 1042) 2024-09-15T03:59:19.097 INFO:tasks.rados.rados.0.smithi122.stdout:2952: done (5 left) 2024-09-15T03:59:19.098 INFO:tasks.rados.rados.0.smithi122.stdout:2953: done (4 left) 2024-09-15T03:59:19.098 INFO:tasks.rados.rados.0.smithi122.stdout:2954: done (3 left) 2024-09-15T03:59:19.098 INFO:tasks.rados.rados.0.smithi122.stdout:2955: done (2 left) 2024-09-15T03:59:19.098 INFO:tasks.rados.rados.0.smithi122.stdout:2956: done (1 left) 2024-09-15T03:59:19.098 INFO:tasks.rados.rados.0.smithi122.stdout:2957: done (0 left) 2024-09-15T03:59:19.098 INFO:tasks.rados.rados.0.smithi122.stdout:2958: delete oid 27 current snap is 340 2024-09-15T03:59:19.104 INFO:tasks.rados.rados.0.smithi122.stdout:2958: done (0 left) 2024-09-15T03:59:19.104 INFO:tasks.rados.rados.0.smithi122.stdout:2959: read oid 7 snap -1 2024-09-15T03:59:19.104 INFO:tasks.rados.rados.0.smithi122.stdout:2959: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:59:19.104 INFO:tasks.rados.rados.0.smithi122.stdout:2960: snap_remove snap 304 2024-09-15T03:59:20.131 INFO:tasks.rados.rados.0.smithi122.stdout:2960: done (1 left) 2024-09-15T03:59:20.131 INFO:tasks.rados.rados.0.smithi122.stdout:2961: delete oid 25 current snap is 340 2024-09-15T03:59:20.131 INFO:tasks.rados.rados.0.smithi122.stdout:2959: expect (ObjNum 1022 snap 320 seq_num 1022) 2024-09-15T03:59:20.237 INFO:tasks.rados.rados.0.smithi122.stdout:2959: done (1 left) 2024-09-15T03:59:20.238 INFO:tasks.rados.rados.0.smithi122.stdout:2961: done (0 left) 2024-09-15T03:59:20.238 INFO:tasks.rados.rados.0.smithi122.stdout:2962: snap_remove snap 323 2024-09-15T03:59:21.086 INFO:tasks.rados.rados.0.smithi122.stdout:2962: done (0 left) 2024-09-15T03:59:21.086 INFO:tasks.rados.rados.0.smithi122.stdout:2963: copy_from oid 22 from oid 42 current snap is 340 2024-09-15T03:59:21.086 INFO:tasks.rados.rados.0.smithi122.stdout:2964: snap_create 2024-09-15T03:59:21.119 INFO:tasks.rados.rados.0.smithi122.stdout:2963: finishing copy_from racing read to smithi12223906-22 2024-09-15T03:59:21.126 INFO:tasks.rados.rados.0.smithi122.stdout:2963: finishing copy_from to smithi12223906-22 2024-09-15T03:59:21.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 2369 (ObjNum 1054 snap 339 seq_num 1054) dirty exists 2024-09-15T03:59:22.090 INFO:tasks.rados.rados.0.smithi122.stdout:2963: done (1 left) 2024-09-15T03:59:22.090 INFO:tasks.rados.rados.0.smithi122.stdout:2964: done (0 left) 2024-09-15T03:59:22.090 INFO:tasks.rados.rados.0.smithi122.stdout:2965: snap_create 2024-09-15T03:59:23.090 INFO:tasks.rados.rados.0.smithi122.stdout:2965: done (0 left) 2024-09-15T03:59:23.090 INFO:tasks.rados.rados.0.smithi122.stdout:2966: delete oid 42 current snap is 342 2024-09-15T03:59:23.103 INFO:tasks.rados.rados.0.smithi122.stdout:2966: done (0 left) 2024-09-15T03:59:23.103 INFO:tasks.rados.rados.0.smithi122.stdout:2967: snap_create 2024-09-15T03:59:24.107 INFO:tasks.rados.rados.0.smithi122.stdout:2967: done (0 left) 2024-09-15T03:59:24.107 INFO:tasks.rados.rados.0.smithi122.stdout:2968: copy_from oid 24 from oid 39 current snap is 343 2024-09-15T03:59:24.107 INFO:tasks.rados.rados.0.smithi122.stdout:2969: read oid 34 snap -1 2024-09-15T03:59:24.107 INFO:tasks.rados.rados.0.smithi122.stdout:2969: expect (ObjNum 982 snap 311 seq_num 982) 2024-09-15T03:59:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2970: read oid 13 snap -1 2024-09-15T03:59:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2970: expect deleted 2024-09-15T03:59:24.108 INFO:tasks.rados.rados.0.smithi122.stdout:2971: snap_remove snap 331 2024-09-15T03:59:25.117 INFO:tasks.rados.rados.0.smithi122.stdout:2971: done (3 left) 2024-09-15T03:59:25.117 INFO:tasks.rados.rados.0.smithi122.stdout:2972: read oid 28 snap 342 2024-09-15T03:59:25.117 INFO:tasks.rados.rados.0.smithi122.stdout:2972: expect (ObjNum 1039 snap 330 seq_num 1039) 2024-09-15T03:59:25.117 INFO:tasks.rados.rados.0.smithi122.stdout:2973: write (excl) oid 8 current snap is 343 2024-09-15T03:59:25.117 INFO:tasks.rados.rados.0.smithi122.stdout:2973: seq_num 1058 ranges {581806=586509,1911596=421369,3087644=403319} 2024-09-15T03:59:25.124 INFO:tasks.rados.rados.0.smithi122.stdout:2973: writing smithi12223906-8 from 581806 to 1168315 tid 1 2024-09-15T03:59:25.132 INFO:tasks.rados.rados.0.smithi122.stdout:2973: writing smithi12223906-8 from 1911596 to 2332965 tid 2 2024-09-15T03:59:25.139 INFO:tasks.rados.rados.0.smithi122.stdout:2973: writing smithi12223906-8 from 3087644 to 3490963 tid 3 2024-09-15T03:59:25.139 INFO:tasks.rados.rados.0.smithi122.stdout:2974: copy_from oid 33 from oid 5 current snap is 343 2024-09-15T03:59:25.139 INFO:tasks.rados.rados.0.smithi122.stdout:2969: expect (ObjNum 982 snap 311 seq_num 982) 2024-09-15T03:59:25.296 INFO:tasks.rados.rados.0.smithi122.stdout:2968: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:59:25.297 INFO:tasks.rados.rados.0.smithi122.stdout:2968: finishing copy_from to smithi12223906-24 2024-09-15T03:59:25.297 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 2323 (ObjNum 1041 snap 330 seq_num 1041) dirty exists 2024-09-15T03:59:25.297 INFO:tasks.rados.rados.0.smithi122.stdout:2972: expect (ObjNum 1039 snap 330 seq_num 1039) 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2973: finishing write tid 1 to smithi12223906-8 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2973: finishing write tid 2 to smithi12223906-8 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2973: finishing write tid 3 to smithi12223906-8 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2973: finishing write tid 4 to smithi12223906-8 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2973: finishing write tid 5 to smithi12223906-8 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1882 (ObjNum 1058 snap 343 seq_num 1058) dirty exists 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2973: left oid 8 (ObjNum 1058 snap 343 seq_num 1058) 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2974: finishing copy_from racing read to smithi12223906-33 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:2974: finishing copy_from to smithi12223906-33 2024-09-15T03:59:25.451 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 903 (ObjNum 1038 snap 327 seq_num 1038) dirty exists 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2968: done (5 left) 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2969: done (4 left) 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2970: done (3 left) 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2972: done (2 left) 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2973: done (1 left) 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2974: done (0 left) 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2975: rollback oid 33 current snap is 343 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 33 to 306 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2976: write (excl) oid 22 current snap is 343 2024-09-15T03:59:25.452 INFO:tasks.rados.rados.0.smithi122.stdout:2976: seq_num 1059 ranges {598613=661758,1970078=565027,3115067=614591} 2024-09-15T03:59:25.468 INFO:tasks.rados.rados.0.smithi122.stdout:2976: writing smithi12223906-22 from 598613 to 1260371 tid 1 2024-09-15T03:59:25.472 INFO:tasks.rados.rados.0.smithi122.stdout:2976: writing smithi12223906-22 from 1970078 to 2535105 tid 2 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2976: writing smithi12223906-22 from 3115067 to 3729658 tid 3 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2977: read oid 21 snap -1 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2977: expect (ObjNum 1040 snap 330 seq_num 1040) 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2978: read oid 9 snap -1 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2978: expect (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2979: delete oid 6 current snap is 343 2024-09-15T03:59:25.480 INFO:tasks.rados.rados.0.smithi122.stdout:2975: finishing rollback tid 0 to smithi12223906-33 2024-09-15T03:59:25.481 INFO:tasks.rados.rados.0.smithi122.stdout:2975: finishing rollback tid 1 to smithi12223906-33 2024-09-15T03:59:25.481 INFO:tasks.rados.rados.0.smithi122.stdout:2975: finishing rollback tid 2 to smithi12223906-33 2024-09-15T03:59:25.481 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2372 (ObjNum 940 snap 295 seq_num 940) dirty exists 2024-09-15T03:59:25.481 INFO:tasks.rados.rados.0.smithi122.stdout:2976: finishing write tid 1 to smithi12223906-22 2024-09-15T03:59:25.481 INFO:tasks.rados.rados.0.smithi122.stdout:2976: finishing write tid 2 to smithi12223906-22 2024-09-15T03:59:25.485 INFO:tasks.rados.rados.0.smithi122.stdout:2976: finishing write tid 3 to smithi12223906-22 2024-09-15T03:59:25.487 INFO:tasks.rados.rados.0.smithi122.stdout:2976: finishing write tid 4 to smithi12223906-22 2024-09-15T03:59:25.487 INFO:tasks.rados.rados.0.smithi122.stdout:2976: finishing write tid 5 to smithi12223906-22 2024-09-15T03:59:25.487 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 2376 (ObjNum 1059 snap 343 seq_num 1059) dirty exists 2024-09-15T03:59:25.487 INFO:tasks.rados.rados.0.smithi122.stdout:2976: left oid 22 (ObjNum 1059 snap 343 seq_num 1059) 2024-09-15T03:59:25.490 INFO:tasks.rados.rados.0.smithi122.stdout:2978: expect (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T03:59:25.592 INFO:tasks.rados.rados.0.smithi122.stdout:2977: expect (ObjNum 1040 snap 330 seq_num 1040) 2024-09-15T03:59:25.812 INFO:tasks.rados.rados.0.smithi122.stdout:2975: done (4 left) 2024-09-15T03:59:25.812 INFO:tasks.rados.rados.0.smithi122.stdout:2976: done (3 left) 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2977: done (2 left) 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2978: done (1 left) 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2979: done (0 left) 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2980: read oid 50 snap -1 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2980: expect (ObjNum 855 snap 277 seq_num 855) 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2981: read oid 25 snap -1 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2981: expect deleted 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2982: copy_from oid 46 from oid 12 current snap is 343 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2983: copy_from oid 43 from oid 11 current snap is 343 2024-09-15T03:59:25.813 INFO:tasks.rados.rados.0.smithi122.stdout:2984: delete oid 22 current snap is 343 2024-09-15T03:59:25.814 INFO:tasks.rados.rados.0.smithi122.stdout:2982: finishing copy_from to smithi12223906-46 2024-09-15T03:59:25.814 INFO:tasks.rados.rados.0.smithi122.stdout:2982: got expected ENOENT (src dne) 2024-09-15T03:59:25.815 INFO:tasks.rados.rados.0.smithi122.stdout:2982: finishing copy_from racing read to smithi12223906-46 2024-09-15T03:59:25.825 INFO:tasks.rados.rados.0.smithi122.stdout:2980: expect (ObjNum 855 snap 277 seq_num 855) 2024-09-15T03:59:25.908 INFO:tasks.rados.rados.0.smithi122.stdout:2983: finishing copy_from racing read to smithi12223906-43 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2983: finishing copy_from to smithi12223906-43 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 1322 (ObjNum 944 snap 295 seq_num 944) dirty exists 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2980: done (4 left) 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2981: done (3 left) 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2982: done (2 left) 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2983: done (1 left) 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2984: done (0 left) 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2985: write oid 34 current snap is 343 2024-09-15T03:59:25.909 INFO:tasks.rados.rados.0.smithi122.stdout:2985: seq_num 1062 ranges {649034=487505,1629171=428707,2680983=564760,3798768=50266} 2024-09-15T03:59:25.920 INFO:tasks.rados.rados.0.smithi122.stdout:2985: writing smithi12223906-34 from 649034 to 1136539 tid 1 2024-09-15T03:59:25.927 INFO:tasks.rados.rados.0.smithi122.stdout:2985: writing smithi12223906-34 from 1629171 to 2057878 tid 2 2024-09-15T03:59:25.934 INFO:tasks.rados.rados.0.smithi122.stdout:2985: writing smithi12223906-34 from 2680983 to 3245743 tid 3 2024-09-15T03:59:25.938 INFO:tasks.rados.rados.0.smithi122.stdout:2985: writing smithi12223906-34 from 3798768 to 3849034 tid 4 2024-09-15T03:59:25.938 INFO:tasks.rados.rados.0.smithi122.stdout:2986: delete oid 47 current snap is 343 2024-09-15T03:59:25.938 INFO:tasks.rados.rados.0.smithi122.stdout:2985: finishing write tid 1 to smithi12223906-34 2024-09-15T03:59:25.938 INFO:tasks.rados.rados.0.smithi122.stdout:2985: finishing write tid 2 to smithi12223906-34 2024-09-15T03:59:25.940 INFO:tasks.rados.rados.0.smithi122.stdout:2985: finishing write tid 3 to smithi12223906-34 2024-09-15T03:59:25.941 INFO:tasks.rados.rados.0.smithi122.stdout:2985: finishing write tid 4 to smithi12223906-34 2024-09-15T03:59:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:2985: finishing write tid 5 to smithi12223906-34 2024-09-15T03:59:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:2985: finishing write tid 6 to smithi12223906-34 2024-09-15T03:59:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1581 (ObjNum 1062 snap 343 seq_num 1062) dirty exists 2024-09-15T03:59:25.942 INFO:tasks.rados.rados.0.smithi122.stdout:2985: left oid 34 (ObjNum 1062 snap 343 seq_num 1062) 2024-09-15T03:59:25.944 INFO:tasks.rados.rados.0.smithi122.stdout:2985: done (1 left) 2024-09-15T03:59:25.944 INFO:tasks.rados.rados.0.smithi122.stdout:2986: done (0 left) 2024-09-15T03:59:25.944 INFO:tasks.rados.rados.0.smithi122.stdout:2987: snap_create 2024-09-15T03:59:26.090 INFO:tasks.rados.rados.0.smithi122.stdout:2987: done (0 left) 2024-09-15T03:59:26.091 INFO:tasks.rados.rados.0.smithi122.stdout:2988: rollback oid 50 current snap is 344 2024-09-15T03:59:26.091 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 303 2024-09-15T03:59:26.091 INFO:tasks.rados.rados.0.smithi122.stdout:2989: delete oid 10 current snap is 344 2024-09-15T03:59:26.105 INFO:tasks.rados.rados.0.smithi122.stdout:2988: finishing rollback tid 0 to smithi12223906-50 2024-09-15T03:59:26.105 INFO:tasks.rados.rados.0.smithi122.stdout:2989: done (1 left) 2024-09-15T03:59:26.105 INFO:tasks.rados.rados.0.smithi122.stdout:2990: rollback oid 8 current snap is 344 2024-09-15T03:59:26.105 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 315 2024-09-15T03:59:26.105 INFO:tasks.rados.rados.0.smithi122.stdout:2991: snap_create 2024-09-15T03:59:26.115 INFO:tasks.rados.rados.0.smithi122.stdout:2988: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:59:26.115 INFO:tasks.rados.rados.0.smithi122.stdout:2988: finishing rollback tid 2 to smithi12223906-50 2024-09-15T03:59:26.115 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2380 (ObjNum 950 snap 300 seq_num 950) dirty exists 2024-09-15T03:59:26.117 INFO:tasks.rados.rados.0.smithi122.stdout:2990: finishing rollback tid 0 to smithi12223906-8 2024-09-15T03:59:26.118 INFO:tasks.rados.rados.0.smithi122.stdout:2990: finishing rollback tid 1 to smithi12223906-8 2024-09-15T03:59:26.118 INFO:tasks.rados.rados.0.smithi122.stdout:2990: finishing rollback tid 2 to smithi12223906-8 2024-09-15T03:59:26.118 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1885 (ObjNum 958 snap 304 seq_num 958) dirty exists 2024-09-15T03:59:27.109 INFO:tasks.rados.rados.0.smithi122.stdout:2988: done (2 left) 2024-09-15T03:59:27.110 INFO:tasks.rados.rados.0.smithi122.stdout:2990: done (1 left) 2024-09-15T03:59:27.110 INFO:tasks.rados.rados.0.smithi122.stdout:2991: done (0 left) 2024-09-15T03:59:27.110 INFO:tasks.rados.rados.0.smithi122.stdout:2992: copy_from oid 29 from oid 27 current snap is 345 2024-09-15T03:59:27.110 INFO:tasks.rados.rados.0.smithi122.stdout:2993: delete oid 9 current snap is 345 2024-09-15T03:59:27.111 INFO:tasks.rados.rados.0.smithi122.stdout:2992: finishing copy_from to smithi12223906-29 2024-09-15T03:59:27.111 INFO:tasks.rados.rados.0.smithi122.stdout:2992: got expected ENOENT (src dne) 2024-09-15T03:59:27.112 INFO:tasks.rados.rados.0.smithi122.stdout:2992: finishing copy_from racing read to smithi12223906-29 2024-09-15T03:59:27.114 INFO:tasks.rados.rados.0.smithi122.stdout:2992: done (1 left) 2024-09-15T03:59:27.114 INFO:tasks.rados.rados.0.smithi122.stdout:2993: done (0 left) 2024-09-15T03:59:27.114 INFO:tasks.rados.rados.0.smithi122.stdout:2994: rollback oid 18 current snap is 345 2024-09-15T03:59:27.115 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 326 2024-09-15T03:59:27.115 INFO:tasks.rados.rados.0.smithi122.stdout:2995: write (excl) oid 37 current snap is 345 2024-09-15T03:59:27.115 INFO:tasks.rados.rados.0.smithi122.stdout:2995: seq_num 1066 ranges {716262=504854,1863099=719692,3074322=726201} 2024-09-15T03:59:27.124 INFO:tasks.rados.rados.0.smithi122.stdout:2995: writing smithi12223906-37 from 716262 to 1221116 tid 1 2024-09-15T03:59:27.133 INFO:tasks.rados.rados.0.smithi122.stdout:2995: writing smithi12223906-37 from 1863099 to 2582791 tid 2 2024-09-15T03:59:27.141 INFO:tasks.rados.rados.0.smithi122.stdout:2995: writing smithi12223906-37 from 3074322 to 3800523 tid 3 2024-09-15T03:59:27.141 INFO:tasks.rados.rados.0.smithi122.stdout:2996: snap_create 2024-09-15T03:59:27.141 INFO:tasks.rados.rados.0.smithi122.stdout:2994: finishing rollback tid 0 to smithi12223906-18 2024-09-15T03:59:27.141 INFO:tasks.rados.rados.0.smithi122.stdout:2994: finishing rollback tid 1 to smithi12223906-18 2024-09-15T03:59:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:2994: finishing rollback tid 2 to smithi12223906-18 2024-09-15T03:59:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2383 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T03:59:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:2995: finishing write tid 1 to smithi12223906-37 2024-09-15T03:59:27.142 INFO:tasks.rados.rados.0.smithi122.stdout:2995: finishing write tid 2 to smithi12223906-37 2024-09-15T03:59:27.146 INFO:tasks.rados.rados.0.smithi122.stdout:2995: finishing write tid 3 to smithi12223906-37 2024-09-15T03:59:27.148 INFO:tasks.rados.rados.0.smithi122.stdout:2995: finishing write tid 4 to smithi12223906-37 2024-09-15T03:59:27.148 INFO:tasks.rados.rados.0.smithi122.stdout:2995: finishing write tid 5 to smithi12223906-37 2024-09-15T03:59:27.148 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 1178 (ObjNum 1066 snap 345 seq_num 1066) dirty exists 2024-09-15T03:59:27.148 INFO:tasks.rados.rados.0.smithi122.stdout:2995: left oid 37 (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T03:59:28.113 INFO:tasks.rados.rados.0.smithi122.stdout:2994: done (2 left) 2024-09-15T03:59:28.113 INFO:tasks.rados.rados.0.smithi122.stdout:2995: done (1 left) 2024-09-15T03:59:28.113 INFO:tasks.rados.rados.0.smithi122.stdout:2996: done (0 left) 2024-09-15T03:59:28.113 INFO:tasks.rados.rados.0.smithi122.stdout:2997: snap_remove snap 287 2024-09-15T03:59:29.123 INFO:tasks.rados.rados.0.smithi122.stdout:2997: done (0 left) 2024-09-15T03:59:29.124 INFO:tasks.rados.rados.0.smithi122.stdout:2998: write (excl) oid 23 current snap is 346 2024-09-15T03:59:29.124 INFO:tasks.rados.rados.0.smithi122.stdout:2998: seq_num 1067 ranges {733069=580103,1921581=579703,3218098=537473} 2024-09-15T03:59:29.136 INFO:tasks.rados.rados.0.smithi122.stdout:2998: writing smithi12223906-23 from 733069 to 1313172 tid 1 2024-09-15T03:59:29.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: writing smithi12223906-23 from 1921581 to 2501284 tid 2 2024-09-15T03:59:29.150 INFO:tasks.rados.rados.0.smithi122.stdout:2998: writing smithi12223906-23 from 3218098 to 3755571 tid 3 2024-09-15T03:59:29.151 INFO:tasks.rados.rados.0.smithi122.stdout:2999: snap_remove snap 338 2024-09-15T03:59:30.142 INFO:tasks.rados.rados.0.smithi122.stdout:2999: done (1 left) 2024-09-15T03:59:30.142 INFO:tasks.rados.rados.0.smithi122.stdout:3000: read oid 38 snap -1 2024-09-15T03:59:30.142 INFO:tasks.rados.rados.0.smithi122.stdout:3000: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: finishing write tid 1 to smithi12223906-23 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: finishing write tid 2 to smithi12223906-23 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: finishing write tid 3 to smithi12223906-23 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: finishing write tid 4 to smithi12223906-23 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: finishing write tid 5 to smithi12223906-23 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2387 (ObjNum 1067 snap 346 seq_num 1067) dirty exists 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: left oid 23 (ObjNum 1067 snap 346 seq_num 1067) 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:2998: done (1 left) 2024-09-15T03:59:30.143 INFO:tasks.rados.rados.0.smithi122.stdout:3001: delete oid 23 current snap is 346 2024-09-15T03:59:30.153 INFO:tasks.rados.rados.0.smithi122.stdout:3001: done (1 left) 2024-09-15T03:59:30.153 INFO:tasks.rados.rados.0.smithi122.stdout:3002: read oid 49 snap -1 2024-09-15T03:59:30.153 INFO:tasks.rados.rados.0.smithi122.stdout:3002: expect (ObjNum 1020 snap 319 seq_num 1020) 2024-09-15T03:59:30.153 INFO:tasks.rados.rados.0.smithi122.stdout:3000: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T03:59:30.245 INFO:tasks.rados.rados.0.smithi122.stdout:3000: done (1 left) 2024-09-15T03:59:30.245 INFO:tasks.rados.rados.0.smithi122.stdout:3003: snap_create 2024-09-15T03:59:30.256 INFO:tasks.rados.rados.0.smithi122.stdout:3002: expect (ObjNum 1020 snap 319 seq_num 1020) 2024-09-15T03:59:31.094 INFO:tasks.rados.rados.0.smithi122.stdout:3002: done (1 left) 2024-09-15T03:59:31.094 INFO:tasks.rados.rados.0.smithi122.stdout:3003: done (0 left) 2024-09-15T03:59:31.094 INFO:tasks.rados.rados.0.smithi122.stdout:3004: write (excl) oid 46 current snap is 347 2024-09-15T03:59:31.095 INFO:tasks.rados.rados.0.smithi122.stdout:3004: seq_num 1069 ranges {766683=730601,1922192=583372,3156591=676370} 2024-09-15T03:59:31.104 INFO:tasks.rados.rados.0.smithi122.stdout:3004: writing smithi12223906-46 from 766683 to 1497284 tid 1 2024-09-15T03:59:31.111 INFO:tasks.rados.rados.0.smithi122.stdout:3004: writing smithi12223906-46 from 1922192 to 2505564 tid 2 2024-09-15T03:59:31.118 INFO:tasks.rados.rados.0.smithi122.stdout:3004: writing smithi12223906-46 from 3156591 to 3832961 tid 3 2024-09-15T03:59:31.119 INFO:tasks.rados.rados.0.smithi122.stdout:3005: snap_create 2024-09-15T03:59:31.119 INFO:tasks.rados.rados.0.smithi122.stdout:3004: finishing write tid 1 to smithi12223906-46 2024-09-15T03:59:31.123 INFO:tasks.rados.rados.0.smithi122.stdout:3004: finishing write tid 2 to smithi12223906-46 2024-09-15T03:59:31.123 INFO:tasks.rados.rados.0.smithi122.stdout:3004: finishing write tid 3 to smithi12223906-46 2024-09-15T03:59:31.124 INFO:tasks.rados.rados.0.smithi122.stdout:3004: finishing write tid 4 to smithi12223906-46 2024-09-15T03:59:31.124 INFO:tasks.rados.rados.0.smithi122.stdout:3004: finishing write tid 5 to smithi12223906-46 2024-09-15T03:59:31.124 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 1596 (ObjNum 1069 snap 347 seq_num 1069) dirty exists 2024-09-15T03:59:31.124 INFO:tasks.rados.rados.0.smithi122.stdout:3004: left oid 46 (ObjNum 1069 snap 347 seq_num 1069) 2024-09-15T03:59:32.108 INFO:tasks.rados.rados.0.smithi122.stdout:3004: done (1 left) 2024-09-15T03:59:32.108 INFO:tasks.rados.rados.0.smithi122.stdout:3005: done (0 left) 2024-09-15T03:59:32.108 INFO:tasks.rados.rados.0.smithi122.stdout:3006: write (excl) oid 17 current snap is 348 2024-09-15T03:59:32.109 INFO:tasks.rados.rados.0.smithi122.stdout:3006: seq_num 1070 ranges {783490=405850,1980674=727030,3184014=487642} 2024-09-15T03:59:32.115 INFO:tasks.rados.rados.0.smithi122.stdout:3006: writing smithi12223906-17 from 783490 to 1189340 tid 1 2024-09-15T03:59:32.124 INFO:tasks.rados.rados.0.smithi122.stdout:3006: writing smithi12223906-17 from 1980674 to 2707704 tid 2 2024-09-15T03:59:32.130 INFO:tasks.rados.rados.0.smithi122.stdout:3006: writing smithi12223906-17 from 3184014 to 3671656 tid 3 2024-09-15T03:59:32.130 INFO:tasks.rados.rados.0.smithi122.stdout:3007: write (excl) oid 8 current snap is 348 2024-09-15T03:59:32.130 INFO:tasks.rados.rados.0.smithi122.stdout:3006: finishing write tid 1 to smithi12223906-17 2024-09-15T03:59:32.130 INFO:tasks.rados.rados.0.smithi122.stdout:3007: seq_num 1071 ranges {400297=481099,1639156=361141} 2024-09-15T03:59:32.135 INFO:tasks.rados.rados.0.smithi122.stdout:3007: writing smithi12223906-8 from 400297 to 881396 tid 1 2024-09-15T03:59:32.142 INFO:tasks.rados.rados.0.smithi122.stdout:3007: writing smithi12223906-8 from 1639156 to 2000297 tid 2 2024-09-15T03:59:32.142 INFO:tasks.rados.rados.0.smithi122.stdout:3008: copy_from oid 24 from oid 40 current snap is 348 2024-09-15T03:59:32.142 INFO:tasks.rados.rados.0.smithi122.stdout:3006: finishing write tid 2 to smithi12223906-17 2024-09-15T03:59:32.142 INFO:tasks.rados.rados.0.smithi122.stdout:3006: finishing write tid 3 to smithi12223906-17 2024-09-15T03:59:32.142 INFO:tasks.rados.rados.0.smithi122.stdout:3006: finishing write tid 4 to smithi12223906-17 2024-09-15T03:59:32.143 INFO:tasks.rados.rados.0.smithi122.stdout:3006: finishing write tid 5 to smithi12223906-17 2024-09-15T03:59:32.143 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1069 (ObjNum 1070 snap 348 seq_num 1070) dirty exists 2024-09-15T03:59:32.143 INFO:tasks.rados.rados.0.smithi122.stdout:3006: left oid 17 (ObjNum 1070 snap 348 seq_num 1070) 2024-09-15T03:59:32.143 INFO:tasks.rados.rados.0.smithi122.stdout:3006: done (2 left) 2024-09-15T03:59:32.143 INFO:tasks.rados.rados.0.smithi122.stdout:3009: snap_create 2024-09-15T03:59:32.149 INFO:tasks.rados.rados.0.smithi122.stdout:3007: finishing write tid 1 to smithi12223906-8 2024-09-15T03:59:32.151 INFO:tasks.rados.rados.0.smithi122.stdout:3007: finishing write tid 2 to smithi12223906-8 2024-09-15T03:59:32.153 INFO:tasks.rados.rados.0.smithi122.stdout:3007: finishing write tid 3 to smithi12223906-8 2024-09-15T03:59:32.154 INFO:tasks.rados.rados.0.smithi122.stdout:3007: finishing write tid 4 to smithi12223906-8 2024-09-15T03:59:32.154 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1888 (ObjNum 1071 snap 348 seq_num 1071) dirty exists 2024-09-15T03:59:32.154 INFO:tasks.rados.rados.0.smithi122.stdout:3007: left oid 8 (ObjNum 1071 snap 348 seq_num 1071) 2024-09-15T03:59:32.164 INFO:tasks.rados.rados.0.smithi122.stdout:3008: finishing copy_from racing read to smithi12223906-24 2024-09-15T03:59:32.165 INFO:tasks.rados.rados.0.smithi122.stdout:3008: finishing copy_from to smithi12223906-24 2024-09-15T03:59:32.165 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 858 (ObjNum 1042 snap 331 seq_num 1042) dirty exists 2024-09-15T03:59:33.112 INFO:tasks.rados.rados.0.smithi122.stdout:3007: done (2 left) 2024-09-15T03:59:33.112 INFO:tasks.rados.rados.0.smithi122.stdout:3008: done (1 left) 2024-09-15T03:59:33.112 INFO:tasks.rados.rados.0.smithi122.stdout:3009: done (0 left) 2024-09-15T03:59:33.112 INFO:tasks.rados.rados.0.smithi122.stdout:3010: delete oid 31 current snap is 349 2024-09-15T03:59:33.120 INFO:tasks.rados.rados.0.smithi122.stdout:3010: done (0 left) 2024-09-15T03:59:33.120 INFO:tasks.rados.rados.0.smithi122.stdout:3011: delete oid 1 current snap is 349 2024-09-15T03:59:33.126 INFO:tasks.rados.rados.0.smithi122.stdout:3011: done (0 left) 2024-09-15T03:59:33.126 INFO:tasks.rados.rados.0.smithi122.stdout:3012: write oid 21 current snap is 349 2024-09-15T03:59:33.126 INFO:tasks.rados.rados.0.smithi122.stdout:3012: seq_num 1074 ranges {450718=423199,1414602=636116} 2024-09-15T03:59:33.131 INFO:tasks.rados.rados.0.smithi122.stdout:3012: writing smithi12223906-21 from 450718 to 873917 tid 1 2024-09-15T03:59:33.139 INFO:tasks.rados.rados.0.smithi122.stdout:3012: writing smithi12223906-21 from 1414602 to 2050718 tid 2 2024-09-15T03:59:33.139 INFO:tasks.rados.rados.0.smithi122.stdout:3013: snap_remove snap 301 2024-09-15T03:59:34.120 INFO:tasks.rados.rados.0.smithi122.stdout:3013: done (1 left) 2024-09-15T03:59:34.120 INFO:tasks.rados.rados.0.smithi122.stdout:3014: copy_from oid 25 from oid 15 current snap is 349 2024-09-15T03:59:34.120 INFO:tasks.rados.rados.0.smithi122.stdout:3012: finishing write tid 1 to smithi12223906-21 2024-09-15T03:59:34.120 INFO:tasks.rados.rados.0.smithi122.stdout:3012: finishing write tid 2 to smithi12223906-21 2024-09-15T03:59:34.120 INFO:tasks.rados.rados.0.smithi122.stdout:3012: finishing write tid 3 to smithi12223906-21 2024-09-15T03:59:34.120 INFO:tasks.rados.rados.0.smithi122.stdout:3012: finishing write tid 4 to smithi12223906-21 2024-09-15T03:59:34.121 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1188 (ObjNum 1074 snap 349 seq_num 1074) dirty exists 2024-09-15T03:59:34.121 INFO:tasks.rados.rados.0.smithi122.stdout:3012: left oid 21 (ObjNum 1074 snap 349 seq_num 1074) 2024-09-15T03:59:34.121 INFO:tasks.rados.rados.0.smithi122.stdout:3012: done (1 left) 2024-09-15T03:59:34.121 INFO:tasks.rados.rados.0.smithi122.stdout:3015: snap_create 2024-09-15T03:59:34.147 INFO:tasks.rados.rados.0.smithi122.stdout:3014: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:59:34.149 INFO:tasks.rados.rados.0.smithi122.stdout:3014: finishing copy_from to smithi12223906-25 2024-09-15T03:59:34.149 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 1886 (ObjNum 1029 snap 321 seq_num 1029) dirty exists 2024-09-15T03:59:35.125 INFO:tasks.rados.rados.0.smithi122.stdout:3014: done (1 left) 2024-09-15T03:59:35.125 INFO:tasks.rados.rados.0.smithi122.stdout:3015: done (0 left) 2024-09-15T03:59:35.125 INFO:tasks.rados.rados.0.smithi122.stdout:3016: copy_from oid 11 from oid 10 current snap is 350 2024-09-15T03:59:35.125 INFO:tasks.rados.rados.0.smithi122.stdout:3017: read oid 31 snap -1 2024-09-15T03:59:35.125 INFO:tasks.rados.rados.0.smithi122.stdout:3017: expect deleted 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3018: read oid 13 snap 345 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3018: expect deleted 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3019: copy_from oid 27 from oid 24 current snap is 350 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3020: rollback oid 19 current snap is 350 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 336 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3021: rollback oid 4 current snap is 350 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 339 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3022: read oid 48 snap -1 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3022: expect (ObjNum 1046 snap 335 seq_num 1046) 2024-09-15T03:59:35.126 INFO:tasks.rados.rados.0.smithi122.stdout:3023: snap_create 2024-09-15T03:59:35.128 INFO:tasks.rados.rados.0.smithi122.stdout:3016: finishing copy_from to smithi12223906-11 2024-09-15T03:59:35.129 INFO:tasks.rados.rados.0.smithi122.stdout:3016: got expected ENOENT (src dne) 2024-09-15T03:59:35.129 INFO:tasks.rados.rados.0.smithi122.stdout:3016: finishing copy_from racing read to smithi12223906-11 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:3021: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:3021: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:3020: finishing rollback tid 0 to smithi12223906-19 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:3020: finishing rollback tid 1 to smithi12223906-19 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:3020: finishing rollback tid 2 to smithi12223906-19 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 1599 (ObjNum 848 snap 276 seq_num 848) dirty exists 2024-09-15T03:59:35.142 INFO:tasks.rados.rados.0.smithi122.stdout:3022: expect (ObjNum 1046 snap 335 seq_num 1046) 2024-09-15T03:59:35.318 INFO:tasks.rados.rados.0.smithi122.stdout:3021: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:59:35.318 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1905 (ObjNum 1051 snap 335 seq_num 1051) dirty exists 2024-09-15T03:59:35.318 INFO:tasks.rados.rados.0.smithi122.stdout:3019: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:59:35.318 INFO:tasks.rados.rados.0.smithi122.stdout:3019: finishing copy_from to smithi12223906-27 2024-09-15T03:59:35.318 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 858 (ObjNum 1042 snap 331 seq_num 1042) dirty exists 2024-09-15T03:59:36.093 INFO:tasks.rados.rados.0.smithi122.stdout:3016: done (7 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3017: done (6 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3018: done (5 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3019: done (4 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3020: done (3 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3021: done (2 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3022: done (1 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3023: done (0 left) 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3024: write oid 45 current snap is 351 2024-09-15T03:59:36.094 INFO:tasks.rados.rados.0.smithi122.stdout:3024: seq_num 1075 ranges {467525=498448,1473084=594379} 2024-09-15T03:59:36.099 INFO:tasks.rados.rados.0.smithi122.stdout:3024: writing smithi12223906-45 from 467525 to 965973 tid 1 2024-09-15T03:59:36.106 INFO:tasks.rados.rados.0.smithi122.stdout:3024: writing smithi12223906-45 from 1473084 to 2067463 tid 2 2024-09-15T03:59:36.106 INFO:tasks.rados.rados.0.smithi122.stdout:3025: read oid 34 snap -1 2024-09-15T03:59:36.106 INFO:tasks.rados.rados.0.smithi122.stdout:3025: expect (ObjNum 1062 snap 343 seq_num 1062) 2024-09-15T03:59:36.106 INFO:tasks.rados.rados.0.smithi122.stdout:3026: snap_create 2024-09-15T03:59:36.110 INFO:tasks.rados.rados.0.smithi122.stdout:3024: finishing write tid 1 to smithi12223906-45 2024-09-15T03:59:36.120 INFO:tasks.rados.rados.0.smithi122.stdout:3024: finishing write tid 2 to smithi12223906-45 2024-09-15T03:59:36.120 INFO:tasks.rados.rados.0.smithi122.stdout:3025: expect (ObjNum 1062 snap 343 seq_num 1062) 2024-09-15T03:59:36.374 INFO:tasks.rados.rados.0.smithi122.stdout:3024: finishing write tid 3 to smithi12223906-45 2024-09-15T03:59:36.374 INFO:tasks.rados.rados.0.smithi122.stdout:3024: finishing write tid 4 to smithi12223906-45 2024-09-15T03:59:36.375 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 1602 (ObjNum 1075 snap 351 seq_num 1075) dirty exists 2024-09-15T03:59:36.375 INFO:tasks.rados.rados.0.smithi122.stdout:3024: left oid 45 (ObjNum 1075 snap 351 seq_num 1075) 2024-09-15T03:59:37.111 INFO:tasks.rados.rados.0.smithi122.stdout:3024: done (2 left) 2024-09-15T03:59:37.111 INFO:tasks.rados.rados.0.smithi122.stdout:3025: done (1 left) 2024-09-15T03:59:37.111 INFO:tasks.rados.rados.0.smithi122.stdout:3026: done (0 left) 2024-09-15T03:59:37.111 INFO:tasks.rados.rados.0.smithi122.stdout:3027: copy_from oid 27 from oid 33 current snap is 352 2024-09-15T03:59:37.111 INFO:tasks.rados.rados.0.smithi122.stdout:3028: delete oid 3 current snap is 352 2024-09-15T03:59:37.117 INFO:tasks.rados.rados.0.smithi122.stdout:3028: done (1 left) 2024-09-15T03:59:37.117 INFO:tasks.rados.rados.0.smithi122.stdout:3029: write oid 44 current snap is 352 2024-09-15T03:59:37.117 INFO:tasks.rados.rados.0.smithi122.stdout:3029: seq_num 1077 ranges {501139=648946,1873695=227444} 2024-09-15T03:59:37.127 INFO:tasks.rados.rados.0.smithi122.stdout:3029: writing smithi12223906-44 from 501139 to 1150085 tid 1 2024-09-15T03:59:37.134 INFO:tasks.rados.rados.0.smithi122.stdout:3029: writing smithi12223906-44 from 1873695 to 2101139 tid 2 2024-09-15T03:59:37.134 INFO:tasks.rados.rados.0.smithi122.stdout:3030: snap_create 2024-09-15T03:59:37.134 INFO:tasks.rados.rados.0.smithi122.stdout:3027: finishing copy_from racing read to smithi12223906-27 2024-09-15T03:59:37.135 INFO:tasks.rados.rados.0.smithi122.stdout:3027: finishing copy_from to smithi12223906-27 2024-09-15T03:59:37.135 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 2372 (ObjNum 940 snap 295 seq_num 940) dirty exists 2024-09-15T03:59:37.135 INFO:tasks.rados.rados.0.smithi122.stdout:3029: finishing write tid 1 to smithi12223906-44 2024-09-15T03:59:37.139 INFO:tasks.rados.rados.0.smithi122.stdout:3029: finishing write tid 2 to smithi12223906-44 2024-09-15T03:59:37.139 INFO:tasks.rados.rados.0.smithi122.stdout:3029: finishing write tid 3 to smithi12223906-44 2024-09-15T03:59:37.140 INFO:tasks.rados.rados.0.smithi122.stdout:3029: finishing write tid 4 to smithi12223906-44 2024-09-15T03:59:37.140 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 2391 (ObjNum 1077 snap 352 seq_num 1077) dirty exists 2024-09-15T03:59:37.140 INFO:tasks.rados.rados.0.smithi122.stdout:3029: left oid 44 (ObjNum 1077 snap 352 seq_num 1077) 2024-09-15T03:59:38.114 INFO:tasks.rados.rados.0.smithi122.stdout:3027: done (2 left) 2024-09-15T03:59:38.114 INFO:tasks.rados.rados.0.smithi122.stdout:3029: done (1 left) 2024-09-15T03:59:38.114 INFO:tasks.rados.rados.0.smithi122.stdout:3030: done (0 left) 2024-09-15T03:59:38.114 INFO:tasks.rados.rados.0.smithi122.stdout:3031: write (excl) oid 9 current snap is 353 2024-09-15T03:59:38.114 INFO:tasks.rados.rados.0.smithi122.stdout:3031: seq_num 1078 ranges {517946=724195,1932177=185769} 2024-09-15T03:59:38.125 INFO:tasks.rados.rados.0.smithi122.stdout:3031: writing smithi12223906-9 from 517946 to 1242141 tid 1 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3031: writing smithi12223906-9 from 1932177 to 2117946 tid 2 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3032: copy_from oid 39 from oid 24 current snap is 353 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3033: read oid 6 snap -1 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3033: expect deleted 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3034: read oid 12 snap -1 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3034: expect deleted 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3035: read oid 18 snap -1 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3035: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3036: snap_create 2024-09-15T03:59:38.131 INFO:tasks.rados.rados.0.smithi122.stdout:3031: finishing write tid 1 to smithi12223906-9 2024-09-15T03:59:38.135 INFO:tasks.rados.rados.0.smithi122.stdout:3031: finishing write tid 2 to smithi12223906-9 2024-09-15T03:59:38.135 INFO:tasks.rados.rados.0.smithi122.stdout:3031: finishing write tid 3 to smithi12223906-9 2024-09-15T03:59:38.136 INFO:tasks.rados.rados.0.smithi122.stdout:3031: finishing write tid 4 to smithi12223906-9 2024-09-15T03:59:38.136 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 1344 (ObjNum 1078 snap 353 seq_num 1078) dirty exists 2024-09-15T03:59:38.136 INFO:tasks.rados.rados.0.smithi122.stdout:3031: left oid 9 (ObjNum 1078 snap 353 seq_num 1078) 2024-09-15T03:59:38.141 INFO:tasks.rados.rados.0.smithi122.stdout:3035: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T03:59:38.306 INFO:tasks.rados.rados.0.smithi122.stdout:3032: finishing copy_from racing read to smithi12223906-39 2024-09-15T03:59:38.307 INFO:tasks.rados.rados.0.smithi122.stdout:3032: finishing copy_from to smithi12223906-39 2024-09-15T03:59:38.307 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 858 (ObjNum 1042 snap 331 seq_num 1042) dirty exists 2024-09-15T03:59:39.135 INFO:tasks.rados.rados.0.smithi122.stdout:3031: done (5 left) 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3032: done (4 left) 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3033: done (3 left) 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3034: done (2 left) 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3035: done (1 left) 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3036: done (0 left) 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3037: write oid 26 current snap is 354 2024-09-15T03:59:39.136 INFO:tasks.rados.rados.0.smithi122.stdout:3037: seq_num 1079 ranges {534753=799444,1990659=144094} 2024-09-15T03:59:39.144 INFO:tasks.rados.rados.0.smithi122.stdout:3037: writing smithi12223906-26 from 534753 to 1334197 tid 1 2024-09-15T03:59:39.151 INFO:tasks.rados.rados.0.smithi122.stdout:3037: writing smithi12223906-26 from 1990659 to 2134753 tid 2 2024-09-15T03:59:39.151 INFO:tasks.rados.rados.0.smithi122.stdout:3038: write (excl) oid 16 current snap is 354 2024-09-15T03:59:39.151 INFO:tasks.rados.rados.0.smithi122.stdout:3038: seq_num 1080 ranges {551560=591046,1765494=386066} 2024-09-15T03:59:39.157 INFO:tasks.rados.rados.0.smithi122.stdout:3038: writing smithi12223906-16 from 551560 to 1142606 tid 1 2024-09-15T03:59:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:3038: writing smithi12223906-16 from 1765494 to 2151560 tid 2 2024-09-15T03:59:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:3039: rollback oid 23 current snap is 354 2024-09-15T03:59:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 303 2024-09-15T03:59:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:3037: finishing write tid 1 to smithi12223906-26 2024-09-15T03:59:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:3037: finishing write tid 2 to smithi12223906-26 2024-09-15T03:59:39.163 INFO:tasks.rados.rados.0.smithi122.stdout:3037: finishing write tid 3 to smithi12223906-26 2024-09-15T03:59:39.164 INFO:tasks.rados.rados.0.smithi122.stdout:3037: finishing write tid 4 to smithi12223906-26 2024-09-15T03:59:39.164 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1891 (ObjNum 1079 snap 354 seq_num 1079) dirty exists 2024-09-15T03:59:39.164 INFO:tasks.rados.rados.0.smithi122.stdout:3037: left oid 26 (ObjNum 1079 snap 354 seq_num 1079) 2024-09-15T03:59:39.164 INFO:tasks.rados.rados.0.smithi122.stdout:3037: done (2 left) 2024-09-15T03:59:39.164 INFO:tasks.rados.rados.0.smithi122.stdout:3040: delete oid 14 current snap is 354 2024-09-15T03:59:39.166 INFO:tasks.rados.rados.0.smithi122.stdout:3038: finishing write tid 1 to smithi12223906-16 2024-09-15T03:59:39.169 INFO:tasks.rados.rados.0.smithi122.stdout:3038: finishing write tid 2 to smithi12223906-16 2024-09-15T03:59:39.169 INFO:tasks.rados.rados.0.smithi122.stdout:3038: finishing write tid 3 to smithi12223906-16 2024-09-15T03:59:39.169 INFO:tasks.rados.rados.0.smithi122.stdout:3039: finishing rollback tid 1 to smithi12223906-23 2024-09-15T03:59:39.169 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2388 (ObjNum 49 snap 0 seq_num 1817663648) dirty dne 2024-09-15T03:59:39.170 INFO:tasks.rados.rados.0.smithi122.stdout:3038: finishing write tid 4 to smithi12223906-16 2024-09-15T03:59:39.170 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1891 (ObjNum 1080 snap 354 seq_num 1080) dirty exists 2024-09-15T03:59:39.170 INFO:tasks.rados.rados.0.smithi122.stdout:3038: left oid 16 (ObjNum 1080 snap 354 seq_num 1080) 2024-09-15T03:59:39.172 INFO:tasks.rados.rados.0.smithi122.stdout:3038: done (2 left) 2024-09-15T03:59:39.172 INFO:tasks.rados.rados.0.smithi122.stdout:3039: done (1 left) 2024-09-15T03:59:39.172 INFO:tasks.rados.rados.0.smithi122.stdout:3040: done (0 left) 2024-09-15T03:59:39.172 INFO:tasks.rados.rados.0.smithi122.stdout:3041: read oid 16 snap -1 2024-09-15T03:59:39.172 INFO:tasks.rados.rados.0.smithi122.stdout:3041: expect (ObjNum 1080 snap 354 seq_num 1080) 2024-09-15T03:59:39.173 INFO:tasks.rados.rados.0.smithi122.stdout:3042: write (excl) oid 5 current snap is 354 2024-09-15T03:59:39.173 INFO:tasks.rados.rados.0.smithi122.stdout:3042: seq_num 1082 ranges {585174=741544,1766105=419069} 2024-09-15T03:59:39.184 INFO:tasks.rados.rados.0.smithi122.stdout:3042: writing smithi12223906-5 from 585174 to 1326718 tid 1 2024-09-15T03:59:39.188 INFO:tasks.rados.rados.0.smithi122.stdout:3042: writing smithi12223906-5 from 1766105 to 2185174 tid 2 2024-09-15T03:59:39.189 INFO:tasks.rados.rados.0.smithi122.stdout:3043: copy_from oid 20 from oid 23 current snap is 354 2024-09-15T03:59:39.189 INFO:tasks.rados.rados.0.smithi122.stdout:3041: expect (ObjNum 1080 snap 354 seq_num 1080) 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3043: finishing copy_from to smithi12223906-20 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3043: got expected ENOENT (src dne) 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3043: finishing copy_from racing read to smithi12223906-20 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3042: finishing write tid 1 to smithi12223906-5 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3042: finishing write tid 2 to smithi12223906-5 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3042: finishing write tid 3 to smithi12223906-5 2024-09-15T03:59:39.266 INFO:tasks.rados.rados.0.smithi122.stdout:3042: finishing write tid 4 to smithi12223906-5 2024-09-15T03:59:39.267 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 909 (ObjNum 1082 snap 354 seq_num 1082) dirty exists 2024-09-15T03:59:39.267 INFO:tasks.rados.rados.0.smithi122.stdout:3042: left oid 5 (ObjNum 1082 snap 354 seq_num 1082) 2024-09-15T03:59:39.267 INFO:tasks.rados.rados.0.smithi122.stdout:3041: done (2 left) 2024-09-15T03:59:39.267 INFO:tasks.rados.rados.0.smithi122.stdout:3042: done (1 left) 2024-09-15T03:59:39.267 INFO:tasks.rados.rados.0.smithi122.stdout:3043: done (0 left) 2024-09-15T03:59:39.267 INFO:tasks.rados.rados.0.smithi122.stdout:3044: snap_create 2024-09-15T03:59:40.150 INFO:tasks.rados.rados.0.smithi122.stdout:3044: done (0 left) 2024-09-15T03:59:40.150 INFO:tasks.rados.rados.0.smithi122.stdout:3045: read oid 8 snap -1 2024-09-15T03:59:40.150 INFO:tasks.rados.rados.0.smithi122.stdout:3045: expect (ObjNum 1071 snap 348 seq_num 1071) 2024-09-15T03:59:40.151 INFO:tasks.rados.rados.0.smithi122.stdout:3046: rollback oid 45 current snap is 355 2024-09-15T03:59:40.151 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 347 2024-09-15T03:59:40.151 INFO:tasks.rados.rados.0.smithi122.stdout:3047: delete oid 20 current snap is 355 2024-09-15T03:59:40.159 INFO:tasks.rados.rados.0.smithi122.stdout:3046: finishing rollback tid 0 to smithi12223906-45 2024-09-15T03:59:40.161 INFO:tasks.rados.rados.0.smithi122.stdout:3047: done (2 left) 2024-09-15T03:59:40.162 INFO:tasks.rados.rados.0.smithi122.stdout:3048: rollback oid 49 current snap is 355 2024-09-15T03:59:40.162 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 302 2024-09-15T03:59:40.162 INFO:tasks.rados.rados.0.smithi122.stdout:3049: delete oid 21 current snap is 355 2024-09-15T03:59:40.162 INFO:tasks.rados.rados.0.smithi122.stdout:3046: finishing rollback tid 1 to smithi12223906-45 2024-09-15T03:59:40.162 INFO:tasks.rados.rados.0.smithi122.stdout:3046: finishing rollback tid 2 to smithi12223906-45 2024-09-15T03:59:40.162 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 1605 (ObjNum 1043 snap 332 seq_num 1043) dirty exists 2024-09-15T03:59:40.169 INFO:tasks.rados.rados.0.smithi122.stdout:3046: done (3 left) 2024-09-15T03:59:40.169 INFO:tasks.rados.rados.0.smithi122.stdout:3049: done (2 left) 2024-09-15T03:59:40.170 INFO:tasks.rados.rados.0.smithi122.stdout:3050: read oid 4 snap 341 2024-09-15T03:59:40.170 INFO:tasks.rados.rados.0.smithi122.stdout:3050: expect (ObjNum 1051 snap 335 seq_num 1051) 2024-09-15T03:59:40.170 INFO:tasks.rados.rados.0.smithi122.stdout:3051: read oid 12 snap -1 2024-09-15T03:59:40.170 INFO:tasks.rados.rados.0.smithi122.stdout:3051: expect deleted 2024-09-15T03:59:40.170 INFO:tasks.rados.rados.0.smithi122.stdout:3052: copy_from oid 6 from oid 50 current snap is 355 2024-09-15T03:59:40.170 INFO:tasks.rados.rados.0.smithi122.stdout:3053: snap_remove snap 306 2024-09-15T03:59:41.098 INFO:tasks.rados.rados.0.smithi122.stdout:3048: finishing rollback tid 0 to smithi12223906-49 2024-09-15T03:59:41.098 INFO:tasks.rados.rados.0.smithi122.stdout:3045: expect (ObjNum 1071 snap 348 seq_num 1071) 2024-09-15T03:59:41.184 INFO:tasks.rados.rados.0.smithi122.stdout:3048: finishing rollback tid 1 to smithi12223906-49 2024-09-15T03:59:41.184 INFO:tasks.rados.rados.0.smithi122.stdout:3048: finishing rollback tid 2 to smithi12223906-49 2024-09-15T03:59:41.184 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1072 (ObjNum 744 snap 226 seq_num 744) dirty exists 2024-09-15T03:59:41.184 INFO:tasks.rados.rados.0.smithi122.stdout:3050: expect (ObjNum 1051 snap 335 seq_num 1051) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3052: finishing copy_from racing read to smithi12223906-6 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3052: finishing copy_from to smithi12223906-6 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2380 (ObjNum 950 snap 300 seq_num 950) dirty exists 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3045: done (5 left) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3048: done (4 left) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3050: done (3 left) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3051: done (2 left) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3052: done (1 left) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3053: done (0 left) 2024-09-15T03:59:41.496 INFO:tasks.rados.rados.0.smithi122.stdout:3054: rollback oid 34 current snap is 355 2024-09-15T03:59:41.497 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 345 2024-09-15T03:59:41.497 INFO:tasks.rados.rados.0.smithi122.stdout:3055: write oid 33 current snap is 355 2024-09-15T03:59:41.497 INFO:tasks.rados.rados.0.smithi122.stdout:3055: seq_num 1085 ranges {635595=567291,1825198=410397} 2024-09-15T03:59:41.512 INFO:tasks.rados.rados.0.smithi122.stdout:3055: writing smithi12223906-33 from 635595 to 1202886 tid 1 2024-09-15T03:59:41.518 INFO:tasks.rados.rados.0.smithi122.stdout:3055: writing smithi12223906-33 from 1825198 to 2235595 tid 2 2024-09-15T03:59:41.518 INFO:tasks.rados.rados.0.smithi122.stdout:3056: delete oid 20 current snap is 355 2024-09-15T03:59:41.518 INFO:tasks.rados.rados.0.smithi122.stdout:3054: finishing rollback tid 0 to smithi12223906-34 2024-09-15T03:59:41.518 INFO:tasks.rados.rados.0.smithi122.stdout:3054: finishing rollback tid 1 to smithi12223906-34 2024-09-15T03:59:41.518 INFO:tasks.rados.rados.0.smithi122.stdout:3054: finishing rollback tid 2 to smithi12223906-34 2024-09-15T03:59:41.518 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1584 (ObjNum 1062 snap 343 seq_num 1062) dirty exists 2024-09-15T03:59:41.520 INFO:tasks.rados.rados.0.smithi122.stdout:3054: done (2 left) 2024-09-15T03:59:41.520 INFO:tasks.rados.rados.0.smithi122.stdout:3056: done (1 left) 2024-09-15T03:59:41.520 INFO:tasks.rados.rados.0.smithi122.stdout:3057: delete oid 50 current snap is 355 2024-09-15T03:59:41.525 INFO:tasks.rados.rados.0.smithi122.stdout:3055: finishing write tid 1 to smithi12223906-33 2024-09-15T03:59:41.525 INFO:tasks.rados.rados.0.smithi122.stdout:3055: finishing write tid 2 to smithi12223906-33 2024-09-15T03:59:41.529 INFO:tasks.rados.rados.0.smithi122.stdout:3055: finishing write tid 3 to smithi12223906-33 2024-09-15T03:59:41.529 INFO:tasks.rados.rados.0.smithi122.stdout:3055: finishing write tid 4 to smithi12223906-33 2024-09-15T03:59:41.530 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2400 (ObjNum 1085 snap 355 seq_num 1085) dirty exists 2024-09-15T03:59:41.530 INFO:tasks.rados.rados.0.smithi122.stdout:3055: left oid 33 (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T03:59:41.530 INFO:tasks.rados.rados.0.smithi122.stdout:3055: done (1 left) 2024-09-15T03:59:41.530 INFO:tasks.rados.rados.0.smithi122.stdout:3057: done (0 left) 2024-09-15T03:59:41.530 INFO:tasks.rados.rados.0.smithi122.stdout:3058: write (excl) oid 31 current snap is 355 2024-09-15T03:59:41.530 INFO:tasks.rados.rados.0.smithi122.stdout:3058: seq_num 1088 ranges {686016=509391,1716997=569019} 2024-09-15T03:59:41.539 INFO:tasks.rados.rados.0.smithi122.stdout:3058: writing smithi12223906-31 from 686016 to 1195407 tid 1 2024-09-15T03:59:41.546 INFO:tasks.rados.rados.0.smithi122.stdout:3058: writing smithi12223906-31 from 1716997 to 2286016 tid 2 2024-09-15T03:59:41.546 INFO:tasks.rados.rados.0.smithi122.stdout:3059: rollback oid 4 current snap is 355 2024-09-15T03:59:41.546 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 4 to 324 2024-09-15T03:59:41.546 INFO:tasks.rados.rados.0.smithi122.stdout:3058: finishing write tid 1 to smithi12223906-31 2024-09-15T03:59:41.546 INFO:tasks.rados.rados.0.smithi122.stdout:3060: write oid 25 current snap is 355 2024-09-15T03:59:41.546 INFO:tasks.rados.rados.0.smithi122.stdout:3060: seq_num 1089 ranges {702823=584640,2059126=243697} 2024-09-15T03:59:41.553 INFO:tasks.rados.rados.0.smithi122.stdout:3060: writing smithi12223906-25 from 702823 to 1287463 tid 1 2024-09-15T03:59:41.559 INFO:tasks.rados.rados.0.smithi122.stdout:3060: writing smithi12223906-25 from 2059126 to 2302823 tid 2 2024-09-15T03:59:41.560 INFO:tasks.rados.rados.0.smithi122.stdout:3061: read oid 44 snap -1 2024-09-15T03:59:41.560 INFO:tasks.rados.rados.0.smithi122.stdout:3061: expect (ObjNum 1077 snap 352 seq_num 1077) 2024-09-15T03:59:41.560 INFO:tasks.rados.rados.0.smithi122.stdout:3062: snap_remove snap 350 2024-09-15T03:59:42.157 INFO:tasks.rados.rados.0.smithi122.stdout:3062: done (4 left) 2024-09-15T03:59:42.157 INFO:tasks.rados.rados.0.smithi122.stdout:3063: rollback oid 11 current snap is 355 2024-09-15T03:59:42.157 INFO:tasks.rados.rados.0.smithi122.stdout:3058: finishing write tid 2 to smithi12223906-31 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3058: finishing write tid 3 to smithi12223906-31 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3058: finishing write tid 4 to smithi12223906-31 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1908 (ObjNum 1088 snap 355 seq_num 1088) dirty exists 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3058: left oid 31 (ObjNum 1088 snap 355 seq_num 1088) 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3059: finishing rollback tid 0 to smithi12223906-4 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3059: finishing rollback tid 1 to smithi12223906-4 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3059: finishing rollback tid 2 to smithi12223906-4 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1911 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:59:42.158 INFO:tasks.rados.rados.0.smithi122.stdout:3061: expect (ObjNum 1077 snap 352 seq_num 1077) 2024-09-15T03:59:42.229 INFO:tasks.rados.rados.0.smithi122.stdout:3060: finishing write tid 1 to smithi12223906-25 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3060: finishing write tid 2 to smithi12223906-25 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3060: finishing write tid 3 to smithi12223906-25 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3060: finishing write tid 4 to smithi12223906-25 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2452 (ObjNum 1089 snap 355 seq_num 1089) dirty exists 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3060: left oid 25 (ObjNum 1089 snap 355 seq_num 1089) 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 11 to 349 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3058: done (4 left) 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3059: done (3 left) 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3060: done (2 left) 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3061: done (1 left) 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3064: rollback oid 1 current snap is 355 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 1 to 347 2024-09-15T03:59:42.230 INFO:tasks.rados.rados.0.smithi122.stdout:3065: write (excl) oid 39 current snap is 355 2024-09-15T03:59:42.231 INFO:tasks.rados.rados.0.smithi122.stdout:3065: seq_num 1090 ranges {719630=659889,2117608=202022} 2024-09-15T03:59:42.244 INFO:tasks.rados.rados.0.smithi122.stdout:3065: writing smithi12223906-39 from 719630 to 1379519 tid 1 2024-09-15T03:59:42.250 INFO:tasks.rados.rados.0.smithi122.stdout:3065: writing smithi12223906-39 from 2117608 to 2319630 tid 2 2024-09-15T03:59:42.250 INFO:tasks.rados.rados.0.smithi122.stdout:3066: write oid 9 current snap is 355 2024-09-15T03:59:42.250 INFO:tasks.rados.rados.0.smithi122.stdout:3066: seq_num 1091 ranges {736437=735138,2176090=160347} 2024-09-15T03:59:42.260 INFO:tasks.rados.rados.0.smithi122.stdout:3066: writing smithi12223906-9 from 736437 to 1471575 tid 1 2024-09-15T03:59:42.262 INFO:tasks.rados.rados.0.smithi122.stdout:3066: writing smithi12223906-9 from 2176090 to 2336437 tid 2 2024-09-15T03:59:42.262 INFO:tasks.rados.rados.0.smithi122.stdout:3067: snap_create 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3063: finishing rollback tid 0 to smithi12223906-11 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3063: finishing rollback tid 1 to smithi12223906-11 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3063: finishing rollback tid 2 to smithi12223906-11 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1894 (ObjNum 944 snap 295 seq_num 944) dirty exists 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3064: finishing rollback tid 1 to smithi12223906-1 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3064: finishing rollback tid 2 to smithi12223906-1 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1896 (ObjNum 1011 snap 316 seq_num 1011) dirty exists 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3065: finishing write tid 1 to smithi12223906-39 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3065: finishing write tid 2 to smithi12223906-39 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3065: finishing write tid 3 to smithi12223906-39 2024-09-15T03:59:42.263 INFO:tasks.rados.rados.0.smithi122.stdout:3065: finishing write tid 4 to smithi12223906-39 2024-09-15T03:59:42.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2464 (ObjNum 1090 snap 355 seq_num 1090) dirty exists 2024-09-15T03:59:42.264 INFO:tasks.rados.rados.0.smithi122.stdout:3065: left oid 39 (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T03:59:42.265 INFO:tasks.rados.rados.0.smithi122.stdout:3066: finishing write tid 1 to smithi12223906-9 2024-09-15T03:59:42.266 INFO:tasks.rados.rados.0.smithi122.stdout:3066: finishing write tid 2 to smithi12223906-9 2024-09-15T03:59:42.266 INFO:tasks.rados.rados.0.smithi122.stdout:3066: finishing write tid 3 to smithi12223906-9 2024-09-15T03:59:42.266 INFO:tasks.rados.rados.0.smithi122.stdout:3066: finishing write tid 4 to smithi12223906-9 2024-09-15T03:59:42.266 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2383 (ObjNum 1091 snap 355 seq_num 1091) dirty exists 2024-09-15T03:59:42.266 INFO:tasks.rados.rados.0.smithi122.stdout:3066: left oid 9 (ObjNum 1091 snap 355 seq_num 1091) 2024-09-15T03:59:43.167 INFO:tasks.rados.rados.0.smithi122.stdout:3063: done (4 left) 2024-09-15T03:59:43.167 INFO:tasks.rados.rados.0.smithi122.stdout:3064: done (3 left) 2024-09-15T03:59:43.167 INFO:tasks.rados.rados.0.smithi122.stdout:3065: done (2 left) 2024-09-15T03:59:43.167 INFO:tasks.rados.rados.0.smithi122.stdout:3066: done (1 left) 2024-09-15T03:59:43.167 INFO:tasks.rados.rados.0.smithi122.stdout:3067: done (0 left) 2024-09-15T03:59:43.167 INFO:tasks.rados.rados.0.smithi122.stdout:3068: snap_remove snap 312 2024-09-15T03:59:44.178 INFO:tasks.rados.rados.0.smithi122.stdout:3068: done (0 left) 2024-09-15T03:59:44.178 INFO:tasks.rados.rados.0.smithi122.stdout:3069: read oid 27 snap -1 2024-09-15T03:59:44.179 INFO:tasks.rados.rados.0.smithi122.stdout:3069: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:59:44.179 INFO:tasks.rados.rados.0.smithi122.stdout:3070: delete oid 5 current snap is 356 2024-09-15T03:59:44.195 INFO:tasks.rados.rados.0.smithi122.stdout:3069: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T03:59:44.317 INFO:tasks.rados.rados.0.smithi122.stdout:3069: done (1 left) 2024-09-15T03:59:44.318 INFO:tasks.rados.rados.0.smithi122.stdout:3070: done (0 left) 2024-09-15T03:59:44.318 INFO:tasks.rados.rados.0.smithi122.stdout:3071: write oid 6 current snap is 356 2024-09-15T03:59:44.318 INFO:tasks.rados.rados.0.smithi122.stdout:3071: seq_num 1093 ranges {770051=485636,1776701=511047} 2024-09-15T03:59:44.333 INFO:tasks.rados.rados.0.smithi122.stdout:3071: writing smithi12223906-6 from 770051 to 1255687 tid 1 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3071: writing smithi12223906-6 from 1776701 to 2287748 tid 2 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3072: rollback oid 30 current snap is 356 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 30 to 231 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3073: read oid 1 snap 334 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3073: expect (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3074: read oid 16 snap -1 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3074: expect (ObjNum 1080 snap 354 seq_num 1080) 2024-09-15T03:59:44.339 INFO:tasks.rados.rados.0.smithi122.stdout:3075: delete oid 13 current snap is 356 2024-09-15T03:59:44.346 INFO:tasks.rados.rados.0.smithi122.stdout:3073: expect (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T03:59:44.444 INFO:tasks.rados.rados.0.smithi122.stdout:3072: finishing rollback tid 0 to smithi12223906-30 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3072: finishing rollback tid 1 to smithi12223906-30 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3071: finishing write tid 1 to smithi12223906-6 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3072: finishing rollback tid 2 to smithi12223906-30 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 2219 (ObjNum 746 snap 226 seq_num 746) dirty exists 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3071: finishing write tid 2 to smithi12223906-6 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3071: finishing write tid 3 to smithi12223906-6 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3071: finishing write tid 4 to smithi12223906-6 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2386 (ObjNum 1093 snap 356 seq_num 1093) dirty exists 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3071: left oid 6 (ObjNum 1093 snap 356 seq_num 1093) 2024-09-15T03:59:44.445 INFO:tasks.rados.rados.0.smithi122.stdout:3074: expect (ObjNum 1080 snap 354 seq_num 1080) 2024-09-15T03:59:44.514 INFO:tasks.rados.rados.0.smithi122.stdout:3071: done (4 left) 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3072: done (3 left) 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3073: done (2 left) 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3074: done (1 left) 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3075: done (0 left) 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3076: read oid 8 snap 315 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3076: expect (ObjNum 958 snap 304 seq_num 958) 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3077: copy_from oid 44 from oid 4 current snap is 356 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3078: write oid 17 current snap is 356 2024-09-15T03:59:44.515 INFO:tasks.rados.rados.0.smithi122.stdout:3078: seq_num 1095 ranges {403665=752487,1610018=514716} 2024-09-15T03:59:44.526 INFO:tasks.rados.rados.0.smithi122.stdout:3078: writing smithi12223906-17 from 403665 to 1156152 tid 1 2024-09-15T03:59:44.533 INFO:tasks.rados.rados.0.smithi122.stdout:3078: writing smithi12223906-17 from 1610018 to 2124734 tid 2 2024-09-15T03:59:44.533 INFO:tasks.rados.rados.0.smithi122.stdout:3079: snap_remove snap 325 2024-09-15T03:59:45.187 INFO:tasks.rados.rados.0.smithi122.stdout:3079: done (3 left) 2024-09-15T03:59:45.187 INFO:tasks.rados.rados.0.smithi122.stdout:3080: copy_from oid 25 from oid 42 current snap is 356 2024-09-15T03:59:45.187 INFO:tasks.rados.rados.0.smithi122.stdout:3076: expect (ObjNum 958 snap 304 seq_num 958) 2024-09-15T03:59:45.263 INFO:tasks.rados.rados.0.smithi122.stdout:3078: finishing write tid 1 to smithi12223906-17 2024-09-15T03:59:45.263 INFO:tasks.rados.rados.0.smithi122.stdout:3077: finishing copy_from racing read to smithi12223906-44 2024-09-15T03:59:45.263 INFO:tasks.rados.rados.0.smithi122.stdout:3077: finishing copy_from to smithi12223906-44 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 1911 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3078: finishing write tid 2 to smithi12223906-17 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3078: finishing write tid 3 to smithi12223906-17 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3078: finishing write tid 4 to smithi12223906-17 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1075 (ObjNum 1095 snap 356 seq_num 1095) dirty exists 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3078: left oid 17 (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3080: finishing copy_from to smithi12223906-25 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3080: got expected ENOENT (src dne) 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3080: finishing copy_from racing read to smithi12223906-25 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3076: done (3 left) 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3077: done (2 left) 2024-09-15T03:59:45.264 INFO:tasks.rados.rados.0.smithi122.stdout:3078: done (1 left) 2024-09-15T03:59:45.265 INFO:tasks.rados.rados.0.smithi122.stdout:3080: done (0 left) 2024-09-15T03:59:45.265 INFO:tasks.rados.rados.0.smithi122.stdout:3081: delete oid 19 current snap is 356 2024-09-15T03:59:45.268 INFO:tasks.rados.rados.0.smithi122.stdout:3081: done (0 left) 2024-09-15T03:59:45.268 INFO:tasks.rados.rados.0.smithi122.stdout:3082: read oid 29 snap 343 2024-09-15T03:59:45.268 INFO:tasks.rados.rados.0.smithi122.stdout:3082: expect (ObjNum 1037 snap 327 seq_num 1037) 2024-09-15T03:59:45.268 INFO:tasks.rados.rados.0.smithi122.stdout:3083: delete oid 9 current snap is 356 2024-09-15T03:59:45.274 INFO:tasks.rados.rados.0.smithi122.stdout:3082: expect (ObjNum 1037 snap 327 seq_num 1037) 2024-09-15T03:59:45.479 INFO:tasks.rados.rados.0.smithi122.stdout:3082: done (1 left) 2024-09-15T03:59:45.479 INFO:tasks.rados.rados.0.smithi122.stdout:3083: done (0 left) 2024-09-15T03:59:45.479 INFO:tasks.rados.rados.0.smithi122.stdout:3084: write oid 36 current snap is 356 2024-09-15T03:59:45.479 INFO:tasks.rados.rados.0.smithi122.stdout:3084: seq_num 1098 ranges {454086=578234,1669111=662043} 2024-09-15T03:59:45.494 INFO:tasks.rados.rados.0.smithi122.stdout:3084: writing smithi12223906-36 from 454086 to 1032320 tid 1 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:3084: writing smithi12223906-36 from 1669111 to 2331154 tid 2 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:3085: rollback oid 21 current snap is 356 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 345 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:3086: rollback oid 26 current snap is 356 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 341 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:3087: rollback oid 25 current snap is 356 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 25 to 302 2024-09-15T03:59:45.502 INFO:tasks.rados.rados.0.smithi122.stdout:3088: snap_create 2024-09-15T03:59:45.504 INFO:tasks.rados.rados.0.smithi122.stdout:3084: finishing write tid 1 to smithi12223906-36 2024-09-15T03:59:45.509 INFO:tasks.rados.rados.0.smithi122.stdout:3085: finishing rollback tid 1 to smithi12223906-21 2024-09-15T03:59:45.509 INFO:tasks.rados.rados.0.smithi122.stdout:3085: finishing rollback tid 2 to smithi12223906-21 2024-09-15T03:59:45.510 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1218 (ObjNum 1040 snap 330 seq_num 1040) dirty exists 2024-09-15T03:59:45.512 INFO:tasks.rados.rados.0.smithi122.stdout:3086: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:59:45.512 INFO:tasks.rados.rados.0.smithi122.stdout:3086: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:59:45.513 INFO:tasks.rados.rados.0.smithi122.stdout:3086: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:59:45.513 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1894 (ObjNum 984 snap 311 seq_num 984) dirty exists 2024-09-15T03:59:45.516 INFO:tasks.rados.rados.0.smithi122.stdout:3084: finishing write tid 2 to smithi12223906-36 2024-09-15T03:59:45.517 INFO:tasks.rados.rados.0.smithi122.stdout:3084: finishing write tid 3 to smithi12223906-36 2024-09-15T03:59:45.517 INFO:tasks.rados.rados.0.smithi122.stdout:3084: finishing write tid 4 to smithi12223906-36 2024-09-15T03:59:45.517 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 2491 (ObjNum 1098 snap 356 seq_num 1098) dirty exists 2024-09-15T03:59:45.517 INFO:tasks.rados.rados.0.smithi122.stdout:3084: left oid 36 (ObjNum 1098 snap 356 seq_num 1098) 2024-09-15T03:59:45.517 INFO:tasks.rados.rados.0.smithi122.stdout:3087: finishing rollback tid 0 to smithi12223906-25 2024-09-15T03:59:45.517 INFO:tasks.rados.rados.0.smithi122.stdout:3087: finishing rollback tid 1 to smithi12223906-25 2024-09-15T03:59:45.518 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2494 (ObjNum 33 snap 0 seq_num 1842828400) dirty dne 2024-09-15T03:59:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:3084: done (4 left) 2024-09-15T03:59:46.100 INFO:tasks.rados.rados.0.smithi122.stdout:3085: done (3 left) 2024-09-15T03:59:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:3086: done (2 left) 2024-09-15T03:59:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:3087: done (1 left) 2024-09-15T03:59:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:3088: done (0 left) 2024-09-15T03:59:46.101 INFO:tasks.rados.rados.0.smithi122.stdout:3089: snap_create 2024-09-15T03:59:47.101 INFO:tasks.rados.rados.0.smithi122.stdout:3089: done (0 left) 2024-09-15T03:59:47.101 INFO:tasks.rados.rados.0.smithi122.stdout:3090: snap_create 2024-09-15T03:59:48.110 INFO:tasks.rados.rados.0.smithi122.stdout:3090: done (0 left) 2024-09-15T03:59:48.110 INFO:tasks.rados.rados.0.smithi122.stdout:3091: write (excl) oid 38 current snap is 359 2024-09-15T03:59:48.110 INFO:tasks.rados.rados.0.smithi122.stdout:3091: seq_num 1099 ranges {470893=653483,1727593=522054} 2024-09-15T03:59:48.117 INFO:tasks.rados.rados.0.smithi122.stdout:3091: writing smithi12223906-38 from 470893 to 1124376 tid 1 2024-09-15T03:59:48.124 INFO:tasks.rados.rados.0.smithi122.stdout:3091: writing smithi12223906-38 from 1727593 to 2249647 tid 2 2024-09-15T03:59:48.124 INFO:tasks.rados.rados.0.smithi122.stdout:3092: snap_remove snap 231 2024-09-15T03:59:49.123 INFO:tasks.rados.rados.0.smithi122.stdout:3091: finishing write tid 1 to smithi12223906-38 2024-09-15T03:59:49.123 INFO:tasks.rados.rados.0.smithi122.stdout:3091: finishing write tid 2 to smithi12223906-38 2024-09-15T03:59:49.123 INFO:tasks.rados.rados.0.smithi122.stdout:3091: finishing write tid 3 to smithi12223906-38 2024-09-15T03:59:49.123 INFO:tasks.rados.rados.0.smithi122.stdout:3091: finishing write tid 4 to smithi12223906-38 2024-09-15T03:59:49.124 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1897 (ObjNum 1099 snap 359 seq_num 1099) dirty exists 2024-09-15T03:59:49.124 INFO:tasks.rados.rados.0.smithi122.stdout:3091: left oid 38 (ObjNum 1099 snap 359 seq_num 1099) 2024-09-15T03:59:49.124 INFO:tasks.rados.rados.0.smithi122.stdout:3091: done (1 left) 2024-09-15T03:59:49.124 INFO:tasks.rados.rados.0.smithi122.stdout:3092: done (0 left) 2024-09-15T03:59:49.124 INFO:tasks.rados.rados.0.smithi122.stdout:3093: snap_remove snap 346 2024-09-15T03:59:50.134 INFO:tasks.rados.rados.0.smithi122.stdout:3093: done (0 left) 2024-09-15T03:59:50.134 INFO:tasks.rados.rados.0.smithi122.stdout:3094: write (excl) oid 23 current snap is 359 2024-09-15T03:59:50.134 INFO:tasks.rados.rados.0.smithi122.stdout:3094: seq_num 1100 ranges {487700=728732,1786075=665712} 2024-09-15T03:59:50.139 INFO:tasks.rados.rados.0.smithi122.stdout:3094: writing smithi12223906-23 from 487700 to 1216432 tid 1 2024-09-15T03:59:50.146 INFO:tasks.rados.rados.0.smithi122.stdout:3094: writing smithi12223906-23 from 1786075 to 2451787 tid 2 2024-09-15T03:59:50.147 INFO:tasks.rados.rados.0.smithi122.stdout:3095: rollback oid 3 current snap is 359 2024-09-15T03:59:50.147 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 342 2024-09-15T03:59:50.148 INFO:tasks.rados.rados.0.smithi122.stdout:3096: write (excl) oid 10 current snap is 359 2024-09-15T03:59:50.148 INFO:tasks.rados.rados.0.smithi122.stdout:3096: seq_num 1101 ranges {504507=403981,1328204=409370,2252362=252145} 2024-09-15T03:59:50.152 INFO:tasks.rados.rados.0.smithi122.stdout:3096: writing smithi12223906-10 from 504507 to 908488 tid 1 2024-09-15T03:59:50.157 INFO:tasks.rados.rados.0.smithi122.stdout:3096: writing smithi12223906-10 from 1328204 to 1737574 tid 2 2024-09-15T03:59:50.161 INFO:tasks.rados.rados.0.smithi122.stdout:3096: writing smithi12223906-10 from 2252362 to 2504507 tid 3 2024-09-15T03:59:50.161 INFO:tasks.rados.rados.0.smithi122.stdout:3097: snap_create 2024-09-15T03:59:50.161 INFO:tasks.rados.rados.0.smithi122.stdout:3096: finishing write tid 1 to smithi12223906-10 2024-09-15T03:59:50.163 INFO:tasks.rados.rados.0.smithi122.stdout:3094: finishing write tid 1 to smithi12223906-23 2024-09-15T03:59:50.168 INFO:tasks.rados.rados.0.smithi122.stdout:3094: finishing write tid 2 to smithi12223906-23 2024-09-15T03:59:50.168 INFO:tasks.rados.rados.0.smithi122.stdout:3094: finishing write tid 3 to smithi12223906-23 2024-09-15T03:59:50.169 INFO:tasks.rados.rados.0.smithi122.stdout:3095: finishing rollback tid 1 to smithi12223906-3 2024-09-15T03:59:50.169 INFO:tasks.rados.rados.0.smithi122.stdout:3095: finishing rollback tid 2 to smithi12223906-3 2024-09-15T03:59:50.169 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2518 (ObjNum 973 snap 307 seq_num 973) dirty exists 2024-09-15T03:59:50.169 INFO:tasks.rados.rados.0.smithi122.stdout:3094: finishing write tid 4 to smithi12223906-23 2024-09-15T03:59:50.169 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2449 (ObjNum 1100 snap 359 seq_num 1100) dirty exists 2024-09-15T03:59:50.169 INFO:tasks.rados.rados.0.smithi122.stdout:3094: left oid 23 (ObjNum 1100 snap 359 seq_num 1100) 2024-09-15T03:59:50.170 INFO:tasks.rados.rados.0.smithi122.stdout:3096: finishing write tid 2 to smithi12223906-10 2024-09-15T03:59:50.171 INFO:tasks.rados.rados.0.smithi122.stdout:3096: finishing write tid 3 to smithi12223906-10 2024-09-15T03:59:50.171 INFO:tasks.rados.rados.0.smithi122.stdout:3096: finishing write tid 4 to smithi12223906-10 2024-09-15T03:59:50.171 INFO:tasks.rados.rados.0.smithi122.stdout:3096: finishing write tid 5 to smithi12223906-10 2024-09-15T03:59:50.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1228 (ObjNum 1101 snap 359 seq_num 1101) dirty exists 2024-09-15T03:59:50.171 INFO:tasks.rados.rados.0.smithi122.stdout:3096: left oid 10 (ObjNum 1101 snap 359 seq_num 1101) 2024-09-15T03:59:51.108 INFO:tasks.rados.rados.0.smithi122.stdout:3094: done (3 left) 2024-09-15T03:59:51.108 INFO:tasks.rados.rados.0.smithi122.stdout:3095: done (2 left) 2024-09-15T03:59:51.108 INFO:tasks.rados.rados.0.smithi122.stdout:3096: done (1 left) 2024-09-15T03:59:51.108 INFO:tasks.rados.rados.0.smithi122.stdout:3097: done (0 left) 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3098: read oid 13 snap -1 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3098: expect deleted 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3099: copy_from oid 7 from oid 34 current snap is 360 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3100: copy_from oid 14 from oid 50 current snap is 360 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3101: read oid 15 snap -1 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3101: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3102: read oid 11 snap -1 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3102: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3098: done (4 left) 2024-09-15T03:59:51.109 INFO:tasks.rados.rados.0.smithi122.stdout:3103: read oid 13 snap 324 2024-09-15T03:59:51.110 INFO:tasks.rados.rados.0.smithi122.stdout:3103: expect deleted 2024-09-15T03:59:51.110 INFO:tasks.rados.rados.0.smithi122.stdout:3104: write oid 6 current snap is 360 2024-09-15T03:59:51.110 INFO:tasks.rados.rados.0.smithi122.stdout:3104: seq_num 1102 ranges {521314=479230,1786686=669381} 2024-09-15T03:59:51.114 INFO:tasks.rados.rados.0.smithi122.stdout:3104: writing smithi12223906-6 from 521314 to 1000544 tid 1 2024-09-15T03:59:51.123 INFO:tasks.rados.rados.0.smithi122.stdout:3104: writing smithi12223906-6 from 1786686 to 2456067 tid 2 2024-09-15T03:59:51.123 INFO:tasks.rados.rados.0.smithi122.stdout:3100: finishing copy_from to smithi12223906-14 2024-09-15T03:59:51.123 INFO:tasks.rados.rados.0.smithi122.stdout:3100: got expected ENOENT (src dne) 2024-09-15T03:59:51.124 INFO:tasks.rados.rados.0.smithi122.stdout:3100: finishing copy_from racing read to smithi12223906-14 2024-09-15T03:59:51.124 INFO:tasks.rados.rados.0.smithi122.stdout:3100: done (5 left) 2024-09-15T03:59:51.124 INFO:tasks.rados.rados.0.smithi122.stdout:3103: done (4 left) 2024-09-15T03:59:51.124 INFO:tasks.rados.rados.0.smithi122.stdout:3105: write (excl) oid 5 current snap is 360 2024-09-15T03:59:51.124 INFO:tasks.rados.rados.0.smithi122.stdout:3105: seq_num 1103 ranges {538121=670832,1961521=413039} 2024-09-15T03:59:51.131 INFO:tasks.rados.rados.0.smithi122.stdout:3105: writing smithi12223906-5 from 538121 to 1208953 tid 1 2024-09-15T03:59:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:3105: writing smithi12223906-5 from 1961521 to 2374560 tid 2 2024-09-15T03:59:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:3106: snap_create 2024-09-15T03:59:51.139 INFO:tasks.rados.rados.0.smithi122.stdout:3101: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T03:59:51.438 INFO:tasks.rados.rados.0.smithi122.stdout:3102: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T03:59:51.633 INFO:tasks.rados.rados.0.smithi122.stdout:3099: finishing copy_from racing read to smithi12223906-7 2024-09-15T03:59:51.633 INFO:tasks.rados.rados.0.smithi122.stdout:3105: finishing write tid 1 to smithi12223906-5 2024-09-15T03:59:51.633 INFO:tasks.rados.rados.0.smithi122.stdout:3099: finishing copy_from to smithi12223906-7 2024-09-15T03:59:51.633 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 1584 (ObjNum 1062 snap 343 seq_num 1062) dirty exists 2024-09-15T03:59:51.633 INFO:tasks.rados.rados.0.smithi122.stdout:3104: finishing write tid 1 to smithi12223906-6 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3104: finishing write tid 2 to smithi12223906-6 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3104: finishing write tid 3 to smithi12223906-6 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3104: finishing write tid 4 to smithi12223906-6 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2390 (ObjNum 1102 snap 360 seq_num 1102) dirty exists 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3104: left oid 6 (ObjNum 1102 snap 360 seq_num 1102) 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3105: finishing write tid 2 to smithi12223906-5 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3105: finishing write tid 3 to smithi12223906-5 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3105: finishing write tid 4 to smithi12223906-5 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 913 (ObjNum 1103 snap 360 seq_num 1103) dirty exists 2024-09-15T03:59:51.634 INFO:tasks.rados.rados.0.smithi122.stdout:3105: left oid 5 (ObjNum 1103 snap 360 seq_num 1103) 2024-09-15T03:59:52.117 INFO:tasks.rados.rados.0.smithi122.stdout:3099: done (5 left) 2024-09-15T03:59:52.117 INFO:tasks.rados.rados.0.smithi122.stdout:3101: done (4 left) 2024-09-15T03:59:52.117 INFO:tasks.rados.rados.0.smithi122.stdout:3102: done (3 left) 2024-09-15T03:59:52.117 INFO:tasks.rados.rados.0.smithi122.stdout:3104: done (2 left) 2024-09-15T03:59:52.117 INFO:tasks.rados.rados.0.smithi122.stdout:3105: done (1 left) 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:3106: done (0 left) 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:3107: rollback oid 22 current snap is 361 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 22 to 297 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:3108: read oid 1 snap -1 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:3108: expect (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:3109: write oid 27 current snap is 361 2024-09-15T03:59:52.118 INFO:tasks.rados.rados.0.smithi122.stdout:3109: seq_num 1104 ranges {554928=746081,2020003=534925} 2024-09-15T03:59:52.127 INFO:tasks.rados.rados.0.smithi122.stdout:3109: writing smithi12223906-27 from 554928 to 1301009 tid 1 2024-09-15T03:59:52.135 INFO:tasks.rados.rados.0.smithi122.stdout:3109: writing smithi12223906-27 from 2020003 to 2554928 tid 2 2024-09-15T03:59:52.135 INFO:tasks.rados.rados.0.smithi122.stdout:3110: snap_create 2024-09-15T03:59:52.135 INFO:tasks.rados.rados.0.smithi122.stdout:3107: finishing rollback tid 1 to smithi12223906-22 2024-09-15T03:59:52.135 INFO:tasks.rados.rados.0.smithi122.stdout:3107: finishing rollback tid 2 to smithi12223906-22 2024-09-15T03:59:52.135 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 2451 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T03:59:52.135 INFO:tasks.rados.rados.0.smithi122.stdout:3108: expect (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T03:59:52.229 INFO:tasks.rados.rados.0.smithi122.stdout:3109: finishing write tid 1 to smithi12223906-27 2024-09-15T03:59:52.230 INFO:tasks.rados.rados.0.smithi122.stdout:3109: finishing write tid 2 to smithi12223906-27 2024-09-15T03:59:52.230 INFO:tasks.rados.rados.0.smithi122.stdout:3109: finishing write tid 3 to smithi12223906-27 2024-09-15T03:59:52.230 INFO:tasks.rados.rados.0.smithi122.stdout:3109: finishing write tid 4 to smithi12223906-27 2024-09-15T03:59:52.230 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 2455 (ObjNum 1104 snap 361 seq_num 1104) dirty exists 2024-09-15T03:59:52.230 INFO:tasks.rados.rados.0.smithi122.stdout:3109: left oid 27 (ObjNum 1104 snap 361 seq_num 1104) 2024-09-15T03:59:53.120 INFO:tasks.rados.rados.0.smithi122.stdout:3107: done (3 left) 2024-09-15T03:59:53.120 INFO:tasks.rados.rados.0.smithi122.stdout:3108: done (2 left) 2024-09-15T03:59:53.120 INFO:tasks.rados.rados.0.smithi122.stdout:3109: done (1 left) 2024-09-15T03:59:53.121 INFO:tasks.rados.rados.0.smithi122.stdout:3110: done (0 left) 2024-09-15T03:59:53.121 INFO:tasks.rados.rados.0.smithi122.stdout:3111: snap_remove snap 316 2024-09-15T03:59:54.126 INFO:tasks.rados.rados.0.smithi122.stdout:3111: done (0 left) 2024-09-15T03:59:54.126 INFO:tasks.rados.rados.0.smithi122.stdout:3112: delete oid 38 current snap is 362 2024-09-15T03:59:54.135 INFO:tasks.rados.rados.0.smithi122.stdout:3112: done (0 left) 2024-09-15T03:59:54.135 INFO:tasks.rados.rados.0.smithi122.stdout:3113: write (excl) oid 48 current snap is 362 2024-09-15T03:59:54.135 INFO:tasks.rados.rados.0.smithi122.stdout:3113: seq_num 1106 ranges {588542=496579,1620614=676719} 2024-09-15T03:59:54.142 INFO:tasks.rados.rados.0.smithi122.stdout:3113: writing smithi12223906-48 from 588542 to 1085121 tid 1 2024-09-15T03:59:54.150 INFO:tasks.rados.rados.0.smithi122.stdout:3113: writing smithi12223906-48 from 1620614 to 2297333 tid 2 2024-09-15T03:59:54.152 INFO:tasks.rados.rados.0.smithi122.stdout:3114: write (excl) oid 29 current snap is 362 2024-09-15T03:59:54.153 INFO:tasks.rados.rados.0.smithi122.stdout:3113: finishing write tid 1 to smithi12223906-48 2024-09-15T03:59:54.153 INFO:tasks.rados.rados.0.smithi122.stdout:3114: seq_num 1107 ranges {605349=571828,1679096=420377} 2024-09-15T03:59:54.156 INFO:tasks.rados.rados.0.smithi122.stdout:3114: writing smithi12223906-29 from 605349 to 1177177 tid 1 2024-09-15T03:59:54.162 INFO:tasks.rados.rados.0.smithi122.stdout:3114: writing smithi12223906-29 from 1679096 to 2099473 tid 2 2024-09-15T03:59:54.162 INFO:tasks.rados.rados.0.smithi122.stdout:3115: rollback oid 50 current snap is 362 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 361 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3113: finishing write tid 2 to smithi12223906-48 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3113: finishing write tid 3 to smithi12223906-48 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3113: finishing write tid 4 to smithi12223906-48 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 1238 (ObjNum 1106 snap 362 seq_num 1106) dirty exists 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3113: left oid 48 (ObjNum 1106 snap 362 seq_num 1106) 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3113: done (2 left) 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3116: copy_from oid 15 from oid 20 current snap is 362 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3117: rollback oid 26 current snap is 362 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 324 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3118: read oid 9 snap -1 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3118: expect deleted 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3119: copy_from oid 16 from oid 28 current snap is 362 2024-09-15T03:59:54.163 INFO:tasks.rados.rados.0.smithi122.stdout:3120: delete oid 48 current snap is 362 2024-09-15T03:59:54.164 INFO:tasks.rados.rados.0.smithi122.stdout:3116: finishing copy_from to smithi12223906-15 2024-09-15T03:59:54.164 INFO:tasks.rados.rados.0.smithi122.stdout:3116: got expected ENOENT (src dne) 2024-09-15T03:59:54.167 INFO:tasks.rados.rados.0.smithi122.stdout:3116: finishing copy_from racing read to smithi12223906-15 2024-09-15T03:59:54.167 INFO:tasks.rados.rados.0.smithi122.stdout:3116: done (6 left) 2024-09-15T03:59:54.167 INFO:tasks.rados.rados.0.smithi122.stdout:3118: done (5 left) 2024-09-15T03:59:54.167 INFO:tasks.rados.rados.0.smithi122.stdout:3120: done (4 left) 2024-09-15T03:59:54.167 INFO:tasks.rados.rados.0.smithi122.stdout:3121: delete oid 8 current snap is 362 2024-09-15T03:59:54.169 INFO:tasks.rados.rados.0.smithi122.stdout:3114: finishing write tid 1 to smithi12223906-29 2024-09-15T03:59:54.169 INFO:tasks.rados.rados.0.smithi122.stdout:3115: finishing rollback tid 1 to smithi12223906-50 2024-09-15T03:59:54.169 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2402 (ObjNum 129 snap 0 seq_num 1843188208) dirty dne 2024-09-15T03:59:54.170 INFO:tasks.rados.rados.0.smithi122.stdout:3114: finishing write tid 2 to smithi12223906-29 2024-09-15T03:59:54.170 INFO:tasks.rados.rados.0.smithi122.stdout:3114: finishing write tid 3 to smithi12223906-29 2024-09-15T03:59:54.170 INFO:tasks.rados.rados.0.smithi122.stdout:3114: finishing write tid 4 to smithi12223906-29 2024-09-15T03:59:54.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 1601 (ObjNum 1107 snap 362 seq_num 1107) dirty exists 2024-09-15T03:59:54.171 INFO:tasks.rados.rados.0.smithi122.stdout:3114: left oid 29 (ObjNum 1107 snap 362 seq_num 1107) 2024-09-15T03:59:54.172 INFO:tasks.rados.rados.0.smithi122.stdout:3117: finishing rollback tid 0 to smithi12223906-26 2024-09-15T03:59:54.172 INFO:tasks.rados.rados.0.smithi122.stdout:3117: finishing rollback tid 1 to smithi12223906-26 2024-09-15T03:59:54.172 INFO:tasks.rados.rados.0.smithi122.stdout:3117: finishing rollback tid 2 to smithi12223906-26 2024-09-15T03:59:54.172 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1901 (ObjNum 984 snap 311 seq_num 984) dirty exists 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:3114: done (4 left) 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:3115: done (3 left) 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:3117: done (2 left) 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:3121: done (1 left) 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:3122: rollback oid 22 current snap is 362 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 22 to 360 2024-09-15T03:59:54.173 INFO:tasks.rados.rados.0.smithi122.stdout:3123: delete oid 47 current snap is 362 2024-09-15T03:59:54.179 INFO:tasks.rados.rados.0.smithi122.stdout:3119: finishing copy_from racing read to smithi12223906-16 2024-09-15T03:59:54.180 INFO:tasks.rados.rados.0.smithi122.stdout:3123: done (2 left) 2024-09-15T03:59:54.180 INFO:tasks.rados.rados.0.smithi122.stdout:3124: snap_remove snap 317 2024-09-15T03:59:55.138 INFO:tasks.rados.rados.0.smithi122.stdout:3124: done (2 left) 2024-09-15T03:59:55.138 INFO:tasks.rados.rados.0.smithi122.stdout:3125: write (excl) oid 21 current snap is 362 2024-09-15T03:59:55.138 INFO:tasks.rados.rados.0.smithi122.stdout:3122: finishing rollback tid 0 to smithi12223906-22 2024-09-15T03:59:55.138 INFO:tasks.rados.rados.0.smithi122.stdout:3125: seq_num 1111 ranges {672577=589177,1913024=427715} 2024-09-15T03:59:55.151 INFO:tasks.rados.rados.0.smithi122.stdout:3125: writing smithi12223906-21 from 672577 to 1261754 tid 1 2024-09-15T03:59:55.158 INFO:tasks.rados.rados.0.smithi122.stdout:3125: writing smithi12223906-21 from 1913024 to 2340739 tid 2 2024-09-15T03:59:55.158 INFO:tasks.rados.rados.0.smithi122.stdout:3122: finishing rollback tid 1 to smithi12223906-22 2024-09-15T03:59:55.159 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 2469 (ObjNum 49 snap 0 seq_num 1817663720) dirty dne 2024-09-15T03:59:55.159 INFO:tasks.rados.rados.0.smithi122.stdout:3119: finishing copy_from to smithi12223906-16 2024-09-15T03:59:55.159 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1598 (ObjNum 1039 snap 330 seq_num 1039) dirty exists 2024-09-15T03:59:55.159 INFO:tasks.rados.rados.0.smithi122.stdout:3119: done (2 left) 2024-09-15T03:59:55.159 INFO:tasks.rados.rados.0.smithi122.stdout:3122: done (1 left) 2024-09-15T03:59:55.159 INFO:tasks.rados.rados.0.smithi122.stdout:3126: snap_remove snap 322 2024-09-15T03:59:56.108 INFO:tasks.rados.rados.0.smithi122.stdout:3126: done (1 left) 2024-09-15T03:59:56.108 INFO:tasks.rados.rados.0.smithi122.stdout:3127: write (excl) oid 44 current snap is 362 2024-09-15T03:59:56.108 INFO:tasks.rados.rados.0.smithi122.stdout:3125: finishing write tid 1 to smithi12223906-21 2024-09-15T03:59:56.108 INFO:tasks.rados.rados.0.smithi122.stdout:3125: finishing write tid 2 to smithi12223906-21 2024-09-15T03:59:56.108 INFO:tasks.rados.rados.0.smithi122.stdout:3125: finishing write tid 3 to smithi12223906-21 2024-09-15T03:59:56.108 INFO:tasks.rados.rados.0.smithi122.stdout:3127: seq_num 1112 ranges {689384=664426,1971506=687726} 2024-09-15T03:59:56.118 INFO:tasks.rados.rados.0.smithi122.stdout:3127: writing smithi12223906-44 from 689384 to 1353810 tid 1 2024-09-15T03:59:56.126 INFO:tasks.rados.rados.0.smithi122.stdout:3127: writing smithi12223906-44 from 1971506 to 2659232 tid 2 2024-09-15T03:59:56.126 INFO:tasks.rados.rados.0.smithi122.stdout:3128: snap_create 2024-09-15T03:59:56.126 INFO:tasks.rados.rados.0.smithi122.stdout:3125: finishing write tid 4 to smithi12223906-21 2024-09-15T03:59:56.126 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1249 (ObjNum 1111 snap 362 seq_num 1111) dirty exists 2024-09-15T03:59:56.126 INFO:tasks.rados.rados.0.smithi122.stdout:3125: left oid 21 (ObjNum 1111 snap 362 seq_num 1111) 2024-09-15T03:59:56.149 INFO:tasks.rados.rados.0.smithi122.stdout:3127: finishing write tid 1 to smithi12223906-44 2024-09-15T03:59:56.149 INFO:tasks.rados.rados.0.smithi122.stdout:3127: finishing write tid 2 to smithi12223906-44 2024-09-15T03:59:56.149 INFO:tasks.rados.rados.0.smithi122.stdout:3127: finishing write tid 3 to smithi12223906-44 2024-09-15T03:59:56.150 INFO:tasks.rados.rados.0.smithi122.stdout:3127: finishing write tid 4 to smithi12223906-44 2024-09-15T03:59:56.150 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 2491 (ObjNum 1112 snap 362 seq_num 1112) dirty exists 2024-09-15T03:59:56.150 INFO:tasks.rados.rados.0.smithi122.stdout:3127: left oid 44 (ObjNum 1112 snap 362 seq_num 1112) 2024-09-15T03:59:57.126 INFO:tasks.rados.rados.0.smithi122.stdout:3125: done (2 left) 2024-09-15T03:59:57.126 INFO:tasks.rados.rados.0.smithi122.stdout:3127: done (1 left) 2024-09-15T03:59:57.126 INFO:tasks.rados.rados.0.smithi122.stdout:3128: done (0 left) 2024-09-15T03:59:57.126 INFO:tasks.rados.rados.0.smithi122.stdout:3129: delete oid 40 current snap is 363 2024-09-15T03:59:57.133 INFO:tasks.rados.rados.0.smithi122.stdout:3129: done (0 left) 2024-09-15T03:59:57.133 INFO:tasks.rados.rados.0.smithi122.stdout:3130: read oid 44 snap -1 2024-09-15T03:59:57.133 INFO:tasks.rados.rados.0.smithi122.stdout:3130: expect (ObjNum 1112 snap 362 seq_num 1112) 2024-09-15T03:59:57.133 INFO:tasks.rados.rados.0.smithi122.stdout:3131: snap_create 2024-09-15T03:59:57.144 INFO:tasks.rados.rados.0.smithi122.stdout:3130: expect (ObjNum 1112 snap 362 seq_num 1112) 2024-09-15T03:59:58.130 INFO:tasks.rados.rados.0.smithi122.stdout:3130: done (1 left) 2024-09-15T03:59:58.130 INFO:tasks.rados.rados.0.smithi122.stdout:3131: done (0 left) 2024-09-15T03:59:58.130 INFO:tasks.rados.rados.0.smithi122.stdout:3132: rollback oid 42 current snap is 364 2024-09-15T03:59:58.131 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 348 2024-09-15T03:59:58.131 INFO:tasks.rados.rados.0.smithi122.stdout:3133: snap_create 2024-09-15T03:59:58.134 INFO:tasks.rados.rados.0.smithi122.stdout:3132: finishing rollback tid 1 to smithi12223906-42 2024-09-15T03:59:58.134 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 2391 (ObjNum 257 snap 0 seq_num 1842754864) dirty dne 2024-09-15T03:59:59.131 INFO:tasks.rados.rados.0.smithi122.stdout:3132: done (1 left) 2024-09-15T03:59:59.131 INFO:tasks.rados.rados.0.smithi122.stdout:3133: done (0 left) 2024-09-15T03:59:59.131 INFO:tasks.rados.rados.0.smithi122.stdout:3134: snap_remove snap 357 2024-09-15T04:00:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:3134: done (0 left) 2024-09-15T04:00:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:3135: copy_from oid 41 from oid 48 current snap is 365 2024-09-15T04:00:00.134 INFO:tasks.rados.rados.0.smithi122.stdout:3136: delete oid 43 current snap is 365 2024-09-15T04:00:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:3136: done (1 left) 2024-09-15T04:00:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:3137: rollback oid 28 current snap is 365 2024-09-15T04:00:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 28 to 364 2024-09-15T04:00:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:3138: copy_from oid 24 from oid 33 current snap is 365 2024-09-15T04:00:00.145 INFO:tasks.rados.rados.0.smithi122.stdout:3139: delete oid 29 current snap is 365 2024-09-15T04:00:00.152 INFO:tasks.rados.rados.0.smithi122.stdout:3137: finishing rollback tid 0 to smithi12223906-28 2024-09-15T04:00:00.152 INFO:tasks.rados.rados.0.smithi122.stdout:3137: finishing rollback tid 1 to smithi12223906-28 2024-09-15T04:00:00.152 INFO:tasks.rados.rados.0.smithi122.stdout:3137: finishing rollback tid 2 to smithi12223906-28 2024-09-15T04:00:00.152 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 1604 (ObjNum 1039 snap 330 seq_num 1039) dirty exists 2024-09-15T04:00:00.155 INFO:tasks.rados.rados.0.smithi122.stdout:3135: finishing copy_from to smithi12223906-41 2024-09-15T04:00:00.155 INFO:tasks.rados.rados.0.smithi122.stdout:3135: got expected ENOENT (src dne) 2024-09-15T04:00:00.155 INFO:tasks.rados.rados.0.smithi122.stdout:3135: finishing copy_from racing read to smithi12223906-41 2024-09-15T04:00:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:3135: done (3 left) 2024-09-15T04:00:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:3137: done (2 left) 2024-09-15T04:00:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:3139: done (1 left) 2024-09-15T04:00:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:3140: copy_from oid 39 from oid 43 current snap is 365 2024-09-15T04:00:00.160 INFO:tasks.rados.rados.0.smithi122.stdout:3141: snap_remove snap 362 2024-09-15T04:00:01.108 INFO:tasks.rados.rados.0.smithi122.stdout:3141: done (2 left) 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:3142: snap_create 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:3138: finishing copy_from racing read to smithi12223906-24 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:3140: finishing copy_from to smithi12223906-39 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:3140: got expected ENOENT (src dne) 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:3140: finishing copy_from racing read to smithi12223906-39 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:3138: finishing copy_from to smithi12223906-24 2024-09-15T04:00:01.109 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 2400 (ObjNum 1085 snap 355 seq_num 1085) dirty exists 2024-09-15T04:00:02.114 INFO:tasks.rados.rados.0.smithi122.stdout:3138: done (2 left) 2024-09-15T04:00:02.114 INFO:tasks.rados.rados.0.smithi122.stdout:3140: done (1 left) 2024-09-15T04:00:02.114 INFO:tasks.rados.rados.0.smithi122.stdout:3142: done (0 left) 2024-09-15T04:00:02.114 INFO:tasks.rados.rados.0.smithi122.stdout:3143: delete oid 22 current snap is 366 2024-09-15T04:00:02.119 INFO:tasks.rados.rados.0.smithi122.stdout:3143: done (0 left) 2024-09-15T04:00:02.119 INFO:tasks.rados.rados.0.smithi122.stdout:3144: snap_create 2024-09-15T04:00:03.130 INFO:tasks.rados.rados.0.smithi122.stdout:3144: done (0 left) 2024-09-15T04:00:03.130 INFO:tasks.rados.rados.0.smithi122.stdout:3145: snap_remove snap 348 2024-09-15T04:00:04.141 INFO:tasks.rados.rados.0.smithi122.stdout:3145: done (0 left) 2024-09-15T04:00:04.141 INFO:tasks.rados.rados.0.smithi122.stdout:3146: copy_from oid 45 from oid 32 current snap is 367 2024-09-15T04:00:04.141 INFO:tasks.rados.rados.0.smithi122.stdout:3147: copy_from oid 49 from oid 40 current snap is 367 2024-09-15T04:00:04.141 INFO:tasks.rados.rados.0.smithi122.stdout:3148: rollback oid 26 current snap is 367 2024-09-15T04:00:04.142 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 363 2024-09-15T04:00:04.142 INFO:tasks.rados.rados.0.smithi122.stdout:3149: snap_create 2024-09-15T04:00:04.148 INFO:tasks.rados.rados.0.smithi122.stdout:3147: finishing copy_from to smithi12223906-49 2024-09-15T04:00:04.148 INFO:tasks.rados.rados.0.smithi122.stdout:3147: got expected ENOENT (src dne) 2024-09-15T04:00:04.148 INFO:tasks.rados.rados.0.smithi122.stdout:3147: finishing copy_from racing read to smithi12223906-49 2024-09-15T04:00:04.166 INFO:tasks.rados.rados.0.smithi122.stdout:3148: finishing rollback tid 0 to smithi12223906-26 2024-09-15T04:00:04.166 INFO:tasks.rados.rados.0.smithi122.stdout:3148: finishing rollback tid 1 to smithi12223906-26 2024-09-15T04:00:04.171 INFO:tasks.rados.rados.0.smithi122.stdout:3148: finishing rollback tid 2 to smithi12223906-26 2024-09-15T04:00:04.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1905 (ObjNum 984 snap 311 seq_num 984) dirty exists 2024-09-15T04:00:04.173 INFO:tasks.rados.rados.0.smithi122.stdout:3146: finishing copy_from racing read to smithi12223906-45 2024-09-15T04:00:04.173 INFO:tasks.rados.rados.0.smithi122.stdout:3146: finishing copy_from to smithi12223906-45 2024-09-15T04:00:04.173 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 2110 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:00:05.151 INFO:tasks.rados.rados.0.smithi122.stdout:3146: done (3 left) 2024-09-15T04:00:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:3147: done (2 left) 2024-09-15T04:00:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:3148: done (1 left) 2024-09-15T04:00:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:3149: done (0 left) 2024-09-15T04:00:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:3150: copy_from oid 29 from oid 33 current snap is 368 2024-09-15T04:00:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:3151: write (excl) oid 1 current snap is 368 2024-09-15T04:00:05.152 INFO:tasks.rados.rados.0.smithi122.stdout:3151: seq_num 1117 ranges {773419=640671,2147563=438722} 2024-09-15T04:00:05.162 INFO:tasks.rados.rados.0.smithi122.stdout:3151: writing smithi12223906-1 from 773419 to 1414090 tid 1 2024-09-15T04:00:05.170 INFO:tasks.rados.rados.0.smithi122.stdout:3151: writing smithi12223906-1 from 2147563 to 2586285 tid 2 2024-09-15T04:00:05.170 INFO:tasks.rados.rados.0.smithi122.stdout:3152: write (excl) oid 18 current snap is 368 2024-09-15T04:00:05.170 INFO:tasks.rados.rados.0.smithi122.stdout:3150: finishing copy_from racing read to smithi12223906-29 2024-09-15T04:00:05.170 INFO:tasks.rados.rados.0.smithi122.stdout:3152: seq_num 1118 ranges {790226=432273,1806045=582380} 2024-09-15T04:00:05.176 INFO:tasks.rados.rados.0.smithi122.stdout:3152: writing smithi12223906-18 from 790226 to 1222499 tid 1 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3152: writing smithi12223906-18 from 1806045 to 2388425 tid 2 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3153: copy_from oid 32 from oid 48 current snap is 368 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3150: finishing copy_from to smithi12223906-29 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 2400 (ObjNum 1085 snap 355 seq_num 1085) dirty exists 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3150: done (3 left) 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3154: read oid 17 snap -1 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3154: expect (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3155: write (excl) oid 50 current snap is 368 2024-09-15T04:00:05.184 INFO:tasks.rados.rados.0.smithi122.stdout:3155: seq_num 1119 ranges {407033=507522,1464527=442391,2345976=461057} 2024-09-15T04:00:05.188 INFO:tasks.rados.rados.0.smithi122.stdout:3155: writing smithi12223906-50 from 407033 to 914555 tid 1 2024-09-15T04:00:05.194 INFO:tasks.rados.rados.0.smithi122.stdout:3155: writing smithi12223906-50 from 1464527 to 1906918 tid 2 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3155: writing smithi12223906-50 from 2345976 to 2807033 tid 3 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3156: copy_from oid 34 from oid 28 current snap is 368 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3151: finishing write tid 1 to smithi12223906-1 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3151: finishing write tid 2 to smithi12223906-1 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3151: finishing write tid 3 to smithi12223906-1 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3151: finishing write tid 4 to smithi12223906-1 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1899 (ObjNum 1117 snap 368 seq_num 1117) dirty exists 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3151: left oid 1 (ObjNum 1117 snap 368 seq_num 1117) 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3151: done (5 left) 2024-09-15T04:00:05.200 INFO:tasks.rados.rados.0.smithi122.stdout:3157: read oid 15 snap -1 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3157: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3158: read oid 5 snap -1 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3158: expect (ObjNum 1103 snap 360 seq_num 1103) 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3159: read oid 26 snap -1 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3159: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3160: delete oid 27 current snap is 368 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3152: finishing write tid 1 to smithi12223906-18 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3152: finishing write tid 2 to smithi12223906-18 2024-09-15T04:00:05.201 INFO:tasks.rados.rados.0.smithi122.stdout:3154: expect (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T04:00:05.319 INFO:tasks.rados.rados.0.smithi122.stdout:3152: finishing write tid 3 to smithi12223906-18 2024-09-15T04:00:05.319 INFO:tasks.rados.rados.0.smithi122.stdout:3152: finishing write tid 4 to smithi12223906-18 2024-09-15T04:00:05.319 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2506 (ObjNum 1118 snap 368 seq_num 1118) dirty exists 2024-09-15T04:00:05.320 INFO:tasks.rados.rados.0.smithi122.stdout:3152: left oid 18 (ObjNum 1118 snap 368 seq_num 1118) 2024-09-15T04:00:05.320 INFO:tasks.rados.rados.0.smithi122.stdout:3153: finishing copy_from to smithi12223906-32 2024-09-15T04:00:05.320 INFO:tasks.rados.rados.0.smithi122.stdout:3153: got expected ENOENT (src dne) 2024-09-15T04:00:05.320 INFO:tasks.rados.rados.0.smithi122.stdout:3153: finishing copy_from racing read to smithi12223906-32 2024-09-15T04:00:05.320 INFO:tasks.rados.rados.0.smithi122.stdout:3155: finishing write tid 1 to smithi12223906-50 2024-09-15T04:00:05.320 INFO:tasks.rados.rados.0.smithi122.stdout:3157: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T04:00:05.614 INFO:tasks.rados.rados.0.smithi122.stdout:3159: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:00:05.790 INFO:tasks.rados.rados.0.smithi122.stdout:3155: finishing write tid 2 to smithi12223906-50 2024-09-15T04:00:05.790 INFO:tasks.rados.rados.0.smithi122.stdout:3155: finishing write tid 3 to smithi12223906-50 2024-09-15T04:00:05.790 INFO:tasks.rados.rados.0.smithi122.stdout:3155: finishing write tid 4 to smithi12223906-50 2024-09-15T04:00:05.790 INFO:tasks.rados.rados.0.smithi122.stdout:3155: finishing write tid 5 to smithi12223906-50 2024-09-15T04:00:05.790 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2510 (ObjNum 1119 snap 368 seq_num 1119) dirty exists 2024-09-15T04:00:05.790 INFO:tasks.rados.rados.0.smithi122.stdout:3155: left oid 50 (ObjNum 1119 snap 368 seq_num 1119) 2024-09-15T04:00:05.791 INFO:tasks.rados.rados.0.smithi122.stdout:3158: expect (ObjNum 1103 snap 360 seq_num 1103) 2024-09-15T04:00:05.866 INFO:tasks.rados.rados.0.smithi122.stdout:3156: finishing copy_from racing read to smithi12223906-34 2024-09-15T04:00:05.866 INFO:tasks.rados.rados.0.smithi122.stdout:3156: finishing copy_from to smithi12223906-34 2024-09-15T04:00:05.866 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1604 (ObjNum 1039 snap 330 seq_num 1039) dirty exists 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3152: done (8 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3153: done (7 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3154: done (6 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3155: done (5 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3156: done (4 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3157: done (3 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3158: done (2 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3159: done (1 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3160: done (0 left) 2024-09-15T04:00:05.867 INFO:tasks.rados.rados.0.smithi122.stdout:3161: snap_create 2024-09-15T04:00:06.109 INFO:tasks.rados.rados.0.smithi122.stdout:3161: done (0 left) 2024-09-15T04:00:06.109 INFO:tasks.rados.rados.0.smithi122.stdout:3162: snap_create 2024-09-15T04:00:07.113 INFO:tasks.rados.rados.0.smithi122.stdout:3162: done (0 left) 2024-09-15T04:00:07.113 INFO:tasks.rados.rados.0.smithi122.stdout:3163: read oid 50 snap 349 2024-09-15T04:00:07.113 INFO:tasks.rados.rados.0.smithi122.stdout:3163: expect (ObjNum 950 snap 300 seq_num 950) 2024-09-15T04:00:07.113 INFO:tasks.rados.rados.0.smithi122.stdout:3164: write oid 36 current snap is 370 2024-09-15T04:00:07.113 INFO:tasks.rados.rados.0.smithi122.stdout:3164: seq_num 1121 ranges {440647=658020,1581491=446060,2800822=39825} 2024-09-15T04:00:07.120 INFO:tasks.rados.rados.0.smithi122.stdout:3164: writing smithi12223906-36 from 440647 to 1098667 tid 1 2024-09-15T04:00:07.127 INFO:tasks.rados.rados.0.smithi122.stdout:3164: writing smithi12223906-36 from 1581491 to 2027551 tid 2 2024-09-15T04:00:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:3164: writing smithi12223906-36 from 2800822 to 2840647 tid 3 2024-09-15T04:00:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:3165: copy_from oid 30 from oid 37 current snap is 370 2024-09-15T04:00:07.132 INFO:tasks.rados.rados.0.smithi122.stdout:3163: expect (ObjNum 950 snap 300 seq_num 950) 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3164: finishing write tid 1 to smithi12223906-36 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3164: finishing write tid 2 to smithi12223906-36 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3164: finishing write tid 3 to smithi12223906-36 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3164: finishing write tid 4 to smithi12223906-36 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3164: finishing write tid 5 to smithi12223906-36 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 2572 (ObjNum 1121 snap 370 seq_num 1121) dirty exists 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3164: left oid 36 (ObjNum 1121 snap 370 seq_num 1121) 2024-09-15T04:00:07.361 INFO:tasks.rados.rados.0.smithi122.stdout:3165: finishing copy_from racing read to smithi12223906-30 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:3165: finishing copy_from to smithi12223906-30 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 1178 (ObjNum 1066 snap 345 seq_num 1066) dirty exists 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:3163: done (2 left) 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:3164: done (1 left) 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:3165: done (0 left) 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:3166: write oid 33 current snap is 370 2024-09-15T04:00:07.362 INFO:tasks.rados.rados.0.smithi122.stdout:3166: seq_num 1122 ranges {457454=733269,1923620=589718} 2024-09-15T04:00:07.374 INFO:tasks.rados.rados.0.smithi122.stdout:3166: writing smithi12223906-33 from 457454 to 1190723 tid 1 2024-09-15T04:00:07.381 INFO:tasks.rados.rados.0.smithi122.stdout:3166: writing smithi12223906-33 from 1923620 to 2513338 tid 2 2024-09-15T04:00:07.381 INFO:tasks.rados.rados.0.smithi122.stdout:3167: write (excl) oid 21 current snap is 370 2024-09-15T04:00:07.381 INFO:tasks.rados.rados.0.smithi122.stdout:3167: seq_num 1123 ranges {474261=408518,1582102=449729,2739315=134946} 2024-09-15T04:00:07.386 INFO:tasks.rados.rados.0.smithi122.stdout:3167: writing smithi12223906-21 from 474261 to 882779 tid 1 2024-09-15T04:00:07.392 INFO:tasks.rados.rados.0.smithi122.stdout:3167: writing smithi12223906-21 from 1582102 to 2031831 tid 2 2024-09-15T04:00:07.397 INFO:tasks.rados.rados.0.smithi122.stdout:3167: writing smithi12223906-21 from 2739315 to 2874261 tid 3 2024-09-15T04:00:07.397 INFO:tasks.rados.rados.0.smithi122.stdout:3168: snap_create 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:3166: finishing write tid 1 to smithi12223906-33 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:3166: finishing write tid 2 to smithi12223906-33 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:3166: finishing write tid 3 to smithi12223906-33 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:3166: finishing write tid 4 to smithi12223906-33 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2516 (ObjNum 1122 snap 370 seq_num 1122) dirty exists 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:3166: left oid 33 (ObjNum 1122 snap 370 seq_num 1122) 2024-09-15T04:00:07.398 INFO:tasks.rados.rados.0.smithi122.stdout:3167: finishing write tid 1 to smithi12223906-21 2024-09-15T04:00:07.400 INFO:tasks.rados.rados.0.smithi122.stdout:3167: finishing write tid 2 to smithi12223906-21 2024-09-15T04:00:07.400 INFO:tasks.rados.rados.0.smithi122.stdout:3167: finishing write tid 3 to smithi12223906-21 2024-09-15T04:00:07.402 INFO:tasks.rados.rados.0.smithi122.stdout:3167: finishing write tid 4 to smithi12223906-21 2024-09-15T04:00:07.403 INFO:tasks.rados.rados.0.smithi122.stdout:3167: finishing write tid 5 to smithi12223906-21 2024-09-15T04:00:07.403 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 1268 (ObjNum 1123 snap 370 seq_num 1123) dirty exists 2024-09-15T04:00:07.403 INFO:tasks.rados.rados.0.smithi122.stdout:3167: left oid 21 (ObjNum 1123 snap 370 seq_num 1123) 2024-09-15T04:00:08.121 INFO:tasks.rados.rados.0.smithi122.stdout:3166: done (2 left) 2024-09-15T04:00:08.121 INFO:tasks.rados.rados.0.smithi122.stdout:3167: done (1 left) 2024-09-15T04:00:08.121 INFO:tasks.rados.rados.0.smithi122.stdout:3168: done (0 left) 2024-09-15T04:00:08.121 INFO:tasks.rados.rados.0.smithi122.stdout:3169: copy_from oid 4 from oid 30 current snap is 371 2024-09-15T04:00:08.121 INFO:tasks.rados.rados.0.smithi122.stdout:3170: write oid 18 current snap is 371 2024-09-15T04:00:08.121 INFO:tasks.rados.rados.0.smithi122.stdout:3170: seq_num 1124 ranges {491068=483767,1640584=593387,2650385=240683} 2024-09-15T04:00:08.127 INFO:tasks.rados.rados.0.smithi122.stdout:3170: writing smithi12223906-18 from 491068 to 974835 tid 1 2024-09-15T04:00:08.137 INFO:tasks.rados.rados.0.smithi122.stdout:3170: writing smithi12223906-18 from 1640584 to 2233971 tid 2 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:3170: writing smithi12223906-18 from 2650385 to 2891068 tid 3 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:3171: read oid 21 snap 369 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:3171: expect (ObjNum 1111 snap 362 seq_num 1111) 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:3172: rollback oid 44 current snap is 371 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 44 to 359 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:3170: finishing write tid 1 to smithi12223906-18 2024-09-15T04:00:08.141 INFO:tasks.rados.rados.0.smithi122.stdout:3173: copy_from oid 26 from oid 22 current snap is 371 2024-09-15T04:00:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:3174: copy_from oid 10 from oid 32 current snap is 371 2024-09-15T04:00:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:3175: write oid 31 current snap is 371 2024-09-15T04:00:08.142 INFO:tasks.rados.rados.0.smithi122.stdout:3175: seq_num 1125 ranges {507875=559016,1699066=737045} 2024-09-15T04:00:08.147 INFO:tasks.rados.rados.0.smithi122.stdout:3175: writing smithi12223906-31 from 507875 to 1066891 tid 1 2024-09-15T04:00:08.156 INFO:tasks.rados.rados.0.smithi122.stdout:3175: writing smithi12223906-31 from 1699066 to 2436111 tid 2 2024-09-15T04:00:08.156 INFO:tasks.rados.rados.0.smithi122.stdout:3176: read oid 36 snap -1 2024-09-15T04:00:08.156 INFO:tasks.rados.rados.0.smithi122.stdout:3176: expect (ObjNum 1121 snap 370 seq_num 1121) 2024-09-15T04:00:08.156 INFO:tasks.rados.rados.0.smithi122.stdout:3177: write oid 27 current snap is 371 2024-09-15T04:00:08.156 INFO:tasks.rados.rados.0.smithi122.stdout:3177: seq_num 1126 ranges {524682=750618,1757548=597056} 2024-09-15T04:00:08.164 INFO:tasks.rados.rados.0.smithi122.stdout:3177: writing smithi12223906-27 from 524682 to 1275300 tid 1 2024-09-15T04:00:08.170 INFO:tasks.rados.rados.0.smithi122.stdout:3177: writing smithi12223906-27 from 1757548 to 2354604 tid 2 2024-09-15T04:00:08.171 INFO:tasks.rados.rados.0.smithi122.stdout:3178: read oid 11 snap -1 2024-09-15T04:00:08.171 INFO:tasks.rados.rados.0.smithi122.stdout:3178: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T04:00:08.171 INFO:tasks.rados.rados.0.smithi122.stdout:3179: write oid 1 current snap is 371 2024-09-15T04:00:08.171 INFO:tasks.rados.rados.0.smithi122.stdout:3169: finishing copy_from racing read to smithi12223906-4 2024-09-15T04:00:08.171 INFO:tasks.rados.rados.0.smithi122.stdout:3179: seq_num 1127 ranges {541489=425867,1416030=740714,2732654=208835} 2024-09-15T04:00:08.176 INFO:tasks.rados.rados.0.smithi122.stdout:3179: writing smithi12223906-1 from 541489 to 967356 tid 1 2024-09-15T04:00:08.183 INFO:tasks.rados.rados.0.smithi122.stdout:3179: writing smithi12223906-1 from 1416030 to 2156744 tid 2 2024-09-15T04:00:08.188 INFO:tasks.rados.rados.0.smithi122.stdout:3179: writing smithi12223906-1 from 2732654 to 2941489 tid 3 2024-09-15T04:00:08.188 INFO:tasks.rados.rados.0.smithi122.stdout:3180: write oid 49 current snap is 371 2024-09-15T04:00:08.188 INFO:tasks.rados.rados.0.smithi122.stdout:3180: seq_num 1128 ranges {558296=501116,1474512=600725,2760077=198219} 2024-09-15T04:00:08.194 INFO:tasks.rados.rados.0.smithi122.stdout:3180: writing smithi12223906-49 from 558296 to 1059412 tid 1 2024-09-15T04:00:08.200 INFO:tasks.rados.rados.0.smithi122.stdout:3180: writing smithi12223906-49 from 1474512 to 2075237 tid 2 2024-09-15T04:00:08.205 INFO:tasks.rados.rados.0.smithi122.stdout:3180: writing smithi12223906-49 from 2760077 to 2958296 tid 3 2024-09-15T04:00:08.205 INFO:tasks.rados.rados.0.smithi122.stdout:3181: write (excl) oid 28 current snap is 371 2024-09-15T04:00:08.205 INFO:tasks.rados.rados.0.smithi122.stdout:3170: finishing write tid 2 to smithi12223906-18 2024-09-15T04:00:08.205 INFO:tasks.rados.rados.0.smithi122.stdout:3181: seq_num 1129 ranges {575103=576365,1932994=744383} 2024-09-15T04:00:08.212 INFO:tasks.rados.rados.0.smithi122.stdout:3181: writing smithi12223906-28 from 575103 to 1151468 tid 1 2024-09-15T04:00:08.221 INFO:tasks.rados.rados.0.smithi122.stdout:3181: writing smithi12223906-28 from 1932994 to 2677377 tid 2 2024-09-15T04:00:08.221 INFO:tasks.rados.rados.0.smithi122.stdout:3182: copy_from oid 33 from oid 17 current snap is 371 2024-09-15T04:00:08.221 INFO:tasks.rados.rados.0.smithi122.stdout:3170: finishing write tid 3 to smithi12223906-18 2024-09-15T04:00:08.221 INFO:tasks.rados.rados.0.smithi122.stdout:3170: finishing write tid 4 to smithi12223906-18 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3170: finishing write tid 5 to smithi12223906-18 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2521 (ObjNum 1124 snap 371 seq_num 1124) dirty exists 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3170: left oid 18 (ObjNum 1124 snap 371 seq_num 1124) 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3170: done (13 left) 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3183: read oid 12 snap -1 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3183: expect deleted 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3184: write oid 48 current snap is 371 2024-09-15T04:00:08.222 INFO:tasks.rados.rados.0.smithi122.stdout:3171: expect (ObjNum 1111 snap 362 seq_num 1111) 2024-09-15T04:00:08.458 INFO:tasks.rados.rados.0.smithi122.stdout:3173: finishing copy_from to smithi12223906-26 2024-09-15T04:00:08.458 INFO:tasks.rados.rados.0.smithi122.stdout:3173: got expected ENOENT (src dne) 2024-09-15T04:00:08.458 INFO:tasks.rados.rados.0.smithi122.stdout:3173: finishing copy_from racing read to smithi12223906-26 2024-09-15T04:00:08.458 INFO:tasks.rados.rados.0.smithi122.stdout:3172: finishing rollback tid 0 to smithi12223906-44 2024-09-15T04:00:08.458 INFO:tasks.rados.rados.0.smithi122.stdout:3172: finishing rollback tid 1 to smithi12223906-44 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3169: finishing copy_from to smithi12223906-4 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1178 (ObjNum 1066 snap 345 seq_num 1066) dirty exists 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3172: finishing rollback tid 2 to smithi12223906-44 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 2525 (ObjNum 926 snap 292 seq_num 926) dirty exists 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3175: finishing write tid 1 to smithi12223906-31 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3175: finishing write tid 2 to smithi12223906-31 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3175: finishing write tid 3 to smithi12223906-31 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3175: finishing write tid 4 to smithi12223906-31 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1914 (ObjNum 1125 snap 371 seq_num 1125) dirty exists 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3175: left oid 31 (ObjNum 1125 snap 371 seq_num 1125) 2024-09-15T04:00:08.459 INFO:tasks.rados.rados.0.smithi122.stdout:3176: expect (ObjNum 1121 snap 370 seq_num 1121) 2024-09-15T04:00:08.589 INFO:tasks.rados.rados.0.smithi122.stdout:3177: finishing write tid 1 to smithi12223906-27 2024-09-15T04:00:08.589 INFO:tasks.rados.rados.0.smithi122.stdout:3177: finishing write tid 2 to smithi12223906-27 2024-09-15T04:00:08.590 INFO:tasks.rados.rados.0.smithi122.stdout:3177: finishing write tid 3 to smithi12223906-27 2024-09-15T04:00:08.590 INFO:tasks.rados.rados.0.smithi122.stdout:3177: finishing write tid 4 to smithi12223906-27 2024-09-15T04:00:08.590 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 2528 (ObjNum 1126 snap 371 seq_num 1126) dirty exists 2024-09-15T04:00:08.590 INFO:tasks.rados.rados.0.smithi122.stdout:3177: left oid 27 (ObjNum 1126 snap 371 seq_num 1126) 2024-09-15T04:00:08.590 INFO:tasks.rados.rados.0.smithi122.stdout:3178: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T04:00:08.777 INFO:tasks.rados.rados.0.smithi122.stdout:3174: finishing copy_from racing read to smithi12223906-10 2024-09-15T04:00:08.777 INFO:tasks.rados.rados.0.smithi122.stdout:3179: finishing write tid 1 to smithi12223906-1 2024-09-15T04:00:08.777 INFO:tasks.rados.rados.0.smithi122.stdout:3174: finishing copy_from to smithi12223906-10 2024-09-15T04:00:08.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 2110 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:00:08.777 INFO:tasks.rados.rados.0.smithi122.stdout:3179: finishing write tid 2 to smithi12223906-1 2024-09-15T04:00:08.778 INFO:tasks.rados.rados.0.smithi122.stdout:3179: finishing write tid 3 to smithi12223906-1 2024-09-15T04:00:08.778 INFO:tasks.rados.rados.0.smithi122.stdout:3179: finishing write tid 4 to smithi12223906-1 2024-09-15T04:00:08.778 INFO:tasks.rados.rados.0.smithi122.stdout:3179: finishing write tid 5 to smithi12223906-1 2024-09-15T04:00:08.778 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1903 (ObjNum 1127 snap 371 seq_num 1127) dirty exists 2024-09-15T04:00:08.778 INFO:tasks.rados.rados.0.smithi122.stdout:3179: left oid 1 (ObjNum 1127 snap 371 seq_num 1127) 2024-09-15T04:00:08.778 INFO:tasks.rados.rados.0.smithi122.stdout:3184: seq_num 1130 ranges {591910=651614,1875123=604394} 2024-09-15T04:00:08.790 INFO:tasks.rados.rados.0.smithi122.stdout:3184: writing smithi12223906-48 from 591910 to 1243524 tid 1 2024-09-15T04:00:08.799 INFO:tasks.rados.rados.0.smithi122.stdout:3184: writing smithi12223906-48 from 1875123 to 2479517 tid 2 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3169: done (14 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3171: done (13 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3172: done (12 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3173: done (11 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3174: done (10 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3175: done (9 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3176: done (8 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3177: done (7 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3178: done (6 left) 2024-09-15T04:00:08.800 INFO:tasks.rados.rados.0.smithi122.stdout:3179: done (5 left) 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3185: copy_from oid 5 from oid 2 current snap is 371 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3180: finishing write tid 1 to smithi12223906-49 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3180: finishing write tid 2 to smithi12223906-49 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3180: finishing write tid 3 to smithi12223906-49 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3180: finishing write tid 4 to smithi12223906-49 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3180: finishing write tid 5 to smithi12223906-49 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1079 (ObjNum 1128 snap 371 seq_num 1128) dirty exists 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3180: left oid 49 (ObjNum 1128 snap 371 seq_num 1128) 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3181: finishing write tid 1 to smithi12223906-28 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3181: finishing write tid 2 to smithi12223906-28 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3181: finishing write tid 3 to smithi12223906-28 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3181: finishing write tid 4 to smithi12223906-28 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 2403 (ObjNum 1129 snap 371 seq_num 1129) dirty exists 2024-09-15T04:00:08.801 INFO:tasks.rados.rados.0.smithi122.stdout:3181: left oid 28 (ObjNum 1129 snap 371 seq_num 1129) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3182: finishing copy_from racing read to smithi12223906-33 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3182: finishing copy_from to smithi12223906-33 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 1075 (ObjNum 1095 snap 356 seq_num 1095) dirty exists 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3180: done (5 left) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3181: done (4 left) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3182: done (3 left) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3183: done (2 left) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3186: read oid 24 snap -1 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3186: expect (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3187: copy_from oid 4 from oid 22 current snap is 371 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3184: finishing write tid 1 to smithi12223906-48 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3188: read oid 27 snap 354 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3188: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T04:00:08.802 INFO:tasks.rados.rados.0.smithi122.stdout:3189: read oid 32 snap -1 2024-09-15T04:00:08.803 INFO:tasks.rados.rados.0.smithi122.stdout:3189: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:00:08.803 INFO:tasks.rados.rados.0.smithi122.stdout:3190: write (excl) oid 15 current snap is 371 2024-09-15T04:00:08.803 INFO:tasks.rados.rados.0.smithi122.stdout:3190: seq_num 1131 ranges {608717=726863,1933605=748052} 2024-09-15T04:00:08.807 INFO:tasks.rados.rados.0.smithi122.stdout:3190: writing smithi12223906-15 from 608717 to 1335580 tid 1 2024-09-15T04:00:08.816 INFO:tasks.rados.rados.0.smithi122.stdout:3190: writing smithi12223906-15 from 1933605 to 2681657 tid 2 2024-09-15T04:00:08.816 INFO:tasks.rados.rados.0.smithi122.stdout:3191: snap_create 2024-09-15T04:00:08.816 INFO:tasks.rados.rados.0.smithi122.stdout:3184: finishing write tid 2 to smithi12223906-48 2024-09-15T04:00:08.816 INFO:tasks.rados.rados.0.smithi122.stdout:3184: finishing write tid 3 to smithi12223906-48 2024-09-15T04:00:08.816 INFO:tasks.rados.rados.0.smithi122.stdout:3184: finishing write tid 4 to smithi12223906-48 2024-09-15T04:00:08.816 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2113 (ObjNum 1130 snap 371 seq_num 1130) dirty exists 2024-09-15T04:00:08.817 INFO:tasks.rados.rados.0.smithi122.stdout:3184: left oid 48 (ObjNum 1130 snap 371 seq_num 1130) 2024-09-15T04:00:08.817 INFO:tasks.rados.rados.0.smithi122.stdout:3186: expect (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T04:00:08.924 INFO:tasks.rados.rados.0.smithi122.stdout:3187: finishing copy_from to smithi12223906-4 2024-09-15T04:00:08.924 INFO:tasks.rados.rados.0.smithi122.stdout:3187: got expected ENOENT (src dne) 2024-09-15T04:00:08.924 INFO:tasks.rados.rados.0.smithi122.stdout:3187: finishing copy_from racing read to smithi12223906-4 2024-09-15T04:00:08.924 INFO:tasks.rados.rados.0.smithi122.stdout:3188: expect (ObjNum 940 snap 295 seq_num 940) 2024-09-15T04:00:09.045 INFO:tasks.rados.rados.0.smithi122.stdout:3185: finishing copy_from racing read to smithi12223906-5 2024-09-15T04:00:09.045 INFO:tasks.rados.rados.0.smithi122.stdout:3185: finishing copy_from to smithi12223906-5 2024-09-15T04:00:09.045 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 842 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:3190: finishing write tid 1 to smithi12223906-15 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:3190: finishing write tid 2 to smithi12223906-15 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:3190: finishing write tid 3 to smithi12223906-15 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:3190: finishing write tid 4 to smithi12223906-15 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1906 (ObjNum 1131 snap 371 seq_num 1131) dirty exists 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:3190: left oid 15 (ObjNum 1131 snap 371 seq_num 1131) 2024-09-15T04:00:09.046 INFO:tasks.rados.rados.0.smithi122.stdout:3189: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:00:09.220 INFO:tasks.rados.rados.0.smithi122.stdout:3184: done (7 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3185: done (6 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3186: done (5 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3187: done (4 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3188: done (3 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3189: done (2 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3190: done (1 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3191: done (0 left) 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3192: rollback oid 17 current snap is 372 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 343 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3193: read oid 20 snap -1 2024-09-15T04:00:09.221 INFO:tasks.rados.rados.0.smithi122.stdout:3193: expect deleted 2024-09-15T04:00:09.222 INFO:tasks.rados.rados.0.smithi122.stdout:3194: delete oid 44 current snap is 372 2024-09-15T04:00:09.226 INFO:tasks.rados.rados.0.smithi122.stdout:3192: finishing rollback tid 0 to smithi12223906-17 2024-09-15T04:00:09.228 INFO:tasks.rados.rados.0.smithi122.stdout:3192: finishing rollback tid 1 to smithi12223906-17 2024-09-15T04:00:09.228 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1081 (ObjNum 33 snap 0 seq_num 1953066355) dirty dne 2024-09-15T04:00:09.230 INFO:tasks.rados.rados.0.smithi122.stdout:3192: done (2 left) 2024-09-15T04:00:09.230 INFO:tasks.rados.rados.0.smithi122.stdout:3193: done (1 left) 2024-09-15T04:00:09.230 INFO:tasks.rados.rados.0.smithi122.stdout:3194: done (0 left) 2024-09-15T04:00:09.230 INFO:tasks.rados.rados.0.smithi122.stdout:3195: write oid 21 current snap is 372 2024-09-15T04:00:09.230 INFO:tasks.rados.rados.0.smithi122.stdout:3195: seq_num 1133 ranges {642331=593714,1766922=751721} 2024-09-15T04:00:09.239 INFO:tasks.rados.rados.0.smithi122.stdout:3195: writing smithi12223906-21 from 642331 to 1236045 tid 1 2024-09-15T04:00:09.248 INFO:tasks.rados.rados.0.smithi122.stdout:3195: writing smithi12223906-21 from 1766922 to 2518643 tid 2 2024-09-15T04:00:09.248 INFO:tasks.rados.rados.0.smithi122.stdout:3195: finishing write tid 1 to smithi12223906-21 2024-09-15T04:00:09.248 INFO:tasks.rados.rados.0.smithi122.stdout:3196: read oid 28 snap -1 2024-09-15T04:00:09.248 INFO:tasks.rados.rados.0.smithi122.stdout:3196: expect (ObjNum 1129 snap 371 seq_num 1129) 2024-09-15T04:00:09.248 INFO:tasks.rados.rados.0.smithi122.stdout:3197: snap_remove snap 367 2024-09-15T04:00:10.144 INFO:tasks.rados.rados.0.smithi122.stdout:3197: done (2 left) 2024-09-15T04:00:10.144 INFO:tasks.rados.rados.0.smithi122.stdout:3198: delete oid 18 current snap is 372 2024-09-15T04:00:10.144 INFO:tasks.rados.rados.0.smithi122.stdout:3195: finishing write tid 2 to smithi12223906-21 2024-09-15T04:00:10.145 INFO:tasks.rados.rados.0.smithi122.stdout:3195: finishing write tid 3 to smithi12223906-21 2024-09-15T04:00:10.145 INFO:tasks.rados.rados.0.smithi122.stdout:3195: finishing write tid 4 to smithi12223906-21 2024-09-15T04:00:10.145 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 2116 (ObjNum 1133 snap 372 seq_num 1133) dirty exists 2024-09-15T04:00:10.145 INFO:tasks.rados.rados.0.smithi122.stdout:3195: left oid 21 (ObjNum 1133 snap 372 seq_num 1133) 2024-09-15T04:00:10.145 INFO:tasks.rados.rados.0.smithi122.stdout:3196: expect (ObjNum 1129 snap 371 seq_num 1129) 2024-09-15T04:00:10.320 INFO:tasks.rados.rados.0.smithi122.stdout:3195: done (2 left) 2024-09-15T04:00:10.320 INFO:tasks.rados.rados.0.smithi122.stdout:3196: done (1 left) 2024-09-15T04:00:10.320 INFO:tasks.rados.rados.0.smithi122.stdout:3198: done (0 left) 2024-09-15T04:00:10.320 INFO:tasks.rados.rados.0.smithi122.stdout:3199: read oid 6 snap -1 2024-09-15T04:00:10.320 INFO:tasks.rados.rados.0.smithi122.stdout:3199: expect (ObjNum 1102 snap 360 seq_num 1102) 2024-09-15T04:00:10.320 INFO:tasks.rados.rados.0.smithi122.stdout:3200: snap_create 2024-09-15T04:00:10.329 INFO:tasks.rados.rados.0.smithi122.stdout:3199: expect (ObjNum 1102 snap 360 seq_num 1102) 2024-09-15T04:00:11.113 INFO:tasks.rados.rados.0.smithi122.stdout:3199: done (1 left) 2024-09-15T04:00:11.114 INFO:tasks.rados.rados.0.smithi122.stdout:3200: done (0 left) 2024-09-15T04:00:11.114 INFO:tasks.rados.rados.0.smithi122.stdout:3201: delete oid 5 current snap is 373 2024-09-15T04:00:11.133 INFO:tasks.rados.rados.0.smithi122.stdout:3201: done (0 left) 2024-09-15T04:00:11.133 INFO:tasks.rados.rados.0.smithi122.stdout:3202: delete oid 50 current snap is 373 2024-09-15T04:00:11.138 INFO:tasks.rados.rados.0.smithi122.stdout:3202: done (0 left) 2024-09-15T04:00:11.138 INFO:tasks.rados.rados.0.smithi122.stdout:3203: read oid 21 snap 345 2024-09-15T04:00:11.138 INFO:tasks.rados.rados.0.smithi122.stdout:3203: expect (ObjNum 1040 snap 330 seq_num 1040) 2024-09-15T04:00:11.138 INFO:tasks.rados.rados.0.smithi122.stdout:3204: read oid 24 snap 314 2024-09-15T04:00:11.138 INFO:tasks.rados.rados.0.smithi122.stdout:3204: expect deleted 2024-09-15T04:00:11.138 INFO:tasks.rados.rados.0.smithi122.stdout:3205: copy_from oid 38 from oid 43 current snap is 373 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3206: read oid 36 snap -1 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3206: expect (ObjNum 1121 snap 370 seq_num 1121) 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3207: read oid 11 snap 332 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3207: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3208: rollback oid 9 current snap is 373 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 347 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3209: write (excl) oid 15 current snap is 373 2024-09-15T04:00:11.139 INFO:tasks.rados.rados.0.smithi122.stdout:3209: seq_num 1137 ranges {709559=494710,1884497=759059} 2024-09-15T04:00:11.146 INFO:tasks.rados.rados.0.smithi122.stdout:3209: writing smithi12223906-15 from 709559 to 1204269 tid 1 2024-09-15T04:00:11.156 INFO:tasks.rados.rados.0.smithi122.stdout:3209: writing smithi12223906-15 from 1884497 to 2643556 tid 2 2024-09-15T04:00:11.156 INFO:tasks.rados.rados.0.smithi122.stdout:3210: snap_create 2024-09-15T04:00:11.156 INFO:tasks.rados.rados.0.smithi122.stdout:3208: finishing rollback tid 1 to smithi12223906-9 2024-09-15T04:00:11.156 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2387 (ObjNum 33 snap 0 seq_num 1817658064) dirty dne 2024-09-15T04:00:11.156 INFO:tasks.rados.rados.0.smithi122.stdout:3207: expect (ObjNum 944 snap 295 seq_num 944) 2024-09-15T04:00:11.357 INFO:tasks.rados.rados.0.smithi122.stdout:3206: expect (ObjNum 1121 snap 370 seq_num 1121) 2024-09-15T04:00:11.488 INFO:tasks.rados.rados.0.smithi122.stdout:3205: finishing copy_from to smithi12223906-38 2024-09-15T04:00:11.489 INFO:tasks.rados.rados.0.smithi122.stdout:3205: got expected ENOENT (src dne) 2024-09-15T04:00:11.489 INFO:tasks.rados.rados.0.smithi122.stdout:3205: finishing copy_from racing read to smithi12223906-38 2024-09-15T04:00:11.489 INFO:tasks.rados.rados.0.smithi122.stdout:3209: finishing write tid 1 to smithi12223906-15 2024-09-15T04:00:11.489 INFO:tasks.rados.rados.0.smithi122.stdout:3203: expect (ObjNum 1040 snap 330 seq_num 1040) 2024-09-15T04:00:11.709 INFO:tasks.rados.rados.0.smithi122.stdout:3209: finishing write tid 2 to smithi12223906-15 2024-09-15T04:00:11.709 INFO:tasks.rados.rados.0.smithi122.stdout:3209: finishing write tid 3 to smithi12223906-15 2024-09-15T04:00:11.710 INFO:tasks.rados.rados.0.smithi122.stdout:3209: finishing write tid 4 to smithi12223906-15 2024-09-15T04:00:11.710 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1909 (ObjNum 1137 snap 373 seq_num 1137) dirty exists 2024-09-15T04:00:11.710 INFO:tasks.rados.rados.0.smithi122.stdout:3209: left oid 15 (ObjNum 1137 snap 373 seq_num 1137) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3203: done (7 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3204: done (6 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3205: done (5 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3206: done (4 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3207: done (3 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3208: done (2 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3209: done (1 left) 2024-09-15T04:00:12.164 INFO:tasks.rados.rados.0.smithi122.stdout:3210: done (0 left) 2024-09-15T04:00:12.165 INFO:tasks.rados.rados.0.smithi122.stdout:3211: read oid 9 snap 341 2024-09-15T04:00:12.165 INFO:tasks.rados.rados.0.smithi122.stdout:3211: expect (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T04:00:12.165 INFO:tasks.rados.rados.0.smithi122.stdout:3212: copy_from oid 10 from oid 49 current snap is 374 2024-09-15T04:00:12.165 INFO:tasks.rados.rados.0.smithi122.stdout:3213: copy_from oid 50 from oid 16 current snap is 374 2024-09-15T04:00:12.165 INFO:tasks.rados.rados.0.smithi122.stdout:3214: write (excl) oid 4 current snap is 374 2024-09-15T04:00:12.165 INFO:tasks.rados.rados.0.smithi122.stdout:3214: seq_num 1138 ranges {726366=569959,1826626=502717,2801601=324765} 2024-09-15T04:00:12.174 INFO:tasks.rados.rados.0.smithi122.stdout:3214: writing smithi12223906-4 from 726366 to 1296325 tid 1 2024-09-15T04:00:12.180 INFO:tasks.rados.rados.0.smithi122.stdout:3214: writing smithi12223906-4 from 1826626 to 2329343 tid 2 2024-09-15T04:00:12.185 INFO:tasks.rados.rados.0.smithi122.stdout:3214: writing smithi12223906-4 from 2801601 to 3126366 tid 3 2024-09-15T04:00:12.185 INFO:tasks.rados.rados.0.smithi122.stdout:3215: snap_remove snap 303 2024-09-15T04:00:13.177 INFO:tasks.rados.rados.0.smithi122.stdout:3215: done (4 left) 2024-09-15T04:00:13.177 INFO:tasks.rados.rados.0.smithi122.stdout:3216: copy_from oid 1 from oid 25 current snap is 374 2024-09-15T04:00:13.177 INFO:tasks.rados.rados.0.smithi122.stdout:3211: expect (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T04:00:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:3212: finishing copy_from racing read to smithi12223906-10 2024-09-15T04:00:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:3214: finishing write tid 1 to smithi12223906-4 2024-09-15T04:00:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:3214: finishing write tid 2 to smithi12223906-4 2024-09-15T04:00:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:3212: finishing copy_from to smithi12223906-10 2024-09-15T04:00:13.282 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 1079 (ObjNum 1128 snap 371 seq_num 1128) dirty exists 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3213: finishing copy_from racing read to smithi12223906-50 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3214: finishing write tid 3 to smithi12223906-4 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3214: finishing write tid 4 to smithi12223906-4 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3213: finishing copy_from to smithi12223906-50 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1598 (ObjNum 1039 snap 330 seq_num 1039) dirty exists 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3214: finishing write tid 5 to smithi12223906-4 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 1918 (ObjNum 1138 snap 374 seq_num 1138) dirty exists 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3214: left oid 4 (ObjNum 1138 snap 374 seq_num 1138) 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3211: done (4 left) 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3212: done (3 left) 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3213: done (2 left) 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3214: done (1 left) 2024-09-15T04:00:13.283 INFO:tasks.rados.rados.0.smithi122.stdout:3217: read oid 19 snap -1 2024-09-15T04:00:13.284 INFO:tasks.rados.rados.0.smithi122.stdout:3217: expect deleted 2024-09-15T04:00:13.284 INFO:tasks.rados.rados.0.smithi122.stdout:3218: snap_create 2024-09-15T04:00:13.284 INFO:tasks.rados.rados.0.smithi122.stdout:3216: finishing copy_from to smithi12223906-1 2024-09-15T04:00:13.284 INFO:tasks.rados.rados.0.smithi122.stdout:3216: got expected ENOENT (src dne) 2024-09-15T04:00:13.284 INFO:tasks.rados.rados.0.smithi122.stdout:3216: finishing copy_from racing read to smithi12223906-1 2024-09-15T04:00:14.189 INFO:tasks.rados.rados.0.smithi122.stdout:3216: done (2 left) 2024-09-15T04:00:14.189 INFO:tasks.rados.rados.0.smithi122.stdout:3217: done (1 left) 2024-09-15T04:00:14.189 INFO:tasks.rados.rados.0.smithi122.stdout:3218: done (0 left) 2024-09-15T04:00:14.189 INFO:tasks.rados.rados.0.smithi122.stdout:3219: snap_remove snap 340 2024-09-15T04:00:14.932 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T04:00:14.932 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T04:00:14.932 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T04:00:14.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T04:00:14.936 INFO:tasks.ceph.mon.g:Started 2024-09-15T04:00:14.936 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T04:00:14.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:00:15.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient: get_monmap_and_config 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient: monmap: 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:15.046035+0000 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:15.046035+0000 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.051 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.051 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.051 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.051 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 1 Processor -- start 2024-09-15T04:00:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 1 -- start start 2024-09-15T04:00:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient: init 2024-09-15T04:00:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient: _add_conns ranks=[1,7,5,0,8,4,3,6,2] 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient(hunting): picked mon.noname-f con 0x7f00d00634e0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient(hunting): picked mon.noname-i con 0x7f00d0099230 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient(hunting): picked mon.noname-h con 0x7f00d0138dd0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d004ca10 con 0x7f00d00634e0 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d0063dc0 con 0x7f00d0138dd0 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d0063f90 con 0x7f00d0099230 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d6589700 10 monclient(hunting): authenticate will time out at 1759.475547s 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d5d88700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f00d0138dd0 0x7f00d013b1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:43308/0 (socket says 172.21.15.26:43308) 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d4d86700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f00d0099230 0x7f00d0099600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:60050/0 (socket says 172.21.15.26:60050) 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d5d88700 1 -- 172.21.15.26:0/1289708047 learned_addr learned my addr 172.21.15.26:0/1289708047 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.042+0000 7f00d5587700 1 --1- 172.21.15.26:0/1289708047 >> v1:172.21.15.122:6789/0 conn(0x7f00d00634e0 0x7f00d0132eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:59558/0 (socket says 172.21.15.26:59558) 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661057926 0 0) 0x7f00d0063f90 con 0x7f00d0099230 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): my global_id is 234286 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc003570 con 0x7f00d0099230 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974840071 0 0) 0x7f00d004ca10 con 0x7f00d00634e0 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): my global_id is 204589 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d0063f90 con 0x7f00d00634e0 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178200772 0 0) 0x7f00d0063dc0 con 0x7f00d0138dd0 2024-09-15T04:00:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient(hunting): my global_id is 6080 2024-09-15T04:00:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d004ca10 con 0x7f00d0138dd0 2024-09-15T04:00:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3396415794 0 0) 0x7f00d0063f90 con 0x7f00d00634e0 2024-09-15T04:00:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 >> v1:172.21.15.122:6792/0 conn(0x7f00d0099230 legacy=0x7f00d0099600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 >> v1:172.21.15.122:6791/0 conn(0x7f00d0138dd0 legacy=0x7f00d013b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 monclient: found mon.noname-f 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0097a60 con 0x7f00d00634e0 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00cc001790 con 0x7f00d00634e0 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: dump: 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 5 monclient: authenticate success, global_id 204589 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f00cc001a40 con 0x7f00d00634e0 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 1 -- 172.21.15.26:0/1289708047 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00cc004b70 con 0x7f00d00634e0 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00c7fff700 10 monclient: dump: 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 4 set_mon_vals no callback set 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 10 monclient: shutdown 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 1 -- 172.21.15.26:0/1289708047 >> v1:172.21.15.122:6789/0 conn(0x7f00d00634e0 legacy=0x7f00d0132eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 1 -- 172.21.15.26:0/1289708047 shutdown_connections 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 1 -- 172.21.15.26:0/1289708047 >> 172.21.15.26:0/1289708047 conn(0x7f00d0071860 msgr2=0x7f00d00a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 1 -- 172.21.15.26:0/1289708047 shutdown_connections 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 1 -- 172.21.15.26:0/1289708047 wait complete. 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.046+0000 7f00d6589700 10 monclient: monmap: 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.066 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 Processor -- start 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 -- start start 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: init 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: _add_conns ranks=[0,7,5,6,2,8,3,1,4] 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient(hunting): picked mon.noname-a con 0x7f00d00634e0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient(hunting): picked mon.noname-i con 0x7f00d0099230 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient(hunting): picked mon.noname-h con 0x7f00d0138dd0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d009ec00 con 0x7f00d00634e0 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d009e830 con 0x7f00d0138dd0 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d5587700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f00d00634e0 0x7f00d012ee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48790/0 (socket says 172.21.15.26:48790) 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d5587700 1 -- 172.21.15.26:0/2583207280 learned_addr learned my addr 172.21.15.26:0/2583207280 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d0130d30 con 0x7f00d0099230 2024-09-15T04:00:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient(hunting): authenticate will time out at 1759.481837s 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239899877 0 0) 0x7f00d009ec00 con 0x7f00d00634e0 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): my global_id is 194706 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b4003570 con 0x7f00d00634e0 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875863914 0 0) 0x7f00d009e830 con 0x7f00d0138dd0 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): my global_id is 6089 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d009ec00 con 0x7f00d0138dd0 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104414464 0 0) 0x7f00d0130d30 con 0x7f00d0099230 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): my global_id is 234295 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d009e830 con 0x7f00d0099230 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3625846682 0 0) 0x7f00b4003570 con 0x7f00d00634e0 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00d0130d30 con 0x7f00d00634e0 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00cc001750 con 0x7f00d00634e0 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): dump: 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2479326947 0 0) 0x7f00d0130d30 con 0x7f00d00634e0 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 >> v1:172.21.15.122:6792/0 conn(0x7f00d0099230 legacy=0x7f00d012f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 >> v1:172.21.15.122:6791/0 conn(0x7f00d0138dd0 legacy=0x7f00d014a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 monclient: found mon.a 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d012fc20 con 0x7f00d00634e0 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 5 monclient: authenticate success, global_id 194706 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: _renew_subs 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d0130820 con 0x7f00d00634e0 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: _renew_subs 2024-09-15T04:00:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d01379d0 con 0x7f00d00634e0 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f00cc004e40 con 0x7f00d00634e0 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f00cc0052f0 con 0x7f00d00634e0 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c77fe700 4 set_mon_vals no callback set 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.050+0000 7f00c57fa700 10 monclient: dump: 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f00cc00fcb0 con 0x7f00d00634e0 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00d6589700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00d6589700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00d6589700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b00050b0 con 0x7f00d00634e0 2024-09-15T04:00:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(726..726 src has 435..726) v4 ==== 3911+0+0 (unknown 341894380 0 0) 0x7f00cc044130 con 0x7f00d00634e0 2024-09-15T04:00:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.054+0000 7f00d4d86700 10 monclient: get_auth_request con 0x7f00b40376f0 auth_method 0 2024-09-15T04:00:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.062+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f00cc014fc0 con 0x7f00d00634e0 2024-09-15T04:00:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.062+0000 7f00c57fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.062+0000 7f00c57fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:15.197 INFO:tasks.rados.rados.0.smithi122.stdout:3219: done (0 left) 2024-09-15T04:00:15.197 INFO:tasks.rados.rados.0.smithi122.stdout:3220: read oid 39 snap -1 2024-09-15T04:00:15.197 INFO:tasks.rados.rados.0.smithi122.stdout:3220: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:00:15.198 INFO:tasks.rados.rados.0.smithi122.stdout:3221: delete oid 9 current snap is 375 2024-09-15T04:00:15.205 INFO:tasks.rados.rados.0.smithi122.stdout:3221: done (1 left) 2024-09-15T04:00:15.205 INFO:tasks.rados.rados.0.smithi122.stdout:3222: copy_from oid 38 from oid 31 current snap is 375 2024-09-15T04:00:15.205 INFO:tasks.rados.rados.0.smithi122.stdout:3223: delete oid 15 current snap is 375 2024-09-15T04:00:15.209 INFO:tasks.rados.rados.0.smithi122.stdout:3220: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:00:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.310+0000 7f00d6589700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:00:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.310+0000 7f00d6589700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:00:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.310+0000 7f00d6589700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.310+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f00b00058e0 con 0x7f00d00634e0 2024-09-15T04:00:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00c57fa700 1 -- 172.21.15.26:0/2583207280 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 2939810303) 0x7f00cc0149c0 con 0x7f00d00634e0 2024-09-15T04:00:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00c57fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:00:15.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00c57fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:15.319 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":104,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 >> v1:172.21.15.122:6812/20895 conn(0x7f00b40376f0 legacy=0x7f00b4039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 10 monclient: shutdown 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 >> v1:172.21.15.26:6789/0 conn(0x7f00d00634e0 legacy=0x7f00d012ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 shutdown_connections 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 >> 172.21.15.26:0/2583207280 conn(0x7f00d0071860 msgr2=0x7f00d00669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 shutdown_connections 2024-09-15T04:00:15.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.314+0000 7f00d6589700 1 -- 172.21.15.26:0/2583207280 wait complete. 2024-09-15T04:00:15.333 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T04:00:15.333 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T04:00:15.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:00:15.378 INFO:tasks.rados.rados.0.smithi122.stdout:3222: finishing copy_from racing read to smithi12223906-38 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:3222: finishing copy_from to smithi12223906-38 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1914 (ObjNum 1125 snap 371 seq_num 1125) dirty exists 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:3220: done (2 left) 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:3222: done (1 left) 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:3223: done (0 left) 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:3224: write (excl) oid 47 current snap is 375 2024-09-15T04:00:15.379 INFO:tasks.rados.rados.0.smithi122.stdout:3224: seq_num 1141 ranges {776787=512059,1718425=766397,3000223=176564} 2024-09-15T04:00:15.396 INFO:tasks.rados.rados.0.smithi122.stdout:3224: writing smithi12223906-47 from 776787 to 1288846 tid 1 2024-09-15T04:00:15.403 INFO:tasks.rados.rados.0.smithi122.stdout:3224: writing smithi12223906-47 from 1718425 to 2484822 tid 2 2024-09-15T04:00:15.407 INFO:tasks.rados.rados.0.smithi122.stdout:3224: writing smithi12223906-47 from 3000223 to 3176787 tid 3 2024-09-15T04:00:15.407 INFO:tasks.rados.rados.0.smithi122.stdout:3225: read oid 20 snap -1 2024-09-15T04:00:15.407 INFO:tasks.rados.rados.0.smithi122.stdout:3225: expect deleted 2024-09-15T04:00:15.407 INFO:tasks.rados.rados.0.smithi122.stdout:3226: snap_create 2024-09-15T04:00:15.407 INFO:tasks.rados.rados.0.smithi122.stdout:3224: finishing write tid 1 to smithi12223906-47 2024-09-15T04:00:15.413 INFO:tasks.rados.rados.0.smithi122.stdout:3224: finishing write tid 2 to smithi12223906-47 2024-09-15T04:00:15.413 INFO:tasks.rados.rados.0.smithi122.stdout:3224: finishing write tid 3 to smithi12223906-47 2024-09-15T04:00:15.413 INFO:tasks.rados.rados.0.smithi122.stdout:3224: finishing write tid 4 to smithi12223906-47 2024-09-15T04:00:15.413 INFO:tasks.rados.rados.0.smithi122.stdout:3224: finishing write tid 5 to smithi12223906-47 2024-09-15T04:00:15.413 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 918 (ObjNum 1141 snap 375 seq_num 1141) dirty exists 2024-09-15T04:00:15.413 INFO:tasks.rados.rados.0.smithi122.stdout:3224: left oid 47 (ObjNum 1141 snap 375 seq_num 1141) 2024-09-15T04:00:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.422+0000 7f439e507700 10 monclient: get_monmap_and_config 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.422+0000 7f439e507700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.422+0000 7f439e507700 10 monclient: monmap: 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:15.427414+0000 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:15.427414+0000 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.432 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 Processor -- start 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- start start 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient: init 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient: _add_conns ranks=[2,4,6,1,5,3,8,7,0] 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient(hunting): picked mon.noname-b con 0x7f439809f600 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient(hunting): picked mon.noname-c con 0x7f439809c640 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient(hunting): picked mon.noname-d con 0x7f439809b2c0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439804ca10 con 0x7f439809f600 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439809af10 con 0x7f439809c640 2024-09-15T04:00:15.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439809f9d0 con 0x7f439809b2c0 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient(hunting): authenticate will time out at 1759.857151s 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439cd04700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f439809c640 0x7f439809e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:52530/0 (socket says 172.21.15.26:52530) 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439dd06700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f439809b2c0 0x7f43981375a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54012/0 (socket says 172.21.15.26:54012) 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439cd04700 1 -- 172.21.15.26:0/1254951825 learned_addr learned my addr 172.21.15.26:0/1254951825 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875946544 0 0) 0x7f439804ca10 con 0x7f439809f600 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): my global_id is 204635 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4384003570 con 0x7f439809f600 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892648004 0 0) 0x7f439809af10 con 0x7f439809c640 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): my global_id is 204601 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f439804ca10 con 0x7f439809c640 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597625778 0 0) 0x7f439809f9d0 con 0x7f439809b2c0 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient(hunting): my global_id is 194838 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f439809af10 con 0x7f439809b2c0 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2720255367 0 0) 0x7f4384003570 con 0x7f439809f600 2024-09-15T04:00:15.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 >> v1:172.21.15.26:6792/0 conn(0x7f439809b2c0 legacy=0x7f43981375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 >> v1:172.21.15.26:6791/0 conn(0x7f439809c640 legacy=0x7f439809e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 monclient: found mon.noname-b 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439809ec20 con 0x7f439809f600 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43940017b0 con 0x7f439809f600 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 5 monclient: authenticate success, global_id 204635 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: dump: 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4394001a20 con 0x7f439809f600 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 1 -- 172.21.15.26:0/1254951825 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4394002240 con 0x7f439809f600 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 4 set_mon_vals no callback set 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f438ffff700 10 monclient: dump: 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 10 monclient: shutdown 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- 172.21.15.26:0/1254951825 >> v1:172.21.15.26:6790/0 conn(0x7f439809f600 legacy=0x7f439809de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- 172.21.15.26:0/1254951825 shutdown_connections 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- 172.21.15.26:0/1254951825 >> 172.21.15.26:0/1254951825 conn(0x7f43980a2ea0 msgr2=0x7f439809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- 172.21.15.26:0/1254951825 shutdown_connections 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.426+0000 7f439e507700 1 -- 172.21.15.26:0/1254951825 wait complete. 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: monmap: 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 Processor -- start 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 -- start start 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: init 2024-09-15T04:00:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: _add_conns ranks=[1,3,6,0,4,2,5,7,8] 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient(hunting): picked mon.noname-f con 0x7f439809b2c0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient(hunting): picked mon.noname-g con 0x7f439809c640 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient(hunting): picked mon.noname-d con 0x7f439809f600 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4398132370 con 0x7f439809b2c0 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439812baa0 con 0x7f439809c640 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439812b7c0 con 0x7f439809f600 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient(hunting): authenticate will time out at 1759.861627s 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439dd06700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f439809f600 0x7f43981463c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54014/0 (socket says 172.21.15.26:54014) 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439dd06700 1 -- 172.21.15.26:0/205524722 learned_addr learned my addr 172.21.15.26:0/205524722 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209158950 0 0) 0x7f439812b7c0 con 0x7f439809f600 2024-09-15T04:00:15.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): my global_id is 194847 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437c003570 con 0x7f439809f600 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065854872 0 0) 0x7f4398132370 con 0x7f439809b2c0 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): my global_id is 204598 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f439812b7c0 con 0x7f439809b2c0 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3533223027 0 0) 0x7f437c003570 con 0x7f439809f600 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4398132370 con 0x7f439809f600 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4390001e30 con 0x7f439809f600 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:15.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): dump: 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1304623847 0 0) 0x7f439812b7c0 con 0x7f439809b2c0 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f437c003570 con 0x7f439809b2c0 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4394001ed0 con 0x7f439809b2c0 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient(hunting): dump: 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 802539068 0 0) 0x7f4398132370 con 0x7f439809f600 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.122:6790/0 conn(0x7f439809c640 legacy=0x7f4398142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.122:6789/0 conn(0x7f439809b2c0 legacy=0x7f43980952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 monclient: found mon.d 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439812a860 con 0x7f439809f600 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 5 monclient: authenticate success, global_id 194847 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: _renew_subs 2024-09-15T04:00:15.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439812ca90 con 0x7f439809f600 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: _renew_subs 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f43981294b0 con 0x7f439809f600 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4390004e60 con 0x7f439809f600 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438f7fe700 4 set_mon_vals no callback set 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4390005390 con 0x7f439809f600 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.430+0000 7f438d7fa700 10 monclient: dump: 2024-09-15T04:00:15.455 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.456 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.457 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.457 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.434+0000 7f439e507700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.434+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4378006590 con 0x7f4378002840 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.434+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4390005570 con 0x7f439809f600 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.434+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(727..727 src has 435..727) v4 ==== 3955+0+0 (unknown 132898664 0 0) 0x7f43900442d0 con 0x7f439809f600 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.434+0000 7f439cd04700 10 monclient: get_auth_request con 0x7f437c037ae0 auth_method 0 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259217037 0 0) 0x7f4378006590 con 0x7f4378002840 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient(hunting): my global_id is 194715 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4398132370 con 0x7f4378002840 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 92750455 0 0) 0x7f4398132370 con 0x7f4378002840 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4378006590 con 0x7f4378002840 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f43940025e0 con 0x7f4378002840 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient: dump: 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.461 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3531633267 0 0) 0x7f4378006590 con 0x7f4378002840 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f437800a130 con 0x7f4378002840 2024-09-15T04:00:15.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f43940048a0 con 0x7f4378002840 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.26:6789/0 conn(0x7f4378002840 legacy=0x7f4378004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.438+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.26:6789/0 conn(0x7f4378002840 legacy=0x7f4378004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f439e507700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f4378009640 con 0x7f43780090b0 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234786000 0 0) 0x7f4378009640 con 0x7f43780090b0 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 10 monclient(hunting): my global_id is 194724 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4378006590 con 0x7f43780090b0 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3108116520 0 0) 0x7f4378006590 con 0x7f43780090b0 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4378009640 con 0x7f43780090b0 2024-09-15T04:00:15.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4394003120 con 0x7f43780090b0 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 10 monclient: dump: 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.464 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4093355408 0 0) 0x7f4378009640 con 0x7f43780090b0 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.446+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43780098f0 con 0x7f43780090b0 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 290133343) 0x7f4394004b50 con 0x7f43780090b0 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f438d7fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f438d7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.26:6789/0 conn(0x7f43780090b0 legacy=0x7f4378008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f438d7fa700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.26:6789/0 conn(0x7f43780090b0 legacy=0x7f4378008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:15.466 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:15.467 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.468 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.472 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:15.472 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:15.473 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:15.474 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.475 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.476 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:15.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.478 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:15.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.482 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:15.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.489 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:15.490 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.122:6812/20895 conn(0x7f437c037ae0 legacy=0x7f437c039f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 10 monclient: shutdown 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 >> v1:172.21.15.26:6792/0 conn(0x7f439809f600 legacy=0x7f43981463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439dd06700 1 -- 172.21.15.26:0/205524722 reap_dead start 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439dd06700 1 -- 172.21.15.26:0/205524722 reap_dead start 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 shutdown_connections 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 >> 172.21.15.26:0/205524722 conn(0x7f43980a2ea0 msgr2=0x7f43980a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 shutdown_connections 2024-09-15T04:00:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.450+0000 7f439e507700 1 -- 172.21.15.26:0/205524722 wait complete. 2024-09-15T04:00:15.493 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:15.493 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T04:00:15.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T04:00:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.594+0000 7fb5d5e2a700 10 monclient: get_monmap_and_config 2024-09-15T04:00:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.594+0000 7fb5d5e2a700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.594+0000 7fb5d5e2a700 10 monclient: monmap: 2024-09-15T04:00:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:15.598925+0000 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:15.598925+0000 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.603 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 Processor -- start 2024-09-15T04:00:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- start start 2024-09-15T04:00:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: init 2024-09-15T04:00:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: _add_conns ranks=[0,2,5,4,6,3,8,1,7] 2024-09-15T04:00:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient(hunting): picked mon.noname-a con 0x7fb5d00982f0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient(hunting): picked mon.noname-b con 0x7fb5d009b460 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient(hunting): picked mon.noname-h con 0x7fb5d009b830 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d004ca10 con 0x7fb5d00982f0 2024-09-15T04:00:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d0099590 con 0x7fb5d009b460 2024-09-15T04:00:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d00986c0 con 0x7fb5d009b830 2024-09-15T04:00:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient(hunting): authenticate will time out at 1760.028831s 2024-09-15T04:00:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ceffd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fb5d009b460 0x7fb5d009d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:55274/0 (socket says 172.21.15.26:55274) 2024-09-15T04:00:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ceffd700 1 -- 172.21.15.26:0/606884361 learned_addr learned my addr 172.21.15.26:0/606884361 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5cf7fe700 1 --1- 172.21.15.26:0/606884361 >> v1:172.21.15.26:6789/0 conn(0x7fb5d00982f0 0x7fb5d009c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48818/0 (socket says 172.21.15.26:48818) 2024-09-15T04:00:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684835050 0 0) 0x7fb5d0099590 con 0x7fb5d009b460 2024-09-15T04:00:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): my global_id is 204644 2024-09-15T04:00:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b8003570 con 0x7fb5d009b460 2024-09-15T04:00:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595499760 0 0) 0x7fb5d004ca10 con 0x7fb5d00982f0 2024-09-15T04:00:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): my global_id is 194733 2024-09-15T04:00:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d0099590 con 0x7fb5d00982f0 2024-09-15T04:00:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388792200 0 0) 0x7fb5d00986c0 con 0x7fb5d009b830 2024-09-15T04:00:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient(hunting): my global_id is 6098 2024-09-15T04:00:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d004ca10 con 0x7fb5d009b830 2024-09-15T04:00:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2751243803 0 0) 0x7fb5b8003570 con 0x7fb5d009b460 2024-09-15T04:00:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 >> v1:172.21.15.122:6791/0 conn(0x7fb5d009b830 legacy=0x7fb5d0137820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 >> v1:172.21.15.26:6789/0 conn(0x7fb5d00982f0 legacy=0x7fb5d009c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 monclient: found mon.noname-b 2024-09-15T04:00:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:00:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0099870 con 0x7fb5d009b460 2024-09-15T04:00:15.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 5 monclient: authenticate success, global_id 204644 2024-09-15T04:00:15.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c40017b0 con 0x7fb5d009b460 2024-09-15T04:00:15.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: dump: 2024-09-15T04:00:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.626 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.628 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.628 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5c4001a90 con 0x7fb5d009b460 2024-09-15T04:00:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 4 set_mon_vals no callback set 2024-09-15T04:00:15.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 1 -- 172.21.15.26:0/606884361 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c4004c30 con 0x7fb5d009b460 2024-09-15T04:00:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5ce7fc700 10 monclient: dump: 2024-09-15T04:00:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.630 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.631 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.632 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.633 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: shutdown 2024-09-15T04:00:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/606884361 >> v1:172.21.15.26:6790/0 conn(0x7fb5d009b460 legacy=0x7fb5d009d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/606884361 shutdown_connections 2024-09-15T04:00:15.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/606884361 >> 172.21.15.26:0/606884361 conn(0x7fb5d0063f30 msgr2=0x7fb5d00a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/606884361 shutdown_connections 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/606884361 wait complete. 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.598+0000 7fb5d5e2a700 10 monclient: monmap: 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:15.634 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.635 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 Processor -- start 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 -- start start 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: init 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: _add_conns ranks=[6,1,2,3,5,4,7,0,8] 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient(hunting): picked mon.noname-d con 0x7fb5d00982f0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient(hunting): picked mon.noname-f con 0x7fb5d009b460 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient(hunting): picked mon.noname-b con 0x7fb5d009b830 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d009c110 con 0x7fb5d009b460 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d013e380 con 0x7fb5d009b830 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d009bc40 con 0x7fb5d00982f0 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient(hunting): authenticate will time out at 1760.033109s 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5cf7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fb5d00982f0 0x7fb5d013dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54018/0 (socket says 172.21.15.26:54018) 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5cf7fe700 1 -- 172.21.15.26:0/1325687224 learned_addr learned my addr 172.21.15.26:0/1325687224 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702597601 0 0) 0x7fb5d013e380 con 0x7fb5d009b830 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): my global_id is 204653 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5bc003570 con 0x7fb5d009b830 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687573640 0 0) 0x7fb5d009bc40 con 0x7fb5d00982f0 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): my global_id is 194856 2024-09-15T04:00:15.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d013e380 con 0x7fb5d00982f0 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419710510 0 0) 0x7fb5d009c110 con 0x7fb5d009b460 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): my global_id is 204607 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d009bc40 con 0x7fb5d009b460 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 311315171 0 0) 0x7fb5d013e380 con 0x7fb5d00982f0 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5d009c110 con 0x7fb5d00982f0 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c0001aa0 con 0x7fb5d00982f0 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:15.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): dump: 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.640 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1557910072 0 0) 0x7fb5bc003570 con 0x7fb5d009b830 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5d013e380 con 0x7fb5d009b830 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c8001cc0 con 0x7fb5d009b830 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient(hunting): dump: 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 467751500 0 0) 0x7fb5d009c110 con 0x7fb5d00982f0 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.26:6790/0 conn(0x7fb5d009b830 legacy=0x7fb5d014a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.122:6789/0 conn(0x7fb5d009b460 legacy=0x7fb5d0146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 monclient: found mon.d 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d013f190 con 0x7fb5d00982f0 2024-09-15T04:00:15.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 5 monclient: authenticate success, global_id 194856 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: _renew_subs 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d0122d10 con 0x7fb5d00982f0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: _renew_subs 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d0120a10 con 0x7fb5d00982f0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb5c0004ba0 con 0x7fb5d00982f0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c00050d0 con 0x7fb5d00982f0 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5cdffb700 4 set_mon_vals no callback set 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.602+0000 7fb5b77fe700 10 monclient: dump: 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.645 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb5c00052b0 con 0x7fb5d00982f0 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5d5e2a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb5b0006590 con 0x7fb5b0002840 2024-09-15T04:00:15.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(727..727 src has 435..727) v4 ==== 3955+0+0 (unknown 132898664 0 0) 0x7fb5c0044120 con 0x7fb5d00982f0 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017543345 0 0) 0x7fb5b0006590 con 0x7fb5b0002840 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient(hunting): my global_id is 204662 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d009c110 con 0x7fb5b0002840 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5cffff700 10 monclient: get_auth_request con 0x7fb5bc037cd0 auth_method 0 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 422191924 0 0) 0x7fb5d009c110 con 0x7fb5b0002840 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5b0006590 con 0x7fb5b0002840 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c4001770 con 0x7fb5b0002840 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:15.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient: dump: 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2682738918 0 0) 0x7fb5b0006590 con 0x7fb5b0002840 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5b000a130 con 0x7fb5b0002840 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb5c40048a0 con 0x7fb5b0002840 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.26:6790/0 conn(0x7fb5b0002840 legacy=0x7fb5b0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.606+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.26:6790/0 conn(0x7fb5b0002840 legacy=0x7fb5b0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb5b0007520 con 0x7fb5b00068d0 2024-09-15T04:00:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073235251 0 0) 0x7fb5b0007520 con 0x7fb5b00068d0 2024-09-15T04:00:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient(hunting): my global_id is 204671 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b0006590 con 0x7fb5b00068d0 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 933777260 0 0) 0x7fb5b0006590 con 0x7fb5b00068d0 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5b0007520 con 0x7fb5b00068d0 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb5c4005340 con 0x7fb5b00068d0 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient: dump: 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.653 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4103032096 0 0) 0x7fb5b0007520 con 0x7fb5b00068d0 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5b0008760 con 0x7fb5b00068d0 2024-09-15T04:00:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3498020540) 0x7fb5c40075e0 con 0x7fb5b00068d0 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.26:6790/0 conn(0x7fb5b00068d0 legacy=0x7fb5b0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5b77fe700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.26:6790/0 conn(0x7fb5b00068d0 legacy=0x7fb5b0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:15.655 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.656 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.657 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.658 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:15.659 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.660 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.662 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:15.666 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.667 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.669 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.670 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.671 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.672 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.675 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.675 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:15.675 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.122:6812/20895 conn(0x7fb5bc037cd0 legacy=0x7fb5bc03a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 10 monclient: shutdown 2024-09-15T04:00:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 >> v1:172.21.15.26:6792/0 conn(0x7fb5d00982f0 legacy=0x7fb5d013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5cffff700 1 -- 172.21.15.26:0/1325687224 reap_dead start 2024-09-15T04:00:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5cffff700 1 -- 172.21.15.26:0/1325687224 reap_dead start 2024-09-15T04:00:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 shutdown_connections 2024-09-15T04:00:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 >> 172.21.15.26:0/1325687224 conn(0x7fb5d0063f30 msgr2=0x7fb5d009a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 shutdown_connections 2024-09-15T04:00:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.618+0000 7fb5d5e2a700 1 -- 172.21.15.26:0/1325687224 wait complete. 2024-09-15T04:00:15.680 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:15.680 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T04:00:15.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: get_monmap_and_config 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: monmap: 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:15.796326+0000 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:15.796326+0000 2024-09-15T04:00:15.800 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 1 Processor -- start 2024-09-15T04:00:15.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 1 -- start start 2024-09-15T04:00:15.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: init 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: _add_conns ranks=[4,1,3,6,7,0,8,5,2] 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient(hunting): picked mon.noname-c con 0x7f0ff00a6880 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient(hunting): picked mon.noname-f con 0x7f0ff009faa0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient(hunting): picked mon.noname-g con 0x7f0ff0138d30 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff004ca10 con 0x7f0ff009faa0 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff00a0580 con 0x7f0ff0138d30 2024-09-15T04:00:15.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0133780 con 0x7f0ff00a6880 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient(hunting): authenticate will time out at 1760.225728s 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feffff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f0ff00a6880 0x7f0ff0133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:52536/0 (socket says 172.21.15.26:52536) 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feffff700 1 -- 172.21.15.26:0/1025688278 learned_addr learned my addr 172.21.15.26:0/1025688278 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001545545 0 0) 0x7f0ff0133780 con 0x7f0ff00a6880 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): my global_id is 204610 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fdc003570 con 0x7f0ff00a6880 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352584355 0 0) 0x7f0ff004ca10 con 0x7f0ff009faa0 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): my global_id is 204616 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff0133780 con 0x7f0ff009faa0 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548348490 0 0) 0x7f0fe8001110 con 0x7f0ff0138d30 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient(hunting): my global_id is 244101 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff004ca10 con 0x7f0ff0138d30 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545596262 0 0) 0x7f0fdc003570 con 0x7f0ff00a6880 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 >> v1:172.21.15.122:6790/0 conn(0x7f0ff0138d30 legacy=0x7f0ff013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 >> v1:172.21.15.122:6789/0 conn(0x7f0ff009faa0 legacy=0x7f0ff009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 monclient: found mon.noname-c 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00a6c50 con 0x7f0ff00a6880 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe0001b30 con 0x7f0ff00a6880 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 5 monclient: authenticate success, global_id 204610 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: dump: 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.808 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0fe0001d90 con 0x7f0ff00a6880 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 1 -- 172.21.15.26:0/1025688278 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe0004fb0 con 0x7f0ff00a6880 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0ff53ea700 4 set_mon_vals no callback set 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:15.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.794+0000 7f0feeffd700 10 monclient: dump: 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient: shutdown 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- 172.21.15.26:0/1025688278 >> v1:172.21.15.26:6791/0 conn(0x7f0ff00a6880 legacy=0x7f0ff0133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- 172.21.15.26:0/1025688278 shutdown_connections 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- 172.21.15.26:0/1025688278 >> 172.21.15.26:0/1025688278 conn(0x7f0ff006e2a0 msgr2=0x7f0ff00a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- 172.21.15.26:0/1025688278 shutdown_connections 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- 172.21.15.26:0/1025688278 wait complete. 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient: monmap: 2024-09-15T04:00:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.812 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.813 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.813 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 Processor -- start 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- start start 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient: init 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient: _add_conns ranks=[2,4,6,0,1,7,8,5,3] 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient(hunting): picked mon.noname-b con 0x7f0ff009faa0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient(hunting): picked mon.noname-c con 0x7f0ff00a6880 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient(hunting): picked mon.noname-d con 0x7f0ff0138d30 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0137b70 con 0x7f0ff009faa0 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff009cf10 con 0x7f0ff00a6880 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0131c60 con 0x7f0ff0138d30 2024-09-15T04:00:15.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff53ea700 10 monclient(hunting): authenticate will time out at 1760.230905s 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0feffff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0ff009faa0 0x7f0ff0135620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:55316/0 (socket says 172.21.15.26:55316) 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0ff4be9700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f0ff0138d30 0x7f0ff0136440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54028/0 (socket says 172.21.15.26:54028) 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0feffff700 1 -- 172.21.15.26:0/2606627668 learned_addr learned my addr 172.21.15.26:0/2606627668 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913108788 0 0) 0x7f0ff0137b70 con 0x7f0ff009faa0 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): my global_id is 204680 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd0003570 con 0x7f0ff009faa0 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171610612 0 0) 0x7f0ff004ca10 con 0x7f0ff0138d30 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): my global_id is 194865 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff0137b70 con 0x7f0ff0138d30 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608754483 0 0) 0x7f0ff009cf10 con 0x7f0ff00a6880 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): my global_id is 204619 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff004ca10 con 0x7f0ff00a6880 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1826203367 0 0) 0x7f0fd0003570 con 0x7f0ff009faa0 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ff009cf10 con 0x7f0ff009faa0 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe0001b70 con 0x7f0ff009faa0 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): dump: 2024-09-15T04:00:15.819 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.820 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.798+0000 7f0fe7fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4174626369 0 0) 0x7f0ff0137b70 con 0x7f0ff0138d30 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fd0003570 con 0x7f0ff0138d30 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe8001b50 con 0x7f0ff0138d30 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient(hunting): dump: 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.821 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.822 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 751673756 0 0) 0x7f0ff009cf10 con 0x7f0ff009faa0 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6792/0 conn(0x7f0ff0138d30 legacy=0x7f0ff0136440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6791/0 conn(0x7f0ff00a6880 legacy=0x7f0ff0135d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 monclient: found mon.b 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0137680 con 0x7f0ff009faa0 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 5 monclient: authenticate success, global_id 204680 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 10 monclient: _renew_subs 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:15.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff00a18c0 con 0x7f0ff009faa0 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 10 monclient: _renew_subs 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff0137d40 con 0x7f0ff009faa0 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0fe0005270 con 0x7f0ff009faa0 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe0005720 con 0x7f0ff009faa0 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fee7fc700 4 set_mon_vals no callback set 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: dump: 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.824 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0fe00100e0 con 0x7f0ff009faa0 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(727..727 src has 435..727) v4 ==== 3955+0+0 (unknown 132898664 0 0) 0x7f0fe00443b0 con 0x7f0ff009faa0 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216590665 0 0) 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient(hunting): my global_id is 204628 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff009cf10 con 0x7f0fd4002840 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0ff4be9700 10 monclient: get_auth_request con 0x7f0fd0037d00 auth_method 0 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3594708228 0 0) 0x7f0ff009cf10 con 0x7f0fd4002840 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fd8001a50 con 0x7f0fd4002840 2024-09-15T04:00:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 10 monclient: dump: 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4166187779 0 0) 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.802+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fd400a130 con 0x7f0fd4002840 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.806+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0fd8001f90 con 0x7f0fd4002840 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.806+0000 7f0fe7fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.806+0000 7f0fe7fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.806+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6791/0 conn(0x7f0fd4002840 legacy=0x7f0fd4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.806+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6791/0 conn(0x7f0fd4002840 legacy=0x7f0fd4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0fd4009620 con 0x7f0fd4009090 2024-09-15T04:00:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223393606 0 0) 0x7f0fd4009620 con 0x7f0fd4009090 2024-09-15T04:00:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient(hunting): my global_id is 204637 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd4006590 con 0x7f0fd4009090 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2564054068 0 0) 0x7f0fd4006590 con 0x7f0fd4009090 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fd4009620 con 0x7f0fd4009090 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fd80027a0 con 0x7f0fd4009090 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient: dump: 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3840675896 0 0) 0x7f0fd4009620 con 0x7f0fd4009090 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0fd40098d0 con 0x7f0fd4009090 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 1507209325) 0x7f0fd8002900 con 0x7f0fd4009090 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6791/0 conn(0x7f0fd4009090 legacy=0x7f0fd4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0fe7fff700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6791/0 conn(0x7f0fd4009090 legacy=0x7f0fd4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:15.833 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.834 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.835 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:15.837 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:15.838 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.840 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.845 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:15.848 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.849 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.851 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:15.852 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.122:6812/20895 conn(0x7f0fd0037d00 legacy=0x7f0fd003a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 10 monclient: shutdown 2024-09-15T04:00:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 >> v1:172.21.15.26:6790/0 conn(0x7f0ff009faa0 legacy=0x7f0ff0135620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff4be9700 1 -- 172.21.15.26:0/2606627668 reap_dead start 2024-09-15T04:00:15.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff4be9700 1 -- 172.21.15.26:0/2606627668 reap_dead start 2024-09-15T04:00:15.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 shutdown_connections 2024-09-15T04:00:15.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 >> 172.21.15.26:0/2606627668 conn(0x7f0ff006e2a0 msgr2=0x7f0ff00a4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 shutdown_connections 2024-09-15T04:00:15.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.814+0000 7f0ff53ea700 1 -- 172.21.15.26:0/2606627668 wait complete. 2024-09-15T04:00:15.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:15.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T04:00:15.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:00:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient: get_monmap_and_config 2024-09-15T04:00:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient: monmap: 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:15.969131+0000 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:15.969131+0000 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.973 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.974 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.974 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 1 Processor -- start 2024-09-15T04:00:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 1 -- start start 2024-09-15T04:00:15.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient: init 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient: _add_conns ranks=[8,0,3,2,1,6,7,5,4] 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient(hunting): picked mon.noname-e con 0x7f41c81335e0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient(hunting): picked mon.noname-a con 0x7f41c809fc80 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient(hunting): picked mon.noname-g con 0x7f41c80a0050 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c804ca10 con 0x7f41c809fc80 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c806eab0 con 0x7f41c80a0050 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c80a1330 con 0x7f41c81335e0 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41cc835700 10 monclient(hunting): authenticate will time out at 1760.398823s 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c659c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41c809fc80 0x7f41c8137ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48832/0 (socket says 172.21.15.26:48832) 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c659c700 1 -- 172.21.15.26:0/3486287027 learned_addr learned my addr 172.21.15.26:0/3486287027 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c6d9d700 1 --1- 172.21.15.26:0/3486287027 >> v1:172.21.15.26:6793/0 conn(0x7f41c81335e0 0x7f41c8132da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:35846/0 (socket says 172.21.15.26:35846) 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592699909 0 0) 0x7f41c804ca10 con 0x7f41c809fc80 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): my global_id is 194742 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b4003570 con 0x7f41c809fc80 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853050388 0 0) 0x7f41c80a1330 con 0x7f41c81335e0 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): my global_id is 135431 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c804ca10 con 0x7f41c81335e0 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512438808 0 0) 0x7f41c806eab0 con 0x7f41c80a0050 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient(hunting): my global_id is 244110 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c80a1330 con 0x7f41c80a0050 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823300217 0 0) 0x7f41c804ca10 con 0x7f41c81335e0 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 >> v1:172.21.15.122:6790/0 conn(0x7f41c80a0050 legacy=0x7f41c813b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 >> v1:172.21.15.26:6789/0 conn(0x7f41c809fc80 legacy=0x7f41c8137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.966+0000 7f41c5d9b700 1 monclient: found mon.noname-e 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c80a10d0 con 0x7f41c81335e0 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 5 monclient: authenticate success, global_id 135431 2024-09-15T04:00:15.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41b0001790 con 0x7f41c81335e0 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: dump: 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.981 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f41b0001a40 con 0x7f41c81335e0 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 4 set_mon_vals no callback set 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 1 -- 172.21.15.26:0/3486287027 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41b0004b70 con 0x7f41c81335e0 2024-09-15T04:00:15.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c5d9b700 10 monclient: dump: 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.983 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: shutdown 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- 172.21.15.26:0/3486287027 >> v1:172.21.15.26:6793/0 conn(0x7f41c81335e0 legacy=0x7f41c8132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- 172.21.15.26:0/3486287027 shutdown_connections 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- 172.21.15.26:0/3486287027 >> 172.21.15.26:0/3486287027 conn(0x7f41c806e6e0 msgr2=0x7f41c80a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- 172.21.15.26:0/3486287027 shutdown_connections 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- 172.21.15.26:0/3486287027 wait complete. 2024-09-15T04:00:15.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: build_initial_monmap 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: monmap: 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:15.985 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 Processor -- start 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- start start 2024-09-15T04:00:15.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: init 2024-09-15T04:00:15.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient: _add_conns ranks=[8,3,6,0,7,1,2,4,5] 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient(hunting): picked mon.noname-e con 0x7f41c809fc80 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient(hunting): picked mon.noname-g con 0x7f41c80a0050 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient(hunting): picked mon.noname-d con 0x7f41c81335e0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c80a1b70 con 0x7f41c80a0050 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c809cf10 con 0x7f41c81335e0 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c81428b0 con 0x7f41c809fc80 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41cc835700 10 monclient(hunting): authenticate will time out at 1760.403406s 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c6d9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f41c809fc80 0x7f41c812fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:35848/0 (socket says 172.21.15.26:35848) 2024-09-15T04:00:15.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41c6d9d700 1 -- 172.21.15.26:0/4271432937 learned_addr learned my addr 172.21.15.26:0/4271432937 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368768211 0 0) 0x7f41c81428b0 con 0x7f41c809fc80 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 10 monclient(hunting): my global_id is 135440 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a8003570 con 0x7f41c809fc80 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438938200 0 0) 0x7f41c809cf10 con 0x7f41c81335e0 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 10 monclient(hunting): my global_id is 194874 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.970+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c81428b0 con 0x7f41c81335e0 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058455717 0 0) 0x7f41c80a1b70 con 0x7f41c80a0050 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): my global_id is 244119 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c809cf10 con 0x7f41c80a0050 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1420151868 0 0) 0x7f41a8003570 con 0x7f41c809fc80 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41c80a1b70 con 0x7f41c809fc80 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41b0001750 con 0x7f41c809fc80 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): dump: 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.994 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2533581198 0 0) 0x7f41c81428b0 con 0x7f41c81335e0 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41a8003570 con 0x7f41c81335e0 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2546193682 0 0) 0x7f41c80a1b70 con 0x7f41c809fc80 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.26:6792/0 conn(0x7f41c81335e0 legacy=0x7f41c814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.122:6790/0 conn(0x7f41c80a0050 legacy=0x7f41c8146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 monclient: found mon.e 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c811e190 con 0x7f41c809fc80 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: _finish_auth 0 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:15.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 5 monclient: authenticate success, global_id 135440 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 10 monclient: _renew_subs 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c811e8e0 con 0x7f41c809fc80 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 10 monclient: _renew_subs 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c8133e80 con 0x7f41c809fc80 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f41b0001cd0 con 0x7f41c809fc80 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41b0005390 con 0x7f41c809fc80 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient: dump: 2024-09-15T04:00:15.997 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:15.998 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41c559a700 4 set_mon_vals no callback set 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f41b000fd50 con 0x7f41c809fc80 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f41c811fd70 con 0x7f41c811d7a0 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(727..727 src has 435..727) v4 ==== 3955+0+0 (unknown 132898664 0 0) 0x7f41b00441d0 con 0x7f41c809fc80 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41c759e700 10 monclient: get_auth_request con 0x7f41a80378b0 auth_method 0 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406030327 0 0) 0x7f41c811fd70 con 0x7f41c811d7a0 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 10 monclient(hunting): my global_id is 194883 2024-09-15T04:00:15.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c80a1b70 con 0x7f41c811d7a0 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.974+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2638542934 0 0) 0x7f41c80a1b70 con 0x7f41c811d7a0 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41c811fd70 con 0x7f41c811d7a0 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41b8004b70 con 0x7f41c811d7a0 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 10 monclient: dump: 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.000 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3256038874 0 0) 0x7f41c811fd70 con 0x7f41c811d7a0 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41c811ebb0 con 0x7f41c811d7a0 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f41b80050a0 con 0x7f41c811d7a0 2024-09-15T04:00:16.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.26:6792/0 conn(0x7f41c811d7a0 legacy=0x7f41c805e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.978+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.26:6792/0 conn(0x7f41c811d7a0 legacy=0x7f41c805e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41cc835700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f41c811db70 con 0x7f41c802ae80 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18609740 0 0) 0x7f41c811db70 con 0x7f41c802ae80 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41beffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41beffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41beffd700 10 monclient(hunting): my global_id is 194892 2024-09-15T04:00:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.986+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c811fd70 con 0x7f41c802ae80 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2863519345 0 0) 0x7f41c811fd70 con 0x7f41c802ae80 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41c811db70 con 0x7f41c802ae80 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41b80058b0 con 0x7f41c802ae80 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 10 monclient: dump: 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3732763547 0 0) 0x7f41c811db70 con 0x7f41c802ae80 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f41c81202a0 con 0x7f41c802ae80 2024-09-15T04:00:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 1850807861) 0x7f41b8006c90 con 0x7f41c802ae80 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.26:6792/0 conn(0x7f41c802ae80 legacy=0x7f41c8098a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41beffd700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.26:6792/0 conn(0x7f41c802ae80 legacy=0x7f41c8098a70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:16.006 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.007 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:16.008 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:16.009 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:16.010 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:16.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:16.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.015 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.019 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.020 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.022 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.024 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:16.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.026 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.026 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:16.026 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.122:6812/20895 conn(0x7f41a80378b0 legacy=0x7f41a8039d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 10 monclient: shutdown 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 >> v1:172.21.15.26:6793/0 conn(0x7f41c809fc80 legacy=0x7f41c812fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41c759e700 1 -- 172.21.15.26:0/4271432937 reap_dead start 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 shutdown_connections 2024-09-15T04:00:16.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 >> 172.21.15.26:0/4271432937 conn(0x7f41c806e6e0 msgr2=0x7f41c809dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 shutdown_connections 2024-09-15T04:00:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:15.990+0000 7f41cc835700 1 -- 172.21.15.26:0/4271432937 wait complete. 2024-09-15T04:00:16.028 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:16.028 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T04:00:16.028 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T04:00:16.117 INFO:tasks.rados.rados.0.smithi122.stdout:3224: done (2 left) 2024-09-15T04:00:16.118 INFO:tasks.rados.rados.0.smithi122.stdout:3225: done (1 left) 2024-09-15T04:00:16.118 INFO:tasks.rados.rados.0.smithi122.stdout:3226: done (0 left) 2024-09-15T04:00:16.118 INFO:tasks.rados.rados.0.smithi122.stdout:3227: read oid 5 snap -1 2024-09-15T04:00:16.118 INFO:tasks.rados.rados.0.smithi122.stdout:3227: expect deleted 2024-09-15T04:00:16.118 INFO:tasks.rados.rados.0.smithi122.stdout:3228: delete oid 27 current snap is 376 2024-09-15T04:00:16.127 INFO:tasks.rados.rados.0.smithi122.stdout:3227: done (1 left) 2024-09-15T04:00:16.127 INFO:tasks.rados.rados.0.smithi122.stdout:3228: done (0 left) 2024-09-15T04:00:16.127 INFO:tasks.rados.rados.0.smithi122.stdout:3229: read oid 28 snap 372 2024-09-15T04:00:16.128 INFO:tasks.rados.rados.0.smithi122.stdout:3229: expect (ObjNum 1129 snap 371 seq_num 1129) 2024-09-15T04:00:16.128 INFO:tasks.rados.rados.0.smithi122.stdout:3230: copy_from oid 9 from oid 42 current snap is 376 2024-09-15T04:00:16.128 INFO:tasks.rados.rados.0.smithi122.stdout:3231: read oid 26 snap -1 2024-09-15T04:00:16.128 INFO:tasks.rados.rados.0.smithi122.stdout:3231: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:00:16.128 INFO:tasks.rados.rados.0.smithi122.stdout:3232: snap_create 2024-09-15T04:00:16.134 INFO:tasks.rados.rados.0.smithi122.stdout:3229: expect (ObjNum 1129 snap 371 seq_num 1129) 2024-09-15T04:00:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.138+0000 7fcfeb0a3700 10 monclient: get_monmap_and_config 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.138+0000 7fcfeb0a3700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.138+0000 7fcfeb0a3700 10 monclient: monmap: 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:16.143601+0000 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:16.143601+0000 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.148 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 Processor -- start 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- start start 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: init 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: _add_conns ranks=[7,5,4,2,3,8,6,0,1] 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient(hunting): picked mon.noname-i con 0x7fcfe4063c60 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient(hunting): picked mon.noname-h con 0x7fcfe409aca0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient(hunting): picked mon.noname-c con 0x7fcfe4136de0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe404ca10 con 0x7fcfe4136de0 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe409c600 con 0x7fcfe409aca0 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe40a0230 con 0x7fcfe4063c60 2024-09-15T04:00:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient(hunting): authenticate will time out at 1760.573291s 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfea8a2700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fcfe4136de0 0x7fcfe41391c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:52586/0 (socket says 172.21.15.26:52586) 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfea8a2700 1 -- 172.21.15.26:0/3318747901 learned_addr learned my addr 172.21.15.26:0/3318747901 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942621392 0 0) 0x7fcfe404ca10 con 0x7fcfe4136de0 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): my global_id is 204646 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd0003570 con 0x7fcfe4136de0 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530350797 0 0) 0x7fcfe40a0230 con 0x7fcfe4063c60 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): my global_id is 234304 2024-09-15T04:00:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe404ca10 con 0x7fcfe4063c60 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436224624 0 0) 0x7fcfe409c600 con 0x7fcfe409aca0 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient(hunting): my global_id is 6107 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe40a0230 con 0x7fcfe409aca0 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741027440 0 0) 0x7fcfd0003570 con 0x7fcfe4136de0 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 >> v1:172.21.15.122:6792/0 conn(0x7fcfe4063c60 legacy=0x7fcfe409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 >> v1:172.21.15.122:6791/0 conn(0x7fcfe409aca0 legacy=0x7fcfe41356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 monclient: found mon.noname-c 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe409c320 con 0x7fcfe4136de0 2024-09-15T04:00:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 5 monclient: authenticate success, global_id 204646 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfdc001b30 con 0x7fcfe4136de0 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: dump: 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.159 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcfdc001da0 con 0x7fcfe4136de0 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:16.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 1 -- 172.21.15.26:0/3318747901 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfdc004fb0 con 0x7fcfe4136de0 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 4 set_mon_vals no callback set 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfe909f700 10 monclient: dump: 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: shutdown 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/3318747901 >> v1:172.21.15.26:6791/0 conn(0x7fcfe4136de0 legacy=0x7fcfe41391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/3318747901 shutdown_connections 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/3318747901 >> 172.21.15.26:0/3318747901 conn(0x7fcfe40a4410 msgr2=0x7fcfe40a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/3318747901 shutdown_connections 2024-09-15T04:00:16.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/3318747901 wait complete. 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: monmap: 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.163 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 Processor -- start 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 1 -- start start 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.142+0000 7fcfeb0a3700 10 monclient: init 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient: _add_conns ranks=[6,1,0,5,2,7,4,8,3] 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient(hunting): picked mon.noname-d con 0x7fcfe4063c60 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient(hunting): picked mon.noname-f con 0x7fcfe409aca0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient(hunting): picked mon.noname-a con 0x7fcfe4136de0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe4136990 con 0x7fcfe4136de0 2024-09-15T04:00:16.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe4123280 con 0x7fcfe409aca0 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe4122cc0 con 0x7fcfe4063c60 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient(hunting): authenticate will time out at 1760.576294s 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfea0a1700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fcfe4063c60 0x7fcfe4067640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54070/0 (socket says 172.21.15.26:54070) 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfea0a1700 1 -- 172.21.15.26:0/2044257054 learned_addr learned my addr 172.21.15.26:0/2044257054 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883565156 0 0) 0x7fcfe4122cc0 con 0x7fcfe4063c60 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): my global_id is 194901 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc8003570 con 0x7fcfe4063c60 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364755381 0 0) 0x7fcfe4136990 con 0x7fcfe4136de0 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): my global_id is 194751 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe4122cc0 con 0x7fcfe4136de0 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615818316 0 0) 0x7fcfe4123280 con 0x7fcfe409aca0 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): my global_id is 204625 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe4136990 con 0x7fcfe409aca0 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2596479201 0 0) 0x7fcfc8003570 con 0x7fcfe4063c60 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcfe4123280 con 0x7fcfe4063c60 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3292983982 0 0) 0x7fcfe4122cc0 con 0x7fcfe4136de0 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcfc8003570 con 0x7fcfe4136de0 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfe0002b50 con 0x7fcfe4063c60 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:16.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.167 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfdc001af0 con 0x7fcfe4136de0 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.168 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1423810334 0 0) 0x7fcfe4136990 con 0x7fcfe409aca0 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcfe4122cc0 con 0x7fcfe409aca0 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfd4001a60 con 0x7fcfe409aca0 2024-09-15T04:00:16.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1996710508 0 0) 0x7fcfe4123280 con 0x7fcfe4063c60 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.122:6789/0 conn(0x7fcfe409aca0 legacy=0x7fcfe4067d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.26:6789/0 conn(0x7fcfe4136de0 legacy=0x7fcfe4068460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 monclient: found mon.d 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe41229e0 con 0x7fcfe4063c60 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 5 monclient: authenticate success, global_id 194901 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient: _renew_subs 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4122fa0 con 0x7fcfe4063c60 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient: _renew_subs 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe4121a80 con 0x7fcfe4063c60 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcfe0001620 con 0x7fcfe4063c60 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfe0002d40 con 0x7fcfe4063c60 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfdbfff700 4 set_mon_vals no callback set 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:16.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: dump: 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.173 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcfe0001b50 con 0x7fcfe4063c60 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcfcc006590 con 0x7fcfcc002840 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(728..728 src has 435..728) v4 ==== 3939+0+0 (unknown 3055368912 0 0) 0x7fcfe0043e70 con 0x7fcfe4063c60 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157744677 0 0) 0x7fcfcc006590 con 0x7fcfcc002840 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient(hunting): my global_id is 135449 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe4123280 con 0x7fcfcc002840 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfea8a2700 10 monclient: get_auth_request con 0x7fcfc8038080 auth_method 0 2024-09-15T04:00:16.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3267700522 0 0) 0x7fcfe4123280 con 0x7fcfcc002840 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcfcc006590 con 0x7fcfcc002840 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfd4002230 con 0x7fcfcc002840 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 10 monclient: dump: 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2236704603 0 0) 0x7fcfcc006590 con 0x7fcfcc002840 2024-09-15T04:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.146+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfcc00a130 con 0x7fcfcc002840 2024-09-15T04:00:16.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.150+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fcfd4002690 con 0x7fcfcc002840 2024-09-15T04:00:16.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.150+0000 7fcfda7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.150+0000 7fcfda7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:16.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.150+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.26:6793/0 conn(0x7fcfcc002840 legacy=0x7fcfcc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.150+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.26:6793/0 conn(0x7fcfcc002840 legacy=0x7fcfcc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfeb0a3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcfcc009640 con 0x7fcfcc0090b0 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265541074 0 0) 0x7fcfcc009640 con 0x7fcfcc0090b0 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient(hunting): my global_id is 135458 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfcc006590 con 0x7fcfcc0090b0 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3849813960 0 0) 0x7fcfcc006590 con 0x7fcfcc0090b0 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcfcc009640 con 0x7fcfcc0090b0 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcfd4002b10 con 0x7fcfcc0090b0 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient: dump: 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1886581467 0 0) 0x7fcfcc009640 con 0x7fcfcc0090b0 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcfcc0098f0 con 0x7fcfcc0090b0 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 292532423) 0x7fcfd40034f0 con 0x7fcfcc0090b0 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.26:6793/0 conn(0x7fcfcc0090b0 legacy=0x7fcfcc008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.158+0000 7fcfda7fc700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.26:6793/0 conn(0x7fcfcc0090b0 legacy=0x7fcfcc008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.183 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.184 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.186 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:16.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:16.188 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.193 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:16.194 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.195 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.196 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.200 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:16.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.203 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:16.203 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.122:6812/20895 conn(0x7fcfc8038080 legacy=0x7fcfc803a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 10 monclient: shutdown 2024-09-15T04:00:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 >> v1:172.21.15.26:6792/0 conn(0x7fcfe4063c60 legacy=0x7fcfe4067640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfea8a2700 1 -- 172.21.15.26:0/2044257054 reap_dead start 2024-09-15T04:00:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 shutdown_connections 2024-09-15T04:00:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 >> 172.21.15.26:0/2044257054 conn(0x7fcfe40a4410 msgr2=0x7fcfe40a2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 shutdown_connections 2024-09-15T04:00:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.162+0000 7fcfeb0a3700 1 -- 172.21.15.26:0/2044257054 wait complete. 2024-09-15T04:00:16.205 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:16.205 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T04:00:16.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T04:00:16.313 INFO:tasks.rados.rados.0.smithi122.stdout:3230: finishing copy_from to smithi12223906-9 2024-09-15T04:00:16.313 INFO:tasks.rados.rados.0.smithi122.stdout:3230: got expected ENOENT (src dne) 2024-09-15T04:00:16.313 INFO:tasks.rados.rados.0.smithi122.stdout:3230: finishing copy_from racing read to smithi12223906-9 2024-09-15T04:00:16.314 INFO:tasks.rados.rados.0.smithi122.stdout:3231: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:00:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.310+0000 7f53cedcf700 10 monclient: get_monmap_and_config 2024-09-15T04:00:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.310+0000 7f53cedcf700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.310+0000 7f53cedcf700 10 monclient: monmap: 2024-09-15T04:00:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:16.314830+0000 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:16.314830+0000 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.320 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.320 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.320 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.320 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.320 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.320 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 1 Processor -- start 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 1 -- start start 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient: init 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient: _add_conns ranks=[3,2,6,8,4,7,0,5,1] 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient(hunting): picked mon.noname-g con 0x7f53c80a4840 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient(hunting): picked mon.noname-b con 0x7f53c809aa90 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient(hunting): picked mon.noname-d con 0x7f53c81392c0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c804ca10 con 0x7f53c809aa90 2024-09-15T04:00:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c8068c10 con 0x7f53c80a4840 2024-09-15T04:00:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c80a4c10 con 0x7f53c81392c0 2024-09-15T04:00:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient(hunting): authenticate will time out at 1760.745163s 2024-09-15T04:00:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bffff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f53c809aa90 0x7f53c8137ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:55320/0 (socket says 172.21.15.26:55320) 2024-09-15T04:00:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bffff700 1 -- 172.21.15.26:0/3660595700 learned_addr learned my addr 172.21.15.26:0/3660595700 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188675570 0 0) 0x7f53c804ca10 con 0x7f53c809aa90 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): my global_id is 204689 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b0003570 con 0x7f53c809aa90 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790632694 0 0) 0x7f53c8068c10 con 0x7f53c80a4840 2024-09-15T04:00:16.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): my global_id is 244128 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c804ca10 con 0x7f53c80a4840 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282789129 0 0) 0x7f53c80a4c10 con 0x7f53c81392c0 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient(hunting): my global_id is 194910 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c8068c10 con 0x7f53c81392c0 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953871654 0 0) 0x7f53b0003570 con 0x7f53c809aa90 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 >> v1:172.21.15.26:6792/0 conn(0x7f53c81392c0 legacy=0x7f53c813b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 >> v1:172.21.15.122:6790/0 conn(0x7f53c80a4840 legacy=0x7f53c809db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 monclient: found mon.noname-b 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8068930 con 0x7f53c809aa90 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 5 monclient: authenticate success, global_id 204689 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53b80017b0 con 0x7f53c809aa90 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: dump: 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53b8001a90 con 0x7f53c809aa90 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 1 -- 172.21.15.26:0/3660595700 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53b8004c30 con 0x7f53c809aa90 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 4 set_mon_vals no callback set 2024-09-15T04:00:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53bf7fe700 10 monclient: dump: 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.314+0000 7f53cedcf700 10 monclient: shutdown 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- 172.21.15.26:0/3660595700 >> v1:172.21.15.26:6790/0 conn(0x7f53c809aa90 legacy=0x7f53c8137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- 172.21.15.26:0/3660595700 shutdown_connections 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- 172.21.15.26:0/3660595700 >> 172.21.15.26:0/3660595700 conn(0x7f53c80639b0 msgr2=0x7f53c80a3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- 172.21.15.26:0/3660595700 shutdown_connections 2024-09-15T04:00:16.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- 172.21.15.26:0/3660595700 wait complete. 2024-09-15T04:00:16.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient: monmap: 2024-09-15T04:00:16.338 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.338 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:16.338 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.339 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 Processor -- start 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- start start 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient: init 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient: _add_conns ranks=[1,8,6,3,4,7,0,5,2] 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient(hunting): picked mon.noname-f con 0x7f53c809aa90 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient(hunting): picked mon.noname-e con 0x7f53c80a4840 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient(hunting): picked mon.noname-d con 0x7f53c81392c0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c806e2f0 con 0x7f53c809aa90 2024-09-15T04:00:16.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c8142250 con 0x7f53c81392c0 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c806ba00 con 0x7f53c80a4840 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53cedcf700 10 monclient(hunting): authenticate will time out at 1760.751808s 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53bffff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f53c80a4840 0x7f53c806d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:35866/0 (socket says 172.21.15.26:35866) 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.318+0000 7f53bffff700 1 -- 172.21.15.26:0/2651214678 learned_addr learned my addr 172.21.15.26:0/2651214678 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644925979 0 0) 0x7f53c806ba00 con 0x7f53c80a4840 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): my global_id is 135467 2024-09-15T04:00:16.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b4003570 con 0x7f53c80a4840 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333653523 0 0) 0x7f53c8142250 con 0x7f53c81392c0 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): my global_id is 194919 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c806ba00 con 0x7f53c81392c0 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409986906 0 0) 0x7f53c806e2f0 con 0x7f53c809aa90 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): my global_id is 204634 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c8142250 con 0x7f53c809aa90 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 901317620 0 0) 0x7f53b4003570 con 0x7f53c80a4840 2024-09-15T04:00:16.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53c806e2f0 con 0x7f53c80a4840 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53b8004b70 con 0x7f53c80a4840 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): dump: 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.343 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1906043667 0 0) 0x7f53c806ba00 con 0x7f53c81392c0 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53b4003570 con 0x7f53c81392c0 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c0001c80 con 0x7f53c81392c0 2024-09-15T04:00:16.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient(hunting): dump: 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.345 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1257241047 0 0) 0x7f53c806e2f0 con 0x7f53c80a4840 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.26:6792/0 conn(0x7f53c81392c0 legacy=0x7f53c8149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.122:6789/0 conn(0x7f53c809aa90 legacy=0x7f53c8141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 monclient: found mon.e 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c806bce0 con 0x7f53c80a4840 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 5 monclient: authenticate success, global_id 135467 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 10 monclient: _renew_subs 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c8099850 con 0x7f53c80a4840 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 10 monclient: _renew_subs 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c8142900 con 0x7f53c80a4840 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53b8004f00 con 0x7f53c80a4840 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53b80053b0 con 0x7f53c80a4840 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53beffd700 4 set_mon_vals no callback set 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 10 monclient: dump: 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.348 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53a8006590 con 0x7f53a8002840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.322+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f53b800fd70 con 0x7f53c80a4840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(728..728 src has 435..728) v4 ==== 3939+0+0 (unknown 3055368912 0 0) 0x7f53b8044130 con 0x7f53c80a4840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636959647 0 0) 0x7f53a8006590 con 0x7f53a8002840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient(hunting): my global_id is 204643 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c806e2f0 con 0x7f53a8002840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53cd36c700 10 monclient: get_auth_request con 0x7f53b4037c70 auth_method 0 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3586728984 0 0) 0x7f53c806e2f0 con 0x7f53a8002840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53a8006590 con 0x7f53a8002840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c40017a0 con 0x7f53a8002840 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:16.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient: dump: 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.351 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4170398171 0 0) 0x7f53a8006590 con 0x7f53a8002840 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53a800a130 con 0x7f53a8002840 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f53c4003a40 con 0x7f53a8002840 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.122:6789/0 conn(0x7f53a8002840 legacy=0x7f53a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.326+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.122:6789/0 conn(0x7f53a8002840 legacy=0x7f53a8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53cedcf700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f53a8009640 con 0x7f53a80090b0 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770365463 0 0) 0x7f53a8009640 con 0x7f53a80090b0 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient(hunting): my global_id is 204652 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a8006590 con 0x7f53a80090b0 2024-09-15T04:00:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2603282198 0 0) 0x7f53a8006590 con 0x7f53a80090b0 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53a8009640 con 0x7f53a80090b0 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c40022a0 con 0x7f53a80090b0 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient: dump: 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2773737763 0 0) 0x7f53a8009640 con 0x7f53a80090b0 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f53a80098f0 con 0x7f53a80090b0 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3084840659) 0x7f53c40037f0 con 0x7f53a80090b0 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.122:6789/0 conn(0x7f53a80090b0 legacy=0x7f53a8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.338+0000 7f53bcff9700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.122:6789/0 conn(0x7f53a80090b0 legacy=0x7f53a8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:16.358 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:16.359 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.360 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:16.361 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:16.362 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.363 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.364 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.366 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.367 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.368 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:16.370 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.371 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.373 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.375 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:16.376 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.122:6812/20895 conn(0x7f53b4037c70 legacy=0x7f53b403a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 10 monclient: shutdown 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 >> v1:172.21.15.26:6793/0 conn(0x7f53c80a4840 legacy=0x7f53c806d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cd36c700 1 -- 172.21.15.26:0/2651214678 reap_dead start 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 shutdown_connections 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 >> 172.21.15.26:0/2651214678 conn(0x7f53c80639b0 msgr2=0x7f53c8130cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 shutdown_connections 2024-09-15T04:00:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.342+0000 7f53cedcf700 1 -- 172.21.15.26:0/2651214678 wait complete. 2024-09-15T04:00:16.379 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:16.379 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T04:00:16.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient: get_monmap_and_config 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient: monmap: 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:16.494150+0000 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:16.494150+0000 2024-09-15T04:00:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 1 Processor -- start 2024-09-15T04:00:16.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 1 -- start start 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient: init 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient: _add_conns ranks=[5,6,8,4,2,7,0,3,1] 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient(hunting): picked mon.noname-h con 0x7f811412fd20 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient(hunting): picked mon.noname-d con 0x7f81141300f0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient(hunting): picked mon.noname-e con 0x7f8114130520 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811404ca10 con 0x7f811412fd20 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81140a2830 con 0x7f81141300f0 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8114097b70 con 0x7f8114130520 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.490+0000 7f811b09c700 10 monclient(hunting): authenticate will time out at 1760.923772s 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f8119639700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8114130520 0x7f811413b5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:35870/0 (socket says 172.21.15.26:35870) 2024-09-15T04:00:16.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810bfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f81141300f0 0x7f8114137e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:54092/0 (socket says 172.21.15.26:54092) 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f8119639700 1 -- 172.21.15.26:0/2872029822 learned_addr learned my addr 172.21.15.26:0/2872029822 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f8118e38700 1 --1- 172.21.15.26:0/2872029822 >> v1:172.21.15.122:6791/0 conn(0x7f811412fd20 0x7f8114133270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:43340/0 (socket says 172.21.15.26:43340) 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81777840 0 0) 0x7f811404ca10 con 0x7f811412fd20 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): my global_id is 6116 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80fc003570 con 0x7f811412fd20 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298719189 0 0) 0x7f81140a2830 con 0x7f81141300f0 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): my global_id is 194928 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811404ca10 con 0x7f81141300f0 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268559433 0 0) 0x7f8114097b70 con 0x7f8114130520 2024-09-15T04:00:16.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient(hunting): my global_id is 135476 2024-09-15T04:00:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81140a2830 con 0x7f8114130520 2024-09-15T04:00:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2877399626 0 0) 0x7f80fc003570 con 0x7f811412fd20 2024-09-15T04:00:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 >> v1:172.21.15.26:6793/0 conn(0x7f8114130520 legacy=0x7f811413b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 >> v1:172.21.15.26:6792/0 conn(0x7f81141300f0 legacy=0x7f8114137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 monclient: found mon.noname-h 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8114133980 con 0x7f811412fd20 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 5 monclient: authenticate success, global_id 6116 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81100017b0 con 0x7f811412fd20 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: dump: 2024-09-15T04:00:16.511 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8110001a20 con 0x7f811412fd20 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 1 -- 172.21.15.26:0/2872029822 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8110004b70 con 0x7f811412fd20 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 4 set_mon_vals no callback set 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f810b7fe700 10 monclient: dump: 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.514 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 10 monclient: shutdown 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 1 -- 172.21.15.26:0/2872029822 >> v1:172.21.15.122:6791/0 conn(0x7f811412fd20 legacy=0x7f8114133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 1 -- 172.21.15.26:0/2872029822 shutdown_connections 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 1 -- 172.21.15.26:0/2872029822 >> 172.21.15.26:0/2872029822 conn(0x7f811406e410 msgr2=0x7f81140a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 1 -- 172.21.15.26:0/2872029822 shutdown_connections 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 1 -- 172.21.15.26:0/2872029822 wait complete. 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 10 monclient: monmap: 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.516 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.494+0000 7f811b09c700 1 Processor -- start 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- start start 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: init 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: _add_conns ranks=[1,3,7,4,5,2,6,8,0] 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient(hunting): picked mon.noname-f con 0x7f811412fd20 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:16.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient(hunting): picked mon.noname-g con 0x7f81141300f0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient(hunting): picked mon.noname-i con 0x7f8114130520 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81140a1b50 con 0x7f811412fd20 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81140a1880 con 0x7f81141300f0 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811411ed30 con 0x7f8114130520 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient(hunting): authenticate will time out at 1760.928911s 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8119639700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f8114130520 0x7f811414a7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:60070/0 (socket says 172.21.15.26:60070) 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8119639700 1 -- 172.21.15.26:0/3162002163 learned_addr learned my addr 172.21.15.26:0/3162002163 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919673339 0 0) 0x7f811411ed30 con 0x7f8114130520 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): my global_id is 234313 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8100003570 con 0x7f8114130520 2024-09-15T04:00:16.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435314653 0 0) 0x7f81140a1880 con 0x7f81141300f0 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): my global_id is 244137 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811411ed30 con 0x7f81141300f0 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544548998 0 0) 0x7f81140a1b50 con 0x7f811412fd20 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): my global_id is 204661 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81140a1880 con 0x7f811412fd20 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 264795958 0 0) 0x7f8100003570 con 0x7f8114130520 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81140a1b50 con 0x7f8114130520 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f810c001bb0 con 0x7f8114130520 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.520 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2351696134 0 0) 0x7f81140a1b50 con 0x7f8114130520 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6790/0 conn(0x7f81141300f0 legacy=0x7f8114147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6789/0 conn(0x7f811412fd20 legacy=0x7f811412ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 monclient: found mon.i 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811411f010 con 0x7f8114130520 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 5 monclient: authenticate success, global_id 234313 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: _renew_subs 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811412f2f0 con 0x7f8114130520 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: _renew_subs 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f811411cda0 con 0x7f8114130520 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f810c004bb0 con 0x7f8114130520 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f810c0050e0 con 0x7f8114130520 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f810affd700 4 set_mon_vals no callback set 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:16.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 10 monclient: dump: 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.523 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.524 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f80f4006590 con 0x7f80f4002840 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.498+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f810c005300 con 0x7f8114130520 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(728..728 src has 435..728) v4 ==== 3939+0+0 (unknown 3055368912 0 0) 0x7f810c0441b0 con 0x7f8114130520 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269593784 0 0) 0x7f80f4006590 con 0x7f80f4002840 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient(hunting): my global_id is 244146 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81140a1b50 con 0x7f80f4002840 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f810bfff700 10 monclient: get_auth_request con 0x7f8100037750 auth_method 0 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 492031957 0 0) 0x7f81140a1b50 con 0x7f80f4002840 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80f4006590 con 0x7f80f4002840 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8110004b70 con 0x7f80f4002840 2024-09-15T04:00:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient: dump: 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2613375263 0 0) 0x7f80f4006590 con 0x7f80f4002840 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80f400a130 con 0x7f80f4002840 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f8110001670 con 0x7f80f4002840 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6790/0 conn(0x7f80f4002840 legacy=0x7f80f4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.502+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6790/0 conn(0x7f80f4002840 legacy=0x7f80f4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f811b09c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f80f4009640 con 0x7f80f40090b0 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483669902 0 0) 0x7f80f4009640 con 0x7f80f40090b0 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient(hunting): my global_id is 244155 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f4006590 con 0x7f80f40090b0 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2449054829 0 0) 0x7f80f4006590 con 0x7f80f40090b0 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80f4009640 con 0x7f80f40090b0 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8110005530 con 0x7f80f40090b0 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient: dump: 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2089246100 0 0) 0x7f80f4009640 con 0x7f80f40090b0 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f80f40098f0 con 0x7f80f40090b0 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 2591571186) 0x7f8110007790 con 0x7f80f40090b0 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6790/0 conn(0x7f80f40090b0 legacy=0x7f80f4008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.510+0000 7f8108ff9700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6790/0 conn(0x7f80f40090b0 legacy=0x7f80f4008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.532 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:16.533 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.534 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.535 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:16.536 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.537 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:16.538 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.539 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:16.540 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.547 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.551 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.552 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.553 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.553 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:16.553 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6812/20895 conn(0x7f8100037750 legacy=0x7f8100039c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 10 monclient: shutdown 2024-09-15T04:00:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 >> v1:172.21.15.122:6792/0 conn(0x7f8114130520 legacy=0x7f811414a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f8119639700 1 -- 172.21.15.26:0/3162002163 reap_dead start 2024-09-15T04:00:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 shutdown_connections 2024-09-15T04:00:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 >> 172.21.15.26:0/3162002163 conn(0x7f811406e410 msgr2=0x7f81140a4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 shutdown_connections 2024-09-15T04:00:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.514+0000 7f811b09c700 1 -- 172.21.15.26:0/3162002163 wait complete. 2024-09-15T04:00:16.555 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:16.555 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T04:00:16.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T04:00:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.666+0000 7f097759e700 10 monclient: get_monmap_and_config 2024-09-15T04:00:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.666+0000 7f097759e700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.666+0000 7f097759e700 10 monclient: monmap: 2024-09-15T04:00:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:16.670962+0000 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:16.670962+0000 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.676 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.666+0000 7f097759e700 1 Processor -- start 2024-09-15T04:00:16.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.666+0000 7f097759e700 1 -- start start 2024-09-15T04:00:16.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.666+0000 7f097759e700 10 monclient: init 2024-09-15T04:00:16.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient: _add_conns ranks=[8,2,5,3,6,1,4,0,7] 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient(hunting): picked mon.noname-e con 0x7f09780a2a70 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient(hunting): picked mon.noname-b con 0x7f097809b750 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient(hunting): picked mon.noname-h con 0x7f097813d0c0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097804ca10 con 0x7f097809b750 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097809e5d0 con 0x7f097813d0c0 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097809e2f0 con 0x7f09780a2a70 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient(hunting): authenticate will time out at 1761.100385s 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097659c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f09780a2a70 0x7f0978099e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37242/0 (socket says 172.21.15.26:37242) 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f0975d9b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f097809b750 0x7f097809bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57916/0 (socket says 172.21.15.26:57916) 2024-09-15T04:00:16.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097659c700 1 -- 172.21.15.26:0/2301018034 learned_addr learned my addr 172.21.15.26:0/2301018034 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395341246 0 0) 0x7f097804ca10 con 0x7f097809b750 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): my global_id is 204698 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0960003570 con 0x7f097809b750 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424286788 0 0) 0x7f097809e2f0 con 0x7f09780a2a70 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): my global_id is 135485 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097804ca10 con 0x7f09780a2a70 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025606659 0 0) 0x7f097809e5d0 con 0x7f097813d0c0 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient(hunting): my global_id is 6125 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097809e2f0 con 0x7f097813d0c0 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 382337670 0 0) 0x7f0960003570 con 0x7f097809b750 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 >> v1:172.21.15.26:6793/0 conn(0x7f09780a2a70 legacy=0x7f0978099e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 >> v1:172.21.15.122:6791/0 conn(0x7f097813d0c0 legacy=0x7f097813f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 monclient: found mon.noname-b 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097809a570 con 0x7f097809b750 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 5 monclient: authenticate success, global_id 204698 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f09680017b0 con 0x7f097809b750 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: dump: 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.688 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0968001a90 con 0x7f097809b750 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 4 set_mon_vals no callback set 2024-09-15T04:00:16.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 1 -- 172.21.15.26:0/2301018034 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0968004b70 con 0x7f097809b750 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097559a700 10 monclient: dump: 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.690 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient: shutdown 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- 172.21.15.26:0/2301018034 >> v1:172.21.15.26:6790/0 conn(0x7f097809b750 legacy=0x7f097809bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- 172.21.15.26:0/2301018034 shutdown_connections 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- 172.21.15.26:0/2301018034 >> 172.21.15.26:0/2301018034 conn(0x7f0978063cd0 msgr2=0x7f09780a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- 172.21.15.26:0/2301018034 shutdown_connections 2024-09-15T04:00:16.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 1 -- 172.21.15.26:0/2301018034 wait complete. 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.670+0000 7f097759e700 10 monclient: monmap: 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.692 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 Processor -- start 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 -- start start 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: init 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: _add_conns ranks=[5,4,2,3,1,8,0,6,7] 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient(hunting): picked mon.noname-h con 0x7f097809b750 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient(hunting): picked mon.noname-c con 0x7f09780a2a70 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient(hunting): picked mon.noname-b con 0x7f097813d0c0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097809d490 con 0x7f097813d0c0 2024-09-15T04:00:16.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0978137e00 con 0x7f09780a2a70 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097806e2f0 con 0x7f097809b750 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient(hunting): authenticate will time out at 1761.105162s 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f0975d9b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f09780a2a70 0x7f097806a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40920/0 (socket says 172.21.15.26:40920) 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f0975d9b700 1 -- 172.21.15.26:0/909502251 learned_addr learned my addr 172.21.15.26:0/909502251 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186624627 0 0) 0x7f097809d490 con 0x7f097813d0c0 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): my global_id is 204707 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0958003570 con 0x7f097813d0c0 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055825967 0 0) 0x7f0978137e00 con 0x7f09780a2a70 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): my global_id is 204655 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097809d490 con 0x7f09780a2a70 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704749546 0 0) 0x7f097806e2f0 con 0x7f097809b750 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): my global_id is 6134 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0978137e00 con 0x7f097809b750 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3307885300 0 0) 0x7f0958003570 con 0x7f097813d0c0 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f097806e2f0 con 0x7f097813d0c0 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0970001d40 con 0x7f097813d0c0 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): dump: 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.695 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.696 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2034352228 0 0) 0x7f097809d490 con 0x7f09780a2a70 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0958003570 con 0x7f09780a2a70 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1179854955 0 0) 0x7f0978137e00 con 0x7f097809b750 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f097809d490 con 0x7f097809b750 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0968001770 con 0x7f09780a2a70 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): dump: 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.697 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.698 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f096c001510 con 0x7f097809b750 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient(hunting): dump: 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.699 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 704610371 0 0) 0x7f097806e2f0 con 0x7f097813d0c0 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.122:6791/0 conn(0x7f097809b750 legacy=0x7f097806a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.26:6791/0 conn(0x7f09780a2a70 legacy=0x7f097806a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 monclient: found mon.b 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097806b0a0 con 0x7f097813d0c0 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f09667fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 5 monclient: authenticate success, global_id 204707 2024-09-15T04:00:16.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: _renew_subs 2024-09-15T04:00:16.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:16.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097806db10 con 0x7f097813d0c0 2024-09-15T04:00:16.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: _renew_subs 2024-09-15T04:00:16.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.674+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f097813bbb0 con 0x7f097813d0c0 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0970001f60 con 0x7f097813d0c0 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0970002490 con 0x7f097813d0c0 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f0974d99700 4 set_mon_vals no callback set 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: dump: 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.705 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f09700025f0 con 0x7f097813d0c0 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f097759e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(728..728 src has 435..728) v4 ==== 3939+0+0 (unknown 3055368912 0 0) 0x7f09700416b0 con 0x7f097813d0c0 2024-09-15T04:00:16.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f095c006590 con 0x7f095c002840 2024-09-15T04:00:16.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f097659c700 10 monclient: get_auth_request con 0x7f09580382a0 auth_method 0 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983392273 0 0) 0x7f095c006590 con 0x7f095c002840 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient(hunting): my global_id is 6143 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f097806e2f0 con 0x7f095c002840 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1606773441 0 0) 0x7f097806e2f0 con 0x7f095c002840 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f095c006590 con 0x7f095c002840 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0968001c00 con 0x7f095c002840 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: dump: 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.710 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2440895824 0 0) 0x7f095c006590 con 0x7f095c002840 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f095c00a130 con 0x7f095c002840 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f09680053c0 con 0x7f095c002840 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.122:6791/0 conn(0x7f095c002840 legacy=0x7f095c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.678+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.122:6791/0 conn(0x7f095c002840 legacy=0x7f095c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f097759e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f095c009640 con 0x7f095c0090b0 2024-09-15T04:00:16.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065338261 0 0) 0x7f095c009640 con 0x7f095c0090b0 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 10 monclient(hunting): my global_id is 6152 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f095c006590 con 0x7f095c0090b0 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3662903476 0 0) 0x7f095c006590 con 0x7f095c0090b0 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f095c009640 con 0x7f095c0090b0 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0968005bd0 con 0x7f095c0090b0 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 10 monclient: dump: 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.712 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1448775644 0 0) 0x7f095c009640 con 0x7f095c0090b0 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.690+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f095c0098f0 con 0x7f095c0090b0 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7741 (unknown 0 0 3721785497) 0x7f09680083b0 con 0x7f095c0090b0 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f09667fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f09667fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.122:6791/0 conn(0x7f095c0090b0 legacy=0x7f095c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f09667fc700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.122:6791/0 conn(0x7f095c0090b0 legacy=0x7f095c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:00:16.714 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:00:16.715 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.716 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:16.717 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:16.718 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:16.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:16.721 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:16.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.726 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.727 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.731 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:16.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.733 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.122:6812/20895 conn(0x7f09580382a0 legacy=0x7f095803a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 10 monclient: shutdown 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 >> v1:172.21.15.26:6790/0 conn(0x7f097813d0c0 legacy=0x7f097814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f0976d9d700 1 -- 172.21.15.26:0/909502251 reap_dead start 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f0976d9d700 1 -- 172.21.15.26:0/909502251 reap_dead start 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 shutdown_connections 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 >> 172.21.15.26:0/909502251 conn(0x7f0978063cd0 msgr2=0x7f097806f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 shutdown_connections 2024-09-15T04:00:16.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.694+0000 7f097759e700 1 -- 172.21.15.26:0/909502251 wait complete. 2024-09-15T04:00:16.737 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:16.737 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T04:00:16.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T04:00:16.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.846+0000 7f195d707700 10 monclient: get_monmap_and_config 2024-09-15T04:00:16.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.846+0000 7f195d707700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.846+0000 7f195d707700 10 monclient: monmap: 2024-09-15T04:00:16.854 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.854 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:16.850998+0000 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:16.850998+0000 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.855 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.846+0000 7f195d707700 1 Processor -- start 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.846+0000 7f195d707700 1 -- start start 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.846+0000 7f195d707700 10 monclient: init 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient: _add_conns ranks=[6,5,7,3,8,4,1,2,0] 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient(hunting): picked mon.noname-d con 0x7f1958064390 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient(hunting): picked mon.noname-h con 0x7f195806c580 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient(hunting): picked mon.noname-i con 0x7f1958138cb0 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195804ca10 con 0x7f195806c580 2024-09-15T04:00:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195806bc80 con 0x7f1958064390 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195806d2b0 con 0x7f1958138cb0 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient(hunting): authenticate will time out at 1761.280389s 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f1957fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1958064390 0x7f195806a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37050/0 (socket says 172.21.15.26:37050) 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f1957fff700 1 -- 172.21.15.26:0/3771894777 learned_addr learned my addr 172.21.15.26:0/3771894777 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556730740 0 0) 0x7f195806bc80 con 0x7f1958064390 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient(hunting): my global_id is 194937 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1944003570 con 0x7f1958064390 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1264571721 0 0) 0x7f1944003570 con 0x7f1958064390 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 >> v1:172.21.15.122:6792/0 conn(0x7f1958138cb0 legacy=0x7f195813b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 >> v1:172.21.15.122:6791/0 conn(0x7f195806c580 legacy=0x7f195806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 monclient: found mon.noname-d 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195806b630 con 0x7f1958064390 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f19480017b0 con 0x7f1958064390 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 5 monclient: authenticate success, global_id 194937 2024-09-15T04:00:16.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: dump: 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.862 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1948001a60 con 0x7f1958064390 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 1 -- 172.21.15.26:0/3771894777 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1948004c30 con 0x7f1958064390 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 4 set_mon_vals no callback set 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f19577fe700 10 monclient: dump: 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.864 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient: shutdown 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- 172.21.15.26:0/3771894777 >> v1:172.21.15.26:6792/0 conn(0x7f1958064390 legacy=0x7f195806a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- 172.21.15.26:0/3771894777 shutdown_connections 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- 172.21.15.26:0/3771894777 >> 172.21.15.26:0/3771894777 conn(0x7f1958063340 msgr2=0x7f1958063740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- 172.21.15.26:0/3771894777 shutdown_connections 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- 172.21.15.26:0/3771894777 wait complete. 2024-09-15T04:00:16.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient: build_initial_monmap 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 10 monclient: monmap: 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:16.866 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:16.867 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 Processor -- start 2024-09-15T04:00:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.850+0000 7f195d707700 1 -- start start 2024-09-15T04:00:16.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: init 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: _add_conns ranks=[0,7,1,4,8,6,2,3,5] 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient(hunting): picked mon.noname-a con 0x7f1958064390 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient(hunting): picked mon.noname-i con 0x7f195806c580 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient(hunting): picked mon.noname-f con 0x7f1958138cb0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195809d330 con 0x7f1958064390 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195806cad0 con 0x7f1958138cb0 2024-09-15T04:00:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195809cfc0 con 0x7f195806c580 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient(hunting): authenticate will time out at 1761.284778s 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1957fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1958064390 0x7f19581314b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59306/0 (socket says 172.21.15.26:59306) 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1957fff700 1 -- 172.21.15.26:0/906902581 learned_addr learned my addr 172.21.15.26:0/906902581 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912847891 0 0) 0x7f195809d330 con 0x7f1958064390 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): my global_id is 194760 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1938003570 con 0x7f1958064390 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827071242 0 0) 0x7f195809cfc0 con 0x7f195806c580 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): my global_id is 234331 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f195809d330 con 0x7f195806c580 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645024254 0 0) 0x7f195806cad0 con 0x7f1958138cb0 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): my global_id is 204670 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f195809cfc0 con 0x7f1958138cb0 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2756287635 0 0) 0x7f1938003570 con 0x7f1958064390 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f195806cad0 con 0x7f1958064390 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1948001770 con 0x7f1958064390 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 958187090 0 0) 0x7f195809d330 con 0x7f195806c580 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1938003570 con 0x7f195806c580 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 407055210 0 0) 0x7f195806cad0 con 0x7f1958064390 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6792/0 conn(0x7f195806c580 legacy=0x7f1958131bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6789/0 conn(0x7f1958138cb0 legacy=0x7f19581322d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 monclient: found mon.a 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195809cce0 con 0x7f1958064390 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: _finish_auth 0 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 5 monclient: authenticate success, global_id 194760 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: _renew_subs 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195809d9a0 con 0x7f1958064390 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: _renew_subs 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1958133110 con 0x7f1958064390 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1948004f00 con 0x7f1958064390 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f19480053b0 con 0x7f1958064390 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1956ffd700 4 set_mon_vals no callback set 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f1954ff9700 10 monclient: dump: 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.854+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f193c006590 con 0x7f193c002840 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f194800fd70 con 0x7f1958064390 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(728..728 src has 435..728) v4 ==== 3939+0+0 (unknown 3055368912 0 0) 0x7f1948044180 con 0x7f1958064390 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764473624 0 0) 0x7f193c006590 con 0x7f193c002840 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient(hunting): my global_id is 234340 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f195806cad0 con 0x7f193c002840 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f195cf06700 10 monclient: get_auth_request con 0x7f1938037950 auth_method 0 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2764843486 0 0) 0x7f195806cad0 con 0x7f193c002840 2024-09-15T04:00:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f193c006590 con 0x7f193c002840 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1940002240 con 0x7f193c002840 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient: dump: 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1865887241 0 0) 0x7f193c006590 con 0x7f193c002840 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f193c00a130 con 0x7f193c002840 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f19400025d0 con 0x7f193c002840 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6792/0 conn(0x7f193c002840 legacy=0x7f193c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.858+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6792/0 conn(0x7f193c002840 legacy=0x7f193c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f195d707700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f193c009600 con 0x7f193c009070 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955877169 0 0) 0x7f193c009600 con 0x7f193c009070 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 10 monclient(hunting): my global_id is 234349 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f193c006590 con 0x7f193c009070 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 931988676 0 0) 0x7f193c006590 con 0x7f193c009070 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f193c009600 con 0x7f193c009070 2024-09-15T04:00:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1940003000 con 0x7f193c009070 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.870+0000 7f1954ff9700 10 monclient: dump: 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3371314649 0 0) 0x7f193c009600 con 0x7f193c009070 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f193c0098b0 con 0x7f193c009070 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3378759422) 0x7f1940005270 con 0x7f193c009070 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6792/0 conn(0x7f193c009070 legacy=0x7f193c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f1954ff9700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6792/0 conn(0x7f193c009070 legacy=0x7f193c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:16.888 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.890 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:16.891 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:16.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:16.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.894 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:16.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.898 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:16.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:16.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:16.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:16.909 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:16.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:16.909 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:16.909 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:16.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.122:6812/20895 conn(0x7f1938037950 legacy=0x7f1938039e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 10 monclient: shutdown 2024-09-15T04:00:16.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:16.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 >> v1:172.21.15.26:6789/0 conn(0x7f1958064390 legacy=0x7f19581314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:16.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195cf06700 1 -- 172.21.15.26:0/906902581 reap_dead start 2024-09-15T04:00:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195cf06700 1 -- 172.21.15.26:0/906902581 reap_dead start 2024-09-15T04:00:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 shutdown_connections 2024-09-15T04:00:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 >> 172.21.15.26:0/906902581 conn(0x7f1958063340 msgr2=0x7f19581291f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 shutdown_connections 2024-09-15T04:00:16.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:16.874+0000 7f195d707700 1 -- 172.21.15.26:0/906902581 wait complete. 2024-09-15T04:00:16.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:16.911 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T04:00:16.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T04:00:17.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: get_monmap_and_config 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: build_initial_monmap 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: monmap: 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:17.024503+0000 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:17.024503+0000 2024-09-15T04:00:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:17.029 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:17.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 1 Processor -- start 2024-09-15T04:00:17.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 1 -- start start 2024-09-15T04:00:17.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: init 2024-09-15T04:00:17.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: _add_conns ranks=[3,4,6,8,1,7,2,0,5] 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient(hunting): picked mon.noname-g con 0x7fd22c070de0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient(hunting): picked mon.noname-c con 0x7fd22c0950b0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient(hunting): picked mon.noname-d con 0x7fd22c095b90 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c04ca10 con 0x7fd22c070de0 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c068980 con 0x7fd22c0950b0 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c0711b0 con 0x7fd22c095b90 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient(hunting): authenticate will time out at 1761.454145s 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd232c10700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd22c095b90 0x7fd22c137f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37064/0 (socket says 172.21.15.26:37064) 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd231c0e700 1 --1- 172.21.15.26:0/1818333015 >> v1:172.21.15.26:6791/0 conn(0x7fd22c0950b0 0x7fd22c095480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40922/0 (socket says 172.21.15.26:40922) 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd232c10700 1 -- 172.21.15.26:0/1818333015 learned_addr learned my addr 172.21.15.26:0/1818333015 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186667443 0 0) 0x7fd22c068980 con 0x7fd22c0950b0 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): my global_id is 204664 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd218003570 con 0x7fd22c0950b0 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285703506 0 0) 0x7fd22c0711b0 con 0x7fd22c095b90 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): my global_id is 194946 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c068980 con 0x7fd22c095b90 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661620889 0 0) 0x7fd22c04ca10 con 0x7fd22c070de0 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient(hunting): my global_id is 244164 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c0711b0 con 0x7fd22c070de0 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2254300029 0 0) 0x7fd218003570 con 0x7fd22c0950b0 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 >> v1:172.21.15.26:6792/0 conn(0x7fd22c095b90 legacy=0x7fd22c137f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 >> v1:172.21.15.122:6790/0 conn(0x7fd22c070de0 legacy=0x7fd22c09bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: _finish_hunting 0 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 monclient: found mon.noname-c 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c09aea0 con 0x7fd22c0950b0 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: _finish_auth 0 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 5 monclient: authenticate success, global_id 204664 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd2280017b0 con 0x7fd22c0950b0 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: dump: 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:17.044 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd228001a90 con 0x7fd22c0950b0 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd233411700 4 set_mon_vals no callback set 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 1 -- 172.21.15.26:0/1818333015 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd228004b70 con 0x7fd22c0950b0 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.022+0000 7fd23140d700 10 monclient: dump: 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient: shutdown 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- 172.21.15.26:0/1818333015 >> v1:172.21.15.26:6791/0 conn(0x7fd22c0950b0 legacy=0x7fd22c095480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- 172.21.15.26:0/1818333015 shutdown_connections 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- 172.21.15.26:0/1818333015 >> 172.21.15.26:0/1818333015 conn(0x7fd22c0a24e0 msgr2=0x7fd22c0a28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- 172.21.15.26:0/1818333015 shutdown_connections 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- 172.21.15.26:0/1818333015 wait complete. 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient: build_initial_monmap 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient: monmap: 2024-09-15T04:00:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 Processor -- start 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- start start 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient: init 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient: _add_conns ranks=[5,6,4,7,8,0,2,1,3] 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient(hunting): picked mon.noname-h con 0x7fd22c070de0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient(hunting): picked mon.noname-d con 0x7fd22c0950b0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient(hunting): picked mon.noname-c con 0x7fd22c095b90 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c12fcc0 con 0x7fd22c095b90 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c09c720 con 0x7fd22c070de0 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c12a5e0 con 0x7fd22c0950b0 2024-09-15T04:00:17.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd233411700 10 monclient(hunting): authenticate will time out at 1761.459880s 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd231c0e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd22c0950b0 0x7fd22c141d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37074/0 (socket says 172.21.15.26:37074) 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.026+0000 7fd231c0e700 1 -- 172.21.15.26:0/1070711181 learned_addr learned my addr 172.21.15.26:0/1070711181 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683133937 0 0) 0x7fd22c12a5e0 con 0x7fd22c0950b0 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): my global_id is 194955 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210003570 con 0x7fd22c0950b0 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663298241 0 0) 0x7fd22c12fcc0 con 0x7fd22c095b90 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): my global_id is 204673 2024-09-15T04:00:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c12a5e0 con 0x7fd22c095b90 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821620733 0 0) 0x7fd22c09c720 con 0x7fd22c070de0 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): my global_id is 6170 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c12fcc0 con 0x7fd22c070de0 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1922288753 0 0) 0x7fd210003570 con 0x7fd22c0950b0 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd22c09c720 con 0x7fd22c0950b0 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1862684229 0 0) 0x7fd22c12a5e0 con 0x7fd22c095b90 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd210003570 con 0x7fd22c095b90 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd228001770 con 0x7fd22c0950b0 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:17.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd224001c80 con 0x7fd22c095b90 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2347739107 0 0) 0x7fd22c12fcc0 con 0x7fd22c070de0 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd22c12a5e0 con 0x7fd22c070de0 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd220001a60 con 0x7fd22c070de0 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:17.055 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 608945401 0 0) 0x7fd210003570 con 0x7fd22c095b90 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 >> v1:172.21.15.26:6792/0 conn(0x7fd22c0950b0 legacy=0x7fd22c141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 >> v1:172.21.15.122:6791/0 conn(0x7fd22c070de0 legacy=0x7fd22c09ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 monclient: found mon.c 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c06a260 con 0x7fd22c095b90 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:17.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 5 monclient: authenticate success, global_id 204673 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 monclient: _renew_subs 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c142ba0 con 0x7fd22c095b90 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 monclient: _renew_subs 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c130c20 con 0x7fd22c095b90 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd224004bb0 con 0x7fd22c095b90 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd224005060 con 0x7fd22c095b90 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:17.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd230c0c700 4 set_mon_vals no callback set 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 10 monclient: dump: 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c04bbf0 con 0x7fd22c095b90 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.030+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd22400fa20 con 0x7fd22c095b90 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.034+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(728..728 src has 435..728) v4 ==== 3939+0+0 (unknown 3055368912 0 0) 0x7fd224044040 con 0x7fd22c095b90 2024-09-15T04:00:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.034+0000 7fd23240f700 10 monclient: get_auth_request con 0x7fd210038170 auth_method 0 2024-09-15T04:00:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.038+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fd224014eb0 con 0x7fd22c095b90 2024-09-15T04:00:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.038+0000 7fd21e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:17.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.038+0000 7fd21e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:3229: done (3 left) 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:3230: done (2 left) 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:3231: done (1 left) 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:3232: done (0 left) 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:3233: rollback oid 35 current snap is 377 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 347 2024-09-15T04:00:17.132 INFO:tasks.rados.rados.0.smithi122.stdout:3234: write oid 8 current snap is 377 2024-09-15T04:00:17.133 INFO:tasks.rados.rados.0.smithi122.stdout:3234: seq_num 1143 ranges {410401=662557,1719036=770066,2938716=271685} 2024-09-15T04:00:17.138 INFO:tasks.rados.rados.0.smithi122.stdout:3234: writing smithi12223906-8 from 410401 to 1072958 tid 1 2024-09-15T04:00:17.146 INFO:tasks.rados.rados.0.smithi122.stdout:3234: writing smithi12223906-8 from 1719036 to 2489102 tid 2 2024-09-15T04:00:17.150 INFO:tasks.rados.rados.0.smithi122.stdout:3234: writing smithi12223906-8 from 2938716 to 3210401 tid 3 2024-09-15T04:00:17.150 INFO:tasks.rados.rados.0.smithi122.stdout:3235: write (excl) oid 2 current snap is 377 2024-09-15T04:00:17.150 INFO:tasks.rados.rados.0.smithi122.stdout:3235: seq_num 1144 ranges {427208=737806,1777518=513724,2966139=261069} 2024-09-15T04:00:17.157 INFO:tasks.rados.rados.0.smithi122.stdout:3235: writing smithi12223906-2 from 427208 to 1165014 tid 1 2024-09-15T04:00:17.163 INFO:tasks.rados.rados.0.smithi122.stdout:3235: writing smithi12223906-2 from 1777518 to 2291242 tid 2 2024-09-15T04:00:17.169 INFO:tasks.rados.rados.0.smithi122.stdout:3235: writing smithi12223906-2 from 2966139 to 3227208 tid 3 2024-09-15T04:00:17.169 INFO:tasks.rados.rados.0.smithi122.stdout:3236: write (excl) oid 3 current snap is 377 2024-09-15T04:00:17.169 INFO:tasks.rados.rados.0.smithi122.stdout:3233: finishing rollback tid 0 to smithi12223906-35 2024-09-15T04:00:17.169 INFO:tasks.rados.rados.0.smithi122.stdout:3236: seq_num 1145 ranges {444015=413055,1436000=773735,2709915=534100} 2024-09-15T04:00:17.174 INFO:tasks.rados.rados.0.smithi122.stdout:3236: writing smithi12223906-3 from 444015 to 857070 tid 1 2024-09-15T04:00:17.182 INFO:tasks.rados.rados.0.smithi122.stdout:3236: writing smithi12223906-3 from 1436000 to 2209735 tid 2 2024-09-15T04:00:17.188 INFO:tasks.rados.rados.0.smithi122.stdout:3236: writing smithi12223906-3 from 2709915 to 3244015 tid 3 2024-09-15T04:00:17.188 INFO:tasks.rados.rados.0.smithi122.stdout:3237: write oid 31 current snap is 377 2024-09-15T04:00:17.188 INFO:tasks.rados.rados.0.smithi122.stdout:3233: finishing rollback tid 1 to smithi12223906-35 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3233: finishing rollback tid 2 to smithi12223906-35 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1843 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3234: finishing write tid 1 to smithi12223906-8 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3234: finishing write tid 2 to smithi12223906-8 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3234: finishing write tid 3 to smithi12223906-8 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3234: finishing write tid 4 to smithi12223906-8 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3234: finishing write tid 5 to smithi12223906-8 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1918 (ObjNum 1143 snap 377 seq_num 1143) dirty exists 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3234: left oid 8 (ObjNum 1143 snap 377 seq_num 1143) 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3235: finishing write tid 1 to smithi12223906-2 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3235: finishing write tid 2 to smithi12223906-2 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3235: finishing write tid 3 to smithi12223906-2 2024-09-15T04:00:17.189 INFO:tasks.rados.rados.0.smithi122.stdout:3235: finishing write tid 4 to smithi12223906-2 2024-09-15T04:00:17.190 INFO:tasks.rados.rados.0.smithi122.stdout:3235: finishing write tid 5 to smithi12223906-2 2024-09-15T04:00:17.190 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1085 (ObjNum 1144 snap 377 seq_num 1144) dirty exists 2024-09-15T04:00:17.190 INFO:tasks.rados.rados.0.smithi122.stdout:3235: left oid 2 (ObjNum 1144 snap 377 seq_num 1144) 2024-09-15T04:00:17.190 INFO:tasks.rados.rados.0.smithi122.stdout:3236: finishing write tid 1 to smithi12223906-3 2024-09-15T04:00:17.190 INFO:tasks.rados.rados.0.smithi122.stdout:3237: seq_num 1146 ranges {460822=488304,1378129=517393,2504632=405962} 2024-09-15T04:00:17.193 INFO:tasks.rados.rados.0.smithi122.stdout:3237: writing smithi12223906-31 from 460822 to 949126 tid 1 2024-09-15T04:00:17.199 INFO:tasks.rados.rados.0.smithi122.stdout:3237: writing smithi12223906-31 from 1378129 to 1895522 tid 2 2024-09-15T04:00:17.206 INFO:tasks.rados.rados.0.smithi122.stdout:3237: writing smithi12223906-31 from 2504632 to 2910594 tid 3 2024-09-15T04:00:17.206 INFO:tasks.rados.rados.0.smithi122.stdout:3233: done (4 left) 2024-09-15T04:00:17.206 INFO:tasks.rados.rados.0.smithi122.stdout:3234: done (3 left) 2024-09-15T04:00:17.206 INFO:tasks.rados.rados.0.smithi122.stdout:3235: done (2 left) 2024-09-15T04:00:17.206 INFO:tasks.rados.rados.0.smithi122.stdout:3238: snap_remove snap 327 2024-09-15T04:00:17.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.294+0000 7fd233411700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T04:00:17.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.294+0000 7fd233411700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T04:00:17.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.294+0000 7fd233411700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:17.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.294+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fd22c130e10 con 0x7fd22c095b90 2024-09-15T04:00:17.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.298+0000 7fd21e7fc700 1 -- 172.21.15.26:0/1070711181 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fd2240148b0 con 0x7fd22c095b90 2024-09-15T04:00:17.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.298+0000 7fd21e7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T04:00:17.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.298+0000 7fd21e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:17.309 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.309 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.309 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=704553441,config=3140514009,health=2096750356,kv=2519098299,logm=482402623}) 2024-09-15T04:00:17.309 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=3498835726,config=3140514009,health=2558632036,kv=2519098299,logm=2277830733}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=704553441,config=3140514009,health=2096750356,kv=2519098299,logm=3946676373}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=704553441,config=3140514009,health=2096750356,kv=2519098299,logm=3178929939}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=26,config=1,health=11,kv=8,logm=54} crc {auth=3325102978,config=1228700967,health=3491979032,kv=2519098299,logm=3410832957}) 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.310 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=11,kv=7,logm=54} crc {auth=2137560646,config=3140514009,health=1866747259,kv=777372614,logm=2348907370}) 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=27,config=2,health=11,kv=7,logm=53} crc {auth=4034656387,config=3140514009,health=208967021,kv=1358559433,logm=885838547}) 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=24,config=2,health=11,kv=8,logm=55} crc {auth=2700738082,config=3140514009,health=208967021,kv=2519098299,logm=3412430673}) 2024-09-15T04:00:17.311 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.303+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=1883548091,config=3140514009,health=3491979032,kv=2519098299,logm=2228996956}) 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 >> v1:172.21.15.122:6812/20895 conn(0x7fd210038170 legacy=0x7fd21003a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 10 monclient: shutdown 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 >> v1:172.21.15.26:6791/0 conn(0x7fd22c095b90 legacy=0x7fd22c142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 shutdown_connections 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 >> 172.21.15.26:0/1070711181 conn(0x7fd22c0a24e0 msgr2=0x7fd22c1295c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 shutdown_connections 2024-09-15T04:00:17.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:17.302+0000 7fd233411700 1 -- 172.21.15.26:0/1070711181 wait complete. 2024-09-15T04:00:17.316 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.316 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.316 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3620218237}) 2024-09-15T04:00:17.316 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3384631677}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2297029414}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=919410217}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3201550618}) 2024-09-15T04:00:17.317 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.318 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.318 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1508418364}) 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2881780413}) 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=200121493}) 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.307+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1046438818}) 2024-09-15T04:00:17.321 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3288226759}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2651865139}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2047377028}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2902351608}) 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.322 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4167044167}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1076574983}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=196699179}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3385807730}) 2024-09-15T04:00:17.323 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3026513518}) 2024-09-15T04:00:17.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T04:00:17.336 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=359099456}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=421047326}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3191083367}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.337 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1008461792}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1910705958}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=20600525}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1004139393}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3651423713}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.311+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4017559507}) 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.338 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1715249530}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1963878824}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2892327182}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=682487058}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.339 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1554711770}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=317507014}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2528629700}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3283211810}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1472182803}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1740858437}) 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.340 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2558306758}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3497384302}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3274985801}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=973619335}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.341 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3936755280}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1826043409}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2865159173}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.315+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2290036847}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3428592946}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1526198216}) 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.342 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1949755229}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=41043283}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4112571932}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=569407864}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.343 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=901929188}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1223281887}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1129233493}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2179283014}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2162120867}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2152235090}) 2024-09-15T04:00:17.344 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3012268007}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2423903207}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1105449809}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4270477390}) 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.345 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3678321606}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.319+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=403479042}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2898282946}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=482323979}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3034966443}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=404083661}) 2024-09-15T04:00:17.346 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4199322179}) 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=773526836}) 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=50370327}) 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.347 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3296619558}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=130205458}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2849706894}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=924695956}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=216739448}) 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.348 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3881699171}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=730507056}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=382617354}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3147866826}) 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.349 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=287875618}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.323+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1617076028}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1707387509}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2729056806}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2846310474}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=497681466}) 2024-09-15T04:00:17.350 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=165315244}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=919180648}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1011036790}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1486720918}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1968295562}) 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.351 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=320488752}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2475011815}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1536177130}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2793057395}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.352 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2436217634}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3060730937}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=205701087}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1367954142}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.327+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4201105367}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=605590161}) 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.353 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2121035309}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3788725338}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=119819955}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1316112422}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1706346174}) 2024-09-15T04:00:17.354 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.355 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.355 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2868049885}) 2024-09-15T04:00:17.355 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.355 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3275542163}) 2024-09-15T04:00:17.355 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2699980385}) 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1477304234}) 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1386913222}) 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.417 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1001734730}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1405590140}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4267653640}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2655354492}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1183223337}) 2024-09-15T04:00:17.418 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4024027863}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.331+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3539992805}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4170478836}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=413757916}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2979542165}) 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.419 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=487288477}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2179402800}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=420130755}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1276330994}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1011214639}) 2024-09-15T04:00:17.420 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1683881141}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=884939714}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=614281182}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3494365159}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1027054276}) 2024-09-15T04:00:17.421 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1960484757}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1521033472}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3533233676}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1372541681}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.335+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=314962546}) 2024-09-15T04:00:17.422 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1946806788}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=979144220}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2962907512}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2028519747}) 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.423 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4201868694}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3779853927}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2180473135}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2668307068}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1747320772}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.424 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2187826325}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1699307698}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1307127920}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1278804253}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1516454076}) 2024-09-15T04:00:17.425 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4279742703}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4119083779}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3953120018}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.339+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=294887657}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2434692072}) 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.426 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3956609539}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2882071002}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=584131472}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2006267219}) 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.427 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2021709831}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=422216123}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2249737312}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2857067179}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=295750134}) 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.428 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=340413695}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3042651040}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2754355383}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=295584688}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.429 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4173703636}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=809734500}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2826651158}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.343+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1347330533}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=122034220}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.430 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2057825711}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2484420169}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3550126563}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2113052228}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4027932565}) 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.431 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2699000003}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4273178692}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3501919481}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3450480142}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=464695893}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.432 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1051672406}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2886049445}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1557101044}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1918510288}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.433 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4053414349}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1802147639}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.347+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1358536818}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2589939000}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3284957577}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3534134610}) 2024-09-15T04:00:17.434 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2069268119}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=835787951}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2441290878}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2590922540}) 2024-09-15T04:00:17.435 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1261458458}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2709299562}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3461758505}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3709486345}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1384376676}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.436 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=829540966}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=618833080}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2300178666}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2660277009}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3503060402}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.351+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1254682509}) 2024-09-15T04:00:17.437 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1845810103}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4234046145}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2205652235}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3150974503}) 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.438 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4989350}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1945571162}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1122106973}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1814523268}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1553014204}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.355+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.439 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2320354092}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4231426885}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3104079670}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3951513821}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=753727232}) 2024-09-15T04:00:17.440 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=344211844}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=880479402}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=409931716}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=229069920}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=134622408}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.441 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=763297730}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3730612587}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3880092900}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1725580261}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=171518432}) 2024-09-15T04:00:17.442 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=91197820}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2629434111}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.359+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3350696744}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2049831021}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3319812180}) 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.443 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3635993700}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2618433436}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3958007677}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4163027810}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2395403697}) 2024-09-15T04:00:17.444 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=688864185}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1862478337}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=69100372}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=724043530}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=653430549}) 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.445 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=168786512}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=505243497}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3874810662}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848705566}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4088128898}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.363+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=192888728}) 2024-09-15T04:00:17.446 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4221372952}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1491457268}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2279539267}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=630939146}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.447 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=292511021}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2234312867}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2416071596}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=610701981}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=210187287}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1349394664}) 2024-09-15T04:00:17.448 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=672523951}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4270323079}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1562793388}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2608257109}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.449 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3068166820}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1770787934}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3033747337}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.367+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=203699566}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3089580846}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4000833795}) 2024-09-15T04:00:17.450 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262375953}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2495636385}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=278577135}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3344320066}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.451 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3481764229}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=965771553}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2064013781}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1601649280}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1120617344}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2887843160}) 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.452 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3977399656}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3085063008}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3504782330}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3043313854}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2506847974}) 2024-09-15T04:00:17.453 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.371+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2041741097}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3829899667}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2833528987}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1943177398}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2689073255}) 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.454 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2736991109}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=501659913}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3858883881}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4198839127}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1598794114}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.455 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1335863866}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2251860611}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3530903032}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4208075782}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1865704338}) 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.456 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3257670495}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3301592967}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3426398308}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.375+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4004436957}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3317618215}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4007598867}) 2024-09-15T04:00:17.457 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2816828265}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1788103364}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4287046109}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2645730939}) 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.458 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1962188484}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1818678079}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.379+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=543370568}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4008950279}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2812541412}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.459 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=7842914}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=120078419}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=637948756}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2119718757}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2896705980}) 2024-09-15T04:00:17.460 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3556223717}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197390909}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2277282865}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3465155754}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1842571339}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.461 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3326651656}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3644832881}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2837136097}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2014255885}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1614591594}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.383+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3006427830}) 2024-09-15T04:00:17.462 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4014249923}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1615073560}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2715265534}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3963351692}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.463 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2500948141}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=359553934}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3060312757}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867351142}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1573798396}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3407162216}) 2024-09-15T04:00:17.464 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1738363162}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1098355389}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2536723232}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=453273580}) 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.465 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=714610305}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2476630179}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=358975331}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.387+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1507916131}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3331692506}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.466 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1566998856}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4090096020}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3898906513}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3721400273}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3693138102}) 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.467 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2504673314}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4291002349}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099215510}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=147248421}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2312693915}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.468 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3058449279}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=201950261}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=730564622}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4180502645}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2856525801}) 2024-09-15T04:00:17.469 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3780909250}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.391+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2510443516}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1609487187}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3400610496}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=700923122}) 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.470 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4089117729}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=52361327}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2075555410}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2134537850}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1154272802}) 2024-09-15T04:00:17.471 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1856985387}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=462462062}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4055542835}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=216326835}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.472 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1564985948}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3271055236}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3737010387}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4165627227}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=383903874}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.395+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2860195094}) 2024-09-15T04:00:17.473 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3358075694}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2774028847}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=807867936}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=253674192}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.474 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=215939467}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=839317295}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=577620223}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3869286693}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2805811431}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.475 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=414993364}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2986750084}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2484364253}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=474760865}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.476 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1699231666}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4290964258}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=219185812}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3811172815}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.399+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3614349816}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2415089459}) 2024-09-15T04:00:17.477 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2172111776}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1386602540}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1127438228}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2929049757}) 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.478 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2266452508}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1442126897}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=468013874}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.403+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=412898293}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=863424181}) 2024-09-15T04:00:17.479 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4114548845}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3621591305}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4280030377}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3649187143}) 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.480 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2281059695}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2684981628}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1182469439}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2611287702}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=869842311}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.481 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1045212379}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1801760556}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3337454590}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=102321804}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.482 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3856835801}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=821111892}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=11922789}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.407+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1348854468}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284505193}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3730766673}) 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.483 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=485968265}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=182166694}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4106654896}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=550684403}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.484 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1342244991}) 2024-09-15T04:00:17.485 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.485 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4085995811}) 2024-09-15T04:00:17.485 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=454746591}) 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=838983869}) 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1634683427}) 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.532 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1862764363}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4167977074}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1744274622}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4148082612}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2446858750}) 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.533 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1519309961}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.411+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3207351307}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2352988292}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3405538317}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1087807466}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.534 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3631815629}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4182503775}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3271603450}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1501506284}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3031746434}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2877910982}) 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.535 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1234723289}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.415+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3682241177}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3507102819}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=171572233}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.536 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2087418036}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3000386769}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1380627673}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=599290704}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3849371939}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1596507548}) 2024-09-15T04:00:17.537 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1638412463}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=714040040}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2494274615}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2775848692}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.538 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=882141479}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3063511199}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2346910172}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.419+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2569728901}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3893329694}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=382467456}) 2024-09-15T04:00:17.539 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2222847556}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=347471703}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1768427154}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3703310979}) 2024-09-15T04:00:17.540 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2614868067}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=471388455}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=953505342}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3610149381}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1216924343}) 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.541 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3475645415}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=343258083}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3386831656}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=108249582}) 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.542 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3647649066}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2078750488}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.423+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3410788010}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2139949910}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1191896734}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3080253776}) 2024-09-15T04:00:17.543 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1455145695}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1255154872}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3314459182}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2901961077}) 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.544 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2097652962}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.427+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2628492963}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3192733507}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1293996217}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3192134204}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.545 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=883603488}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2229204409}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2828608388}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1514222976}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=548469128}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=475920270}) 2024-09-15T04:00:17.546 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1022015490}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1828128457}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=201532461}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3642884901}) 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.547 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=929842902}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4153940575}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1285813326}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3208606613}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.431+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=511035837}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.548 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=177498389}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=770120058}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=559265179}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1201609732}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1396150753}) 2024-09-15T04:00:17.549 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1593715636}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=935863277}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3732293782}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=484217046}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=747737201}) 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.550 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2433271343}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2749913437}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4282881742}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3868462708}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.551 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3152044312}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1648923702}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=161200426}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.435+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1333450265}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2746574489}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3771280644}) 2024-09-15T04:00:17.552 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1256001424}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1780061756}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1152075159}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2788073931}) 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.553 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=453530530}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1617832453}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=173806818}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3613474908}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1861606790}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1135437158}) 2024-09-15T04:00:17.554 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4156958990}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.439+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3739743139}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1823952061}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3566224784}) 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.555 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=625336337}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2540686983}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1662099544}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2027252754}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4205767036}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.556 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2716336537}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1749048240}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=429553465}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4271445210}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1358142154}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.443+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2134443556}) 2024-09-15T04:00:17.557 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3958176404}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3682532958}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2942098269}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=885023547}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.558 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=657979609}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2717015636}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3678382818}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=865495642}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3360501307}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2013782157}) 2024-09-15T04:00:17.559 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3117950098}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2999726899}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=632864327}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=907559568}) 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.560 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=529683409}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=295641569}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=800645613}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.447+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=439266340}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3463331545}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.561 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1031442099}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3452744432}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1021773049}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=117679952}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1211495774}) 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.562 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2561462206}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1724340814}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.451+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1222004687}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=80,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=8} crc {logm=1143458224,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=3962887818}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=78,mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=1} crc {logm=4099359502,mdsmap=1228700967,mgr=1601764793,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=331471414,mon_config_key=551709721}) 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.563 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=80,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9} crc {logm=1125751864,mdsmap=3140514009,mgr=3669162908,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=4246148643}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=77,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=1} crc {logm=4192740152,mdsmap=1442564250,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=694698746,mon_config_key=551709721}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=79,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=3261686708,mdsmap=1531151339,mgr=3364684290,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2500276518}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=80,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=8} crc {logm=3570564075,mdsmap=1531151339,mgr=436553390,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2500276518}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.564 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=78,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9} crc {logm=3460604895,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=694698746}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=81,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=7} crc {logm=736409242,mdsmap=1531151339,mgr=3364684289,mgr_metadata=4176136816,mgrstat=2898930648}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=78,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=10} crc {logm=1424053491,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=3050569131}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=82} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2166149264,osd_snap=4280310641}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.455+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=83} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=74616947}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=82} crc {mgrstat=195126667,mon_config_key=763007189,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2166149264,osd_snap=3220815939}) 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.565 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=83} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=2270045001,osd_pg_creating=2166149264,osd_snap=3320296879}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=83} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_snap=4179320054}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=82} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=2166149264,osd_snap=1926055886}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=83} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=231491462,osd_pg_creating=2166149264,osd_snap=2938485187}) 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.566 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=84} crc {mgrstat=195126667,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=2166149264,osd_snap=1086951826}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mgrstat=3,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=83} crc {mgrstat=195126667,mon_config_key=733955277,monmap=4273209975,osd_metadata=2459566930,osd_pg_creating=2166149264,osd_snap=3111879710}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=58,osdmap=42} crc {osd_snap=1324041874,osdmap=840103163}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=57,osdmap=43} crc {osd_snap=3515131047,osdmap=692081114}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=53,osdmap=47} crc {osd_snap=2661333836,osdmap=786704444}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.567 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=56,osdmap=44} crc {osd_snap=4059812153,osdmap=2342464362}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=58,osdmap=42} crc {osd_snap=3685716152,osdmap=3782610466}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=4011482080,osdmap=3215506973}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=58,osdmap=42} crc {osd_snap=771489728,osdmap=127218301}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=55,osdmap=45} crc {osd_snap=2832640065,osdmap=2307597124}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.459+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=56,osdmap=44} crc {osd_snap=1587338231,osdmap=3454432502}) 2024-09-15T04:00:17.568 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2646578541}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1384811331}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=110027984}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2224050596}) 2024-09-15T04:00:17.569 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3104366689}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3622338098}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2803619722}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3674652105}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1185834268}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.570 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=506309932}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=234885240}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1790749938}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1644029042}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.571 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3899995863}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3303118533}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1586935870}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3660385650}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.463+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3760649734}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1115289173}) 2024-09-15T04:00:17.572 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1782537431}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=559559637}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2902843110}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2921287034}) 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.573 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3732126971}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1432297774}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3911675931}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.467+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=364910443}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=94} crc {osdmap=552104874}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.574 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=92} crc {osdmap=4130031293}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=92} crc {osdmap=1260394654}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=88} crc {osdmap=722289052}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=92} crc {osdmap=1257374193}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=92} crc {osdmap=1413235157}) 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.575 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.576 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=90} crc {osdmap=3022437954}) 2024-09-15T04:00:17.576 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:00:17.576 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=89} crc {osdmap=3487467963}) 2024-09-15T04:00:17.576 INFO:tasks.ceph.mon.h.smithi122.stderr:2024-09-15T04:00:17.471+0000 7f3304a28700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=90} crc {osdmap=366403052}) 2024-09-15T04:00:18.326 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T04:00:18.326 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:00:18.328 INFO:tasks.rados.rados.0.smithi122.stdout:3238: done (2 left) 2024-09-15T04:00:18.328 INFO:tasks.rados.rados.0.smithi122.stdout:3239: read oid 39 snap -1 2024-09-15T04:00:18.328 INFO:tasks.rados.rados.0.smithi122.stdout:3239: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:00:18.328 INFO:tasks.rados.rados.0.smithi122.stdout:3240: write oid 36 current snap is 377 2024-09-15T04:00:18.328 INFO:tasks.rados.rados.0.smithi122.stdout:3236: finishing write tid 2 to smithi12223906-3 2024-09-15T04:00:18.328 INFO:tasks.rados.rados.0.smithi122.stdout:3240: seq_num 1147 ranges {477629=563553,1836611=777404,3048408=229221} 2024-09-15T04:00:18.337 INFO:tasks.rados.rados.0.smithi122.stdout:3240: writing smithi12223906-36 from 477629 to 1041182 tid 1 2024-09-15T04:00:18.349 INFO:tasks.rados.rados.0.smithi122.stdout:3240: writing smithi12223906-36 from 1836611 to 2614015 tid 2 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3240: writing smithi12223906-36 from 3048408 to 3277629 tid 3 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3241: copy_from oid 29 from oid 33 current snap is 377 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3236: finishing write tid 3 to smithi12223906-3 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3236: finishing write tid 4 to smithi12223906-3 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3236: finishing write tid 5 to smithi12223906-3 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2597 (ObjNum 1145 snap 377 seq_num 1145) dirty exists 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3236: left oid 3 (ObjNum 1145 snap 377 seq_num 1145) 2024-09-15T04:00:18.353 INFO:tasks.rados.rados.0.smithi122.stdout:3237: finishing write tid 1 to smithi12223906-31 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3237: finishing write tid 2 to smithi12223906-31 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3237: finishing write tid 3 to smithi12223906-31 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3237: finishing write tid 4 to smithi12223906-31 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3237: finishing write tid 5 to smithi12223906-31 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 1922 (ObjNum 1146 snap 377 seq_num 1146) dirty exists 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3237: left oid 31 (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3236: done (4 left) 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3237: done (3 left) 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3242: snap_create 2024-09-15T04:00:18.354 INFO:tasks.rados.rados.0.smithi122.stdout:3239: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:00:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.418+0000 7ff7d0faf700 10 monclient: get_monmap_and_config 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.418+0000 7ff7d0faf700 10 monclient: build_initial_monmap 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.418+0000 7ff7d0faf700 10 monclient: monmap: 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:18.423466+0000 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:18.423466+0000 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:18.427 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:18.428 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 Processor -- start 2024-09-15T04:00:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- start start 2024-09-15T04:00:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient: init 2024-09-15T04:00:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient: _add_conns ranks=[3,0,1,2,6,5,7,8,4] 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient(hunting): picked mon.noname-g con 0x7ff7cc099910 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient(hunting): picked mon.noname-a con 0x7ff7cc12ff80 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient(hunting): picked mon.noname-f con 0x7ff7cc1303d0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc04ca10 con 0x7ff7cc12ff80 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc098be0 con 0x7ff7cc1303d0 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc099460 con 0x7ff7cc099910 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient(hunting): authenticate will time out at 1762.853443s 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7caffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff7cc12ff80 0x7ff7cc134ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59314/0 (socket says 172.21.15.26:59314) 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7cbfff700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff7cc1303d0 0x7ff7cc1382d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:34450/0 (socket says 172.21.15.26:34450) 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7caffd700 1 -- 172.21.15.26:0/763580400 learned_addr learned my addr 172.21.15.26:0/763580400 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:18.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798506807 0 0) 0x7ff7cc099460 con 0x7ff7cc099910 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): my global_id is 244173 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b4003570 con 0x7ff7cc099910 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090802634 0 0) 0x7ff7cc04ca10 con 0x7ff7cc12ff80 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): my global_id is 194769 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7cc099460 con 0x7ff7cc12ff80 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933933822 0 0) 0x7ff7cc098be0 con 0x7ff7cc1303d0 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient(hunting): my global_id is 204679 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7cc04ca10 con 0x7ff7cc1303d0 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231653130 0 0) 0x7ff7cc099460 con 0x7ff7cc12ff80 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 >> v1:172.21.15.122:6790/0 conn(0x7ff7cc099910 legacy=0x7ff7cc12f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 >> v1:172.21.15.122:6789/0 conn(0x7ff7cc1303d0 legacy=0x7ff7cc1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 monclient: found mon.noname-a 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc099110 con 0x7ff7cc12ff80 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0001790 con 0x7ff7cc12ff80 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: dump: 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 5 monclient: authenticate success, global_id 194769 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7c0001a40 con 0x7ff7cc12ff80 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 1 -- 172.21.15.26:0/763580400 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0001c30 con 0x7ff7cc12ff80 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7ca7fc700 10 monclient: dump: 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.438 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 4 set_mon_vals no callback set 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 10 monclient: shutdown 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- 172.21.15.26:0/763580400 >> v1:172.21.15.26:6789/0 conn(0x7ff7cc12ff80 legacy=0x7ff7cc134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- 172.21.15.26:0/763580400 shutdown_connections 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- 172.21.15.26:0/763580400 >> 172.21.15.26:0/763580400 conn(0x7ff7cc071860 msgr2=0x7ff7cc0a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- 172.21.15.26:0/763580400 shutdown_connections 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.422+0000 7ff7d0faf700 1 -- 172.21.15.26:0/763580400 wait complete. 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient: build_initial_monmap 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient: monmap: 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 1 Processor -- start 2024-09-15T04:00:18.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 1 -- start start 2024-09-15T04:00:18.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient: init 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient: _add_conns ranks=[5,4,2,8,1,3,0,7,6] 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient(hunting): picked mon.noname-h con 0x7ff7cc099910 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient(hunting): picked mon.noname-c con 0x7ff7cc12ff80 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient(hunting): picked mon.noname-b con 0x7ff7cc1303d0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc097ea0 con 0x7ff7cc1303d0 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc06b660 con 0x7ff7cc12ff80 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7caffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7ff7cc12ff80 0x7ff7cc09c2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40946/0 (socket says 172.21.15.26:40946) 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7caffd700 1 -- 172.21.15.26:0/1257003451 learned_addr learned my addr 172.21.15.26:0/1257003451 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc09ca00 con 0x7ff7cc099910 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7d0faf700 10 monclient(hunting): authenticate will time out at 1762.859100s 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564759971 0 0) 0x7ff7cc09ca00 con 0x7ff7cc099910 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): my global_id is 6179 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b8003570 con 0x7ff7cc099910 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722376411 0 0) 0x7ff7cc097ea0 con 0x7ff7cc1303d0 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): my global_id is 204716 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7cc09ca00 con 0x7ff7cc1303d0 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361435466 0 0) 0x7ff7cc099110 con 0x7ff7cc12ff80 2024-09-15T04:00:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 10 monclient(hunting): my global_id is 204682 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7cc097ea0 con 0x7ff7cc12ff80 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.426+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1653810782 0 0) 0x7ff7b8003570 con 0x7ff7cc099910 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7cc099110 con 0x7ff7cc099910 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7bc001ec0 con 0x7ff7cc099910 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): dump: 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1499552617 0 0) 0x7ff7cc097ea0 con 0x7ff7cc12ff80 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7b8003570 con 0x7ff7cc12ff80 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1677193229 0 0) 0x7ff7cc09ca00 con 0x7ff7cc1303d0 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff7cc097ea0 con 0x7ff7cc1303d0 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c0001880 con 0x7ff7cc12ff80 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): dump: 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7c40018c0 con 0x7ff7cc1303d0 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient(hunting): dump: 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3661490580 0 0) 0x7ff7cc099110 con 0x7ff7cc099910 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 >> v1:172.21.15.26:6791/0 conn(0x7ff7cc12ff80 legacy=0x7ff7cc09c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 >> v1:172.21.15.26:6790/0 conn(0x7ff7cc1303d0 legacy=0x7ff7cc145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 monclient: found mon.h 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc09cca0 con 0x7ff7cc099910 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 5 monclient: authenticate success, global_id 6179 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 monclient: _renew_subs 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc06ad50 con 0x7ff7cc099910 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 monclient: _renew_subs 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc06c110 con 0x7ff7cc099910 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff7bc002390 con 0x7ff7cc099910 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff7bc0028c0 con 0x7ff7cc099910 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7c9ffb700 4 set_mon_vals no callback set 2024-09-15T04:00:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 10 monclient: dump: 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff7bc002aa0 con 0x7ff7cc099910 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac0050b0 con 0x7ff7cc099910 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(730..730 src has 435..730) v4 ==== 3955+0+0 (unknown 1533685656 0 0) 0x7ff7bc041a00 con 0x7ff7cc099910 2024-09-15T04:00:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.430+0000 7ff7caffd700 10 monclient: get_auth_request con 0x7ff7b8038150 auth_method 0 2024-09-15T04:00:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.442+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ff7bc012800 con 0x7ff7cc099910 2024-09-15T04:00:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.442+0000 7ff7b37fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.442+0000 7ff7b37fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:18.522 INFO:tasks.rados.rados.0.smithi122.stdout:3240: finishing write tid 1 to smithi12223906-36 2024-09-15T04:00:18.522 INFO:tasks.rados.rados.0.smithi122.stdout:3240: finishing write tid 2 to smithi12223906-36 2024-09-15T04:00:18.522 INFO:tasks.rados.rados.0.smithi122.stdout:3240: finishing write tid 3 to smithi12223906-36 2024-09-15T04:00:18.522 INFO:tasks.rados.rados.0.smithi122.stdout:3240: finishing write tid 4 to smithi12223906-36 2024-09-15T04:00:18.522 INFO:tasks.rados.rados.0.smithi122.stdout:3240: finishing write tid 5 to smithi12223906-36 2024-09-15T04:00:18.523 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 2614 (ObjNum 1147 snap 377 seq_num 1147) dirty exists 2024-09-15T04:00:18.523 INFO:tasks.rados.rados.0.smithi122.stdout:3240: left oid 36 (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:00:18.523 INFO:tasks.rados.rados.0.smithi122.stdout:3241: finishing copy_from racing read to smithi12223906-29 2024-09-15T04:00:18.523 INFO:tasks.rados.rados.0.smithi122.stdout:3241: finishing copy_from to smithi12223906-29 2024-09-15T04:00:18.523 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 1075 (ObjNum 1095 snap 356 seq_num 1095) dirty exists 2024-09-15T04:00:18.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.666+0000 7ff7d0faf700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:00:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.666+0000 7ff7d0faf700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:00:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.666+0000 7ff7d0faf700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:00:18.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.666+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff7ac0058e0 con 0x7ff7cc099910 2024-09-15T04:00:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7b37fe700 1 -- 172.21.15.26:0/1257003451 <== mon.5 v1:172.21.15.122:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 571152224) 0x7ff7bc012200 con 0x7ff7cc099910 2024-09-15T04:00:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7b37fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:00:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7b37fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:18.676 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":104,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"h","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:00:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 >> v1:172.21.15.122:6812/20895 conn(0x7ff7b8038150 legacy=0x7ff7b803a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 10 monclient: shutdown 2024-09-15T04:00:18.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:18.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 >> v1:172.21.15.122:6791/0 conn(0x7ff7cc099910 legacy=0x7ff7cc13dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 shutdown_connections 2024-09-15T04:00:18.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 >> 172.21.15.26:0/1257003451 conn(0x7ff7cc071860 msgr2=0x7ff7cc12bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:18.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 shutdown_connections 2024-09-15T04:00:18.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.670+0000 7ff7d0faf700 1 -- 172.21.15.26:0/1257003451 wait complete. 2024-09-15T04:00:18.690 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T04:00:18.690 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T04:00:18.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T04:00:18.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:00:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.794+0000 7fcbee86c700 10 monclient: get_monmap_and_config 2024-09-15T04:00:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.794+0000 7fcbee86c700 10 monclient: build_initial_monmap 2024-09-15T04:00:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.794+0000 7fcbee86c700 10 monclient: monmap: 2024-09-15T04:00:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:18.799005+0000 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:18.799005+0000 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:18.803 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.794+0000 7fcbee86c700 1 Processor -- start 2024-09-15T04:00:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.794+0000 7fcbee86c700 1 -- start start 2024-09-15T04:00:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.794+0000 7fcbee86c700 10 monclient: init 2024-09-15T04:00:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient: _add_conns ranks=[7,0,3,6,2,4,1,8,5] 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient(hunting): picked mon.noname-i con 0x7fcbe80a6880 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient(hunting): picked mon.noname-a con 0x7fcbe809faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient(hunting): picked mon.noname-g con 0x7fcbe8138d30 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe804ca10 con 0x7fcbe809faa0 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80a0580 con 0x7fcbe8138d30 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe8133780 con 0x7fcbe80a6880 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient(hunting): authenticate will time out at 1763.228397s 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbed069700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcbe809faa0 0x7fcbe809fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59326/0 (socket says 172.21.15.26:59326) 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbed069700 1 -- 172.21.15.26:0/1266605524 learned_addr learned my addr 172.21.15.26:0/1266605524 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:18.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849315434 0 0) 0x7fcbe804ca10 con 0x7fcbe809faa0 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): my global_id is 194778 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd4003570 con 0x7fcbe809faa0 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3928062458 0 0) 0x7fcbe0001110 con 0x7fcbe8138d30 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): my global_id is 244182 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe804ca10 con 0x7fcbe8138d30 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235213712 0 0) 0x7fcbe8133780 con 0x7fcbe80a6880 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient(hunting): my global_id is 234358 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe0001110 con 0x7fcbe80a6880 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1398622017 0 0) 0x7fcbd4003570 con 0x7fcbe809faa0 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 >> v1:172.21.15.122:6792/0 conn(0x7fcbe80a6880 legacy=0x7fcbe8133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 >> v1:172.21.15.122:6790/0 conn(0x7fcbe8138d30 legacy=0x7fcbe813b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: _finish_hunting 0 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 monclient: found mon.noname-a 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80a6c50 con 0x7fcbe809faa0 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: _finish_auth 0 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbe4001b30 con 0x7fcbe809faa0 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:18.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: dump: 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.811 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 5 monclient: authenticate success, global_id 194778 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcbe4001e10 con 0x7fcbe809faa0 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 4 set_mon_vals no callback set 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 1 -- 172.21.15.26:0/1266605524 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbe4004fb0 con 0x7fcbe809faa0 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbec868700 10 monclient: dump: 2024-09-15T04:00:18.812 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.813 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 10 monclient: shutdown 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- 172.21.15.26:0/1266605524 >> v1:172.21.15.26:6789/0 conn(0x7fcbe809faa0 legacy=0x7fcbe809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- 172.21.15.26:0/1266605524 shutdown_connections 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- 172.21.15.26:0/1266605524 >> 172.21.15.26:0/1266605524 conn(0x7fcbe806e2a0 msgr2=0x7fcbe80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- 172.21.15.26:0/1266605524 shutdown_connections 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.798+0000 7fcbee86c700 1 -- 172.21.15.26:0/1266605524 wait complete. 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient: build_initial_monmap 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient: monmap: 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:18.814 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:18.815 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 1 Processor -- start 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 1 -- start start 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient: init 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient: _add_conns ranks=[6,7,4,1,5,3,2,0,8] 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient(hunting): picked mon.noname-d con 0x7fcbe809faa0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient(hunting): picked mon.noname-i con 0x7fcbe80a6880 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient(hunting): picked mon.noname-c con 0x7fcbe8138d30 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe809cf10 con 0x7fcbe8138d30 2024-09-15T04:00:18.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80a18c0 con 0x7fcbe809faa0 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe809b160 con 0x7fcbe80a6880 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbee86c700 10 monclient(hunting): authenticate will time out at 1763.234697s 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbed86a700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fcbe809faa0 0x7fcbe80998d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37080/0 (socket says 172.21.15.26:37080) 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbed86a700 1 -- 172.21.15.26:0/3317471929 learned_addr learned my addr 172.21.15.26:0/3317471929 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083861142 0 0) 0x7fcbe80a18c0 con 0x7fcbe809faa0 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): my global_id is 194964 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbcc003570 con 0x7fcbe809faa0 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440240227 0 0) 0x7fcbe809cf10 con 0x7fcbe8138d30 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): my global_id is 204691 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe80a18c0 con 0x7fcbe8138d30 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3721166145 0 0) 0x7fcbcc003570 con 0x7fcbe809faa0 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbe809cf10 con 0x7fcbe809faa0 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901692481 0 0) 0x7fcbe809b160 con 0x7fcbe80a6880 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): my global_id is 234367 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbcc003570 con 0x7fcbe80a6880 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbd8001b20 con 0x7fcbe809faa0 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): dump: 2024-09-15T04:00:18.822 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.823 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1377233004 0 0) 0x7fcbe80a18c0 con 0x7fcbe8138d30 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.802+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbe809b160 con 0x7fcbe8138d30 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbe0001ad0 con 0x7fcbe8138d30 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient(hunting): dump: 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.824 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3058846814 0 0) 0x7fcbe809cf10 con 0x7fcbe809faa0 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.122:6792/0 conn(0x7fcbe80a6880 legacy=0x7fcbe8099fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.26:6791/0 conn(0x7fcbe8138d30 legacy=0x7fcbe814e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: _finish_hunting 0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 monclient: found mon.d 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe809b670 con 0x7fcbe809faa0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: _finish_auth 0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 5 monclient: authenticate success, global_id 194964 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 10 monclient: _renew_subs 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe809ba60 con 0x7fcbe809faa0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 10 monclient: _renew_subs 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe8132910 con 0x7fcbe809faa0 2024-09-15T04:00:18.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcbd8001fc0 con 0x7fcbe809faa0 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbd80024f0 con 0x7fcbe809faa0 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbdffff700 4 set_mon_vals no callback set 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: dump: 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.827 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcbd80026d0 con 0x7fcbe809faa0 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcbc8006590 con 0x7fcbc8002840 2024-09-15T04:00:18.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(730..730 src has 435..730) v4 ==== 3955+0+0 (unknown 1533685656 0 0) 0x7fcbd8043ec0 con 0x7fcbe809faa0 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161127720 0 0) 0x7fcbc8006590 con 0x7fcbc8002840 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient(hunting): my global_id is 194787 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe809cf10 con 0x7fcbc8002840 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbee06b700 10 monclient: get_auth_request con 0x7fcbcc037d50 auth_method 0 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 206232214 0 0) 0x7fcbe809cf10 con 0x7fcbc8002840 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbc8006590 con 0x7fcbc8002840 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbe4001af0 con 0x7fcbc8002840 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: dump: 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.829 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.830 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3000152750 0 0) 0x7fcbc8006590 con 0x7fcbc8002840 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbc800a130 con 0x7fcbc8002840 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fcbe4004df0 con 0x7fcbc8002840 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.26:6789/0 conn(0x7fcbc8002840 legacy=0x7fcbc8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.806+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.26:6789/0 conn(0x7fcbc8002840 legacy=0x7fcbc8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.814+0000 7fcbee86c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.814+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fcbc8009640 con 0x7fcbc80090b0 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623027943 0 0) 0x7fcbc8009640 con 0x7fcbc80090b0 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient(hunting): my global_id is 194796 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc8006590 con 0x7fcbc80090b0 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3995197265 0 0) 0x7fcbc8006590 con 0x7fcbc80090b0 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcbc8009640 con 0x7fcbc80090b0 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcbe40057b0 con 0x7fcbc80090b0 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient: dump: 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.834 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1764582075 0 0) 0x7fcbc8009640 con 0x7fcbc80090b0 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcbc80098f0 con 0x7fcbc80090b0 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 456514837) 0x7fcbe4008270 con 0x7fcbc80090b0 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.26:6789/0 conn(0x7fcbc80090b0 legacy=0x7fcbc8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbddffb700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.26:6789/0 conn(0x7fcbc80090b0 legacy=0x7fcbc8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:18.836 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:18.837 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:18.838 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.839 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:18.840 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:18.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.842 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:18.843 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:18.844 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:18.846 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.847 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.848 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:18.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.852 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:18.853 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.854 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:18.855 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.122:6812/20895 conn(0x7fcbcc037d50 legacy=0x7fcbcc03a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 10 monclient: shutdown 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 >> v1:172.21.15.26:6792/0 conn(0x7fcbe809faa0 legacy=0x7fcbe80998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee06b700 1 -- 172.21.15.26:0/3317471929 reap_dead start 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee06b700 1 -- 172.21.15.26:0/3317471929 reap_dead start 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 shutdown_connections 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 >> 172.21.15.26:0/3317471929 conn(0x7fcbe806e2a0 msgr2=0x7fcbe80a4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 shutdown_connections 2024-09-15T04:00:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.818+0000 7fcbee86c700 1 -- 172.21.15.26:0/3317471929 wait complete. 2024-09-15T04:00:18.858 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:18.858 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T04:00:18.858 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.966+0000 7f26b66d6700 10 monclient: get_monmap_and_config 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.966+0000 7f26b66d6700 10 monclient: build_initial_monmap 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.966+0000 7f26b66d6700 10 monclient: monmap: 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:18.970860+0000 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:18.970860+0000 2024-09-15T04:00:18.974 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:18.975 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 1 Processor -- start 2024-09-15T04:00:18.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 1 -- start start 2024-09-15T04:00:18.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient: init 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient: _add_conns ranks=[8,0,5,4,6,3,1,7,2] 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient(hunting): picked mon.noname-e con 0x7f26b006e280 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient(hunting): picked mon.noname-a con 0x7f26b0099b90 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient(hunting): picked mon.noname-h con 0x7f26b0135db0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b004ca10 con 0x7f26b0099b90 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b00989e0 con 0x7f26b0135db0 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b0130680 con 0x7f26b006e280 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient(hunting): authenticate will time out at 1763.401944s 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b56d4700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f26b006e280 0x7f26b012f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37254/0 (socket says 172.21.15.26:37254) 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b56d4700 1 -- 172.21.15.26:0/1560963041 learned_addr learned my addr 172.21.15.26:0/1560963041 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049417771 0 0) 0x7f26b0130680 con 0x7f26b006e280 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): my global_id is 135494 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c003570 con 0x7f26b006e280 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166011648 0 0) 0x7f26b004ca10 con 0x7f26b0099b90 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): my global_id is 194805 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b0130680 con 0x7f26b0099b90 2024-09-15T04:00:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898201150 0 0) 0x7f26b00989e0 con 0x7f26b0135db0 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient(hunting): my global_id is 6188 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b004ca10 con 0x7f26b0135db0 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3289242206 0 0) 0x7f269c003570 con 0x7f26b006e280 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 >> v1:172.21.15.122:6791/0 conn(0x7f26b0135db0 legacy=0x7f26b0138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 >> v1:172.21.15.26:6789/0 conn(0x7f26b0099b90 legacy=0x7f26b012ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 monclient: found mon.noname-e 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0098c50 con 0x7f26b006e280 2024-09-15T04:00:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: _finish_auth 0 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26ac0017b0 con 0x7f26b006e280 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: dump: 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 5 monclient: authenticate success, global_id 135494 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26ac001a20 con 0x7f26b006e280 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 1 -- 172.21.15.26:0/1560963041 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26ac001c10 con 0x7f26b006e280 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26b66d6700 4 set_mon_vals no callback set 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.970+0000 7f26a7fff700 10 monclient: dump: 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.985 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient: shutdown 2024-09-15T04:00:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- 172.21.15.26:0/1560963041 >> v1:172.21.15.26:6793/0 conn(0x7f26b006e280 legacy=0x7f26b012f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- 172.21.15.26:0/1560963041 shutdown_connections 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- 172.21.15.26:0/1560963041 >> 172.21.15.26:0/1560963041 conn(0x7f26b00a3a00 msgr2=0x7f26b009fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- 172.21.15.26:0/1560963041 shutdown_connections 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- 172.21.15.26:0/1560963041 wait complete. 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient: build_initial_monmap 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient: monmap: 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 Processor -- start 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- start start 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient: init 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient: _add_conns ranks=[7,0,1,4,3,5,2,8,6] 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient(hunting): picked mon.noname-i con 0x7f26b0099b90 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient(hunting): picked mon.noname-a con 0x7f26b0135db0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 10 monclient(hunting): picked mon.noname-f con 0x7f26b0144980 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b0097d10 con 0x7f26b0135db0 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b006ddb0 con 0x7f26b0144980 2024-09-15T04:00:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b66d6700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b0097fe0 con 0x7f26b0099b90 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b4ed3700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f26b0135db0 0x7f26b0134ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59356/0 (socket says 172.21.15.26:59356) 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.974+0000 7f26b4ed3700 1 -- 172.21.15.26:0/563774368 learned_addr learned my addr 172.21.15.26:0/563774368 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 monclient(hunting): authenticate will time out at 1763.408053s 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403693504 0 0) 0x7f26b0097d10 con 0x7f26b0135db0 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): my global_id is 194814 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2694003570 con 0x7f26b0135db0 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1735609269 0 0) 0x7f2694003570 con 0x7f26b0135db0 2024-09-15T04:00:18.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26b0097d10 con 0x7f26b0135db0 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26a0001b20 con 0x7f26b0135db0 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): dump: 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:18.996 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:18.997 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:18.997 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:18.997 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1130132614 0 0) 0x7f26b0097fe0 con 0x7f26b0099b90 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): my global_id is 234376 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2694003570 con 0x7f26b0099b90 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654235353 0 0) 0x7f26b006ddb0 con 0x7f26b0144980 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): my global_id is 204688 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b0097fe0 con 0x7f26b0144980 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1495913944 0 0) 0x7f26b0097d10 con 0x7f26b0135db0 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.122:6792/0 conn(0x7f26b0099b90 legacy=0x7f26b013e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.122:6789/0 conn(0x7f26b0144980 legacy=0x7f26b0146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: _finish_hunting 0 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 monclient: found mon.a 2024-09-15T04:00:18.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b006d7f0 con 0x7f26b0135db0 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 5 monclient: authenticate success, global_id 194814 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 monclient: _renew_subs 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b013ed10 con 0x7f26b0135db0 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 monclient: _renew_subs 2024-09-15T04:00:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b013f3c0 con 0x7f26b0135db0 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f26b004bbf0 con 0x7f26b006e280 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26a0002070 con 0x7f26b0135db0 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26a00025a0 con 0x7f26b0135db0 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient: dump: 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a77fe700 4 set_mon_vals no callback set 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f26a0002700 con 0x7f26b0135db0 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(730..730 src has 435..730) v4 ==== 3955+0+0 (unknown 1533685656 0 0) 0x7f26a0043e80 con 0x7f26b0135db0 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483237087 0 0) 0x7f26b004bbf0 con 0x7f26b006e280 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 10 monclient(hunting): my global_id is 204725 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.978+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b0097d10 con 0x7f26b006e280 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26b5ed5700 10 monclient: get_auth_request con 0x7f2694037660 auth_method 0 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3565518605 0 0) 0x7f26b0097d10 con 0x7f26b006e280 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26b004bbf0 con 0x7f26b006e280 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26ac001770 con 0x7f26b006e280 2024-09-15T04:00:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 10 monclient: dump: 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3016608299 0 0) 0x7f26b004bbf0 con 0x7f26b006e280 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26b006bd50 con 0x7f26b006e280 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f26ac006950 con 0x7f26b006e280 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.26:6790/0 conn(0x7f26b006e280 legacy=0x7f26b009b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.982+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.26:6790/0 conn(0x7f26b006e280 legacy=0x7f26b009b1d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26b66d6700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f26b0067d90 con 0x7f26b002ad10 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447109306 0 0) 0x7f26b0067d90 con 0x7f26b002ad10 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 10 monclient(hunting): my global_id is 204734 2024-09-15T04:00:19.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b004bbf0 con 0x7f26b002ad10 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3283257469 0 0) 0x7f26b004bbf0 con 0x7f26b002ad10 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26b0067d90 con 0x7f26b002ad10 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26ac005190 con 0x7f26b002ad10 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.990+0000 7f26a57fa700 10 monclient: dump: 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.008 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3934756959 0 0) 0x7f26b0067d90 con 0x7f26b002ad10 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f26b006fac0 con 0x7f26b002ad10 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 4257662863) 0x7f26ac0066c0 con 0x7f26b002ad10 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.26:6790/0 conn(0x7f26b002ad10 legacy=0x7f26b009b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26a57fa700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.26:6790/0 conn(0x7f26b002ad10 legacy=0x7f26b009b8e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.010 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:19.011 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.012 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.013 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.014 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.015 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:19.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.017 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.020 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.021 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.022 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.024 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.025 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:19.026 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.027 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.028 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.029 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:19.030 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.122:6812/20895 conn(0x7f2694037660 legacy=0x7f2694039b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 10 monclient: shutdown 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 >> v1:172.21.15.26:6789/0 conn(0x7f26b0135db0 legacy=0x7f26b0134ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b5ed5700 1 -- 172.21.15.26:0/563774368 reap_dead start 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b5ed5700 1 -- 172.21.15.26:0/563774368 reap_dead start 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 shutdown_connections 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 >> 172.21.15.26:0/563774368 conn(0x7f26b00a3a00 msgr2=0x7f26b009a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 shutdown_connections 2024-09-15T04:00:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:18.994+0000 7f26b66d6700 1 -- 172.21.15.26:0/563774368 wait complete. 2024-09-15T04:00:19.033 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:19.033 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T04:00:19.033 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient: get_monmap_and_config 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient: monmap: 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:19.144741+0000 2024-09-15T04:00:19.148 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:19.144741+0000 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.149 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 1 Processor -- start 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 1 -- start start 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient: init 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient: _add_conns ranks=[4,8,6,1,2,3,0,7,5] 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient(hunting): picked mon.noname-c con 0x7fa9300a6880 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient(hunting): picked mon.noname-e con 0x7fa93009faa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient(hunting): picked mon.noname-d con 0x7fa930138d30 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93004ca10 con 0x7fa9300a6880 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9300a0580 con 0x7fa930138d30 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa930133780 con 0x7fa93009faa0 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935a72700 10 monclient(hunting): authenticate will time out at 1763.574916s 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa927fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fa93009faa0 0x7fa93009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37270/0 (socket says 172.21.15.26:37270) 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa927fff700 1 -- 172.21.15.26:0/3448873660 learned_addr learned my addr 172.21.15.26:0/3448873660 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa935271700 1 --1- 172.21.15.26:0/3448873660 >> v1:172.21.15.26:6792/0 conn(0x7fa930138d30 0x7fa93013b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37094/0 (socket says 172.21.15.26:37094) 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa934a70700 1 --1- 172.21.15.26:0/3448873660 >> v1:172.21.15.26:6791/0 conn(0x7fa9300a6880 0x7fa930133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40970/0 (socket says 172.21.15.26:40970) 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207511238 0 0) 0x7fa928001490 con 0x7fa930138d30 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): my global_id is 194973 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa918003570 con 0x7fa930138d30 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35578243 0 0) 0x7fa93004ca10 con 0x7fa9300a6880 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): my global_id is 204700 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa928001490 con 0x7fa9300a6880 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099669594 0 0) 0x7fa930133780 con 0x7fa93009faa0 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 10 monclient(hunting): my global_id is 135503 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93004ca10 con 0x7fa93009faa0 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.142+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1718265138 0 0) 0x7fa918003570 con 0x7fa930138d30 2024-09-15T04:00:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 >> v1:172.21.15.26:6793/0 conn(0x7fa93009faa0 legacy=0x7fa93009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 >> v1:172.21.15.26:6791/0 conn(0x7fa9300a6880 legacy=0x7fa930133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 monclient: found mon.noname-d 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300a6c50 con 0x7fa930138d30 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 5 monclient: authenticate success, global_id 194973 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa928001cc0 con 0x7fa930138d30 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: dump: 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa928001fa0 con 0x7fa930138d30 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 1 -- 172.21.15.26:0/3448873660 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa928004ef0 con 0x7fa930138d30 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 4 set_mon_vals no callback set 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa9277fe700 10 monclient: dump: 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.159 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: shutdown 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- 172.21.15.26:0/3448873660 >> v1:172.21.15.26:6792/0 conn(0x7fa930138d30 legacy=0x7fa93013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- 172.21.15.26:0/3448873660 shutdown_connections 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- 172.21.15.26:0/3448873660 >> 172.21.15.26:0/3448873660 conn(0x7fa93006e2a0 msgr2=0x7fa9300a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- 172.21.15.26:0/3448873660 shutdown_connections 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- 172.21.15.26:0/3448873660 wait complete. 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: monmap: 2024-09-15T04:00:19.160 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.162 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 Processor -- start 2024-09-15T04:00:19.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- start start 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: init 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient: _add_conns ranks=[6,0,1,7,3,8,4,2,5] 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient(hunting): picked mon.noname-d con 0x7fa93009faa0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient(hunting): picked mon.noname-a con 0x7fa9300a6880 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient(hunting): picked mon.noname-f con 0x7fa930138d30 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9300a18c0 con 0x7fa9300a6880 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93009cf10 con 0x7fa930138d30 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa934a70700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fa93009faa0 0x7fa930099630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37110/0 (socket says 172.21.15.26:37110) 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa934a70700 1 -- 172.21.15.26:0/4261849506 learned_addr learned my addr 172.21.15.26:0/4261849506 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93011cce0 con 0x7fa93009faa0 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.146+0000 7fa935a72700 10 monclient(hunting): authenticate will time out at 1763.579798s 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440890543 0 0) 0x7fa93011cce0 con 0x7fa93009faa0 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): my global_id is 194982 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003570 con 0x7fa93009faa0 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320246149 0 0) 0x7fa9300a18c0 con 0x7fa9300a6880 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): my global_id is 194823 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93011cce0 con 0x7fa9300a6880 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3970457799 0 0) 0x7fa91c003570 con 0x7fa93009faa0 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9300a18c0 con 0x7fa93009faa0 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa92c001c20 con 0x7fa93009faa0 2024-09-15T04:00:19.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.169 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344121216 0 0) 0x7fa93009cf10 con 0x7fa930138d30 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): my global_id is 204697 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003570 con 0x7fa930138d30 2024-09-15T04:00:19.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2654537458 0 0) 0x7fa93011cce0 con 0x7fa9300a6880 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa93009cf10 con 0x7fa9300a6880 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa920001a60 con 0x7fa9300a6880 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.171 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3730304498 0 0) 0x7fa9300a18c0 con 0x7fa93009faa0 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.122:6789/0 conn(0x7fa930138d30 legacy=0x7fa93009a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.26:6789/0 conn(0x7fa9300a6880 legacy=0x7fa930099d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 monclient: found mon.d 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93009b6f0 con 0x7fa93009faa0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 5 monclient: authenticate success, global_id 194982 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 10 monclient: _renew_subs 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93011c720 con 0x7fa93009faa0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 10 monclient: _renew_subs 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa930132830 con 0x7fa93009faa0 2024-09-15T04:00:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa92c004ae0 con 0x7fa93009faa0 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa92c005010 con 0x7fa93009faa0 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: dump: 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa926ffd700 4 set_mon_vals no callback set 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa910006590 con 0x7fa910002840 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa92c0051f0 con 0x7fa93009faa0 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(730..730 src has 435..730) v4 ==== 3955+0+0 (unknown 1533685656 0 0) 0x7fa92c044000 con 0x7fa93009faa0 2024-09-15T04:00:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291610422 0 0) 0x7fa910006590 con 0x7fa910002840 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient(hunting): my global_id is 204709 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9300a18c0 con 0x7fa910002840 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa935271700 10 monclient: get_auth_request con 0x7fa91c037d20 auth_method 0 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2392058964 0 0) 0x7fa9300a18c0 con 0x7fa910002840 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa910006590 con 0x7fa910002840 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa920004bf0 con 0x7fa910002840 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.150+0000 7fa924ff9700 10 monclient: dump: 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.177 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2936879616 0 0) 0x7fa910006590 con 0x7fa910002840 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa91000a130 con 0x7fa910002840 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa920005100 con 0x7fa910002840 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.26:6791/0 conn(0x7fa910002840 legacy=0x7fa910004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.154+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.26:6791/0 conn(0x7fa910002840 legacy=0x7fa910004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa935a72700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa910009640 con 0x7fa9100090b0 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961141788 0 0) 0x7fa910009640 con 0x7fa9100090b0 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa924ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa924ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa924ff9700 10 monclient(hunting): my global_id is 204718 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.162+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa910006590 con 0x7fa9100090b0 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2256377677 0 0) 0x7fa910006590 con 0x7fa9100090b0 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa910009640 con 0x7fa9100090b0 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa9200059f0 con 0x7fa9100090b0 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 10 monclient: dump: 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2318363653 0 0) 0x7fa910009640 con 0x7fa9100090b0 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa9100098f0 con 0x7fa9100090b0 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3369140614) 0x7fa920005de0 con 0x7fa9100090b0 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.26:6791/0 conn(0x7fa9100090b0 legacy=0x7fa910008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa924ff9700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.26:6791/0 conn(0x7fa9100090b0 legacy=0x7fa910008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.184 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.185 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:19.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.187 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.189 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.190 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.194 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.195 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.196 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:19.197 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.198 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.199 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.200 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:19.201 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.122:6812/20895 conn(0x7fa91c037d20 legacy=0x7fa91c03a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 10 monclient: shutdown 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 >> v1:172.21.15.26:6792/0 conn(0x7fa93009faa0 legacy=0x7fa930099630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935271700 1 -- 172.21.15.26:0/4261849506 reap_dead start 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935271700 1 -- 172.21.15.26:0/4261849506 reap_dead start 2024-09-15T04:00:19.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 shutdown_connections 2024-09-15T04:00:19.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 >> 172.21.15.26:0/4261849506 conn(0x7fa93006e2a0 msgr2=0x7fa9300a4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 shutdown_connections 2024-09-15T04:00:19.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.166+0000 7fa935a72700 1 -- 172.21.15.26:0/4261849506 wait complete. 2024-09-15T04:00:19.204 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:19.204 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T04:00:19.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3239: done (3 left) 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3240: done (2 left) 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3241: done (1 left) 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3242: done (0 left) 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3243: read oid 1 snap -1 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3243: expect (ObjNum 1127 snap 371 seq_num 1127) 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3244: write (excl) oid 42 current snap is 378 2024-09-15T04:00:19.267 INFO:tasks.rados.rados.0.smithi122.stdout:3244: seq_num 1148 ranges {494436=755155,2011446=521062,3075831=218605} 2024-09-15T04:00:19.275 INFO:tasks.rados.rados.0.smithi122.stdout:3244: writing smithi12223906-42 from 494436 to 1249591 tid 1 2024-09-15T04:00:19.282 INFO:tasks.rados.rados.0.smithi122.stdout:3244: writing smithi12223906-42 from 2011446 to 2532508 tid 2 2024-09-15T04:00:19.286 INFO:tasks.rados.rados.0.smithi122.stdout:3244: writing smithi12223906-42 from 3075831 to 3294436 tid 3 2024-09-15T04:00:19.287 INFO:tasks.rados.rados.0.smithi122.stdout:3245: delete oid 17 current snap is 378 2024-09-15T04:00:19.287 INFO:tasks.rados.rados.0.smithi122.stdout:3243: expect (ObjNum 1127 snap 371 seq_num 1127) 2024-09-15T04:00:19.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.310+0000 7f85b610f700 10 monclient: get_monmap_and_config 2024-09-15T04:00:19.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.310+0000 7f85b610f700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.310+0000 7f85b610f700 10 monclient: monmap: 2024-09-15T04:00:19.318 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:19.314843+0000 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:19.314843+0000 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.320 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.320 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.320 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.320 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.320 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 Processor -- start 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- start start 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient: init 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient: _add_conns ranks=[7,8,5,3,1,0,4,6,2] 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient(hunting): picked mon.noname-i con 0x7f85b00990c0 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient(hunting): picked mon.noname-e con 0x7f85b00a1310 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient(hunting): picked mon.noname-h con 0x7f85b01335a0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b004ca10 con 0x7f85b01335a0 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00a1b20 con 0x7f85b00990c0 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00a10b0 con 0x7f85b00a1310 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient(hunting): authenticate will time out at 1763.745458s 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85a7fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f85b00a1310 0x7f85b0099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37284/0 (socket says 172.21.15.26:37284) 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85a7fff700 1 -- 172.21.15.26:0/1123518787 learned_addr learned my addr 172.21.15.26:0/1123518787 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532227255 0 0) 0x7f85b00a10b0 con 0x7f85b00a1310 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): my global_id is 135512 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f859c003570 con 0x7f85b00a1310 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574703151 0 0) 0x7f85b00a1b20 con 0x7f85b00990c0 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): my global_id is 234385 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b00a10b0 con 0x7f85b00990c0 2024-09-15T04:00:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062767783 0 0) 0x7f85b004ca10 con 0x7f85b01335a0 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient(hunting): my global_id is 6197 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b00a1b20 con 0x7f85b01335a0 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 512973246 0 0) 0x7f859c003570 con 0x7f85b00a1310 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 >> v1:172.21.15.122:6792/0 conn(0x7f85b00990c0 legacy=0x7f85b0132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 >> v1:172.21.15.122:6791/0 conn(0x7f85b01335a0 legacy=0x7f85b013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 monclient: found mon.noname-e 2024-09-15T04:00:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0099ba0 con 0x7f85b00a1310 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f85980017b0 con 0x7f85b00a1310 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 5 monclient: authenticate success, global_id 135512 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: dump: 2024-09-15T04:00:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.337 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8598001a90 con 0x7f85b00a1310 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 1 -- 172.21.15.26:0/1123518787 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8598004c30 con 0x7f85b00a1310 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 4 set_mon_vals no callback set 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:19.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85aeffd700 10 monclient: dump: 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 10 monclient: shutdown 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- 172.21.15.26:0/1123518787 >> v1:172.21.15.26:6793/0 conn(0x7f85b00a1310 legacy=0x7f85b0099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- 172.21.15.26:0/1123518787 shutdown_connections 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.314+0000 7f85b610f700 1 -- 172.21.15.26:0/1123518787 >> 172.21.15.26:0/1123518787 conn(0x7f85b006e370 msgr2=0x7f85b00a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- 172.21.15.26:0/1123518787 shutdown_connections 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- 172.21.15.26:0/1123518787 wait complete. 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: monmap: 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.340 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.341 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 Processor -- start 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- start start 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: init 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: _add_conns ranks=[3,5,2,8,0,4,1,6,7] 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient(hunting): picked mon.noname-g con 0x7f85b00990c0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient(hunting): picked mon.noname-h con 0x7f85b00a1310 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient(hunting): picked mon.noname-b con 0x7f85b01335a0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00a1ce0 con 0x7f85b01335a0 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b009a010 con 0x7f85b00990c0 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b0121b30 con 0x7f85b00a1310 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient(hunting): authenticate will time out at 1763.750024s 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85affff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f85b01335a0 0x7f85b014a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57970/0 (socket says 172.21.15.26:57970) 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85affff700 1 -- 172.21.15.26:0/1667503754 learned_addr learned my addr 172.21.15.26:0/1667503754 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893731017 0 0) 0x7f85b00a1ce0 con 0x7f85b01335a0 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): my global_id is 204743 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8590003570 con 0x7f85b01335a0 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395027884 0 0) 0x7f85b009a010 con 0x7f85b00990c0 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): my global_id is 244191 2024-09-15T04:00:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b00a1ce0 con 0x7f85b00990c0 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11309622 0 0) 0x7f85b0121b30 con 0x7f85b00a1310 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): my global_id is 6206 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b009a010 con 0x7f85b00a1310 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3310962414 0 0) 0x7f8590003570 con 0x7f85b01335a0 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85b0121b30 con 0x7f85b01335a0 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f85a8002d90 con 0x7f85b01335a0 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): dump: 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2121751300 0 0) 0x7f85b00a1ce0 con 0x7f85b00990c0 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8590003570 con 0x7f85b00990c0 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2071660222 0 0) 0x7f85b009a010 con 0x7f85b00a1310 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85b00a1ce0 con 0x7f85b00a1310 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f85a00019e0 con 0x7f85b00990c0 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): dump: 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.346 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8598004b70 con 0x7f85b00a1310 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient(hunting): dump: 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1497890251 0 0) 0x7f85b0121b30 con 0x7f85b01335a0 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.122:6791/0 conn(0x7f85b00a1310 legacy=0x7f85b0147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.122:6790/0 conn(0x7f85b00990c0 legacy=0x7f85b01319e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 monclient: found mon.b 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b01220f0 con 0x7f85b01335a0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85a77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 5 monclient: authenticate success, global_id 204743 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: _renew_subs 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.318+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0120070 con 0x7f85b01335a0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85b610f700 10 monclient: _renew_subs 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85b610f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b0120770 con 0x7f85b01335a0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f85a8003120 con 0x7f85b01335a0 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f85a8001c40 con 0x7f85b01335a0 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85ae7fc700 4 set_mon_vals no callback set 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: dump: 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85b610f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8594006590 con 0x7f8594002840 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f85a8001e20 con 0x7f85b01335a0 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f85a8044270 con 0x7f85b01335a0 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858179814 0 0) 0x7f8594006590 con 0x7f8594002840 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient(hunting): my global_id is 194991 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b0121b30 con 0x7f8594002840 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a7fff700 10 monclient: get_auth_request con 0x7f85900381b0 auth_method 0 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1058325547 0 0) 0x7f85b0121b30 con 0x7f8594002840 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8594006590 con 0x7f8594002840 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f85a00021f0 con 0x7f8594002840 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: dump: 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3735386608 0 0) 0x7f8594006590 con 0x7f8594002840 2024-09-15T04:00:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f859400a130 con 0x7f8594002840 2024-09-15T04:00:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f85a00045a0 con 0x7f8594002840 2024-09-15T04:00:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.26:6792/0 conn(0x7f8594002840 legacy=0x7f8594004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.322+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.26:6792/0 conn(0x7f8594002840 legacy=0x7f8594004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f8594007520 con 0x7f85940068d0 2024-09-15T04:00:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044017640 0 0) 0x7f8594007520 con 0x7f85940068d0 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient(hunting): my global_id is 195000 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8594006590 con 0x7f85940068d0 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3053511020 0 0) 0x7f8594006590 con 0x7f85940068d0 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8594007520 con 0x7f85940068d0 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f85a0002d90 con 0x7f85940068d0 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient: dump: 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 997314566 0 0) 0x7f8594007520 con 0x7f85940068d0 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8594008760 con 0x7f85940068d0 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 1995966765) 0x7f85a0004340 con 0x7f85940068d0 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.26:6792/0 conn(0x7f85940068d0 legacy=0x7f8594006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85a77fe700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.26:6792/0 conn(0x7f85940068d0 legacy=0x7f8594006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.359 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.360 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.361 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.362 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:19.363 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.364 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.366 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.367 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.369 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.371 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:19.372 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.373 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.374 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.375 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:19.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.378 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:19.379 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.122:6812/20895 conn(0x7f85900381b0 legacy=0x7f859003a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 10 monclient: shutdown 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 >> v1:172.21.15.26:6790/0 conn(0x7f85b01335a0 legacy=0x7f85b014a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85affff700 1 -- 172.21.15.26:0/1667503754 reap_dead start 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85affff700 1 -- 172.21.15.26:0/1667503754 reap_dead start 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 shutdown_connections 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 >> 172.21.15.26:0/1667503754 conn(0x7f85b006e370 msgr2=0x7f85b00a4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 shutdown_connections 2024-09-15T04:00:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.334+0000 7f85b610f700 1 -- 172.21.15.26:0/1667503754 wait complete. 2024-09-15T04:00:19.382 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:19.382 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T04:00:19.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T04:00:19.414 INFO:tasks.rados.rados.0.smithi122.stdout:3244: finishing write tid 1 to smithi12223906-42 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3244: finishing write tid 2 to smithi12223906-42 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3244: finishing write tid 3 to smithi12223906-42 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3244: finishing write tid 4 to smithi12223906-42 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3244: finishing write tid 5 to smithi12223906-42 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 2618 (ObjNum 1148 snap 378 seq_num 1148) dirty exists 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3244: left oid 42 (ObjNum 1148 snap 378 seq_num 1148) 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3243: done (2 left) 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3244: done (1 left) 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3245: done (0 left) 2024-09-15T04:00:19.415 INFO:tasks.rados.rados.0.smithi122.stdout:3246: snap_create 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.490+0000 7f66717e3700 10 monclient: get_monmap_and_config 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.490+0000 7f66717e3700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.490+0000 7f66717e3700 10 monclient: monmap: 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:19.493735+0000 2024-09-15T04:00:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:19.493735+0000 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.498 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.490+0000 7f66717e3700 1 Processor -- start 2024-09-15T04:00:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.490+0000 7f66717e3700 1 -- start start 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.490+0000 7f66717e3700 10 monclient: init 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient: _add_conns ranks=[7,8,1,3,5,6,4,2,0] 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient(hunting): picked mon.noname-i con 0x7f666c132b40 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient(hunting): picked mon.noname-e con 0x7f666c09faa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient(hunting): picked mon.noname-f con 0x7f666c138d40 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c04ca10 con 0x7f666c138d40 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c09d5d0 con 0x7f666c132b40 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c133620 con 0x7f666c09faa0 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient(hunting): authenticate will time out at 1763.924529s 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f6663fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f666c09faa0 0x7f666c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37300/0 (socket says 172.21.15.26:37300) 2024-09-15T04:00:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f6663fff700 1 -- 172.21.15.26:0/1322032792 learned_addr learned my addr 172.21.15.26:0/1322032792 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725393091 0 0) 0x7f666c133620 con 0x7f666c09faa0 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): my global_id is 135521 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6658003570 con 0x7f666c09faa0 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584665911 0 0) 0x7f666c04ca10 con 0x7f666c138d40 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): my global_id is 204706 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c133620 con 0x7f666c138d40 2024-09-15T04:00:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092780712 0 0) 0x7f666c09d5d0 con 0x7f666c132b40 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient(hunting): my global_id is 234394 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c04ca10 con 0x7f666c132b40 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737338616 0 0) 0x7f666c04ca10 con 0x7f666c132b40 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 >> v1:172.21.15.26:6793/0 conn(0x7f666c09faa0 legacy=0x7f666c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 >> v1:172.21.15.122:6789/0 conn(0x7f666c138d40 legacy=0x7f666c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 monclient: found mon.noname-i 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T04:00:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0a0cc0 con 0x7f666c132b40 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 5 monclient: authenticate success, global_id 234394 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f665c0016d0 con 0x7f666c132b40 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: dump: 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f665c0018c0 con 0x7f666c132b40 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 1 -- 172.21.15.26:0/1322032792 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f665c001df0 con 0x7f666c132b40 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f666b7fe700 10 monclient: dump: 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 4 set_mon_vals no callback set 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 10 monclient: shutdown 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- 172.21.15.26:0/1322032792 >> v1:172.21.15.122:6792/0 conn(0x7f666c132b40 legacy=0x7f666c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- 172.21.15.26:0/1322032792 shutdown_connections 2024-09-15T04:00:19.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- 172.21.15.26:0/1322032792 >> 172.21.15.26:0/1322032792 conn(0x7f666c06e370 msgr2=0x7f666c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- 172.21.15.26:0/1322032792 shutdown_connections 2024-09-15T04:00:19.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.494+0000 7f66717e3700 1 -- 172.21.15.26:0/1322032792 wait complete. 2024-09-15T04:00:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient: monmap: 2024-09-15T04:00:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 1 Processor -- start 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 1 -- start start 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient: init 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient: _add_conns ranks=[0,1,3,4,8,2,7,5,6] 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient(hunting): picked mon.noname-a con 0x7f666c09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient(hunting): picked mon.noname-f con 0x7f666c132b40 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient(hunting): picked mon.noname-g con 0x7f666c138d40 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c0998c0 con 0x7f666c09faa0 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c098880 con 0x7f666c132b40 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f666bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f666c09faa0 0x7f666c097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59374/0 (socket says 172.21.15.26:59374) 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c09cfa0 con 0x7f666c138d40 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f666bfff700 1 -- 172.21.15.26:0/3188462139 learned_addr learned my addr 172.21.15.26:0/3188462139 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.498+0000 7f66717e3700 10 monclient(hunting): authenticate will time out at 1763.931802s 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122844734 0 0) 0x7f666c0998c0 con 0x7f666c09faa0 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): my global_id is 194832 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f664c003570 con 0x7f666c09faa0 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117966483 0 0) 0x7f666c098880 con 0x7f666c132b40 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): my global_id is 204715 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c0998c0 con 0x7f666c132b40 2024-09-15T04:00:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643223799 0 0) 0x7f666c09cfa0 con 0x7f666c138d40 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): my global_id is 244200 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c098880 con 0x7f666c138d40 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2035736177 0 0) 0x7f664c003570 con 0x7f666c09faa0 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f666c09cfa0 con 0x7f666c09faa0 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f665c001db0 con 0x7f666c09faa0 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.518 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.519 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4064779180 0 0) 0x7f666c0998c0 con 0x7f666c132b40 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f664c003570 con 0x7f666c132b40 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6654001570 con 0x7f666c132b40 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3560672070 0 0) 0x7f666c098880 con 0x7f666c138d40 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f666c0998c0 con 0x7f666c138d40 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6664001c00 con 0x7f666c138d40 2024-09-15T04:00:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2708425719 0 0) 0x7f666c09cfa0 con 0x7f666c09faa0 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.122:6790/0 conn(0x7f666c138d40 legacy=0x7f666c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.122:6789/0 conn(0x7f666c132b40 legacy=0x7f666c098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 monclient: found mon.a 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c09ccc0 con 0x7f666c09faa0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 5 monclient: authenticate success, global_id 194832 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 10 monclient: _renew_subs 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c09afb0 con 0x7f666c09faa0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 10 monclient: _renew_subs 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c0995c0 con 0x7f666c09faa0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f665c0024e0 con 0x7f666c09faa0 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f665c002a10 con 0x7f666c09faa0 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: dump: 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f666affd700 4 set_mon_vals no callback set 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6650006590 con 0x7f6650002840 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f665c002bf0 con 0x7f666c09faa0 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f665c044110 con 0x7f666c09faa0 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062010157 0 0) 0x7f6650006590 con 0x7f6650002840 2024-09-15T04:00:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient(hunting): my global_id is 135530 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c09cfa0 con 0x7f6650002840 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6670fe2700 10 monclient: get_auth_request con 0x7f664c038000 auth_method 0 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1933966128 0 0) 0x7f666c09cfa0 con 0x7f6650002840 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6650006590 con 0x7f6650002840 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6654004b70 con 0x7f6650002840 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.502+0000 7f6668ff9700 10 monclient: dump: 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 989998880 0 0) 0x7f6650006590 con 0x7f6650002840 2024-09-15T04:00:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f665000a130 con 0x7f6650002840 2024-09-15T04:00:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6654005110 con 0x7f6650002840 2024-09-15T04:00:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.26:6793/0 conn(0x7f6650002840 legacy=0x7f6650004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.506+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.26:6793/0 conn(0x7f6650002840 legacy=0x7f6650004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f66717e3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6650007520 con 0x7f66500068d0 2024-09-15T04:00:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931654635 0 0) 0x7f6650007520 con 0x7f66500068d0 2024-09-15T04:00:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient(hunting): my global_id is 135539 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6650006590 con 0x7f66500068d0 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2391266942 0 0) 0x7f6650006590 con 0x7f66500068d0 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6650007520 con 0x7f66500068d0 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6654005920 con 0x7f66500068d0 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient: dump: 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1246922962 0 0) 0x7f6650007520 con 0x7f66500068d0 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6650008760 con 0x7f66500068d0 2024-09-15T04:00:19.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 267184153) 0x7f6654005ac0 con 0x7f66500068d0 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.26:6793/0 conn(0x7f66500068d0 legacy=0x7f6650006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6668ff9700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.26:6793/0 conn(0x7f66500068d0 legacy=0x7f6650006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:19.533 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.534 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:19.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.536 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:19.537 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.538 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.539 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.542 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:19.543 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.544 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.547 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.548 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.551 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.553 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:19.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.554 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.554 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:19.554 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:19.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.122:6812/20895 conn(0x7f664c038000 legacy=0x7f664c03a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f66717e3700 10 monclient: shutdown 2024-09-15T04:00:19.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f66717e3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 >> v1:172.21.15.26:6789/0 conn(0x7f666c09faa0 legacy=0x7f666c097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.518+0000 7f6670fe2700 1 -- 172.21.15.26:0/3188462139 reap_dead start 2024-09-15T04:00:19.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.522+0000 7f6670fe2700 1 -- 172.21.15.26:0/3188462139 reap_dead start 2024-09-15T04:00:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.522+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 shutdown_connections 2024-09-15T04:00:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.522+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 >> 172.21.15.26:0/3188462139 conn(0x7f666c06e370 msgr2=0x7f666c12ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.522+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 shutdown_connections 2024-09-15T04:00:19.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.522+0000 7f66717e3700 1 -- 172.21.15.26:0/3188462139 wait complete. 2024-09-15T04:00:19.556 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:19.556 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T04:00:19.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T04:00:19.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: get_monmap_and_config 2024-09-15T04:00:19.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: monmap: 2024-09-15T04:00:19.675 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:19.671983+0000 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:19.671983+0000 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.676 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.677 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.677 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 1 Processor -- start 2024-09-15T04:00:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 1 -- start start 2024-09-15T04:00:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: init 2024-09-15T04:00:19.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: _add_conns ranks=[8,1,2,5,3,7,4,0,6] 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient(hunting): picked mon.noname-e con 0x7f1334132b40 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient(hunting): picked mon.noname-f con 0x7f133409faa0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient(hunting): picked mon.noname-b con 0x7f1334138d40 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133404ca10 con 0x7f133409faa0 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133409d5d0 con 0x7f1334138d40 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1334133620 con 0x7f1334132b40 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient(hunting): authenticate will time out at 1764.101939s 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1339026700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f1334132b40 0x7f1334132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37334/0 (socket says 172.21.15.26:37334) 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1339827700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f1334138d40 0x7f133413b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57976/0 (socket says 172.21.15.26:57976) 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1339026700 1 -- 172.21.15.26:0/601306591 learned_addr learned my addr 172.21.15.26:0/601306591 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420672758 0 0) 0x7f1334133620 con 0x7f1334132b40 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): my global_id is 135548 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320003570 con 0x7f1334132b40 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621603981 0 0) 0x7f133409d5d0 con 0x7f1334138d40 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): my global_id is 204752 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1334133620 con 0x7f1334138d40 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194780446 0 0) 0x7f133404ca10 con 0x7f133409faa0 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient(hunting): my global_id is 204724 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133409d5d0 con 0x7f133409faa0 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364428564 0 0) 0x7f1320003570 con 0x7f1334132b40 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 >> v1:172.21.15.26:6790/0 conn(0x7f1334138d40 legacy=0x7f133413b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 >> v1:172.21.15.122:6789/0 conn(0x7f133409faa0 legacy=0x7f133409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 monclient: found mon.noname-e 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340a0cc0 con 0x7f1334132b40 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 5 monclient: authenticate success, global_id 135548 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1324001b30 con 0x7f1334132b40 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: dump: 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.688 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1324001da0 con 0x7f1334132b40 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 4 set_mon_vals no callback set 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 1 -- 172.21.15.26:0/601306591 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1324004fb0 con 0x7f1334132b40 2024-09-15T04:00:19.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f1333fff700 10 monclient: dump: 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 10 monclient: shutdown 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.670+0000 7f133a028700 1 -- 172.21.15.26:0/601306591 >> v1:172.21.15.26:6793/0 conn(0x7f1334132b40 legacy=0x7f1334132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- 172.21.15.26:0/601306591 shutdown_connections 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- 172.21.15.26:0/601306591 >> 172.21.15.26:0/601306591 conn(0x7f133406e370 msgr2=0x7f13340a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- 172.21.15.26:0/601306591 shutdown_connections 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- 172.21.15.26:0/601306591 wait complete. 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient: monmap: 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.692 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 Processor -- start 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- start start 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient: init 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient: _add_conns ranks=[4,1,0,8,2,5,7,3,6] 2024-09-15T04:00:19.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient(hunting): picked mon.noname-c con 0x7f133409faa0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient(hunting): picked mon.noname-f con 0x7f1334132b40 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient(hunting): picked mon.noname-a con 0x7f1334138d40 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133409cfa0 con 0x7f1334138d40 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133411c890 con 0x7f1334132b40 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133409b5f0 con 0x7f133409faa0 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f133a028700 10 monclient(hunting): authenticate will time out at 1764.106382s 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f1339026700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f133409faa0 0x7f13340995f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41002/0 (socket says 172.21.15.26:41002) 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f1339026700 1 -- 172.21.15.26:0/1926121327 learned_addr learned my addr 172.21.15.26:0/1926121327 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506548383 0 0) 0x7f133409cfa0 con 0x7f1334138d40 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): my global_id is 194841 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1314003570 con 0x7f1334138d40 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249362759 0 0) 0x7f133409b5f0 con 0x7f133409faa0 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): my global_id is 204727 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133409cfa0 con 0x7f133409faa0 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892419487 0 0) 0x7f133411c890 con 0x7f1334132b40 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): my global_id is 204733 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133409b5f0 con 0x7f1334132b40 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 60265943 0 0) 0x7f133409cfa0 con 0x7f133409faa0 2024-09-15T04:00:19.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f133411c890 con 0x7f133409faa0 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1324001b70 con 0x7f133409faa0 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): dump: 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.696 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2146806421 0 0) 0x7f1314003570 con 0x7f1334138d40 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f133409cfa0 con 0x7f1334138d40 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f132c001900 con 0x7f1334138d40 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 10 monclient(hunting): dump: 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2315417102 0 0) 0x7f133411c890 con 0x7f133409faa0 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.122:6789/0 conn(0x7f1334132b40 legacy=0x7f1334099d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.674+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.26:6789/0 conn(0x7f1334138d40 legacy=0x7f133409a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 monclient: found mon.c 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13341326a0 con 0x7f133409faa0 2024-09-15T04:00:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 5 monclient: authenticate success, global_id 204727 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 10 monclient: _renew_subs 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133409cc40 con 0x7f133409faa0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 10 monclient: _renew_subs 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f133409b0e0 con 0x7f133409faa0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1324005050 con 0x7f133409faa0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1324001530 con 0x7f133409faa0 2024-09-15T04:00:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13337fe700 4 set_mon_vals no callback set 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: dump: 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.701 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f133404bbf0 con 0x7f1334136a80 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f132400fe10 con 0x7f133409faa0 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f1324044520 con 0x7f133409faa0 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947745223 0 0) 0x7f133404bbf0 con 0x7f1334136a80 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient(hunting): my global_id is 204742 2024-09-15T04:00:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133411c890 con 0x7f1334136a80 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f1339827700 10 monclient: get_auth_request con 0x7f1314037d30 auth_method 0 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1562156930 0 0) 0x7f133411c890 con 0x7f1334136a80 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f133404bbf0 con 0x7f1334136a80 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f131c001a30 con 0x7f1334136a80 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 10 monclient: dump: 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2556038068 0 0) 0x7f133404bbf0 con 0x7f1334136a80 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.678+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f133411cad0 con 0x7f1334136a80 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.682+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f131c001f90 con 0x7f1334136a80 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.682+0000 7f13317fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.682+0000 7f13317fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.682+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.122:6789/0 conn(0x7f1334136a80 legacy=0x7f1334136e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.682+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.122:6789/0 conn(0x7f1334136a80 legacy=0x7f1334136e50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.690+0000 7f133a028700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.690+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f133406fac0 con 0x7f1334137560 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732324235 0 0) 0x7f133406fac0 con 0x7f1334137560 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient(hunting): my global_id is 204751 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133404bbf0 con 0x7f1334137560 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4182165209 0 0) 0x7f133404bbf0 con 0x7f1334137560 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f133406fac0 con 0x7f1334137560 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f131c002820 con 0x7f1334137560 2024-09-15T04:00:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient: dump: 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 209761293 0 0) 0x7f133406fac0 con 0x7f1334137560 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f13340a1ee0 con 0x7f1334137560 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3239392646) 0x7f131c0029c0 con 0x7f1334137560 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.122:6789/0 conn(0x7f1334137560 legacy=0x7f133402ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f13317fa700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.122:6789/0 conn(0x7f1334137560 legacy=0x7f133402ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.710 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:19.711 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.712 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.713 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.714 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.715 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:19.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.718 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.721 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:19.722 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:19.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.730 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:19.730 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.122:6812/20895 conn(0x7f1314037d30 legacy=0x7f131403a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 10 monclient: shutdown 2024-09-15T04:00:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 >> v1:172.21.15.26:6791/0 conn(0x7f133409faa0 legacy=0x7f13340995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f1339827700 1 -- 172.21.15.26:0/1926121327 reap_dead start 2024-09-15T04:00:19.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f1339827700 1 -- 172.21.15.26:0/1926121327 reap_dead start 2024-09-15T04:00:19.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 shutdown_connections 2024-09-15T04:00:19.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 >> 172.21.15.26:0/1926121327 conn(0x7f133406e370 msgr2=0x7f133409da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 shutdown_connections 2024-09-15T04:00:19.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.694+0000 7f133a028700 1 -- 172.21.15.26:0/1926121327 wait complete. 2024-09-15T04:00:19.732 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:19.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T04:00:19.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.842+0000 7f64ba3e8700 10 monclient: get_monmap_and_config 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.842+0000 7f64ba3e8700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.842+0000 7f64ba3e8700 10 monclient: monmap: 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:19.847641+0000 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:19.847641+0000 2024-09-15T04:00:19.851 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.852 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 1 Processor -- start 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 1 -- start start 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient: init 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient: _add_conns ranks=[0,2,1,3,6,5,8,7,4] 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient(hunting): picked mon.noname-a con 0x7f64b412fd20 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient(hunting): picked mon.noname-b con 0x7f64b41300f0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient(hunting): picked mon.noname-f con 0x7f64b4130520 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b404ca10 con 0x7f64b412fd20 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b40a2830 con 0x7f64b4130520 2024-09-15T04:00:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b4097b70 con 0x7f64b41300f0 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient(hunting): authenticate will time out at 1764.277619s 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b37fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f64b41300f0 0x7f64b4137e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57992/0 (socket says 172.21.15.26:57992) 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b3fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f64b412fd20 0x7f64b4133270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59388/0 (socket says 172.21.15.26:59388) 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b37fe700 1 -- 172.21.15.26:0/1350879222 learned_addr learned my addr 172.21.15.26:0/1350879222 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615960323 0 0) 0x7f64b4097b70 con 0x7f64b41300f0 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): my global_id is 204761 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f649c003570 con 0x7f64b41300f0 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810452057 0 0) 0x7f64b404ca10 con 0x7f64b412fd20 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): my global_id is 194850 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b4097b70 con 0x7f64b412fd20 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 53604733 0 0) 0x7f64b40a2830 con 0x7f64b4130520 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient(hunting): my global_id is 204760 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b404ca10 con 0x7f64b4130520 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373114455 0 0) 0x7f64b4097b70 con 0x7f64b412fd20 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 >> v1:172.21.15.26:6790/0 conn(0x7f64b41300f0 legacy=0x7f64b4137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 >> v1:172.21.15.122:6789/0 conn(0x7f64b4130520 legacy=0x7f64b413b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 monclient: found mon.noname-a 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4133980 con 0x7f64b412fd20 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a4001790 con 0x7f64b412fd20 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 5 monclient: authenticate success, global_id 194850 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: dump: 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:19.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64a40019b0 con 0x7f64b412fd20 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 4 set_mon_vals no callback set 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 1 -- 172.21.15.26:0/1350879222 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a4004830 con 0x7f64b412fd20 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64b2ffd700 10 monclient: dump: 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.863 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 10 monclient: shutdown 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.846+0000 7f64ba3e8700 1 -- 172.21.15.26:0/1350879222 >> v1:172.21.15.26:6789/0 conn(0x7f64b412fd20 legacy=0x7f64b4133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- 172.21.15.26:0/1350879222 shutdown_connections 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- 172.21.15.26:0/1350879222 >> 172.21.15.26:0/1350879222 conn(0x7f64b406e410 msgr2=0x7f64b40a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- 172.21.15.26:0/1350879222 shutdown_connections 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- 172.21.15.26:0/1350879222 wait complete. 2024-09-15T04:00:19.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient: build_initial_monmap 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient: monmap: 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 Processor -- start 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- start start 2024-09-15T04:00:19.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient: init 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient: _add_conns ranks=[3,0,4,8,7,2,5,6,1] 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient(hunting): picked mon.noname-g con 0x7f64b412fd20 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient(hunting): picked mon.noname-a con 0x7f64b41300f0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient(hunting): picked mon.noname-c con 0x7f64b4130520 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b411e820 con 0x7f64b41300f0 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b411db20 con 0x7f64b412fd20 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b411ea80 con 0x7f64b4130520 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64ba3e8700 10 monclient(hunting): authenticate will time out at 1764.282836s 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64b37fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f64b41300f0 0x7f64b4146f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59396/0 (socket says 172.21.15.26:59396) 2024-09-15T04:00:19.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64b8985700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f64b4130520 0x7f64b414a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41006/0 (socket says 172.21.15.26:41006) 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f64b37fe700 1 -- 172.21.15.26:0/970245971 learned_addr learned my addr 172.21.15.26:0/970245971 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219055674 0 0) 0x7f64b411e820 con 0x7f64b41300f0 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): my global_id is 194859 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a0003570 con 0x7f64b41300f0 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335240734 0 0) 0x7f64b411db20 con 0x7f64b412fd20 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): my global_id is 244209 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b411e820 con 0x7f64b412fd20 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403143952 0 0) 0x7f64b411ea80 con 0x7f64b4130520 2024-09-15T04:00:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 10 monclient(hunting): my global_id is 204736 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.850+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b411db20 con 0x7f64b4130520 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3331487949 0 0) 0x7f64b411db20 con 0x7f64b4130520 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64b411ea80 con 0x7f64b4130520 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3517058929 0 0) 0x7f64a0003570 con 0x7f64b41300f0 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64b411db20 con 0x7f64b41300f0 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64ac001c80 con 0x7f64b4130520 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): dump: 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.873 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a8001a40 con 0x7f64b41300f0 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): dump: 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.875 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1529843450 0 0) 0x7f64b411e820 con 0x7f64b412fd20 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64a0003570 con 0x7f64b412fd20 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a4001690 con 0x7f64b412fd20 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): dump: 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.876 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.877 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 381338227 0 0) 0x7f64b411db20 con 0x7f64b41300f0 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.26:6791/0 conn(0x7f64b4130520 legacy=0x7f64b414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.122:6790/0 conn(0x7f64b412fd20 legacy=0x7f64b412e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 monclient: found mon.a 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b411ed60 con 0x7f64b41300f0 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: _finish_auth 0 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 5 monclient: authenticate success, global_id 194859 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 10 monclient: _renew_subs 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b411f040 con 0x7f64b41300f0 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 10 monclient: _renew_subs 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b41340b0 con 0x7f64b41300f0 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f64a8001d20 con 0x7f64b41300f0 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a8004ee0 con 0x7f64b41300f0 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64b27fc700 4 set_mon_vals no callback set 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: dump: 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6494006590 con 0x7f6494002840 2024-09-15T04:00:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f64a800f8a0 con 0x7f64b41300f0 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f64a8043e60 con 0x7f64b41300f0 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063514250 0 0) 0x7f6494006590 con 0x7f6494002840 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient(hunting): my global_id is 244218 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b411db20 con 0x7f6494002840 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f64b8985700 10 monclient: get_auth_request con 0x7f64a0038160 auth_method 0 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2666403961 0 0) 0x7f64b411db20 con 0x7f6494002840 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6494006590 con 0x7f6494002840 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a4004830 con 0x7f6494002840 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.854+0000 7f649bfff700 10 monclient: dump: 2024-09-15T04:00:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 72047052 0 0) 0x7f6494006590 con 0x7f6494002840 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f649400a130 con 0x7f6494002840 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f64a40050c0 con 0x7f6494002840 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.122:6790/0 conn(0x7f6494002840 legacy=0x7f6494004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.858+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.122:6790/0 conn(0x7f6494002840 legacy=0x7f6494004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f64ba3e8700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6494009600 con 0x7f6494009070 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659057833 0 0) 0x7f6494009600 con 0x7f6494009070 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient(hunting): my global_id is 244227 2024-09-15T04:00:19.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6494006590 con 0x7f6494009070 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 150043998 0 0) 0x7f6494006590 con 0x7f6494009070 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6494009600 con 0x7f6494009070 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64a4005950 con 0x7f6494009070 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient: dump: 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1623013795 0 0) 0x7f6494009600 con 0x7f6494009070 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f64940098b0 con 0x7f6494009070 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3971213223) 0x7f64a4007e20 con 0x7f6494009070 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.122:6790/0 conn(0x7f6494009070 legacy=0x7f6494008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f649bfff700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.122:6790/0 conn(0x7f6494009070 legacy=0x7f6494008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:19.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.888 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:19.890 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:19.891 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.892 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:19.893 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:19.898 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:19.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:19.901 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:19.904 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:19.905 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:19.906 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.870+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.122:6812/20895 conn(0x7f64a0038160 legacy=0x7f64a003a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 10 monclient: shutdown 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 >> v1:172.21.15.26:6789/0 conn(0x7f64b41300f0 legacy=0x7f64b4146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64b8985700 1 -- 172.21.15.26:0/970245971 reap_dead start 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 shutdown_connections 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 >> 172.21.15.26:0/970245971 conn(0x7f64b406e410 msgr2=0x7f64b40a49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 shutdown_connections 2024-09-15T04:00:19.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:19.874+0000 7f64ba3e8700 1 -- 172.21.15.26:0/970245971 wait complete. 2024-09-15T04:00:19.909 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:19.909 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T04:00:19.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T04:00:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient: get_monmap_and_config 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient: build_initial_monmap 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient: monmap: 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:20.024423+0000 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:20.024423+0000 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:20.029 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:20.029 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 1 Processor -- start 2024-09-15T04:00:20.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 1 -- start start 2024-09-15T04:00:20.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient: init 2024-09-15T04:00:20.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:20.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient: _add_conns ranks=[8,0,4,1,5,3,7,2,6] 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient(hunting): picked mon.noname-e con 0x7f41f8070de0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient(hunting): picked mon.noname-a con 0x7f41f806cdc0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient(hunting): picked mon.noname-c con 0x7f41f8137060 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f804ca10 con 0x7f41f806cdc0 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f806baa0 con 0x7f41f8137060 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f806c3e0 con 0x7f41f8070de0 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41fd67e700 10 monclient(hunting): authenticate will time out at 1764.455611s 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41f67fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41f806cdc0 0x7f41f809d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59408/0 (socket says 172.21.15.26:59408) 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41f67fc700 1 -- 172.21.15.26:0/1264717285 learned_addr learned my addr 172.21.15.26:0/1264717285 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41f77fe700 1 --1- 172.21.15.26:0/1264717285 >> v1:172.21.15.26:6791/0 conn(0x7f41f8137060 0x7f41f8139440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41020/0 (socket says 172.21.15.26:41020) 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.022+0000 7f41f6ffd700 1 --1- 172.21.15.26:0/1264717285 >> v1:172.21.15.26:6793/0 conn(0x7f41f8070de0 0x7f41f809d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37346/0 (socket says 172.21.15.26:37346) 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216095601 0 0) 0x7f41f806baa0 con 0x7f41f8137060 2024-09-15T04:00:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): my global_id is 204745 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e0003570 con 0x7f41f8137060 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273463373 0 0) 0x7f41f804ca10 con 0x7f41f806cdc0 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): my global_id is 194868 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f806baa0 con 0x7f41f806cdc0 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406420490 0 0) 0x7f41f806c3e0 con 0x7f41f8070de0 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient(hunting): my global_id is 135557 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f804ca10 con 0x7f41f8070de0 2024-09-15T04:00:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1139316035 0 0) 0x7f41e0003570 con 0x7f41f8137060 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 >> v1:172.21.15.26:6793/0 conn(0x7f41f8070de0 legacy=0x7f41f809d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 >> v1:172.21.15.26:6789/0 conn(0x7f41f806cdc0 legacy=0x7f41f809d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 monclient: found mon.noname-c 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80711b0 con 0x7f41f8137060 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: _finish_auth 0 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41f0001a60 con 0x7f41f8137060 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: dump: 2024-09-15T04:00:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.037 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 5 monclient: authenticate success, global_id 204745 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f41f0001d40 con 0x7f41f8137060 2024-09-15T04:00:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 1 -- 172.21.15.26:0/1264717285 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41f0001f30 con 0x7f41f8137060 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 4 set_mon_vals no callback set 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41f5ffb700 10 monclient: dump: 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.040 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 10 monclient: shutdown 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 1 -- 172.21.15.26:0/1264717285 >> v1:172.21.15.26:6791/0 conn(0x7f41f8137060 legacy=0x7f41f8139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 1 -- 172.21.15.26:0/1264717285 shutdown_connections 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 1 -- 172.21.15.26:0/1264717285 >> 172.21.15.26:0/1264717285 conn(0x7f41f8063e10 msgr2=0x7f41f80a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 1 -- 172.21.15.26:0/1264717285 shutdown_connections 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 1 -- 172.21.15.26:0/1264717285 wait complete. 2024-09-15T04:00:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 10 monclient: build_initial_monmap 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.026+0000 7f41fd67e700 10 monclient: monmap: 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 Processor -- start 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 -- start start 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: init 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: _add_conns ranks=[0,6,7,1,4,3,2,8,5] 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient(hunting): picked mon.noname-a con 0x7f41f806cdc0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient(hunting): picked mon.noname-d con 0x7f41f8070de0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient(hunting): picked mon.noname-i con 0x7f41f8137060 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f811b5a0 con 0x7f41f806cdc0 2024-09-15T04:00:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f811abc0 con 0x7f41f8070de0 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f811aea0 con 0x7f41f8137060 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient(hunting): authenticate will time out at 1764.461353s 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41f6ffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41f806cdc0 0x7f41f8099fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59424/0 (socket says 172.21.15.26:59424) 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41f6ffd700 1 -- 172.21.15.26:0/229441490 learned_addr learned my addr 172.21.15.26:0/229441490 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339728675 0 0) 0x7f41f811b5a0 con 0x7f41f806cdc0 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): my global_id is 194877 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e4003570 con 0x7f41f806cdc0 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252228572 0 0) 0x7f41f811abc0 con 0x7f41f8070de0 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): my global_id is 195009 2024-09-15T04:00:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f811b5a0 con 0x7f41f8070de0 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220975381 0 0) 0x7f41f811aea0 con 0x7f41f8137060 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): my global_id is 234403 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f811abc0 con 0x7f41f8137060 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3333860867 0 0) 0x7f41e4003570 con 0x7f41f806cdc0 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41f811aea0 con 0x7f41f806cdc0 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41e8001a80 con 0x7f41f806cdc0 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): dump: 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1304417496 0 0) 0x7f41f811b5a0 con 0x7f41f8070de0 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41e4003570 con 0x7f41f8070de0 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41ec001a60 con 0x7f41f8070de0 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): dump: 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3664530967 0 0) 0x7f41f811abc0 con 0x7f41f8137060 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41f811b5a0 con 0x7f41f8137060 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41f0001a20 con 0x7f41f8137060 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient(hunting): dump: 2024-09-15T04:00:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2407017116 0 0) 0x7f41e4003570 con 0x7f41f8070de0 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.122:6792/0 conn(0x7f41f8137060 legacy=0x7f41f8148510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.26:6789/0 conn(0x7f41f806cdc0 legacy=0x7f41f8099fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 monclient: found mon.d 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f809ae00 con 0x7f41f8070de0 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: _finish_auth 0 2024-09-15T04:00:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 5 monclient: authenticate success, global_id 195009 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: _renew_subs 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f81192c0 con 0x7f41f8070de0 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: _renew_subs 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f812fdc0 con 0x7f41f8070de0 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f41ec001c80 con 0x7f41f8070de0 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41ec004e20 con 0x7f41f8070de0 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41f57fa700 4 set_mon_vals no callback set 2024-09-15T04:00:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.030+0000 7f41deffd700 10 monclient: dump: 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f41ec005000 con 0x7f41f8070de0 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41fd67e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f41d8006590 con 0x7f41d8002840 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f41ec043e70 con 0x7f41f8070de0 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41f77fe700 10 monclient: get_auth_request con 0x7f41e4037fb0 auth_method 0 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958914193 0 0) 0x7f41d8006590 con 0x7f41d8002840 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 10 monclient(hunting): my global_id is 6215 2024-09-15T04:00:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e4003570 con 0x7f41d8002840 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2118972817 0 0) 0x7f41e4003570 con 0x7f41d8002840 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41d8006590 con 0x7f41d8002840 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41e8005010 con 0x7f41d8002840 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 10 monclient: dump: 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 753296608 0 0) 0x7f41d8006590 con 0x7f41d8002840 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.034+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41d800a130 con 0x7f41d8002840 2024-09-15T04:00:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.038+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f41e8007250 con 0x7f41d8002840 2024-09-15T04:00:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.038+0000 7f41deffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.038+0000 7f41deffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.038+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.122:6791/0 conn(0x7f41d8002840 legacy=0x7f41d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.038+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.122:6791/0 conn(0x7f41d8002840 legacy=0x7f41d8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41fd67e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f41d8007520 con 0x7f41d80068d0 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376041030 0 0) 0x7f41d8007520 con 0x7f41d80068d0 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41deffd700 10 monclient(hunting): my global_id is 6224 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.046+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d8006590 con 0x7f41d80068d0 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1947688665 0 0) 0x7f41d8006590 con 0x7f41d80068d0 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41d8007520 con 0x7f41d80068d0 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41e8005a90 con 0x7f41d80068d0 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 10 monclient: dump: 2024-09-15T04:00:20.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 798005940 0 0) 0x7f41d8007520 con 0x7f41d80068d0 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f41d8008760 con 0x7f41d80068d0 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7741 (unknown 0 0 2800717804) 0x7f41e800a240 con 0x7f41d80068d0 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.122:6791/0 conn(0x7f41d80068d0 legacy=0x7f41d8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41deffd700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.122:6791/0 conn(0x7f41d80068d0 legacy=0x7f41d8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:20.067 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:20.068 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:20.069 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.070 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:20.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:20.072 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.073 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.076 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.077 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.078 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:20.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.083 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:20.086 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:20.087 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:20.088 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:20.088 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.088 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.088 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:20.088 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:20.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.122:6812/20895 conn(0x7f41e4037fb0 legacy=0x7f41e403a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 10 monclient: shutdown 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 >> v1:172.21.15.26:6792/0 conn(0x7f41f8070de0 legacy=0x7f41f809a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41f77fe700 1 -- 172.21.15.26:0/229441490 reap_dead start 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41f77fe700 1 -- 172.21.15.26:0/229441490 reap_dead start 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 shutdown_connections 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 >> 172.21.15.26:0/229441490 conn(0x7f41f8063e10 msgr2=0x7f41f80a0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 shutdown_connections 2024-09-15T04:00:20.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.050+0000 7f41fd67e700 1 -- 172.21.15.26:0/229441490 wait complete. 2024-09-15T04:00:20.091 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:20.091 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T04:00:20.091 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.198+0000 7f524e7aa700 10 monclient: get_monmap_and_config 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.198+0000 7f524e7aa700 10 monclient: build_initial_monmap 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.198+0000 7f524e7aa700 10 monclient: monmap: 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:20.203577+0000 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:20.203577+0000 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:20.208 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 1 Processor -- start 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 1 -- start start 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient: init 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient: _add_conns ranks=[6,0,4,7,1,2,8,3,5] 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient(hunting): picked mon.noname-d con 0x7f5248132b40 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient(hunting): picked mon.noname-a con 0x7f524809faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient(hunting): picked mon.noname-c con 0x7f5248138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524804ca10 con 0x7f524809faa0 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524809d5d0 con 0x7f5248138d40 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5248133620 con 0x7f5248132b40 2024-09-15T04:00:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient(hunting): authenticate will time out at 1764.633670s 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524d7a8700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5248132b40 0x7f5248132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:37124/0 (socket says 172.21.15.26:37124) 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524d7a8700 1 -- 172.21.15.26:0/208754198 learned_addr learned my addr 172.21.15.26:0/208754198 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327546536 0 0) 0x7f5248133620 con 0x7f5248132b40 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): my global_id is 195018 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5230003570 con 0x7f5248132b40 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856774082 0 0) 0x7f524809d5d0 con 0x7f5248138d40 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): my global_id is 204754 2024-09-15T04:00:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5248133620 con 0x7f5248138d40 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166792332 0 0) 0x7f524804ca10 con 0x7f524809faa0 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient(hunting): my global_id is 194886 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f524809d5d0 con 0x7f524809faa0 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747058291 0 0) 0x7f5230003570 con 0x7f5248132b40 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 >> v1:172.21.15.26:6791/0 conn(0x7f5248138d40 legacy=0x7f524813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 >> v1:172.21.15.26:6789/0 conn(0x7f524809faa0 legacy=0x7f524809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 monclient: found mon.noname-d 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52480a0cc0 con 0x7f5248132b40 2024-09-15T04:00:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: _finish_auth 0 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 5 monclient: authenticate success, global_id 195018 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5240001b30 con 0x7f5248132b40 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: dump: 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5240001da0 con 0x7f5248132b40 2024-09-15T04:00:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 1 -- 172.21.15.26:0/208754198 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5240004fb0 con 0x7f5248132b40 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f524e7aa700 4 set_mon_vals no callback set 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.202+0000 7f5247fff700 10 monclient: dump: 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient: shutdown 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- 172.21.15.26:0/208754198 >> v1:172.21.15.26:6792/0 conn(0x7f5248132b40 legacy=0x7f5248132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- 172.21.15.26:0/208754198 shutdown_connections 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- 172.21.15.26:0/208754198 >> 172.21.15.26:0/208754198 conn(0x7f524806e370 msgr2=0x7f52480a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- 172.21.15.26:0/208754198 shutdown_connections 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- 172.21.15.26:0/208754198 wait complete. 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient: build_initial_monmap 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient: monmap: 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:20.220 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 Processor -- start 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- start start 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient: init 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient: _add_conns ranks=[4,1,8,3,6,7,2,0,5] 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient(hunting): picked mon.noname-c con 0x7f524809faa0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient(hunting): picked mon.noname-f con 0x7f5248132b40 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient(hunting): picked mon.noname-e con 0x7f5248138d40 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:20.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524809b530 con 0x7f5248132b40 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524809cfa0 con 0x7f524809faa0 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524811c750 con 0x7f5248138d40 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524e7aa700 10 monclient(hunting): authenticate will time out at 1764.638762s 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524dfa9700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5248138d40 0x7f524814a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37350/0 (socket says 172.21.15.26:37350) 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f524dfa9700 1 -- 172.21.15.26:0/3797505448 learned_addr learned my addr 172.21.15.26:0/3797505448 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482986098 0 0) 0x7f524811c750 con 0x7f5248138d40 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): my global_id is 135566 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5234003570 con 0x7f5248138d40 2024-09-15T04:00:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685032464 0 0) 0x7f524809cfa0 con 0x7f524809faa0 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): my global_id is 204763 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f524811c750 con 0x7f524809faa0 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394106191 0 0) 0x7f524809b530 con 0x7f5248132b40 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): my global_id is 204769 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f524809cfa0 con 0x7f5248132b40 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1105441873 0 0) 0x7f5234003570 con 0x7f5248138d40 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f524809b530 con 0x7f5248138d40 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5238001a90 con 0x7f5248138d40 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): dump: 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.229 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.206+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1514750236 0 0) 0x7f524811c750 con 0x7f524809faa0 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5234003570 con 0x7f524809faa0 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2471427172 0 0) 0x7f524809b530 con 0x7f5248138d40 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.26:6791/0 conn(0x7f524809faa0 legacy=0x7f5248099680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.122:6789/0 conn(0x7f5248132b40 legacy=0x7f5248099d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: _finish_hunting 0 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 monclient: found mon.e 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248132660 con 0x7f5248138d40 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: _finish_auth 0 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 5 monclient: authenticate success, global_id 135566 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 10 monclient: _renew_subs 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524809ca70 con 0x7f5248138d40 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 10 monclient: _renew_subs 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f524809bda0 con 0x7f5248138d40 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5238002030 con 0x7f5248138d40 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5238004e80 con 0x7f5248138d40 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f5246ffd700 4 set_mon_vals no callback set 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: dump: 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5228006590 con 0x7f5228002840 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f523800f8c0 con 0x7f5248138d40 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f5238043dc0 con 0x7f5248138d40 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170104085 0 0) 0x7f5228006590 con 0x7f5228002840 2024-09-15T04:00:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient(hunting): my global_id is 234412 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f524809b530 con 0x7f5228002840 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f524cfa7700 10 monclient: get_auth_request con 0x7f5234037990 auth_method 0 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1277680052 0 0) 0x7f524809b530 con 0x7f5228002840 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5228006590 con 0x7f5228002840 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5240004ee0 con 0x7f5228002840 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 10 monclient: dump: 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2903851147 0 0) 0x7f5228006590 con 0x7f5228002840 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.210+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f522800a130 con 0x7f5228002840 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.214+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5240005700 con 0x7f5228002840 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.214+0000 7f52457fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.214+0000 7f52457fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.214+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.122:6792/0 conn(0x7f5228002840 legacy=0x7f5228004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.214+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.122:6792/0 conn(0x7f5228002840 legacy=0x7f5228004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f524e7aa700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5228007520 con 0x7f52280068d0 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871357678 0 0) 0x7f5228007520 con 0x7f52280068d0 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 10 monclient(hunting): my global_id is 234421 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5228006590 con 0x7f52280068d0 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 869744754 0 0) 0x7f5228006590 con 0x7f52280068d0 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5228007520 con 0x7f52280068d0 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5240005ec0 con 0x7f52280068d0 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.222+0000 7f52457fa700 10 monclient: dump: 2024-09-15T04:00:20.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:20.239 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 972705697 0 0) 0x7f5228007520 con 0x7f52280068d0 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5228008760 con 0x7f52280068d0 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 1483011861) 0x7f5240006250 con 0x7f52280068d0 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.122:6792/0 conn(0x7f52280068d0 legacy=0x7f5228006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f52457fa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.122:6792/0 conn(0x7f52280068d0 legacy=0x7f5228006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:20.240 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 104, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 5, 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:20.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:20.259 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:20.260 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.122:6812/20895 conn(0x7f5234037990 legacy=0x7f5234039e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 10 monclient: shutdown 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 >> v1:172.21.15.26:6793/0 conn(0x7f5248138d40 legacy=0x7f524814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524dfa9700 1 -- 172.21.15.26:0/3797505448 reap_dead start 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524dfa9700 1 -- 172.21.15.26:0/3797505448 reap_dead start 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 shutdown_connections 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 >> 172.21.15.26:0/3797505448 conn(0x7f524806e370 msgr2=0x7f52480669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 shutdown_connections 2024-09-15T04:00:20.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.226+0000 7f524e7aa700 1 -- 172.21.15.26:0/3797505448 wait complete. 2024-09-15T04:00:20.262 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:00:20.262 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h', 'i', 'g', 'b'] 2024-09-15T04:00:20.262 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-15T04:00:20.262 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-09-15T04:00:20.262 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-15T04:00:20.262 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-15T04:00:20.262 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:00:20.286 INFO:tasks.ceph.mon.h:Stopped 2024-09-15T04:00:20.286 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-09-15T04:00:20.286 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T04:00:20.286 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T04:00:20.286 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:00:20.349 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T04:00:20.350 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.g 2024-09-15T04:00:20.350 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T04:00:20.350 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T04:00:20.350 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:00:20.367 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T04:00:20.367 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-15T04:00:20.367 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T04:00:20.367 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T04:00:20.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:00:20.386 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T04:00:20.386 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-09-15T04:00:20.386 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:00:20.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient: get_monmap_and_config 2024-09-15T04:00:20.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient: build_initial_monmap 2024-09-15T04:00:20.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient: monmap: 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:20.493053+0000 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:20.493053+0000 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:20.497 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:20.498 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:20.498 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 1 Processor -- start 2024-09-15T04:00:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 1 -- start start 2024-09-15T04:00:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient: init 2024-09-15T04:00:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient: _add_conns ranks=[2,3,8,6,0,4,7,1,5] 2024-09-15T04:00:20.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient(hunting): picked mon.noname-b con 0x7f5f8c099310 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient(hunting): picked mon.noname-g con 0x7f5f8c132fa0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient(hunting): picked mon.noname-e con 0x7f5f8c1333f0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c04ca10 con 0x7f5f8c099310 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c097e50 con 0x7f5f8c132fa0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c0a1410 con 0x7f5f8c1333f0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f94238700 10 monclient(hunting): authenticate will time out at 1764.922455s 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f91fd4700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7f5f8c099310 legacy=0x7f5f8c0996e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f917d3700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f5f8c132fa0 legacy=0x7f5f8c1380a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f927d5700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5f8c1333f0 0x7f5f8c13b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37354/0 (socket says 172.21.15.26:37354) 2024-09-15T04:00:20.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.490+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 learned_addr learned my addr 172.21.15.26:0/913541801 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.694+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f8c099310 legacy=0x7f5f8c0996e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:20.694+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f8c132fa0 legacy=0x7f5f8c1380a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:21.098+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f8c099310 legacy=0x7f5f8c0996e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:21.098+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f8c132fa0 legacy=0x7f5f8c1380a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:21.902+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f8c099310 legacy=0x7f5f8c0996e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:21.902+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f8c132fa0 legacy=0x7f5f8c1380a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient(hunting): tick 2024-09-15T04:00:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6793/0 conn(0x7f5f8c1333f0 legacy=0x7f5f8c13b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f8c132fa0 legacy=0x7f5f8c1380a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f8c099310 legacy=0x7f5f8c0996e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient: _add_conns ranks=[7,6,4,1,5,0] 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-i con 0x7f5f700037f0 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-d con 0x7f5f70007470 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-c con 0x7f5f7000b1d0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c0a2710 con 0x7f5f7000b1d0 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c04ca10 con 0x7f5f70007470 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c097e50 con 0x7f5f700037f0 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f837fe700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.490+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f700037f0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:23.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:23.694+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f700037f0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:24.098+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f700037f0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:24.902+0000 7f5f91fd4700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f700037f0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient(hunting): tick 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f700037f0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6792/0 conn(0x7f5f70007470 legacy=0x7f5f70009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6791/0 conn(0x7f5f7000b1d0 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient: _add_conns ranks=[0,1,5] 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-a con 0x7f5f7000efd0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-f con 0x7f5f7000ff20 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-h con 0x7f5f70010ab0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700120b0 con 0x7f5f7000efd0 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c097e50 con 0x7f5f7000ff20 2024-09-15T04:00:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7001cc10 con 0x7f5f70010ab0 2024-09-15T04:00:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f837fe700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 reap_dead start 2024-09-15T04:00:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 reap_dead start 2024-09-15T04:00:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f700193e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:26.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:26.698+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f700193e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:27.102+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f700193e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:27.906+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f700193e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient(hunting): tick 2024-09-15T04:00:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f700193e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6789/0 conn(0x7f5f7000ff20 legacy=0x7f5f700103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6789/0 conn(0x7f5f7000efd0 legacy=0x7f5f7000f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient: _add_conns ranks=[6,5,4,0,2,7,3,8,1] 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-d con 0x7f5f70006400 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-h con 0x7f5f70010ab0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-c con 0x7f5f70006f80 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7001de40 con 0x7f5f70006f80 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7001cc10 con 0x7f5f70010ab0 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f70009930 con 0x7f5f70006400 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f837fe700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.490+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:29.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:29.698+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:30.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:30.102+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:30.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:30.906+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient(hunting): tick 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6792/0 conn(0x7f5f70006400 legacy=0x7f5f700067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f70010ab0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6791/0 conn(0x7f5f70006f80 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient: _add_conns ranks=[1,7,2,3,0,8] 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 reap_dead start 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-f con 0x7f5f7000ff20 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-i con 0x7f5f70006400 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-b con 0x7f5f7000efd0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f70009ac0 con 0x7f5f7000ff20 2024-09-15T04:00:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7001cc10 con 0x7f5f7000efd0 2024-09-15T04:00:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f70009e20 con 0x7f5f70006400 2024-09-15T04:00:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f837fe700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f70006400 legacy=0x7f5f700067d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:32.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.698+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:32.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:32.698+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f70006400 legacy=0x7f5f700067d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:33.102+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:33.102+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f70006400 legacy=0x7f5f700067d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:33.906+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:33.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:33.906+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f70006400 legacy=0x7f5f700067d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:35.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient(hunting): tick 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6792/0 conn(0x7f5f70006400 legacy=0x7f5f700067d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f70005c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6789/0 conn(0x7f5f7000ff20 legacy=0x7f5f700103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient: _add_conns ranks=[0,3,8] 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-a con 0x7f5f7000f3f0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-g con 0x7f5f7000f7c0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-e con 0x7f5f7000a7e0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f70006f80 con 0x7f5f7000f3f0 2024-09-15T04:00:35.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7001cc10 con 0x7f5f7000f7c0 2024-09-15T04:00:35.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f70009e20 con 0x7f5f7000a7e0 2024-09-15T04:00:35.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f837fe700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:35.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.494+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000a0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:35.698+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000a0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:36.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:36.102+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000a0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:36.906+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000a0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:38.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient(hunting): tick 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6793/0 conn(0x7f5f7000a7e0 legacy=0x7f5f7001a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000a0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6789/0 conn(0x7f5f7000f3f0 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient: _add_conns ranks=[6,5,3,8,0,1,7,2,4] 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 reap_dead start 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-d con 0x7f5f7001bb10 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 reap_dead start 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-h con 0x7f5f7000efd0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient(hunting): picked mon.noname-g con 0x7f5f7000f7c0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7000ac90 con 0x7f5f7000f7c0 2024-09-15T04:00:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7001cc10 con 0x7f5f7000efd0 2024-09-15T04:00:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7000af40 con 0x7f5f7001bb10 2024-09-15T04:00:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f837fe700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.494+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f7000efd0 legacy=0x7f5f700103a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:38.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.698+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:38.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:38.698+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f7000efd0 legacy=0x7f5f700103a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.102+0000 7f5f927d5700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.102+0000 7f5f917d3700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f7000efd0 legacy=0x7f5f700103a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.302+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153276399 0 0) 0x7f5f7000af40 con 0x7f5f7001bb10 2024-09-15T04:00:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.302+0000 7f5f90fd2700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.302+0000 7f5f90fd2700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.302+0000 7f5f90fd2700 10 monclient(hunting): my global_id is 195027 2024-09-15T04:00:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.302+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f78003570 con 0x7f5f7001bb10 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335999440 0 0) 0x7f5f78003570 con 0x7f5f7001bb10 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6791/0 conn(0x7f5f7000efd0 legacy=0x7f5f700103a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000f7c0 legacy=0x7f5f7000d680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: _finish_hunting 0 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 monclient: found mon.noname-d 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:00:39.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7001c5f0 con 0x7f5f7001bb10 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: _finish_auth 0 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 5 monclient: authenticate success, global_id 195027 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5f7c004270 con 0x7f5f7001bb10 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: dump: 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f7c004510 con 0x7f5f7001bb10 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 4 set_mon_vals no callback set 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 1 -- 172.21.15.26:0/913541801 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5f7c004a40 con 0x7f5f7001bb10 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f90fd2700 10 monclient: dump: 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.334 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.334 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 10 monclient: shutdown 2024-09-15T04:00:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 1 -- 172.21.15.26:0/913541801 >> v1:172.21.15.26:6792/0 conn(0x7f5f7001bb10 legacy=0x7f5f7001bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 1 -- 172.21.15.26:0/913541801 shutdown_connections 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 1 -- 172.21.15.26:0/913541801 >> 172.21.15.26:0/913541801 conn(0x7f5f8c0a6c30 msgr2=0x7f5f8c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 1 -- 172.21.15.26:0/913541801 shutdown_connections 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 1 -- 172.21.15.26:0/913541801 wait complete. 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 10 monclient: build_initial_monmap 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.322+0000 7f5f94238700 10 monclient: monmap: 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:39.340 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 1 Processor -- start 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 1 -- start start 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient: init 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient: _add_conns ranks=[3,5,8,2,1,4,7,0,6] 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient(hunting): picked mon.noname-g con 0x7f5f7000efd0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient(hunting): picked mon.noname-h con 0x7f5f7000f7c0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient(hunting): picked mon.noname-e con 0x7f5f7001bb10 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c11e610 con 0x7f5f7000efd0 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c1371d0 con 0x7f5f7000f7c0 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c11d440 con 0x7f5f7001bb10 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f91fd4700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f8c131210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f94238700 10 monclient(hunting): authenticate will time out at 1783.758098s 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f917d3700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f5f7000f7c0 legacy=0x7f5f8c146fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f927d5700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5f7001bb10 0x7f5f8c14a6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:34394/0 (socket says 172.21.15.26:34394) 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f927d5700 1 -- 172.21.15.26:0/2946401863 learned_addr learned my addr 172.21.15.26:0/2946401863 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965080886 0 0) 0x7f5f8c11d440 con 0x7f5f7001bb10 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): my global_id is 135575 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f70009c90 con 0x7f5f7001bb10 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 969161076 0 0) 0x7f5f70009c90 con 0x7f5f7001bb10 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5f8c11d440 con 0x7f5f7001bb10 2024-09-15T04:00:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5f840017d0 con 0x7f5f7001bb10 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): dump: 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.326+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 527963817 0 0) 0x7f5f8c11d440 con 0x7f5f7001bb10 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 >> v1:172.21.15.122:6791/0 conn(0x7f5f7000f7c0 legacy=0x7f5f8c146fb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 >> v1:172.21.15.122:6790/0 conn(0x7f5f7000efd0 legacy=0x7f5f8c131210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:00:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 monclient: found mon.e 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c11e330 con 0x7f5f7001bb10 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: _finish_auth 0 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 5 monclient: authenticate success, global_id 135575 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 monclient: _renew_subs 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c11e8f0 con 0x7f5f7001bb10 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 monclient: _renew_subs 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c131920 con 0x7f5f7001bb10 2024-09-15T04:00:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5f84004830 con 0x7f5f7001bb10 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5f84004ce0 con 0x7f5f7001bb10 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 10 monclient: dump: 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f83fff700 4 set_mon_vals no callback set 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5f84004ee0 con 0x7f5f7001bb10 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f5f84043d70 con 0x7f5f7001bb10 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c04bbf0 con 0x7f5f7001bb10 2024-09-15T04:00:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.330+0000 7f5f91fd4700 10 monclient: get_auth_request con 0x7f5f7003a7f0 auth_method 0 2024-09-15T04:00:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.334+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f5f84014b90 con 0x7f5f7001bb10 2024-09-15T04:00:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.334+0000 7f5f81ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.334+0000 7f5f81ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.562+0000 7f5f94238700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:00:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.562+0000 7f5f94238700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:00:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.562+0000 7f5f94238700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.562+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5f8c04bbf0 con 0x7f5f7001bb10 2024-09-15T04:00:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f81ffb700 1 -- 172.21.15.26:0/2946401863 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2677 (unknown 254746657 0 1379282551) 0x7f5f84014770 con 0x7f5f7001bb10 2024-09-15T04:00:39.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f81ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:00:39.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f81ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:39.572 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":108,"quorum":[0,1,4,6,8],"quorum_names":["a","f","c","d","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 >> v1:172.21.15.122:6812/20895 conn(0x7f5f7003a7f0 legacy=0x7f5f7003cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 10 monclient: shutdown 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 >> v1:172.21.15.26:6793/0 conn(0x7f5f7001bb10 legacy=0x7f5f8c14a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 shutdown_connections 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 >> 172.21.15.26:0/2946401863 conn(0x7f5f8c0a6c30 msgr2=0x7f5f8c0679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 shutdown_connections 2024-09-15T04:00:39.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.566+0000 7f5f94238700 1 -- 172.21.15.26:0/2946401863 wait complete. 2024-09-15T04:00:39.586 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-09-15T04:00:39.586 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon a 2024-09-15T04:00:39.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:00:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient: get_monmap_and_config 2024-09-15T04:00:39.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient: build_initial_monmap 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient: monmap: 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:39.693147+0000 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:39.693147+0000 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:39.697 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:39.698 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 1 Processor -- start 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 1 -- start start 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient: init 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient: _add_conns ranks=[4,1,0,2,3,5,8,6,7] 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient(hunting): picked mon.noname-c con 0x7f21dc09b8d0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient(hunting): picked mon.noname-f con 0x7f21dc09c250 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:39.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient(hunting): picked mon.noname-a con 0x7f21dc09e490 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc04ca10 con 0x7f21dc09e490 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc09d8a0 con 0x7f21dc09c250 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc1317e0 con 0x7f21dc09b8d0 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e2300700 10 monclient(hunting): authenticate will time out at 1784.123309s 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e089d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21dc09e490 0x7f21dc138f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38294/0 (socket says 172.21.15.26:38294) 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21e089d700 1 -- 172.21.15.26:0/1529465915 learned_addr learned my addr 172.21.15.26:0/1529465915 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21dbfff700 1 --1- 172.21.15.26:0/1529465915 >> v1:172.21.15.26:6791/0 conn(0x7f21dc09b8d0 0x7f21dc1309c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:34816/0 (socket says 172.21.15.26:34816) 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269206252 0 0) 0x7f21dc04ca10 con 0x7f21dc09e490 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21db7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21db7fe700 10 monclient(hunting): my global_id is 194895 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.690+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c8003570 con 0x7f21dc09e490 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642084941 0 0) 0x7f21dc1317e0 con 0x7f21dc09b8d0 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient(hunting): my global_id is 204772 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21dc04ca10 con 0x7f21dc09b8d0 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108826483 0 0) 0x7f21dc09d8a0 con 0x7f21dc09c250 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient(hunting): my global_id is 204778 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21dc1317e0 con 0x7f21dc09c250 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993776661 0 0) 0x7f21c8003570 con 0x7f21dc09e490 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 >> v1:172.21.15.26:6791/0 conn(0x7f21dc09b8d0 legacy=0x7f21dc1309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 >> v1:172.21.15.122:6789/0 conn(0x7f21dc09c250 legacy=0x7f21dc1310d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 monclient: found mon.noname-a 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc09d5c0 con 0x7f21dc09e490 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 5 monclient: authenticate success, global_id 194895 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21d4001b50 con 0x7f21dc09e490 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: dump: 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21d4001e30 con 0x7f21dc09e490 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 4 set_mon_vals no callback set 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 1 -- 172.21.15.26:0/1529465915 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21d4004fb0 con 0x7f21dc09e490 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21db7fe700 10 monclient: dump: 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 10 monclient: shutdown 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 -- 172.21.15.26:0/1529465915 >> v1:172.21.15.26:6789/0 conn(0x7f21dc09e490 legacy=0x7f21dc138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 -- 172.21.15.26:0/1529465915 shutdown_connections 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 -- 172.21.15.26:0/1529465915 >> 172.21.15.26:0/1529465915 conn(0x7f21dc063f30 msgr2=0x7f21dc0a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 -- 172.21.15.26:0/1529465915 shutdown_connections 2024-09-15T04:00:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 -- 172.21.15.26:0/1529465915 wait complete. 2024-09-15T04:00:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 10 monclient: build_initial_monmap 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 10 monclient: monmap: 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 Processor -- start 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 1 -- start start 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.694+0000 7f21e2300700 10 monclient: init 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient: _add_conns ranks=[3,0,5,1,2,8,7,6,4] 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient(hunting): picked mon.noname-g con 0x7f21dc09c250 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient(hunting): picked mon.noname-a con 0x7f21dc09e490 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient(hunting): picked mon.noname-h con 0x7f21dc11e380 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc11e790 con 0x7f21dc09e490 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc09cd20 con 0x7f21dc09c250 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc1315d0 con 0x7f21dc11e380 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient(hunting): authenticate will time out at 1784.128361s 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d3fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21dc09e490 0x7f21dc144d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38296/0 (socket says 172.21.15.26:38296) 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e089d700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f21dc11e380 legacy=0x7f21dc148460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d3fff700 1 -- 172.21.15.26:0/2053562100 learned_addr learned my addr 172.21.15.26:0/2053562100 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21dbfff700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f21dc09c250 legacy=0x7f21dc130830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392540331 0 0) 0x7f21dc1317e0 con 0x7f21dc09e490 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): my global_id is 194904 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21bc003570 con 0x7f21dc09e490 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2355453461 0 0) 0x7f21bc003570 con 0x7f21dc09e490 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21dc1317e0 con 0x7f21dc09e490 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21c4001b20 con 0x7f21dc09e490 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): dump: 2024-09-15T04:00:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1094870975 0 0) 0x7f21dc1317e0 con 0x7f21dc09e490 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.122:6791/0 conn(0x7f21dc11e380 legacy=0x7f21dc148460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.122:6790/0 conn(0x7f21dc09c250 legacy=0x7f21dc130830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 monclient: found mon.a 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc11fa90 con 0x7f21dc09e490 2024-09-15T04:00:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: _finish_auth 0 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 5 monclient: authenticate success, global_id 194904 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient: _renew_subs 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc11eec0 con 0x7f21dc09e490 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient: _renew_subs 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc130f40 con 0x7f21dc09e490 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f21c40020b0 con 0x7f21dc09e490 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21c40025e0 con 0x7f21dc09e490 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21da7fc700 4 set_mon_vals no callback set 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient: dump: 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f21c40027c0 con 0x7f21dc09e490 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f21c0006590 con 0x7f21c0002840 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f21c4043e10 con 0x7f21dc09e490 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918045522 0 0) 0x7f21c0006590 con 0x7f21c0002840 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 10 monclient(hunting): my global_id is 194913 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21dc1317e0 con 0x7f21c0002840 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.698+0000 7f21e089d700 10 monclient: get_auth_request con 0x7f21bc037570 auth_method 0 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 501357555 0 0) 0x7f21dc1317e0 con 0x7f21c0002840 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21c0006590 con 0x7f21c0002840 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21cc0018e0 con 0x7f21c0002840 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 10 monclient: dump: 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 182745022 0 0) 0x7f21c0006590 con 0x7f21c0002840 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21c000a130 con 0x7f21c0002840 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f21cc005860 con 0x7f21c0002840 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.26:6789/0 conn(0x7f21c0002840 legacy=0x7f21c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.702+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.26:6789/0 conn(0x7f21c0002840 legacy=0x7f21c0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f21c0007520 con 0x7f21c00068d0 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138491828 0 0) 0x7f21c0007520 con 0x7f21c00068d0 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient(hunting): my global_id is 194922 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c0006590 con 0x7f21c00068d0 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 246380813 0 0) 0x7f21c0006590 con 0x7f21c00068d0 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21c0007520 con 0x7f21c00068d0 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21cc004e60 con 0x7f21c00068d0 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient: dump: 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3555750743 0 0) 0x7f21c0007520 con 0x7f21c00068d0 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f21c0008760 con 0x7f21c00068d0 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7695 (unknown 0 0 2697094221) 0x7f21cc005af0 con 0x7f21c00068d0 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.26:6789/0 conn(0x7f21c00068d0 legacy=0x7f21c0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21d8ff9700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.26:6789/0 conn(0x7f21c00068d0 legacy=0x7f21c0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:39.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 108, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:39.730 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:39.731 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:39.732 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:39.733 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.734 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.737 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.738 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.739 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.740 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.741 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.742 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:39.745 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.746 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.747 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:39.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:39.749 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.750 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:39.750 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.122:6812/20895 conn(0x7f21bc037570 legacy=0x7f21bc039a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 10 monclient: shutdown 2024-09-15T04:00:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 >> v1:172.21.15.26:6789/0 conn(0x7f21dc09e490 legacy=0x7f21dc144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e089d700 1 -- 172.21.15.26:0/2053562100 reap_dead start 2024-09-15T04:00:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 shutdown_connections 2024-09-15T04:00:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 >> 172.21.15.26:0/2053562100 conn(0x7f21dc063f30 msgr2=0x7f21dc06bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 shutdown_connections 2024-09-15T04:00:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.714+0000 7f21e2300700 1 -- 172.21.15.26:0/2053562100 wait complete. 2024-09-15T04:00:39.752 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T04:00:39.752 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon c 2024-09-15T04:00:39.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:00:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.858+0000 7f139bfff700 10 monclient: get_monmap_and_config 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.858+0000 7f139bfff700 10 monclient: build_initial_monmap 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.858+0000 7f139bfff700 10 monclient: monmap: 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:39.863381+0000 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:39.863381+0000 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:39.868 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 Processor -- start 2024-09-15T04:00:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- start start 2024-09-15T04:00:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: init 2024-09-15T04:00:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: _add_conns ranks=[3,4,6,0,1,7,8,5,2] 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient(hunting): picked mon.noname-g con 0x7f139c063ac0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f139c09dcf0 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient(hunting): picked mon.noname-d con 0x7f139c09e0c0 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c04ca10 con 0x7f139c063ac0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c09cc00 con 0x7f139c09dcf0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c063e90 con 0x7f139c09e0c0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient(hunting): authenticate will time out at 1784.293120s 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139affd700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7f139c063ac0 legacy=0x7f139c09d5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f13927fc700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f139c09dcf0 0x7f139c1356c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:34820/0 (socket says 172.21.15.26:34820) 2024-09-15T04:00:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f13927fc700 1 -- 172.21.15.26:0/3493347313 learned_addr learned my addr 172.21.15.26:0/3493347313 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947875283 0 0) 0x7f139c09cc00 con 0x7f139c09dcf0 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient(hunting): my global_id is 204781 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1388003570 con 0x7f139c09dcf0 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194274613 0 0) 0x7f139c063e90 con 0x7f139c09e0c0 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient(hunting): my global_id is 195036 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f139c09cc00 con 0x7f139c09e0c0 2024-09-15T04:00:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 800310279 0 0) 0x7f1388003570 con 0x7f139c09dcf0 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 >> v1:172.21.15.26:6792/0 conn(0x7f139c09e0c0 legacy=0x7f139c138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 >> v1:172.21.15.122:6790/0 conn(0x7f139c063ac0 legacy=0x7f139c09d5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 monclient: found mon.noname-c 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c09c2f0 con 0x7f139c09dcf0 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 5 monclient: authenticate success, global_id 204781 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f13840017b0 con 0x7f139c09dcf0 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: dump: 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.881 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f13840019d0 con 0x7f139c09dcf0 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 4 set_mon_vals no callback set 2024-09-15T04:00:39.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 1 -- 172.21.15.26:0/3493347313 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f13840048f0 con 0x7f139c09dcf0 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139a7fc700 10 monclient: dump: 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.883 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: shutdown 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- 172.21.15.26:0/3493347313 >> v1:172.21.15.26:6791/0 conn(0x7f139c09dcf0 legacy=0x7f139c1356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- 172.21.15.26:0/3493347313 shutdown_connections 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- 172.21.15.26:0/3493347313 >> 172.21.15.26:0/3493347313 conn(0x7f139c0a4410 msgr2=0x7f139c0a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:39.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- 172.21.15.26:0/3493347313 shutdown_connections 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 1 -- 172.21.15.26:0/3493347313 wait complete. 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: build_initial_monmap 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.862+0000 7f139bfff700 10 monclient: monmap: 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:39.885 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 Processor -- start 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- start start 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: init 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: _add_conns ranks=[7,2,8,6,4,1,3,0,5] 2024-09-15T04:00:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient(hunting): picked mon.noname-i con 0x7f139c063ac0 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient(hunting): picked mon.noname-b con 0x7f139c09dcf0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient(hunting): picked mon.noname-e con 0x7f139c09e0c0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c06e0d0 con 0x7f139c09dcf0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c122de0 con 0x7f139c063ac0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c09ae50 con 0x7f139c09e0c0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient(hunting): authenticate will time out at 1784.297269s 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13927fc700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7f139c09dcf0 legacy=0x7f139c139a20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139affd700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7f139c063ac0 legacy=0x7f139c141ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139b7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f139c09e0c0 0x7f139c149e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:34406/0 (socket says 172.21.15.26:34406) 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139b7fe700 1 -- 172.21.15.26:0/3006571772 learned_addr learned my addr 172.21.15.26:0/3006571772 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990418188 0 0) 0x7f139c09ae50 con 0x7f139c09e0c0 2024-09-15T04:00:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): my global_id is 135584 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f137c003570 con 0x7f139c09e0c0 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3079812744 0 0) 0x7f137c003570 con 0x7f139c09e0c0 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f139c09ae50 con 0x7f139c09e0c0 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1394001e10 con 0x7f139c09e0c0 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): dump: 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.888 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:39.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1919025589 0 0) 0x7f139c09ae50 con 0x7f139c09e0c0 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.122:6792/0 conn(0x7f139c063ac0 legacy=0x7f139c141ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.26:6790/0 conn(0x7f139c09dcf0 legacy=0x7f139c139a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: _finish_hunting 0 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 monclient: found mon.e 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c122b00 con 0x7f139c09e0c0 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: _finish_auth 0 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 5 monclient: authenticate success, global_id 135584 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: _renew_subs 2024-09-15T04:00:39.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c1230c0 con 0x7f139c09e0c0 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: _renew_subs 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c13a130 con 0x7f139c09e0c0 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f13940023e0 con 0x7f139c09e0c0 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1394002910 con 0x7f139c09e0c0 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f1399ffb700 4 set_mon_vals no callback set 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f13937fe700 10 monclient: dump: 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.891 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.892 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.866+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1380006590 con 0x7f1380002840 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337123897 0 0) 0x7f1380006590 con 0x7f1380002840 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient(hunting): my global_id is 204790 2024-09-15T04:00:39.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f139c09ae50 con 0x7f1380002840 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1394002af0 con 0x7f139c09e0c0 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7f1394044350 con 0x7f139c09e0c0 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4244781150 0 0) 0x7f139c09ae50 con 0x7f1380002840 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1380006590 con 0x7f1380002840 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f138c004090 con 0x7f1380002840 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient: dump: 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.894 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13927fc700 10 monclient: get_auth_request con 0x7f137c0373b0 auth_method 0 2024-09-15T04:00:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2693901891 0 0) 0x7f1380006590 con 0x7f1380002840 2024-09-15T04:00:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f138000a130 con 0x7f1380002840 2024-09-15T04:00:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f138c0063d0 con 0x7f1380002840 2024-09-15T04:00:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.26:6791/0 conn(0x7f1380002840 legacy=0x7f1380004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.870+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.26:6791/0 conn(0x7f1380002840 legacy=0x7f1380004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:39.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1380007520 con 0x7f13800068d0 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237015365 0 0) 0x7f1380007520 con 0x7f13800068d0 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient(hunting): my global_id is 204799 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1380006590 con 0x7f13800068d0 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3809915428 0 0) 0x7f1380006590 con 0x7f13800068d0 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1380007520 con 0x7f13800068d0 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f138c004be0 con 0x7f13800068d0 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient: dump: 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4013180627 0 0) 0x7f1380007520 con 0x7f13800068d0 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1380008760 con 0x7f13800068d0 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7355 (unknown 0 0 2646046208) 0x7f138c0093c0 con 0x7f13800068d0 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.26:6791/0 conn(0x7f13800068d0 legacy=0x7f1380006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f13937fe700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.26:6791/0 conn(0x7f13800068d0 legacy=0x7f1380006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 108, 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:39.901 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:39.902 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:39.903 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:39.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:39.905 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.907 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:39.908 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:39.909 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:39.910 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.911 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.914 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:39.915 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:39.916 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.917 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:39.918 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.919 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:39.920 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.122:6812/20895 conn(0x7f137c0373b0 legacy=0x7f137c039860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139bfff700 10 monclient: shutdown 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 >> v1:172.21.15.26:6793/0 conn(0x7f139c09e0c0 legacy=0x7f139c149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139b7fe700 1 -- 172.21.15.26:0/3006571772 reap_dead start 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.882+0000 7f139b7fe700 1 -- 172.21.15.26:0/3006571772 reap_dead start 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.886+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 shutdown_connections 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.886+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 >> 172.21.15.26:0/3006571772 conn(0x7f139c0a4410 msgr2=0x7f139c0a08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.886+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 shutdown_connections 2024-09-15T04:00:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:39.886+0000 7f139bfff700 1 -- 172.21.15.26:0/3006571772 wait complete. 2024-09-15T04:00:39.923 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T04:00:39.923 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-09-15T04:00:39.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:00:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient: get_monmap_and_config 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient: build_initial_monmap 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient: monmap: 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:40.040194+0000 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:40.040194+0000 2024-09-15T04:00:40.044 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:40.045 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 1 Processor -- start 2024-09-15T04:00:40.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 1 -- start start 2024-09-15T04:00:40.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient: init 2024-09-15T04:00:40.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:40.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient: _add_conns ranks=[5,2,1,0,8,4,3,7,6] 2024-09-15T04:00:40.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient(hunting): picked mon.noname-h con 0x7fdd0c0990c0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient(hunting): picked mon.noname-b con 0x7fdd0c0a1310 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient(hunting): picked mon.noname-f con 0x7fdd0c1335a0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c04ca10 con 0x7fdd0c1335a0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c0a1b20 con 0x7fdd0c0a1310 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c0a10b0 con 0x7fdd0c0990c0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 10 monclient(hunting): authenticate will time out at 1784.470300s 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0b7fe700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fdd0c0a1310 legacy=0x7fdd0c099490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0bfff700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fdd0c0990c0 legacy=0x7fdd0c132e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd10a01700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fdd0c1335a0 0x7fdd0c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:42526/0 (socket says 172.21.15.26:42526) 2024-09-15T04:00:40.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd10a01700 1 -- 172.21.15.26:0/1551604559 learned_addr learned my addr 172.21.15.26:0/1551604559 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434513492 0 0) 0x7fdd0c04ca10 con 0x7fdd0c1335a0 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient(hunting): my global_id is 204787 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf8003570 con 0x7fdd0c1335a0 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2886418209 0 0) 0x7fdcf8003570 con 0x7fdd0c1335a0 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 >> v1:172.21.15.122:6791/0 conn(0x7fdd0c0990c0 legacy=0x7fdd0c132e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 >> v1:172.21.15.26:6790/0 conn(0x7fdd0c0a1310 legacy=0x7fdd0c099490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 monclient: found mon.noname-f 2024-09-15T04:00:40.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.122:6789/0 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c099ba0 con 0x7fdd0c1335a0 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient: _finish_auth 0 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd04001b30 con 0x7fdd0c1335a0 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd12464700 5 monclient: authenticate success, global_id 204787 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.038+0000 7fdd0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 10 monclient: dump: 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.052 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.053 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 <== mon.1 v1:172.21.15.122:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdd04001d50 con 0x7fdd0c1335a0 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 1 -- 172.21.15.26:0/1551604559 <== mon.1 v1:172.21.15.122:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdd04004fb0 con 0x7fdd0c1335a0 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 4 set_mon_vals no callback set 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0affd700 10 monclient: dump: 2024-09-15T04:00:40.054 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.055 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: shutdown 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- 172.21.15.26:0/1551604559 >> v1:172.21.15.122:6789/0 conn(0x7fdd0c1335a0 legacy=0x7fdd0c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- 172.21.15.26:0/1551604559 shutdown_connections 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- 172.21.15.26:0/1551604559 >> 172.21.15.26:0/1551604559 conn(0x7fdd0c06e370 msgr2=0x7fdd0c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- 172.21.15.26:0/1551604559 shutdown_connections 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- 172.21.15.26:0/1551604559 wait complete. 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: build_initial_monmap 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: monmap: 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:40.056 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:40.057 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 Processor -- start 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- start start 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: init 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient: _add_conns ranks=[1,8,3,7,5,6,2,0,4] 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient(hunting): picked mon.noname-f con 0x7fdd0c0990c0 addr v1:172.21.15.122:6789/0 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient(hunting): picked mon.noname-e con 0x7fdd0c0a1310 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient(hunting): picked mon.noname-g con 0x7fdd0c1335a0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c142850 con 0x7fdd0c0990c0 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c1302c0 con 0x7fdd0c1335a0 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0c09a010 con 0x7fdd0c0a1310 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd12464700 10 monclient(hunting): authenticate will time out at 1784.475693s 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd10a01700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fdd0c1335a0 legacy=0x7fdd0c149d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0b7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fdd0c0a1310 0x7fdd0c132090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:34416/0 (socket says 172.21.15.26:34416) 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.042+0000 7fdd0b7fe700 1 -- 172.21.15.26:0/882226868 learned_addr learned my addr 172.21.15.26:0/882226868 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896527284 0 0) 0x7fdd0c09a010 con 0x7fdd0c0a1310 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): my global_id is 135593 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcec003570 con 0x7fdd0c0a1310 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023331089 0 0) 0x7fdd0c142850 con 0x7fdd0c0990c0 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): my global_id is 204796 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd0c09a010 con 0x7fdd0c0990c0 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2656489640 0 0) 0x7fdcec003570 con 0x7fdd0c0a1310 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd0c142850 con 0x7fdd0c0a1310 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdcfc004350 con 0x7fdd0c0a1310 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): dump: 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:40.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 438918795 0 0) 0x7fdd0c09a010 con 0x7fdd0c0990c0 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcec003570 con 0x7fdd0c0990c0 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdcf4004340 con 0x7fdd0c0990c0 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): dump: 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.067 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2626703279 0 0) 0x7fdd0c142850 con 0x7fdd0c0a1310 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.122:6790/0 conn(0x7fdd0c1335a0 legacy=0x7fdd0c149d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:40.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.122:6789/0 conn(0x7fdd0c0990c0 legacy=0x7fdd0c1419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: _finish_hunting 0 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 monclient: found mon.e 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c12f360 con 0x7fdd0c0a1310 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: _finish_auth 0 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 5 monclient: authenticate success, global_id 135593 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 10 monclient: _renew_subs 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0c1305a0 con 0x7fdd0c0a1310 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 10 monclient: _renew_subs 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:00:40.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c142260 con 0x7fdd0c0a1310 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdcfc004630 con 0x7fdd0c0a1310 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdcfc004b60 con 0x7fdd0c0a1310 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd0a7fc700 4 set_mon_vals no callback set 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: dump: 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.070 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdcf0006590 con 0x7fdcf0002840 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fdcfc004d40 con 0x7fdd0c0a1310 2024-09-15T04:00:40.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7fdcfc043f00 con 0x7fdd0c0a1310 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977726942 0 0) 0x7fdcf0006590 con 0x7fdcf0002840 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient(hunting): my global_id is 195045 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd0c142850 con 0x7fdcf0002840 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd10a01700 10 monclient: get_auth_request con 0x7fdcec037ba0 auth_method 0 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1859533224 0 0) 0x7fdd0c142850 con 0x7fdcf0002840 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcf0006590 con 0x7fdcf0002840 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdcf4004b50 con 0x7fdcf0002840 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 10 monclient: dump: 2024-09-15T04:00:40.072 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.073 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2048008460 0 0) 0x7fdcf0006590 con 0x7fdcf0002840 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.046+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdcf000a130 con 0x7fdcf0002840 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.050+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fdcf4005150 con 0x7fdcf0002840 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.050+0000 7fdd03fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.050+0000 7fdd03fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.050+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.26:6792/0 conn(0x7fdcf0002840 legacy=0x7fdcf0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.050+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.26:6792/0 conn(0x7fdcf0002840 legacy=0x7fdcf0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:40.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd12464700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fdcf0007520 con 0x7fdcf00068d0 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399360944 0 0) 0x7fdcf0007520 con 0x7fdcf00068d0 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 10 monclient(hunting): my global_id is 195054 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf0006590 con 0x7fdcf00068d0 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2537145263 0 0) 0x7fdcf0006590 con 0x7fdcf00068d0 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcf0007520 con 0x7fdcf00068d0 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdcf4005a00 con 0x7fdcf00068d0 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.058+0000 7fdd03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 10 monclient: dump: 2024-09-15T04:00:40.076 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.077 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1231952250 0 0) 0x7fdcf0007520 con 0x7fdcf00068d0 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdcf0008760 con 0x7fdcf00068d0 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7525 (unknown 0 0 2881690683) 0x7fdcf4005d70 con 0x7fdcf00068d0 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.26:6792/0 conn(0x7fdcf00068d0 legacy=0x7fdcf0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd03fff700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.26:6792/0 conn(0x7fdcf00068d0 legacy=0x7fdcf0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:40.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 108, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:40.079 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.080 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.081 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:40.082 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.083 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.084 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.085 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.086 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.087 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.088 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.089 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.090 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:40.091 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:40.092 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:40.093 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.094 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.095 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:40.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.097 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:40.098 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.122:6812/20895 conn(0x7fdcec037ba0 legacy=0x7fdcec03a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 10 monclient: shutdown 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 >> v1:172.21.15.26:6793/0 conn(0x7fdd0c0a1310 legacy=0x7fdd0c132090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd10a01700 1 -- 172.21.15.26:0/882226868 reap_dead start 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd10a01700 1 -- 172.21.15.26:0/882226868 reap_dead start 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 shutdown_connections 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 >> 172.21.15.26:0/882226868 conn(0x7fdd0c06e370 msgr2=0x7fdd0c0a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 shutdown_connections 2024-09-15T04:00:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.062+0000 7fdd12464700 1 -- 172.21.15.26:0/882226868 wait complete. 2024-09-15T04:00:40.101 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T04:00:40.101 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon e 2024-09-15T04:00:40.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T04:00:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient: get_monmap_and_config 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient: build_initial_monmap 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient: monmap: 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:40.216488+0000 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:40.216488+0000 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:40.221 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 1 Processor -- start 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 1 -- start start 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient: init 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient: _add_conns ranks=[5,0,6,1,2,7,8,3,4] 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient(hunting): picked mon.noname-h con 0x7fab90132b40 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient(hunting): picked mon.noname-a con 0x7fab9009faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient(hunting): picked mon.noname-d con 0x7fab90138d40 addr v1:172.21.15.26:6792/0 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9004ca10 con 0x7fab9009faa0 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9009d5d0 con 0x7fab90132b40 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab90133620 con 0x7fab90138d40 2024-09-15T04:00:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 10 monclient(hunting): authenticate will time out at 1784.646894s 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8f7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fab90138d40 0x7fab9013b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:38316/0 (socket says 172.21.15.26:38316) 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8f7fe700 1 -- 172.21.15.26:0/3838604773 learned_addr learned my addr 172.21.15.26:0/3838604773 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8e7fc700 1 --1- 172.21.15.26:0/3838604773 >> v1:172.21.15.26:6789/0 conn(0x7fab9009faa0 0x7fab9009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38318/0 (socket says 172.21.15.26:38318) 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8effd700 1 -- 172.21.15.26:0/3838604773 >> v1:172.21.15.122:6791/0 conn(0x7fab90132b40 legacy=0x7fab90132f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533907840 0 0) 0x7fab90133620 con 0x7fab90138d40 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient(hunting): my global_id is 195063 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab78003570 con 0x7fab90138d40 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004857585 0 0) 0x7fab9004ca10 con 0x7fab9009faa0 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient(hunting): my global_id is 194931 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab90133620 con 0x7fab9009faa0 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4031079709 0 0) 0x7fab78003570 con 0x7fab90138d40 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 >> v1:172.21.15.122:6791/0 conn(0x7fab90132b40 legacy=0x7fab90132f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 >> v1:172.21.15.26:6789/0 conn(0x7fab9009faa0 legacy=0x7fab9009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient: _finish_hunting 0 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 monclient: found mon.noname-d 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900a0cc0 con 0x7fab90138d40 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient: _finish_auth 0 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab88001b30 con 0x7fab90138d40 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 5 monclient: authenticate success, global_id 195063 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8dffb700 10 monclient: dump: 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.214+0000 7fab8ffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fab88001da0 con 0x7fab90138d40 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:40.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 4 set_mon_vals no callback set 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8dffb700 1 -- 172.21.15.26:0/3838604773 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab88004fb0 con 0x7fab90138d40 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8dffb700 10 monclient: dump: 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: shutdown 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- 172.21.15.26:0/3838604773 >> v1:172.21.15.26:6792/0 conn(0x7fab90138d40 legacy=0x7fab9013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- 172.21.15.26:0/3838604773 shutdown_connections 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- 172.21.15.26:0/3838604773 >> 172.21.15.26:0/3838604773 conn(0x7fab9006e370 msgr2=0x7fab900a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- 172.21.15.26:0/3838604773 shutdown_connections 2024-09-15T04:00:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- 172.21.15.26:0/3838604773 wait complete. 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: build_initial_monmap 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: monmap: 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 Processor -- start 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- start start 2024-09-15T04:00:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: init 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient: _add_conns ranks=[8,4,0,2,1,5,3,7,6] 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient(hunting): picked mon.noname-e con 0x7fab9009faa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient(hunting): picked mon.noname-c con 0x7fab90132b40 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient(hunting): picked mon.noname-a con 0x7fab90138d40 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9009cfa0 con 0x7fab90138d40 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab901374b0 con 0x7fab90132b40 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9011c980 con 0x7fab9009faa0 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8effd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fab9009faa0 0x7fab901354a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:34430/0 (socket says 172.21.15.26:34430) 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8effd700 1 -- 172.21.15.26:0/4279548646 learned_addr learned my addr 172.21.15.26:0/4279548646 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab8ffff700 10 monclient(hunting): authenticate will time out at 1784.651347s 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679127051 0 0) 0x7fab901374b0 con 0x7fab90132b40 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): my global_id is 204808 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab7c003570 con 0x7fab90132b40 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730591755 0 0) 0x7fab9009cfa0 con 0x7fab90138d40 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): my global_id is 194940 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab901374b0 con 0x7fab90138d40 2024-09-15T04:00:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242467372 0 0) 0x7fab9011c980 con 0x7fab9009faa0 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 10 monclient(hunting): my global_id is 135602 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.218+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9009cfa0 con 0x7fab9009faa0 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2115230656 0 0) 0x7fab901374b0 con 0x7fab90138d40 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab9011c980 con 0x7fab90138d40 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab88001b30 con 0x7fab90138d40 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): dump: 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.245 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 706818762 0 0) 0x7fab7c003570 con 0x7fab90132b40 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab901374b0 con 0x7fab90132b40 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab84001a60 con 0x7fab90132b40 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): dump: 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4207199807 0 0) 0x7fab9009cfa0 con 0x7fab9009faa0 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab7c003570 con 0x7fab9009faa0 2024-09-15T04:00:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab80004280 con 0x7fab9009faa0 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): dump: 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3537807834 0 0) 0x7fab9011c980 con 0x7fab90138d40 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6793/0 conn(0x7fab9009faa0 legacy=0x7fab901354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6791/0 conn(0x7fab90132b40 legacy=0x7fab90135bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 monclient: found mon.a 2024-09-15T04:00:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab90138a60 con 0x7fab90138d40 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: _finish_auth 0 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 5 monclient: authenticate success, global_id 194940 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 10 monclient: _renew_subs 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9009cc40 con 0x7fab90138d40 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 10 monclient: _renew_subs 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901324b0 con 0x7fab90138d40 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fab880015b0 con 0x7fab90138d40 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:40.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab88005520 con 0x7fab90138d40 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: dump: 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.251 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8d7fa700 4 set_mon_vals no callback set 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fab70006590 con 0x7fab70002840 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669025209 0 0) 0x7fab70006590 con 0x7fab70002840 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient(hunting): my global_id is 135611 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9011c980 con 0x7fab70002840 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fab8800fee0 con 0x7fab90138d40 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(731..731 src has 435..731) v4 ==== 3927+0+0 (unknown 722504526 0 0) 0x7fab88044380 con 0x7fab90138d40 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 501144325 0 0) 0x7fab9011c980 con 0x7fab70002840 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab70006590 con 0x7fab70002840 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab8e7fc700 10 monclient: get_auth_request con 0x7fab7c038130 auth_method 0 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab80004950 con 0x7fab70002840 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.222+0000 7fab76ffd700 10 monclient: dump: 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.253 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.254 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3724954654 0 0) 0x7fab70006590 con 0x7fab70002840 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab7000a130 con 0x7fab70002840 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fab80004f00 con 0x7fab70002840 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6793/0 conn(0x7fab70002840 legacy=0x7fab70004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.226+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6793/0 conn(0x7fab70002840 legacy=0x7fab70004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:40.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fab70007520 con 0x7fab700068d0 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775925710 0 0) 0x7fab70007520 con 0x7fab700068d0 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient(hunting): my global_id is 135620 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab70006590 con 0x7fab700068d0 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1185162449 0 0) 0x7fab70006590 con 0x7fab700068d0 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab70007520 con 0x7fab700068d0 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fab80005730 con 0x7fab700068d0 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient: dump: 2024-09-15T04:00:40.257 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.258 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2334870512 0 0) 0x7fab70007520 con 0x7fab700068d0 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fab70008760 con 0x7fab700068d0 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7527 (unknown 0 0 1573249391) 0x7fab80005920 con 0x7fab700068d0 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6793/0 conn(0x7fab700068d0 legacy=0x7fab70006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab76ffd700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6793/0 conn(0x7fab700068d0 legacy=0x7fab70006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:40.259 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 108, 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:40.260 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:40.261 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:40.262 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:40.263 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:40.264 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.265 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.267 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:40.268 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:40.269 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.271 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.272 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:40.273 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.274 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:40.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.277 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.278 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:40.279 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:40.280 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.122:6812/20895 conn(0x7fab7c038130 legacy=0x7fab7c03a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 10 monclient: shutdown 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8f7fe700 1 -- 172.21.15.26:0/4279548646 reap_dead start 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 >> v1:172.21.15.26:6789/0 conn(0x7fab90138d40 legacy=0x7fab901362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 shutdown_connections 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 >> 172.21.15.26:0/4279548646 conn(0x7fab9006e370 msgr2=0x7fab9009da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:40.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 shutdown_connections 2024-09-15T04:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.234+0000 7fab8ffff700 1 -- 172.21.15.26:0/4279548646 wait complete. 2024-09-15T04:00:40.282 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T04:00:40.282 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon f 2024-09-15T04:00:40.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T04:00:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient: get_monmap_and_config 2024-09-15T04:00:40.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient: build_initial_monmap 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient: monmap: 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:00:40.393332+0000 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:00:40.393332+0000 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:40.397 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:40.398 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:40.398 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 1 Processor -- start 2024-09-15T04:00:40.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 1 -- start start 2024-09-15T04:00:40.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient: init 2024-09-15T04:00:40.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:40.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient: _add_conns ranks=[5,0,8,6,7,1,2,3,4] 2024-09-15T04:00:40.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient(hunting): picked mon.noname-h con 0x7fa0f0070de0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient(hunting): picked mon.noname-a con 0x7fa0f00989b0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient(hunting): picked mon.noname-e con 0x7fa0f0098dc0 addr v1:172.21.15.26:6793/0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f004ca10 con 0x7fa0f00989b0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f00964e0 con 0x7fa0f0070de0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f006e780 con 0x7fa0f0098dc0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ef59e700 10 monclient(hunting): authenticate will time out at 1784.823013s 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ee59c700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f009de90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0edd9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0f00989b0 0x7fa0f0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38326/0 (socket says 172.21.15.26:38326) 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0eed9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fa0f0098dc0 0x7fa0f013b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:34466/0 (socket says 172.21.15.26:34466) 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0edd9b700 1 -- 172.21.15.26:0/2236022081 learned_addr learned my addr 172.21.15.26:0/2236022081 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871590880 0 0) 0x7fa0f006e780 con 0x7fa0f0098dc0 2024-09-15T04:00:40.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient(hunting): my global_id is 135629 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d8003570 con 0x7fa0f0098dc0 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420133720 0 0) 0x7fa0f004ca10 con 0x7fa0f00989b0 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient(hunting): my global_id is 194949 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f006e780 con 0x7fa0f00989b0 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 641820230 0 0) 0x7fa0d8003570 con 0x7fa0f0098dc0 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f009de90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 >> v1:172.21.15.26:6789/0 conn(0x7fa0f00989b0 legacy=0x7fa0f0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient: _finish_hunting 0 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 monclient: found mon.noname-e 2024-09-15T04:00:40.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f009e5a0 con 0x7fa0f0098dc0 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient: _finish_auth 0 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e8001b30 con 0x7fa0f0098dc0 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.390+0000 7fa0ed59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 10 monclient: dump: 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.404 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 5 monclient: authenticate success, global_id 135629 2024-09-15T04:00:40.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa0e8001da0 con 0x7fa0f0098dc0 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 1 -- 172.21.15.26:0/2236022081 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e8004ef0 con 0x7fa0f0098dc0 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: get_monmap_and_config success 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 4 set_mon_vals no callback set 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ed59a700 10 monclient: dump: 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:40.406 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: shutdown 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:40.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2236022081 >> v1:172.21.15.26:6793/0 conn(0x7fa0f0098dc0 legacy=0x7fa0f013b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2236022081 shutdown_connections 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2236022081 >> 172.21.15.26:0/2236022081 conn(0x7fa0f00a2ea0 msgr2=0x7fa0f009eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2236022081 shutdown_connections 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2236022081 wait complete. 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: build_initial_monmap 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: monmap: 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:00:40.408 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:00:40.409 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 Processor -- start 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- start start 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: init 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: _reopen_session rank -1 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient: _add_conns ranks=[5,3,7,2,4,0,1,8,6] 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient(hunting): picked mon.noname-h con 0x7fa0f0070de0 addr v1:172.21.15.122:6791/0 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient(hunting): picked mon.noname-g con 0x7fa0f00989b0 addr v1:172.21.15.122:6790/0 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient(hunting): picked mon.noname-i con 0x7fa0f0098dc0 addr v1:172.21.15.122:6792/0 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f009fa30 con 0x7fa0f00989b0 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f012e7c0 con 0x7fa0f0070de0 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f009f800 con 0x7fa0f0098dc0 2024-09-15T04:00:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ef59e700 10 monclient(hunting): authenticate will time out at 1784.827900s 2024-09-15T04:00:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0ee59c700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f0141d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0edd9b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fa0f00989b0 legacy=0x7fa0f0147020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:40.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.394+0000 7fa0eed9d700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa0f0098dc0 legacy=0x7fa0f014a750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:40.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.598+0000 7fa0ee59c700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f0141d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:40.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.602+0000 7fa0edd9b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fa0f00989b0 legacy=0x7fa0f0147020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:40.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:40.602+0000 7fa0eed9d700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa0f0098dc0 legacy=0x7fa0f014a750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:41.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:41.002+0000 7fa0ee59c700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f0141d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:41.006+0000 7fa0edd9b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fa0f00989b0 legacy=0x7fa0f0147020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:41.006+0000 7fa0eed9d700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa0f0098dc0 legacy=0x7fa0f014a750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:41.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:41.806+0000 7fa0ee59c700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f0141d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:00:41.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:41.810+0000 7fa0eed9d700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa0f0098dc0 legacy=0x7fa0f014a750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6792/0 2024-09-15T04:00:41.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:41.810+0000 7fa0edd9b700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fa0f00989b0 legacy=0x7fa0f0147020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6790/0 2024-09-15T04:00:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 10 monclient(hunting): tick 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 1 monclient(hunting): continuing hunt 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 1 -- >> v1:172.21.15.122:6792/0 conn(0x7fa0f0098dc0 legacy=0x7fa0f014a750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fa0f0070de0 legacy=0x7fa0f0141d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 1 -- >> v1:172.21.15.122:6790/0 conn(0x7fa0f00989b0 legacy=0x7fa0f0147020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.394+0000 7fa0dd7fa700 10 monclient: _add_conns ranks=[0,4,2,6,1,8] 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 10 monclient(hunting): picked mon.noname-a con 0x7fa0d40037d0 addr v1:172.21.15.26:6789/0 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 10 monclient(hunting): picked mon.noname-c con 0x7fa0d4007450 addr v1:172.21.15.26:6791/0 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 10 monclient(hunting): picked mon.noname-b con 0x7fa0d400b1d0 addr v1:172.21.15.26:6790/0 2024-09-15T04:00:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f012ebd0 con 0x7fa0d40037d0 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f009fa30 con 0x7fa0d400b1d0 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f012e7c0 con 0x7fa0d4007450 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dd7fa700 10 monclient(hunting): _renew_subs 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0eed9d700 1 -- >> v1:172.21.15.26:6790/0 conn(0x7fa0d400b1d0 legacy=0x7fa0d400d680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6790/0 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0edd9b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fa0d4007450 0x7fa0d4009900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:34878/0 (socket says 172.21.15.26:34878) 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ee59c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0d40037d0 0x7fa0d4005c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38338/0 (socket says 172.21.15.26:38338) 2024-09-15T04:00:43.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0edd9b700 1 -- 172.21.15.26:0/2991789322 learned_addr learned my addr 172.21.15.26:0/2991789322 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:00:43.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250034932 0 0) 0x7fa0f012e7c0 con 0x7fa0d4007450 2024-09-15T04:00:43.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:43.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:43.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): my global_id is 204817 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d0003570 con 0x7fa0d4007450 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530387735 0 0) 0x7fa0f012ebd0 con 0x7fa0d40037d0 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): my global_id is 194958 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f012e7c0 con 0x7fa0d40037d0 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 78434976 0 0) 0x7fa0f012e7c0 con 0x7fa0d40037d0 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0f012ebd0 con 0x7fa0d40037d0 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e00044b0 con 0x7fa0d40037d0 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:00:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2956337743 0 0) 0x7fa0d0003570 con 0x7fa0d4007450 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0f012e7c0 con 0x7fa0d4007450 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e4001b40 con 0x7fa0d4007450 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient(hunting): dump: 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1130916401 0 0) 0x7fa0f012ebd0 con 0x7fa0d40037d0 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.26:6791/0 conn(0x7fa0d4007450 legacy=0x7fa0d4009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.26:6790/0 conn(0x7fa0d400b1d0 legacy=0x7fa0d400d680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 monclient: found mon.a 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0eed9d700 1 -- 172.21.15.26:0/2991789322 reap_dead start 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f009f800 con 0x7fa0d40037d0 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: _finish_auth 0 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 5 monclient: authenticate success, global_id 194958 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 10 monclient: _renew_subs 2024-09-15T04:00:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f012e000 con 0x7fa0d40037d0 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 10 monclient: _renew_subs 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f0130ec0 con 0x7fa0d40037d0 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa0e00049d0 con 0x7fa0d40037d0 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e0004f00 con 0x7fa0d40037d0 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0dffff700 4 set_mon_vals no callback set 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.398+0000 7fa0de7fc700 10 monclient: dump: 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0ef59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa0c8006590 con 0x7fa0c8002840 2024-09-15T04:00:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa0e0005060 con 0x7fa0d40037d0 2024-09-15T04:00:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(732..732 src has 435..732) v4 ==== 3927+0+0 (unknown 1882415559 0 0) 0x7fa0e0043e80 con 0x7fa0d40037d0 2024-09-15T04:00:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738124981 0 0) 0x7fa0c8006590 con 0x7fa0c8002840 2024-09-15T04:00:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 10 monclient(hunting): my global_id is 204805 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0f012ebd0 con 0x7fa0c8002840 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0eed9d700 10 monclient: get_auth_request con 0x7fa0d0037a10 auth_method 0 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 246669359 0 0) 0x7fa0f012ebd0 con 0x7fa0c8002840 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0c8006590 con 0x7fa0c8002840 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e4001d60 con 0x7fa0c8002840 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 10 monclient: dump: 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:43.419 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 964083612 0 0) 0x7fa0c8006590 con 0x7fa0c8002840 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.402+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0c800a130 con 0x7fa0c8002840 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.406+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa0e4005170 con 0x7fa0c8002840 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.406+0000 7fa0de7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.406+0000 7fa0de7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.406+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.122:6789/0 conn(0x7fa0c8002840 legacy=0x7fa0c8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.406+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.122:6789/0 conn(0x7fa0c8002840 legacy=0x7fa0c8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.414+0000 7fa0ef59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:00:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.414+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa0c8009640 con 0x7fa0c80090b0 2024-09-15T04:00:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802028343 0 0) 0x7fa0c8009640 con 0x7fa0c80090b0 2024-09-15T04:00:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:00:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:00:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient(hunting): my global_id is 204814 2024-09-15T04:00:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c8006590 con 0x7fa0c80090b0 2024-09-15T04:00:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1257603218 0 0) 0x7fa0c8006590 con 0x7fa0c80090b0 2024-09-15T04:00:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0c8009640 con 0x7fa0c80090b0 2024-09-15T04:00:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0e40058d0 con 0x7fa0c80090b0 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient: dump: 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:00:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:00:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:00:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:00:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:00:43.429 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:00:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3103126510 0 0) 0x7fa0c8009640 con 0x7fa0c80090b0 2024-09-15T04:00:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa0c80098f0 con 0x7fa0c80090b0 2024-09-15T04:00:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7525 (unknown 0 0 2656599632) 0x7fa0e4005d40 con 0x7fa0c80090b0 2024-09-15T04:00:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.122:6789/0 conn(0x7fa0c80090b0 legacy=0x7fa0c8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0de7fc700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.122:6789/0 conn(0x7fa0c80090b0 legacy=0x7fa0c8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 108, 2024-09-15T04:00:43.430 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:43.431 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:43.432 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:00:43.433 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:00:43.434 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.435 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:00:43.436 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:00:43.437 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:00:43.438 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.439 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.441 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:00:43.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.443 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:00:43.447 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.448 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:00:43.449 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:00:43.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:00:43.450 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:00:43.450 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.122:6812/20895 conn(0x7fa0d0037a10 legacy=0x7fa0d0039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 10 monclient: shutdown 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 >> v1:172.21.15.26:6789/0 conn(0x7fa0d40037d0 legacy=0x7fa0d4005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 shutdown_connections 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 >> 172.21.15.26:0/2991789322 conn(0x7fa0f00a2ea0 msgr2=0x7fa0f00a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 shutdown_connections 2024-09-15T04:00:43.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:00:43.418+0000 7fa0ef59e700 1 -- 172.21.15.26:0/2991789322 wait complete. 2024-09-15T04:00:43.452 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-09-15T04:00:43.452 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T04:00:53.739 INFO:tasks.rados.rados.0.smithi122.stdout:3246: done (0 left) 2024-09-15T04:00:53.739 INFO:tasks.rados.rados.0.smithi122.stdout:3247: snap_remove snap 324 2024-09-15T04:00:54.754 INFO:tasks.rados.rados.0.smithi122.stdout:3247: done (0 left) 2024-09-15T04:00:54.754 INFO:tasks.rados.rados.0.smithi122.stdout:3248: read oid 29 snap -1 2024-09-15T04:00:54.754 INFO:tasks.rados.rados.0.smithi122.stdout:3248: expect (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T04:00:54.755 INFO:tasks.rados.rados.0.smithi122.stdout:3249: write (excl) oid 38 current snap is 379 2024-09-15T04:00:54.755 INFO:tasks.rados.rados.0.smithi122.stdout:3249: seq_num 1150 ranges {528050=505653,1612057=524731,2614324=567403} 2024-09-15T04:00:54.758 INFO:tasks.rados.rados.0.smithi122.stdout:3249: writing smithi12223906-38 from 528050 to 1033703 tid 1 2024-09-15T04:00:54.767 INFO:tasks.rados.rados.0.smithi122.stdout:3249: writing smithi12223906-38 from 1612057 to 2136788 tid 2 2024-09-15T04:00:54.773 INFO:tasks.rados.rados.0.smithi122.stdout:3249: writing smithi12223906-38 from 2614324 to 3181727 tid 3 2024-09-15T04:00:54.773 INFO:tasks.rados.rados.0.smithi122.stdout:3250: snap_create 2024-09-15T04:00:54.773 INFO:tasks.rados.rados.0.smithi122.stdout:3248: expect (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:3249: finishing write tid 1 to smithi12223906-38 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:3249: finishing write tid 2 to smithi12223906-38 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:3249: finishing write tid 3 to smithi12223906-38 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:3249: finishing write tid 4 to smithi12223906-38 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:3249: finishing write tid 5 to smithi12223906-38 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1922 (ObjNum 1150 snap 379 seq_num 1150) dirty exists 2024-09-15T04:00:54.886 INFO:tasks.rados.rados.0.smithi122.stdout:3249: left oid 38 (ObjNum 1150 snap 379 seq_num 1150) 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3248: done (2 left) 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3249: done (1 left) 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3250: done (0 left) 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3251: read oid 30 snap -1 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3251: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3252: write oid 49 current snap is 380 2024-09-15T04:00:55.752 INFO:tasks.rados.rados.0.smithi122.stdout:3252: seq_num 1151 ranges {544857=580902,1670539=668389,3041747=303110} 2024-09-15T04:00:55.763 INFO:tasks.rados.rados.0.smithi122.stdout:3252: writing smithi12223906-49 from 544857 to 1125759 tid 1 2024-09-15T04:00:55.771 INFO:tasks.rados.rados.0.smithi122.stdout:3252: writing smithi12223906-49 from 1670539 to 2338928 tid 2 2024-09-15T04:00:55.778 INFO:tasks.rados.rados.0.smithi122.stdout:3252: writing smithi12223906-49 from 3041747 to 3344857 tid 3 2024-09-15T04:00:55.778 INFO:tasks.rados.rados.0.smithi122.stdout:3253: read oid 5 snap 354 2024-09-15T04:00:55.778 INFO:tasks.rados.rados.0.smithi122.stdout:3253: expect (ObjNum 1082 snap 354 seq_num 1082) 2024-09-15T04:00:55.778 INFO:tasks.rados.rados.0.smithi122.stdout:3254: delete oid 22 current snap is 380 2024-09-15T04:00:55.778 INFO:tasks.rados.rados.0.smithi122.stdout:3251: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:00:55.903 INFO:tasks.rados.rados.0.smithi122.stdout:3252: finishing write tid 1 to smithi12223906-49 2024-09-15T04:00:55.903 INFO:tasks.rados.rados.0.smithi122.stdout:3252: finishing write tid 2 to smithi12223906-49 2024-09-15T04:00:55.903 INFO:tasks.rados.rados.0.smithi122.stdout:3251: done (3 left) 2024-09-15T04:00:55.903 INFO:tasks.rados.rados.0.smithi122.stdout:3254: done (2 left) 2024-09-15T04:00:55.903 INFO:tasks.rados.rados.0.smithi122.stdout:3255: write oid 16 current snap is 380 2024-09-15T04:00:55.904 INFO:tasks.rados.rados.0.smithi122.stdout:3252: finishing write tid 3 to smithi12223906-49 2024-09-15T04:00:55.904 INFO:tasks.rados.rados.0.smithi122.stdout:3252: finishing write tid 4 to smithi12223906-49 2024-09-15T04:00:55.904 INFO:tasks.rados.rados.0.smithi122.stdout:3255: seq_num 1153 ranges {578471=731400,1787503=672058,3096593=281878} 2024-09-15T04:00:55.910 INFO:tasks.rados.rados.0.smithi122.stdout:3255: writing smithi12223906-16 from 578471 to 1309871 tid 1 2024-09-15T04:00:55.916 INFO:tasks.rados.rados.0.smithi122.stdout:3255: writing smithi12223906-16 from 1787503 to 2459561 tid 2 2024-09-15T04:00:55.922 INFO:tasks.rados.rados.0.smithi122.stdout:3255: writing smithi12223906-16 from 3096593 to 3378471 tid 3 2024-09-15T04:00:55.922 INFO:tasks.rados.rados.0.smithi122.stdout:3256: rollback oid 10 current snap is 380 2024-09-15T04:00:55.922 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 337 2024-09-15T04:00:55.922 INFO:tasks.rados.rados.0.smithi122.stdout:3252: finishing write tid 5 to smithi12223906-49 2024-09-15T04:00:55.922 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1089 (ObjNum 1151 snap 380 seq_num 1151) dirty exists 2024-09-15T04:00:55.922 INFO:tasks.rados.rados.0.smithi122.stdout:3252: left oid 49 (ObjNum 1151 snap 380 seq_num 1151) 2024-09-15T04:00:55.923 INFO:tasks.rados.rados.0.smithi122.stdout:3253: expect (ObjNum 1082 snap 354 seq_num 1082) 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:3255: finishing write tid 1 to smithi12223906-16 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:3255: finishing write tid 2 to smithi12223906-16 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:3255: finishing write tid 3 to smithi12223906-16 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:3255: finishing write tid 4 to smithi12223906-16 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:3255: finishing write tid 5 to smithi12223906-16 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1914 (ObjNum 1153 snap 380 seq_num 1153) dirty exists 2024-09-15T04:00:56.039 INFO:tasks.rados.rados.0.smithi122.stdout:3255: left oid 16 (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3256: finishing rollback tid 0 to smithi12223906-10 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3256: finishing rollback tid 1 to smithi12223906-10 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3256: finishing rollback tid 2 to smithi12223906-10 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 2119 (ObjNum 1014 snap 318 seq_num 1014) dirty exists 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3252: done (3 left) 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3253: done (2 left) 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3255: done (1 left) 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3256: done (0 left) 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3257: write (excl) oid 3 current snap is 380 2024-09-15T04:00:56.040 INFO:tasks.rados.rados.0.smithi122.stdout:3257: seq_num 1154 ranges {595278=406649,1729632=532069,3007663=387615} 2024-09-15T04:00:56.044 INFO:tasks.rados.rados.0.smithi122.stdout:3257: writing smithi12223906-3 from 595278 to 1001927 tid 1 2024-09-15T04:00:56.052 INFO:tasks.rados.rados.0.smithi122.stdout:3257: writing smithi12223906-3 from 1729632 to 2261701 tid 2 2024-09-15T04:00:56.059 INFO:tasks.rados.rados.0.smithi122.stdout:3257: writing smithi12223906-3 from 3007663 to 3395278 tid 3 2024-09-15T04:00:56.059 INFO:tasks.rados.rados.0.smithi122.stdout:3258: snap_remove snap 297 2024-09-15T04:00:56.734 INFO:tasks.rados.rados.0.smithi122.stdout:3258: done (1 left) 2024-09-15T04:00:56.734 INFO:tasks.rados.rados.0.smithi122.stdout:3259: write oid 50 current snap is 380 2024-09-15T04:00:56.734 INFO:tasks.rados.rados.0.smithi122.stdout:3257: finishing write tid 1 to smithi12223906-3 2024-09-15T04:00:56.734 INFO:tasks.rados.rados.0.smithi122.stdout:3259: seq_num 1155 ranges {612085=481898,1788114=675727,3035086=376999} 2024-09-15T04:00:56.741 INFO:tasks.rados.rados.0.smithi122.stdout:3259: writing smithi12223906-50 from 612085 to 1093983 tid 1 2024-09-15T04:00:56.749 INFO:tasks.rados.rados.0.smithi122.stdout:3259: writing smithi12223906-50 from 1788114 to 2463841 tid 2 2024-09-15T04:00:56.754 INFO:tasks.rados.rados.0.smithi122.stdout:3259: writing smithi12223906-50 from 3035086 to 3412085 tid 3 2024-09-15T04:00:56.755 INFO:tasks.rados.rados.0.smithi122.stdout:3260: read oid 36 snap -1 2024-09-15T04:00:56.755 INFO:tasks.rados.rados.0.smithi122.stdout:3260: expect (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:00:56.755 INFO:tasks.rados.rados.0.smithi122.stdout:3261: snap_remove snap 332 2024-09-15T04:00:57.777 INFO:tasks.rados.rados.0.smithi122.stdout:3261: done (3 left) 2024-09-15T04:00:57.777 INFO:tasks.rados.rados.0.smithi122.stdout:3262: rollback oid 9 current snap is 380 2024-09-15T04:00:57.782 INFO:tasks.rados.rados.0.smithi122.stdout:3257: finishing write tid 2 to smithi12223906-3 2024-09-15T04:00:57.782 INFO:tasks.rados.rados.0.smithi122.stdout:3257: finishing write tid 3 to smithi12223906-3 2024-09-15T04:00:57.782 INFO:tasks.rados.rados.0.smithi122.stdout:3257: finishing write tid 4 to smithi12223906-3 2024-09-15T04:00:57.782 INFO:tasks.rados.rados.0.smithi122.stdout:3257: finishing write tid 5 to smithi12223906-3 2024-09-15T04:00:57.782 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2633 (ObjNum 1154 snap 380 seq_num 1154) dirty exists 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3257: left oid 3 (ObjNum 1154 snap 380 seq_num 1154) 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3259: finishing write tid 1 to smithi12223906-50 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3259: finishing write tid 2 to smithi12223906-50 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3259: finishing write tid 3 to smithi12223906-50 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3259: finishing write tid 4 to smithi12223906-50 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3259: finishing write tid 5 to smithi12223906-50 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2597 (ObjNum 1155 snap 380 seq_num 1155) dirty exists 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3259: left oid 50 (ObjNum 1155 snap 380 seq_num 1155) 2024-09-15T04:00:57.783 INFO:tasks.rados.rados.0.smithi122.stdout:3260: expect (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:00:57.972 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 355 2024-09-15T04:00:57.972 INFO:tasks.rados.rados.0.smithi122.stdout:3257: done (3 left) 2024-09-15T04:00:57.972 INFO:tasks.rados.rados.0.smithi122.stdout:3259: done (2 left) 2024-09-15T04:00:57.972 INFO:tasks.rados.rados.0.smithi122.stdout:3260: done (1 left) 2024-09-15T04:00:57.973 INFO:tasks.rados.rados.0.smithi122.stdout:3263: snap_remove snap 288 2024-09-15T04:00:58.802 INFO:tasks.rados.rados.0.smithi122.stdout:3263: done (1 left) 2024-09-15T04:00:58.802 INFO:tasks.rados.rados.0.smithi122.stdout:3264: copy_from oid 4 from oid 36 current snap is 380 2024-09-15T04:00:58.802 INFO:tasks.rados.rados.0.smithi122.stdout:3262: finishing rollback tid 1 to smithi12223906-9 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:3262: finishing rollback tid 2 to smithi12223906-9 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2392 (ObjNum 1091 snap 355 seq_num 1091) dirty exists 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:3262: done (1 left) 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:3265: rollback oid 18 current snap is 380 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 315 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:3266: write (excl) oid 15 current snap is 380 2024-09-15T04:00:58.803 INFO:tasks.rados.rados.0.smithi122.stdout:3266: seq_num 1156 ranges {628892=673500,1962949=535738,3178862=250030} 2024-09-15T04:00:58.810 INFO:tasks.rados.rados.0.smithi122.stdout:3266: writing smithi12223906-15 from 628892 to 1302392 tid 1 2024-09-15T04:00:58.819 INFO:tasks.rados.rados.0.smithi122.stdout:3266: writing smithi12223906-15 from 1962949 to 2498687 tid 2 2024-09-15T04:00:58.824 INFO:tasks.rados.rados.0.smithi122.stdout:3266: writing smithi12223906-15 from 3178862 to 3428892 tid 3 2024-09-15T04:00:58.824 INFO:tasks.rados.rados.0.smithi122.stdout:3267: snap_create 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:3264: finishing copy_from racing read to smithi12223906-4 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:3264: finishing copy_from to smithi12223906-4 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2614 (ObjNum 1147 snap 377 seq_num 1147) dirty exists 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:3266: finishing write tid 1 to smithi12223906-15 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:3266: finishing write tid 2 to smithi12223906-15 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:3265: finishing rollback tid 1 to smithi12223906-18 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:3265: finishing rollback tid 2 to smithi12223906-18 2024-09-15T04:00:58.825 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2621 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T04:00:58.827 INFO:tasks.rados.rados.0.smithi122.stdout:3266: finishing write tid 3 to smithi12223906-15 2024-09-15T04:00:58.829 INFO:tasks.rados.rados.0.smithi122.stdout:3266: finishing write tid 4 to smithi12223906-15 2024-09-15T04:00:58.829 INFO:tasks.rados.rados.0.smithi122.stdout:3266: finishing write tid 5 to smithi12223906-15 2024-09-15T04:00:58.829 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1918 (ObjNum 1156 snap 380 seq_num 1156) dirty exists 2024-09-15T04:00:58.829 INFO:tasks.rados.rados.0.smithi122.stdout:3266: left oid 15 (ObjNum 1156 snap 380 seq_num 1156) 2024-09-15T04:00:59.805 INFO:tasks.rados.rados.0.smithi122.stdout:3264: done (3 left) 2024-09-15T04:00:59.805 INFO:tasks.rados.rados.0.smithi122.stdout:3265: done (2 left) 2024-09-15T04:00:59.805 INFO:tasks.rados.rados.0.smithi122.stdout:3266: done (1 left) 2024-09-15T04:00:59.805 INFO:tasks.rados.rados.0.smithi122.stdout:3267: done (0 left) 2024-09-15T04:00:59.805 INFO:tasks.rados.rados.0.smithi122.stdout:3268: write (excl) oid 28 current snap is 381 2024-09-15T04:00:59.805 INFO:tasks.rados.rados.0.smithi122.stdout:3268: seq_num 1157 ranges {645699=748749,2021431=679396,3206285=239414} 2024-09-15T04:00:59.817 INFO:tasks.rados.rados.0.smithi122.stdout:3268: writing smithi12223906-28 from 645699 to 1394448 tid 1 2024-09-15T04:00:59.826 INFO:tasks.rados.rados.0.smithi122.stdout:3268: writing smithi12223906-28 from 2021431 to 2700827 tid 2 2024-09-15T04:00:59.831 INFO:tasks.rados.rados.0.smithi122.stdout:3268: writing smithi12223906-28 from 3206285 to 3445699 tid 3 2024-09-15T04:00:59.831 INFO:tasks.rados.rados.0.smithi122.stdout:3269: delete oid 41 current snap is 381 2024-09-15T04:00:59.832 INFO:tasks.rados.rados.0.smithi122.stdout:3268: finishing write tid 1 to smithi12223906-28 2024-09-15T04:00:59.835 INFO:tasks.rados.rados.0.smithi122.stdout:3268: finishing write tid 2 to smithi12223906-28 2024-09-15T04:00:59.836 INFO:tasks.rados.rados.0.smithi122.stdout:3269: done (1 left) 2024-09-15T04:00:59.837 INFO:tasks.rados.rados.0.smithi122.stdout:3270: write oid 19 current snap is 381 2024-09-15T04:00:59.837 INFO:tasks.rados.rados.0.smithi122.stdout:3270: seq_num 1159 ranges {679313=499247,1622042=683065,2744778=701557} 2024-09-15T04:00:59.844 INFO:tasks.rados.rados.0.smithi122.stdout:3270: writing smithi12223906-19 from 679313 to 1178560 tid 1 2024-09-15T04:00:59.850 INFO:tasks.rados.rados.0.smithi122.stdout:3270: writing smithi12223906-19 from 1622042 to 2305107 tid 2 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3270: writing smithi12223906-19 from 2744778 to 3446335 tid 3 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3271: rollback oid 20 current snap is 381 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 355 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3268: finishing write tid 3 to smithi12223906-28 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3268: finishing write tid 4 to smithi12223906-28 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3268: finishing write tid 5 to smithi12223906-28 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 2407 (ObjNum 1157 snap 381 seq_num 1157) dirty exists 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3268: left oid 28 (ObjNum 1157 snap 381 seq_num 1157) 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3270: finishing write tid 1 to smithi12223906-19 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3268: done (2 left) 2024-09-15T04:00:59.858 INFO:tasks.rados.rados.0.smithi122.stdout:3272: snap_remove snap 376 2024-09-15T04:01:00.823 INFO:tasks.rados.rados.0.smithi122.stdout:3272: done (2 left) 2024-09-15T04:01:00.823 INFO:tasks.rados.rados.0.smithi122.stdout:3273: snap_remove snap 314 2024-09-15T04:01:01.735 INFO:tasks.rados.rados.0.smithi122.stdout:3270: finishing write tid 2 to smithi12223906-19 2024-09-15T04:01:01.735 INFO:tasks.rados.rados.0.smithi122.stdout:3271: finishing rollback tid 1 to smithi12223906-20 2024-09-15T04:01:01.735 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1580 (ObjNum 33 snap 0 seq_num 1842457120) dirty dne 2024-09-15T04:01:01.735 INFO:tasks.rados.rados.0.smithi122.stdout:3270: finishing write tid 3 to smithi12223906-19 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3270: finishing write tid 4 to smithi12223906-19 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3270: finishing write tid 5 to smithi12223906-19 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 2114 (ObjNum 1159 snap 381 seq_num 1159) dirty exists 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3270: left oid 19 (ObjNum 1159 snap 381 seq_num 1159) 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3270: done (2 left) 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3271: done (1 left) 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3273: done (0 left) 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3274: read oid 32 snap -1 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3274: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3275: copy_from oid 9 from oid 39 current snap is 381 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3276: write oid 40 current snap is 381 2024-09-15T04:01:01.736 INFO:tasks.rados.rados.0.smithi122.stdout:3276: seq_num 1160 ranges {696120=574496,1680524=543076,2888554=512829} 2024-09-15T04:01:01.744 INFO:tasks.rados.rados.0.smithi122.stdout:3276: writing smithi12223906-40 from 696120 to 1270616 tid 1 2024-09-15T04:01:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:3276: writing smithi12223906-40 from 1680524 to 2223600 tid 2 2024-09-15T04:01:01.758 INFO:tasks.rados.rados.0.smithi122.stdout:3276: writing smithi12223906-40 from 2888554 to 3401383 tid 3 2024-09-15T04:01:01.758 INFO:tasks.rados.rados.0.smithi122.stdout:3277: copy_from oid 47 from oid 36 current snap is 381 2024-09-15T04:01:01.758 INFO:tasks.rados.rados.0.smithi122.stdout:3274: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3275: finishing copy_from racing read to smithi12223906-9 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3275: finishing copy_from to smithi12223906-9 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2464 (ObjNum 1090 snap 355 seq_num 1090) dirty exists 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3276: finishing write tid 1 to smithi12223906-40 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3274: done (3 left) 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3275: done (2 left) 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3278: read oid 48 snap -1 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3278: expect (ObjNum 1130 snap 371 seq_num 1130) 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3279: read oid 9 snap -1 2024-09-15T04:01:01.938 INFO:tasks.rados.rados.0.smithi122.stdout:3279: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3280: read oid 26 snap -1 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3280: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3281: snap_create 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3276: finishing write tid 2 to smithi12223906-40 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3276: finishing write tid 3 to smithi12223906-40 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3276: finishing write tid 4 to smithi12223906-40 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3276: finishing write tid 5 to smithi12223906-40 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2225 (ObjNum 1160 snap 381 seq_num 1160) dirty exists 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3276: left oid 40 (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3277: finishing copy_from racing read to smithi12223906-47 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:3277: finishing copy_from to smithi12223906-47 2024-09-15T04:01:01.939 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 2614 (ObjNum 1147 snap 377 seq_num 1147) dirty exists 2024-09-15T04:01:01.947 INFO:tasks.rados.rados.0.smithi122.stdout:3278: expect (ObjNum 1130 snap 371 seq_num 1130) 2024-09-15T04:01:02.047 INFO:tasks.rados.rados.0.smithi122.stdout:3279: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:01:02.212 INFO:tasks.rados.rados.0.smithi122.stdout:3280: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:01:02.829 INFO:tasks.rados.rados.0.smithi122.stdout:3276: done (5 left) 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3277: done (4 left) 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3278: done (3 left) 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3279: done (2 left) 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3280: done (1 left) 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3281: done (0 left) 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3282: write oid 11 current snap is 382 2024-09-15T04:01:02.830 INFO:tasks.rados.rados.0.smithi122.stdout:3282: seq_num 1161 ranges {712927=649745,2139006=686734} 2024-09-15T04:01:02.840 INFO:tasks.rados.rados.0.smithi122.stdout:3282: writing smithi12223906-11 from 712927 to 1362672 tid 1 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3282: writing smithi12223906-11 from 2139006 to 2825740 tid 2 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3283: copy_from oid 4 from oid 39 current snap is 382 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3284: rollback oid 15 current snap is 382 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 347 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3285: copy_from oid 6 from oid 45 current snap is 382 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3286: read oid 44 snap 326 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3286: expect deleted 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3287: read oid 22 snap -1 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3287: expect deleted 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3288: write oid 48 current snap is 382 2024-09-15T04:01:02.850 INFO:tasks.rados.rados.0.smithi122.stdout:3288: seq_num 1162 ranges {729734=724994,2197488=430392,3227047=302687} 2024-09-15T04:01:02.858 INFO:tasks.rados.rados.0.smithi122.stdout:3288: writing smithi12223906-48 from 729734 to 1454728 tid 1 2024-09-15T04:01:02.866 INFO:tasks.rados.rados.0.smithi122.stdout:3288: writing smithi12223906-48 from 2197488 to 2627880 tid 2 2024-09-15T04:01:02.872 INFO:tasks.rados.rados.0.smithi122.stdout:3288: writing smithi12223906-48 from 3227047 to 3529734 tid 3 2024-09-15T04:01:02.872 INFO:tasks.rados.rados.0.smithi122.stdout:3289: delete oid 23 current snap is 382 2024-09-15T04:01:02.872 INFO:tasks.rados.rados.0.smithi122.stdout:3282: finishing write tid 1 to smithi12223906-11 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3282: finishing write tid 2 to smithi12223906-11 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3282: finishing write tid 3 to smithi12223906-11 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3282: finishing write tid 4 to smithi12223906-11 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1921 (ObjNum 1161 snap 382 seq_num 1161) dirty exists 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3282: left oid 11 (ObjNum 1161 snap 382 seq_num 1161) 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3284: finishing rollback tid 0 to smithi12223906-15 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3284: finishing rollback tid 1 to smithi12223906-15 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3284: finishing rollback tid 2 to smithi12223906-15 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1924 (ObjNum 1029 snap 321 seq_num 1029) dirty exists 2024-09-15T04:01:02.873 INFO:tasks.rados.rados.0.smithi122.stdout:3283: finishing copy_from racing read to smithi12223906-4 2024-09-15T04:01:02.874 INFO:tasks.rados.rados.0.smithi122.stdout:3285: finishing copy_from racing read to smithi12223906-6 2024-09-15T04:01:02.875 INFO:tasks.rados.rados.0.smithi122.stdout:3283: finishing copy_from to smithi12223906-4 2024-09-15T04:01:02.875 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2464 (ObjNum 1090 snap 355 seq_num 1090) dirty exists 2024-09-15T04:01:02.875 INFO:tasks.rados.rados.0.smithi122.stdout:3285: finishing copy_from to smithi12223906-6 2024-09-15T04:01:02.875 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2110 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:01:02.876 INFO:tasks.rados.rados.0.smithi122.stdout:3282: done (7 left) 2024-09-15T04:01:02.876 INFO:tasks.rados.rados.0.smithi122.stdout:3283: done (6 left) 2024-09-15T04:01:02.876 INFO:tasks.rados.rados.0.smithi122.stdout:3284: done (5 left) 2024-09-15T04:01:02.876 INFO:tasks.rados.rados.0.smithi122.stdout:3285: done (4 left) 2024-09-15T04:01:02.876 INFO:tasks.rados.rados.0.smithi122.stdout:3286: done (3 left) 2024-09-15T04:01:02.877 INFO:tasks.rados.rados.0.smithi122.stdout:3287: done (2 left) 2024-09-15T04:01:02.877 INFO:tasks.rados.rados.0.smithi122.stdout:3289: done (1 left) 2024-09-15T04:01:02.877 INFO:tasks.rados.rados.0.smithi122.stdout:3290: write (excl) oid 8 current snap is 382 2024-09-15T04:01:02.877 INFO:tasks.rados.rados.0.smithi122.stdout:3290: seq_num 1164 ranges {763348=591845,1914452=434061,2881893=674270} 2024-09-15T04:01:02.884 INFO:tasks.rados.rados.0.smithi122.stdout:3290: writing smithi12223906-8 from 763348 to 1355193 tid 1 2024-09-15T04:01:02.889 INFO:tasks.rados.rados.0.smithi122.stdout:3290: writing smithi12223906-8 from 1914452 to 2348513 tid 2 2024-09-15T04:01:02.897 INFO:tasks.rados.rados.0.smithi122.stdout:3290: writing smithi12223906-8 from 2881893 to 3556163 tid 3 2024-09-15T04:01:02.897 INFO:tasks.rados.rados.0.smithi122.stdout:3291: write oid 29 current snap is 382 2024-09-15T04:01:02.897 INFO:tasks.rados.rados.0.smithi122.stdout:3291: seq_num 1165 ranges {780155=667094,1972934=694072,3309316=270839} 2024-09-15T04:01:02.904 INFO:tasks.rados.rados.0.smithi122.stdout:3291: writing smithi12223906-29 from 780155 to 1447249 tid 1 2024-09-15T04:01:02.911 INFO:tasks.rados.rados.0.smithi122.stdout:3291: writing smithi12223906-29 from 1972934 to 2667006 tid 2 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3291: writing smithi12223906-29 from 3309316 to 3580155 tid 3 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3292: read oid 40 snap -1 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3292: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3293: copy_from oid 34 from oid 49 current snap is 382 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3288: finishing write tid 1 to smithi12223906-48 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3288: finishing write tid 2 to smithi12223906-48 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3288: finishing write tid 3 to smithi12223906-48 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3288: finishing write tid 4 to smithi12223906-48 2024-09-15T04:01:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3288: finishing write tid 5 to smithi12223906-48 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2123 (ObjNum 1162 snap 382 seq_num 1162) dirty exists 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3288: left oid 48 (ObjNum 1162 snap 382 seq_num 1162) 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: finishing write tid 1 to smithi12223906-8 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: finishing write tid 2 to smithi12223906-8 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: finishing write tid 3 to smithi12223906-8 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: finishing write tid 4 to smithi12223906-8 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: finishing write tid 5 to smithi12223906-8 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1926 (ObjNum 1164 snap 382 seq_num 1164) dirty exists 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: left oid 8 (ObjNum 1164 snap 382 seq_num 1164) 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3291: finishing write tid 1 to smithi12223906-29 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3288: done (4 left) 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3290: done (3 left) 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3294: write (excl) oid 23 current snap is 382 2024-09-15T04:01:02.918 INFO:tasks.rados.rados.0.smithi122.stdout:3294: seq_num 1166 ranges {796962=742343,2031416=437730,2936739=413167} 2024-09-15T04:01:02.926 INFO:tasks.rados.rados.0.smithi122.stdout:3294: writing smithi12223906-23 from 796962 to 1539305 tid 1 2024-09-15T04:01:02.931 INFO:tasks.rados.rados.0.smithi122.stdout:3294: writing smithi12223906-23 from 2031416 to 2469146 tid 2 2024-09-15T04:01:02.936 INFO:tasks.rados.rados.0.smithi122.stdout:3294: writing smithi12223906-23 from 2936739 to 3349906 tid 3 2024-09-15T04:01:02.936 INFO:tasks.rados.rados.0.smithi122.stdout:3295: snap_remove snap 379 2024-09-15T04:01:03.855 INFO:tasks.rados.rados.0.smithi122.stdout:3295: done (4 left) 2024-09-15T04:01:03.855 INFO:tasks.rados.rados.0.smithi122.stdout:3296: read oid 44 snap -1 2024-09-15T04:01:03.856 INFO:tasks.rados.rados.0.smithi122.stdout:3296: expect deleted 2024-09-15T04:01:03.857 INFO:tasks.rados.rados.0.smithi122.stdout:3297: rollback oid 10 current snap is 382 2024-09-15T04:01:03.857 INFO:tasks.rados.rados.0.smithi122.stdout:3291: finishing write tid 2 to smithi12223906-29 2024-09-15T04:01:03.857 INFO:tasks.rados.rados.0.smithi122.stdout:3292: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3291: finishing write tid 3 to smithi12223906-29 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3291: finishing write tid 4 to smithi12223906-29 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3291: finishing write tid 5 to smithi12223906-29 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 2411 (ObjNum 1165 snap 382 seq_num 1165) dirty exists 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3291: left oid 29 (ObjNum 1165 snap 382 seq_num 1165) 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3293: finishing copy_from racing read to smithi12223906-34 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3294: finishing write tid 1 to smithi12223906-23 2024-09-15T04:01:03.973 INFO:tasks.rados.rados.0.smithi122.stdout:3293: finishing copy_from to smithi12223906-34 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1089 (ObjNum 1151 snap 380 seq_num 1151) dirty exists 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:3294: finishing write tid 2 to smithi12223906-23 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:3294: finishing write tid 3 to smithi12223906-23 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:3294: finishing write tid 4 to smithi12223906-23 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:3294: finishing write tid 5 to smithi12223906-23 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2647 (ObjNum 1166 snap 382 seq_num 1166) dirty exists 2024-09-15T04:01:03.974 INFO:tasks.rados.rados.0.smithi122.stdout:3294: left oid 23 (ObjNum 1166 snap 382 seq_num 1166) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 355 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3291: done (5 left) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3292: done (4 left) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3293: done (3 left) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3294: done (2 left) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3296: done (1 left) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3298: rollback oid 21 current snap is 382 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 21 to 336 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3299: read oid 8 snap -1 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3299: expect (ObjNum 1164 snap 382 seq_num 1164) 2024-09-15T04:01:03.975 INFO:tasks.rados.rados.0.smithi122.stdout:3300: snap_create 2024-09-15T04:01:03.992 INFO:tasks.rados.rados.0.smithi122.stdout:3297: finishing rollback tid 0 to smithi12223906-10 2024-09-15T04:01:03.992 INFO:tasks.rados.rados.0.smithi122.stdout:3297: finishing rollback tid 1 to smithi12223906-10 2024-09-15T04:01:03.992 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 2125 (ObjNum 273 snap 0 seq_num 1006658512) dirty dne 2024-09-15T04:01:03.992 INFO:tasks.rados.rados.0.smithi122.stdout:3298: finishing rollback tid 0 to smithi12223906-21 2024-09-15T04:01:03.992 INFO:tasks.rados.rados.0.smithi122.stdout:3298: finishing rollback tid 1 to smithi12223906-21 2024-09-15T04:01:03.992 INFO:tasks.rados.rados.0.smithi122.stdout:3299: expect (ObjNum 1164 snap 382 seq_num 1164) 2024-09-15T04:01:04.163 INFO:tasks.rados.rados.0.smithi122.stdout:3298: finishing rollback tid 2 to smithi12223906-21 2024-09-15T04:01:04.163 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 2128 (ObjNum 1040 snap 330 seq_num 1040) dirty exists 2024-09-15T04:01:04.864 INFO:tasks.rados.rados.0.smithi122.stdout:3297: done (3 left) 2024-09-15T04:01:04.864 INFO:tasks.rados.rados.0.smithi122.stdout:3298: done (2 left) 2024-09-15T04:01:04.864 INFO:tasks.rados.rados.0.smithi122.stdout:3299: done (1 left) 2024-09-15T04:01:04.864 INFO:tasks.rados.rados.0.smithi122.stdout:3300: done (0 left) 2024-09-15T04:01:04.864 INFO:tasks.rados.rados.0.smithi122.stdout:3301: read oid 49 snap 372 2024-09-15T04:01:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:3301: expect (ObjNum 1128 snap 371 seq_num 1128) 2024-09-15T04:01:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:3302: read oid 46 snap 336 2024-09-15T04:01:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:3302: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:01:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:3303: write (excl) oid 32 current snap is 383 2024-09-15T04:01:04.865 INFO:tasks.rados.rados.0.smithi122.stdout:3303: seq_num 1167 ranges {413769=417592,1573545=697741,2847809=624439} 2024-09-15T04:01:04.876 INFO:tasks.rados.rados.0.smithi122.stdout:3303: writing smithi12223906-32 from 413769 to 831361 tid 1 2024-09-15T04:01:04.883 INFO:tasks.rados.rados.0.smithi122.stdout:3303: writing smithi12223906-32 from 1573545 to 2271286 tid 2 2024-09-15T04:01:04.890 INFO:tasks.rados.rados.0.smithi122.stdout:3303: writing smithi12223906-32 from 2847809 to 3472248 tid 3 2024-09-15T04:01:04.891 INFO:tasks.rados.rados.0.smithi122.stdout:3304: copy_from oid 6 from oid 25 current snap is 383 2024-09-15T04:01:04.891 INFO:tasks.rados.rados.0.smithi122.stdout:3302: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:01:05.087 INFO:tasks.rados.rados.0.smithi122.stdout:3301: expect (ObjNum 1128 snap 371 seq_num 1128) 2024-09-15T04:01:05.331 INFO:tasks.rados.rados.0.smithi122.stdout:3303: finishing write tid 1 to smithi12223906-32 2024-09-15T04:01:05.331 INFO:tasks.rados.rados.0.smithi122.stdout:3303: finishing write tid 2 to smithi12223906-32 2024-09-15T04:01:05.331 INFO:tasks.rados.rados.0.smithi122.stdout:3304: finishing copy_from to smithi12223906-6 2024-09-15T04:01:05.331 INFO:tasks.rados.rados.0.smithi122.stdout:3304: got expected ENOENT (src dne) 2024-09-15T04:01:05.331 INFO:tasks.rados.rados.0.smithi122.stdout:3304: finishing copy_from racing read to smithi12223906-6 2024-09-15T04:01:05.331 INFO:tasks.rados.rados.0.smithi122.stdout:3303: finishing write tid 3 to smithi12223906-32 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3301: done (3 left) 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3302: done (2 left) 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3304: done (1 left) 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3305: delete oid 50 current snap is 383 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3303: finishing write tid 4 to smithi12223906-32 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3303: finishing write tid 5 to smithi12223906-32 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 2656 (ObjNum 1167 snap 383 seq_num 1167) dirty exists 2024-09-15T04:01:05.332 INFO:tasks.rados.rados.0.smithi122.stdout:3303: left oid 32 (ObjNum 1167 snap 383 seq_num 1167) 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3303: done (1 left) 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3305: done (0 left) 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3306: read oid 14 snap 373 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3306: expect deleted 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3307: read oid 9 snap -1 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3307: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3308: write (excl) oid 49 current snap is 383 2024-09-15T04:01:05.341 INFO:tasks.rados.rados.0.smithi122.stdout:3308: seq_num 1169 ranges {447383=568090,1690509=701410,2902655=646983} 2024-09-15T04:01:05.353 INFO:tasks.rados.rados.0.smithi122.stdout:3308: writing smithi12223906-49 from 447383 to 1015473 tid 1 2024-09-15T04:01:05.361 INFO:tasks.rados.rados.0.smithi122.stdout:3308: writing smithi12223906-49 from 1690509 to 2391919 tid 2 2024-09-15T04:01:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3308: writing smithi12223906-49 from 2902655 to 3549638 tid 3 2024-09-15T04:01:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3309: read oid 43 snap -1 2024-09-15T04:01:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3309: expect deleted 2024-09-15T04:01:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3310: copy_from oid 25 from oid 42 current snap is 383 2024-09-15T04:01:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3307: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:01:05.532 INFO:tasks.rados.rados.0.smithi122.stdout:3308: finishing write tid 1 to smithi12223906-49 2024-09-15T04:01:05.532 INFO:tasks.rados.rados.0.smithi122.stdout:3308: finishing write tid 2 to smithi12223906-49 2024-09-15T04:01:05.532 INFO:tasks.rados.rados.0.smithi122.stdout:3308: finishing write tid 3 to smithi12223906-49 2024-09-15T04:01:05.532 INFO:tasks.rados.rados.0.smithi122.stdout:3308: finishing write tid 4 to smithi12223906-49 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3308: finishing write tid 5 to smithi12223906-49 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1093 (ObjNum 1169 snap 383 seq_num 1169) dirty exists 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3308: left oid 49 (ObjNum 1169 snap 383 seq_num 1169) 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3310: finishing copy_from racing read to smithi12223906-25 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3310: finishing copy_from to smithi12223906-25 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2618 (ObjNum 1148 snap 378 seq_num 1148) dirty exists 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3306: done (4 left) 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3307: done (3 left) 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3308: done (2 left) 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3309: done (1 left) 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3310: done (0 left) 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3311: read oid 24 snap -1 2024-09-15T04:01:05.533 INFO:tasks.rados.rados.0.smithi122.stdout:3311: expect (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T04:01:05.534 INFO:tasks.rados.rados.0.smithi122.stdout:3312: write oid 28 current snap is 383 2024-09-15T04:01:05.534 INFO:tasks.rados.rados.0.smithi122.stdout:3312: seq_num 1170 ranges {464190=643339,1748991=445068,2930078=574608} 2024-09-15T04:01:05.547 INFO:tasks.rados.rados.0.smithi122.stdout:3312: writing smithi12223906-28 from 464190 to 1107529 tid 1 2024-09-15T04:01:05.553 INFO:tasks.rados.rados.0.smithi122.stdout:3312: writing smithi12223906-28 from 1748991 to 2194059 tid 2 2024-09-15T04:01:05.561 INFO:tasks.rados.rados.0.smithi122.stdout:3312: writing smithi12223906-28 from 2930078 to 3504686 tid 3 2024-09-15T04:01:05.561 INFO:tasks.rados.rados.0.smithi122.stdout:3313: snap_remove snap 334 2024-09-15T04:01:05.875 INFO:tasks.rados.rados.0.smithi122.stdout:3313: done (2 left) 2024-09-15T04:01:05.875 INFO:tasks.rados.rados.0.smithi122.stdout:3314: delete oid 7 current snap is 383 2024-09-15T04:01:05.875 INFO:tasks.rados.rados.0.smithi122.stdout:3311: expect (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T04:01:05.981 INFO:tasks.rados.rados.0.smithi122.stdout:3312: finishing write tid 1 to smithi12223906-28 2024-09-15T04:01:05.981 INFO:tasks.rados.rados.0.smithi122.stdout:3312: finishing write tid 2 to smithi12223906-28 2024-09-15T04:01:05.982 INFO:tasks.rados.rados.0.smithi122.stdout:3312: finishing write tid 3 to smithi12223906-28 2024-09-15T04:01:05.982 INFO:tasks.rados.rados.0.smithi122.stdout:3312: finishing write tid 4 to smithi12223906-28 2024-09-15T04:01:05.982 INFO:tasks.rados.rados.0.smithi122.stdout:3312: finishing write tid 5 to smithi12223906-28 2024-09-15T04:01:05.982 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 2415 (ObjNum 1170 snap 383 seq_num 1170) dirty exists 2024-09-15T04:01:05.982 INFO:tasks.rados.rados.0.smithi122.stdout:3312: left oid 28 (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:01:05.987 INFO:tasks.rados.rados.0.smithi122.stdout:3311: done (2 left) 2024-09-15T04:01:05.987 INFO:tasks.rados.rados.0.smithi122.stdout:3312: done (1 left) 2024-09-15T04:01:05.988 INFO:tasks.rados.rados.0.smithi122.stdout:3314: done (0 left) 2024-09-15T04:01:05.988 INFO:tasks.rados.rados.0.smithi122.stdout:3315: rollback oid 39 current snap is 383 2024-09-15T04:01:05.988 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 39 to 366 2024-09-15T04:01:05.988 INFO:tasks.rados.rados.0.smithi122.stdout:3316: write oid 19 current snap is 383 2024-09-15T04:01:05.988 INFO:tasks.rados.rados.0.smithi122.stdout:3316: seq_num 1172 ranges {497804=510190,1465955=448737,2584924=597152} 2024-09-15T04:01:05.998 INFO:tasks.rados.rados.0.smithi122.stdout:3316: writing smithi12223906-19 from 497804 to 1007994 tid 1 2024-09-15T04:01:06.003 INFO:tasks.rados.rados.0.smithi122.stdout:3316: writing smithi12223906-19 from 1465955 to 1914692 tid 2 2024-09-15T04:01:06.011 INFO:tasks.rados.rados.0.smithi122.stdout:3316: writing smithi12223906-19 from 2584924 to 3182076 tid 3 2024-09-15T04:01:06.011 INFO:tasks.rados.rados.0.smithi122.stdout:3317: snap_remove snap 344 2024-09-15T04:01:06.011 INFO:tasks.rados.rados.0.smithi122.stdout:3315: finishing rollback tid 0 to smithi12223906-39 2024-09-15T04:01:06.011 INFO:tasks.rados.rados.0.smithi122.stdout:3315: finishing rollback tid 1 to smithi12223906-39 2024-09-15T04:01:06.011 INFO:tasks.rados.rados.0.smithi122.stdout:3315: finishing rollback tid 2 to smithi12223906-39 2024-09-15T04:01:06.011 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2704 (ObjNum 1090 snap 355 seq_num 1090) dirty exists 2024-09-15T04:01:06.012 INFO:tasks.rados.rados.0.smithi122.stdout:3316: finishing write tid 1 to smithi12223906-19 2024-09-15T04:01:06.012 INFO:tasks.rados.rados.0.smithi122.stdout:3316: finishing write tid 2 to smithi12223906-19 2024-09-15T04:01:06.739 INFO:tasks.rados.rados.0.smithi122.stdout:3315: done (2 left) 2024-09-15T04:01:06.739 INFO:tasks.rados.rados.0.smithi122.stdout:3317: done (1 left) 2024-09-15T04:01:06.739 INFO:tasks.rados.rados.0.smithi122.stdout:3318: snap_create 2024-09-15T04:01:06.740 INFO:tasks.rados.rados.0.smithi122.stdout:3316: finishing write tid 3 to smithi12223906-19 2024-09-15T04:01:06.740 INFO:tasks.rados.rados.0.smithi122.stdout:3316: finishing write tid 4 to smithi12223906-19 2024-09-15T04:01:06.741 INFO:tasks.rados.rados.0.smithi122.stdout:3316: finishing write tid 5 to smithi12223906-19 2024-09-15T04:01:06.741 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 2118 (ObjNum 1172 snap 383 seq_num 1172) dirty exists 2024-09-15T04:01:06.741 INFO:tasks.rados.rados.0.smithi122.stdout:3316: left oid 19 (ObjNum 1172 snap 383 seq_num 1172) 2024-09-15T04:01:07.750 INFO:tasks.rados.rados.0.smithi122.stdout:3316: done (1 left) 2024-09-15T04:01:07.750 INFO:tasks.rados.rados.0.smithi122.stdout:3318: done (0 left) 2024-09-15T04:01:07.750 INFO:tasks.rados.rados.0.smithi122.stdout:3319: copy_from oid 50 from oid 31 current snap is 384 2024-09-15T04:01:07.750 INFO:tasks.rados.rados.0.smithi122.stdout:3320: read oid 18 snap -1 2024-09-15T04:01:07.750 INFO:tasks.rados.rados.0.smithi122.stdout:3320: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T04:01:07.751 INFO:tasks.rados.rados.0.smithi122.stdout:3321: rollback oid 2 current snap is 384 2024-09-15T04:01:07.751 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 360 2024-09-15T04:01:07.751 INFO:tasks.rados.rados.0.smithi122.stdout:3322: rollback oid 3 current snap is 384 2024-09-15T04:01:07.751 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 366 2024-09-15T04:01:07.751 INFO:tasks.rados.rados.0.smithi122.stdout:3323: write (excl) oid 38 current snap is 384 2024-09-15T04:01:07.751 INFO:tasks.rados.rados.0.smithi122.stdout:3323: seq_num 1173 ranges {514611=585439,1524437=592395,2895994=408424} 2024-09-15T04:01:07.761 INFO:tasks.rados.rados.0.smithi122.stdout:3323: writing smithi12223906-38 from 514611 to 1100050 tid 1 2024-09-15T04:01:07.769 INFO:tasks.rados.rados.0.smithi122.stdout:3323: writing smithi12223906-38 from 1524437 to 2116832 tid 2 2024-09-15T04:01:07.776 INFO:tasks.rados.rados.0.smithi122.stdout:3323: writing smithi12223906-38 from 2895994 to 3304418 tid 3 2024-09-15T04:01:07.776 INFO:tasks.rados.rados.0.smithi122.stdout:3324: read oid 7 snap 382 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3324: expect (ObjNum 1062 snap 343 seq_num 1062) 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3325: snap_create 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3321: finishing rollback tid 0 to smithi12223906-2 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3322: finishing rollback tid 0 to smithi12223906-3 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3322: finishing rollback tid 1 to smithi12223906-3 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3322: finishing rollback tid 2 to smithi12223906-3 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2718 (ObjNum 973 snap 307 seq_num 973) dirty exists 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3321: finishing rollback tid 1 to smithi12223906-2 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3321: finishing rollback tid 2 to smithi12223906-2 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1096 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T04:01:07.777 INFO:tasks.rados.rados.0.smithi122.stdout:3320: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T04:01:07.943 INFO:tasks.rados.rados.0.smithi122.stdout:3319: finishing copy_from racing read to smithi12223906-50 2024-09-15T04:01:07.943 INFO:tasks.rados.rados.0.smithi122.stdout:3319: finishing copy_from to smithi12223906-50 2024-09-15T04:01:07.943 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 1922 (ObjNum 1146 snap 377 seq_num 1146) dirty exists 2024-09-15T04:01:07.943 INFO:tasks.rados.rados.0.smithi122.stdout:3323: finishing write tid 1 to smithi12223906-38 2024-09-15T04:01:07.943 INFO:tasks.rados.rados.0.smithi122.stdout:3323: finishing write tid 2 to smithi12223906-38 2024-09-15T04:01:07.943 INFO:tasks.rados.rados.0.smithi122.stdout:3323: finishing write tid 3 to smithi12223906-38 2024-09-15T04:01:07.944 INFO:tasks.rados.rados.0.smithi122.stdout:3323: finishing write tid 4 to smithi12223906-38 2024-09-15T04:01:07.944 INFO:tasks.rados.rados.0.smithi122.stdout:3323: finishing write tid 5 to smithi12223906-38 2024-09-15T04:01:07.944 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 1930 (ObjNum 1173 snap 384 seq_num 1173) dirty exists 2024-09-15T04:01:07.944 INFO:tasks.rados.rados.0.smithi122.stdout:3323: left oid 38 (ObjNum 1173 snap 384 seq_num 1173) 2024-09-15T04:01:07.944 INFO:tasks.rados.rados.0.smithi122.stdout:3324: expect (ObjNum 1062 snap 343 seq_num 1062) 2024-09-15T04:01:08.757 INFO:tasks.rados.rados.0.smithi122.stdout:3319: done (6 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3320: done (5 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3321: done (4 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3322: done (3 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3323: done (2 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3324: done (1 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3325: done (0 left) 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3326: copy_from oid 43 from oid 32 current snap is 385 2024-09-15T04:01:08.758 INFO:tasks.rados.rados.0.smithi122.stdout:3327: snap_remove snap 319 2024-09-15T04:01:09.767 INFO:tasks.rados.rados.0.smithi122.stdout:3326: finishing copy_from racing read to smithi12223906-43 2024-09-15T04:01:09.767 INFO:tasks.rados.rados.0.smithi122.stdout:3326: finishing copy_from to smithi12223906-43 2024-09-15T04:01:09.767 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 2656 (ObjNum 1167 snap 383 seq_num 1167) dirty exists 2024-09-15T04:01:09.767 INFO:tasks.rados.rados.0.smithi122.stdout:3326: done (1 left) 2024-09-15T04:01:09.768 INFO:tasks.rados.rados.0.smithi122.stdout:3327: done (0 left) 2024-09-15T04:01:09.768 INFO:tasks.rados.rados.0.smithi122.stdout:3328: rollback oid 17 current snap is 385 2024-09-15T04:01:09.768 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 353 2024-09-15T04:01:09.768 INFO:tasks.rados.rados.0.smithi122.stdout:3329: read oid 50 snap -1 2024-09-15T04:01:09.768 INFO:tasks.rados.rados.0.smithi122.stdout:3329: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:01:09.771 INFO:tasks.rados.rados.0.smithi122.stdout:3330: snap_remove snap 361 2024-09-15T04:01:10.771 INFO:tasks.rados.rados.0.smithi122.stdout:3330: done (2 left) 2024-09-15T04:01:10.771 INFO:tasks.rados.rados.0.smithi122.stdout:3331: read oid 44 snap -1 2024-09-15T04:01:10.771 INFO:tasks.rados.rados.0.smithi122.stdout:3331: expect deleted 2024-09-15T04:01:10.772 INFO:tasks.rados.rados.0.smithi122.stdout:3332: write (excl) oid 49 current snap is 385 2024-09-15T04:01:10.772 INFO:tasks.rados.rados.0.smithi122.stdout:3328: finishing rollback tid 1 to smithi12223906-17 2024-09-15T04:01:10.772 INFO:tasks.rados.rados.0.smithi122.stdout:3328: finishing rollback tid 2 to smithi12223906-17 2024-09-15T04:01:10.772 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1098 (ObjNum 1070 snap 348 seq_num 1070) dirty exists 2024-09-15T04:01:10.772 INFO:tasks.rados.rados.0.smithi122.stdout:3329: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:01:10.963 INFO:tasks.rados.rados.0.smithi122.stdout:3332: seq_num 1174 ranges {531418=660688,1982919=452406,3039770=691648} 2024-09-15T04:01:10.980 INFO:tasks.rados.rados.0.smithi122.stdout:3332: writing smithi12223906-49 from 531418 to 1192106 tid 1 2024-09-15T04:01:10.987 INFO:tasks.rados.rados.0.smithi122.stdout:3332: writing smithi12223906-49 from 1982919 to 2435325 tid 2 2024-09-15T04:01:10.995 INFO:tasks.rados.rados.0.smithi122.stdout:3332: writing smithi12223906-49 from 3039770 to 3731418 tid 3 2024-09-15T04:01:10.995 INFO:tasks.rados.rados.0.smithi122.stdout:3328: done (3 left) 2024-09-15T04:01:10.995 INFO:tasks.rados.rados.0.smithi122.stdout:3329: done (2 left) 2024-09-15T04:01:10.995 INFO:tasks.rados.rados.0.smithi122.stdout:3331: done (1 left) 2024-09-15T04:01:10.995 INFO:tasks.rados.rados.0.smithi122.stdout:3333: write (excl) oid 18 current snap is 385 2024-09-15T04:01:10.995 INFO:tasks.rados.rados.0.smithi122.stdout:3332: finishing write tid 1 to smithi12223906-49 2024-09-15T04:01:10.996 INFO:tasks.rados.rados.0.smithi122.stdout:3332: finishing write tid 2 to smithi12223906-49 2024-09-15T04:01:10.996 INFO:tasks.rados.rados.0.smithi122.stdout:3333: seq_num 1175 ranges {548225=735937,1925048=596064,2950840=547321} 2024-09-15T04:01:11.002 INFO:tasks.rados.rados.0.smithi122.stdout:3333: writing smithi12223906-18 from 548225 to 1284162 tid 1 2024-09-15T04:01:11.009 INFO:tasks.rados.rados.0.smithi122.stdout:3333: writing smithi12223906-18 from 1925048 to 2521112 tid 2 2024-09-15T04:01:11.015 INFO:tasks.rados.rados.0.smithi122.stdout:3333: writing smithi12223906-18 from 2950840 to 3498161 tid 3 2024-09-15T04:01:11.015 INFO:tasks.rados.rados.0.smithi122.stdout:3334: snap_create 2024-09-15T04:01:11.015 INFO:tasks.rados.rados.0.smithi122.stdout:3332: finishing write tid 3 to smithi12223906-49 2024-09-15T04:01:11.015 INFO:tasks.rados.rados.0.smithi122.stdout:3332: finishing write tid 4 to smithi12223906-49 2024-09-15T04:01:11.016 INFO:tasks.rados.rados.0.smithi122.stdout:3332: finishing write tid 5 to smithi12223906-49 2024-09-15T04:01:11.016 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1102 (ObjNum 1174 snap 385 seq_num 1174) dirty exists 2024-09-15T04:01:11.016 INFO:tasks.rados.rados.0.smithi122.stdout:3332: left oid 49 (ObjNum 1174 snap 385 seq_num 1174) 2024-09-15T04:01:11.024 INFO:tasks.rados.rados.0.smithi122.stdout:3333: finishing write tid 1 to smithi12223906-18 2024-09-15T04:01:11.024 INFO:tasks.rados.rados.0.smithi122.stdout:3333: finishing write tid 2 to smithi12223906-18 2024-09-15T04:01:11.024 INFO:tasks.rados.rados.0.smithi122.stdout:3333: finishing write tid 3 to smithi12223906-18 2024-09-15T04:01:11.024 INFO:tasks.rados.rados.0.smithi122.stdout:3333: finishing write tid 4 to smithi12223906-18 2024-09-15T04:01:11.025 INFO:tasks.rados.rados.0.smithi122.stdout:3333: finishing write tid 5 to smithi12223906-18 2024-09-15T04:01:11.025 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2710 (ObjNum 1175 snap 385 seq_num 1175) dirty exists 2024-09-15T04:01:11.025 INFO:tasks.rados.rados.0.smithi122.stdout:3333: left oid 18 (ObjNum 1175 snap 385 seq_num 1175) 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:3332: done (2 left) 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:3333: done (1 left) 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:3334: done (0 left) 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:3335: rollback oid 34 current snap is 386 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 355 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:3336: write (excl) oid 6 current snap is 386 2024-09-15T04:01:11.790 INFO:tasks.rados.rados.0.smithi122.stdout:3336: seq_num 1176 ranges {565032=411186,1583530=456075,2578263=758593} 2024-09-15T04:01:11.802 INFO:tasks.rados.rados.0.smithi122.stdout:3336: writing smithi12223906-6 from 565032 to 976218 tid 1 2024-09-15T04:01:11.808 INFO:tasks.rados.rados.0.smithi122.stdout:3336: writing smithi12223906-6 from 1583530 to 2039605 tid 2 2024-09-15T04:01:11.816 INFO:tasks.rados.rados.0.smithi122.stdout:3336: writing smithi12223906-6 from 2578263 to 3336856 tid 3 2024-09-15T04:01:11.816 INFO:tasks.rados.rados.0.smithi122.stdout:3337: read oid 35 snap 336 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3337: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3338: write oid 18 current snap is 386 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3335: finishing rollback tid 0 to smithi12223906-34 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3335: finishing rollback tid 1 to smithi12223906-34 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3335: finishing rollback tid 2 to smithi12223906-34 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 1607 (ObjNum 1062 snap 343 seq_num 1062) dirty exists 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3336: finishing write tid 1 to smithi12223906-6 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3336: finishing write tid 2 to smithi12223906-6 2024-09-15T04:01:11.817 INFO:tasks.rados.rados.0.smithi122.stdout:3338: seq_num 1177 ranges {581839=486435,1642012=599733,3005686=569865} 2024-09-15T04:01:11.822 INFO:tasks.rados.rados.0.smithi122.stdout:3338: writing smithi12223906-18 from 581839 to 1068274 tid 1 2024-09-15T04:01:11.830 INFO:tasks.rados.rados.0.smithi122.stdout:3338: writing smithi12223906-18 from 1642012 to 2241745 tid 2 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3338: writing smithi12223906-18 from 3005686 to 3575551 tid 3 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3335: done (3 left) 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3339: snap_create 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3336: finishing write tid 3 to smithi12223906-6 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3336: finishing write tid 4 to smithi12223906-6 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3336: finishing write tid 5 to smithi12223906-6 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2468 (ObjNum 1176 snap 386 seq_num 1176) dirty exists 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3336: left oid 6 (ObjNum 1176 snap 386 seq_num 1176) 2024-09-15T04:01:11.838 INFO:tasks.rados.rados.0.smithi122.stdout:3337: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:3338: finishing write tid 1 to smithi12223906-18 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:3338: finishing write tid 2 to smithi12223906-18 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:3338: finishing write tid 3 to smithi12223906-18 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:3338: finishing write tid 4 to smithi12223906-18 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:3338: finishing write tid 5 to smithi12223906-18 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2715 (ObjNum 1177 snap 386 seq_num 1177) dirty exists 2024-09-15T04:01:11.948 INFO:tasks.rados.rados.0.smithi122.stdout:3338: left oid 18 (ObjNum 1177 snap 386 seq_num 1177) 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3336: done (3 left) 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3337: done (2 left) 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3338: done (1 left) 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3339: done (0 left) 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3340: read oid 45 snap -1 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3340: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3341: write oid 24 current snap is 387 2024-09-15T04:01:12.803 INFO:tasks.rados.rados.0.smithi122.stdout:3341: seq_num 1178 ranges {598646=561684,1700494=459744,2633109=781137} 2024-09-15T04:01:12.813 INFO:tasks.rados.rados.0.smithi122.stdout:3341: writing smithi12223906-24 from 598646 to 1160330 tid 1 2024-09-15T04:01:12.819 INFO:tasks.rados.rados.0.smithi122.stdout:3341: writing smithi12223906-24 from 1700494 to 2160238 tid 2 2024-09-15T04:01:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3341: writing smithi12223906-24 from 2633109 to 3414246 tid 3 2024-09-15T04:01:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3342: read oid 23 snap -1 2024-09-15T04:01:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3342: expect (ObjNum 1166 snap 382 seq_num 1166) 2024-09-15T04:01:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3343: rollback oid 3 current snap is 387 2024-09-15T04:01:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 326 2024-09-15T04:01:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3340: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:01:13.015 INFO:tasks.rados.rados.0.smithi122.stdout:3341: finishing write tid 1 to smithi12223906-24 2024-09-15T04:01:13.016 INFO:tasks.rados.rados.0.smithi122.stdout:3341: finishing write tid 2 to smithi12223906-24 2024-09-15T04:01:13.016 INFO:tasks.rados.rados.0.smithi122.stdout:3342: expect (ObjNum 1166 snap 382 seq_num 1166) 2024-09-15T04:01:13.128 INFO:tasks.rados.rados.0.smithi122.stdout:3341: finishing write tid 3 to smithi12223906-24 2024-09-15T04:01:13.128 INFO:tasks.rados.rados.0.smithi122.stdout:3341: finishing write tid 4 to smithi12223906-24 2024-09-15T04:01:13.128 INFO:tasks.rados.rados.0.smithi122.stdout:3341: finishing write tid 5 to smithi12223906-24 2024-09-15T04:01:13.128 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 2744 (ObjNum 1178 snap 387 seq_num 1178) dirty exists 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3341: left oid 24 (ObjNum 1178 snap 387 seq_num 1178) 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3343: finishing rollback tid 0 to smithi12223906-3 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3343: finishing rollback tid 1 to smithi12223906-3 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3343: finishing rollback tid 2 to smithi12223906-3 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2748 (ObjNum 973 snap 307 seq_num 973) dirty exists 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3340: done (3 left) 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3341: done (2 left) 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3342: done (1 left) 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3343: done (0 left) 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3344: write oid 25 current snap is 387 2024-09-15T04:01:13.129 INFO:tasks.rados.rados.0.smithi122.stdout:3344: seq_num 1179 ranges {615453=753286,2158976=603402,3460532=354921} 2024-09-15T04:01:13.145 INFO:tasks.rados.rados.0.smithi122.stdout:3344: writing smithi12223906-25 from 615453 to 1368739 tid 1 2024-09-15T04:01:13.153 INFO:tasks.rados.rados.0.smithi122.stdout:3344: writing smithi12223906-25 from 2158976 to 2762378 tid 2 2024-09-15T04:01:13.159 INFO:tasks.rados.rados.0.smithi122.stdout:3344: writing smithi12223906-25 from 3460532 to 3815453 tid 3 2024-09-15T04:01:13.159 INFO:tasks.rados.rados.0.smithi122.stdout:3345: write (excl) oid 4 current snap is 387 2024-09-15T04:01:13.159 INFO:tasks.rados.rados.0.smithi122.stdout:3345: seq_num 1180 ranges {632260=428535,1817458=463413,2687955=520034,3709324=122936} 2024-09-15T04:01:13.165 INFO:tasks.rados.rados.0.smithi122.stdout:3345: writing smithi12223906-4 from 632260 to 1060795 tid 1 2024-09-15T04:01:13.172 INFO:tasks.rados.rados.0.smithi122.stdout:3345: writing smithi12223906-4 from 1817458 to 2280871 tid 2 2024-09-15T04:01:13.177 INFO:tasks.rados.rados.0.smithi122.stdout:3345: writing smithi12223906-4 from 2687955 to 3207989 tid 3 2024-09-15T04:01:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:3345: writing smithi12223906-4 from 3709324 to 3832260 tid 4 2024-09-15T04:01:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:3346: copy_from oid 39 from oid 7 current snap is 387 2024-09-15T04:01:13.181 INFO:tasks.rados.rados.0.smithi122.stdout:3344: finishing write tid 1 to smithi12223906-25 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3344: finishing write tid 2 to smithi12223906-25 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3344: finishing write tid 3 to smithi12223906-25 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3344: finishing write tid 4 to smithi12223906-25 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3344: finishing write tid 5 to smithi12223906-25 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2753 (ObjNum 1179 snap 387 seq_num 1179) dirty exists 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3344: left oid 25 (ObjNum 1179 snap 387 seq_num 1179) 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3344: done (2 left) 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3347: copy_from oid 38 from oid 20 current snap is 387 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3345: finishing write tid 1 to smithi12223906-4 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3345: finishing write tid 2 to smithi12223906-4 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:3348: rollback oid 33 current snap is 387 2024-09-15T04:01:13.182 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 33 to 368 2024-09-15T04:01:13.183 INFO:tasks.rados.rados.0.smithi122.stdout:3349: read oid 24 snap 315 2024-09-15T04:01:13.183 INFO:tasks.rados.rados.0.smithi122.stdout:3349: expect deleted 2024-09-15T04:01:13.183 INFO:tasks.rados.rados.0.smithi122.stdout:3350: snap_create 2024-09-15T04:01:13.183 INFO:tasks.rados.rados.0.smithi122.stdout:3347: finishing copy_from to smithi12223906-38 2024-09-15T04:01:13.184 INFO:tasks.rados.rados.0.smithi122.stdout:3347: got expected ENOENT (src dne) 2024-09-15T04:01:13.184 INFO:tasks.rados.rados.0.smithi122.stdout:3346: finishing copy_from to smithi12223906-39 2024-09-15T04:01:13.184 INFO:tasks.rados.rados.0.smithi122.stdout:3346: got expected ENOENT (src dne) 2024-09-15T04:01:13.184 INFO:tasks.rados.rados.0.smithi122.stdout:3347: finishing copy_from racing read to smithi12223906-38 2024-09-15T04:01:13.184 INFO:tasks.rados.rados.0.smithi122.stdout:3346: finishing copy_from racing read to smithi12223906-39 2024-09-15T04:01:13.184 INFO:tasks.rados.rados.0.smithi122.stdout:3345: finishing write tid 3 to smithi12223906-4 2024-09-15T04:01:13.185 INFO:tasks.rados.rados.0.smithi122.stdout:3345: finishing write tid 4 to smithi12223906-4 2024-09-15T04:01:13.186 INFO:tasks.rados.rados.0.smithi122.stdout:3345: finishing write tid 5 to smithi12223906-4 2024-09-15T04:01:13.187 INFO:tasks.rados.rados.0.smithi122.stdout:3345: finishing write tid 6 to smithi12223906-4 2024-09-15T04:01:13.187 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2619 (ObjNum 1180 snap 387 seq_num 1180) dirty exists 2024-09-15T04:01:13.187 INFO:tasks.rados.rados.0.smithi122.stdout:3345: left oid 4 (ObjNum 1180 snap 387 seq_num 1180) 2024-09-15T04:01:13.191 INFO:tasks.rados.rados.0.smithi122.stdout:3348: finishing rollback tid 0 to smithi12223906-33 2024-09-15T04:01:13.191 INFO:tasks.rados.rados.0.smithi122.stdout:3348: finishing rollback tid 1 to smithi12223906-33 2024-09-15T04:01:13.192 INFO:tasks.rados.rados.0.smithi122.stdout:3348: finishing rollback tid 2 to smithi12223906-33 2024-09-15T04:01:13.192 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2719 (ObjNum 1085 snap 355 seq_num 1085) dirty exists 2024-09-15T04:01:13.813 INFO:tasks.rados.rados.0.smithi122.stdout:3345: done (5 left) 2024-09-15T04:01:13.813 INFO:tasks.rados.rados.0.smithi122.stdout:3346: done (4 left) 2024-09-15T04:01:13.813 INFO:tasks.rados.rados.0.smithi122.stdout:3347: done (3 left) 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3348: done (2 left) 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3349: done (1 left) 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3350: done (0 left) 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3351: rollback oid 26 current snap is 388 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 26 to 378 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3352: read oid 39 snap -1 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3352: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3353: read oid 3 snap -1 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3353: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T04:01:13.814 INFO:tasks.rados.rados.0.smithi122.stdout:3354: delete oid 13 current snap is 388 2024-09-15T04:01:13.817 INFO:tasks.rados.rados.0.smithi122.stdout:3354: done (3 left) 2024-09-15T04:01:13.817 INFO:tasks.rados.rados.0.smithi122.stdout:3355: read oid 24 snap -1 2024-09-15T04:01:13.817 INFO:tasks.rados.rados.0.smithi122.stdout:3355: expect (ObjNum 1178 snap 387 seq_num 1178) 2024-09-15T04:01:13.817 INFO:tasks.rados.rados.0.smithi122.stdout:3356: copy_from oid 12 from oid 19 current snap is 388 2024-09-15T04:01:13.818 INFO:tasks.rados.rados.0.smithi122.stdout:3357: rollback oid 14 current snap is 388 2024-09-15T04:01:13.818 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 330 2024-09-15T04:01:13.818 INFO:tasks.rados.rados.0.smithi122.stdout:3358: read oid 41 snap -1 2024-09-15T04:01:13.818 INFO:tasks.rados.rados.0.smithi122.stdout:3358: expect deleted 2024-09-15T04:01:13.818 INFO:tasks.rados.rados.0.smithi122.stdout:3359: copy_from oid 33 from oid 38 current snap is 388 2024-09-15T04:01:13.818 INFO:tasks.rados.rados.0.smithi122.stdout:3360: snap_remove snap 351 2024-09-15T04:01:14.822 INFO:tasks.rados.rados.0.smithi122.stdout:3360: done (8 left) 2024-09-15T04:01:14.822 INFO:tasks.rados.rados.0.smithi122.stdout:3361: write (excl) oid 25 current snap is 388 2024-09-15T04:01:14.822 INFO:tasks.rados.rados.0.smithi122.stdout:3361: seq_num 1182 ranges {665874=579033,1934422=467082,3142801=542578} 2024-09-15T04:01:14.833 INFO:tasks.rados.rados.0.smithi122.stdout:3361: writing smithi12223906-25 from 665874 to 1244907 tid 1 2024-09-15T04:01:14.840 INFO:tasks.rados.rados.0.smithi122.stdout:3361: writing smithi12223906-25 from 1934422 to 2401504 tid 2 2024-09-15T04:01:14.847 INFO:tasks.rados.rados.0.smithi122.stdout:3361: writing smithi12223906-25 from 3142801 to 3685379 tid 3 2024-09-15T04:01:14.847 INFO:tasks.rados.rados.0.smithi122.stdout:3362: snap_create 2024-09-15T04:01:14.847 INFO:tasks.rados.rados.0.smithi122.stdout:3352: expect (ObjNum 1090 snap 355 seq_num 1090) 2024-09-15T04:01:15.020 INFO:tasks.rados.rados.0.smithi122.stdout:3351: finishing rollback tid 0 to smithi12223906-26 2024-09-15T04:01:15.020 INFO:tasks.rados.rados.0.smithi122.stdout:3351: finishing rollback tid 1 to smithi12223906-26 2024-09-15T04:01:15.020 INFO:tasks.rados.rados.0.smithi122.stdout:3351: finishing rollback tid 2 to smithi12223906-26 2024-09-15T04:01:15.020 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1933 (ObjNum 984 snap 311 seq_num 984) dirty exists 2024-09-15T04:01:15.020 INFO:tasks.rados.rados.0.smithi122.stdout:3353: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T04:01:15.153 INFO:tasks.rados.rados.0.smithi122.stdout:3356: finishing copy_from racing read to smithi12223906-12 2024-09-15T04:01:15.153 INFO:tasks.rados.rados.0.smithi122.stdout:3355: expect (ObjNum 1178 snap 387 seq_num 1178) 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3357: finishing rollback tid 1 to smithi12223906-14 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3359: finishing copy_from racing read to smithi12223906-33 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3357: finishing rollback tid 2 to smithi12223906-14 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 2755 (ObjNum 1036 snap 327 seq_num 1036) dirty exists 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3356: finishing copy_from to smithi12223906-12 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 2118 (ObjNum 1172 snap 383 seq_num 1172) dirty exists 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3359: finishing copy_from to smithi12223906-33 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 1930 (ObjNum 1173 snap 384 seq_num 1173) dirty exists 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3361: finishing write tid 1 to smithi12223906-25 2024-09-15T04:01:15.338 INFO:tasks.rados.rados.0.smithi122.stdout:3361: finishing write tid 2 to smithi12223906-25 2024-09-15T04:01:15.339 INFO:tasks.rados.rados.0.smithi122.stdout:3361: finishing write tid 3 to smithi12223906-25 2024-09-15T04:01:15.339 INFO:tasks.rados.rados.0.smithi122.stdout:3361: finishing write tid 4 to smithi12223906-25 2024-09-15T04:01:15.339 INFO:tasks.rados.rados.0.smithi122.stdout:3361: finishing write tid 5 to smithi12223906-25 2024-09-15T04:01:15.339 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2768 (ObjNum 1182 snap 388 seq_num 1182) dirty exists 2024-09-15T04:01:15.339 INFO:tasks.rados.rados.0.smithi122.stdout:3361: left oid 25 (ObjNum 1182 snap 388 seq_num 1182) 2024-09-15T04:01:15.831 INFO:tasks.rados.rados.0.smithi122.stdout:3351: done (9 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3352: done (8 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3353: done (7 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3355: done (6 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3356: done (5 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3357: done (4 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3358: done (3 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3359: done (2 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3361: done (1 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3362: done (0 left) 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3363: copy_from oid 38 from oid 21 current snap is 389 2024-09-15T04:01:15.832 INFO:tasks.rados.rados.0.smithi122.stdout:3364: rollback oid 29 current snap is 389 2024-09-15T04:01:15.833 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 29 to 374 2024-09-15T04:01:15.833 INFO:tasks.rados.rados.0.smithi122.stdout:3365: write (excl) oid 22 current snap is 389 2024-09-15T04:01:15.833 INFO:tasks.rados.rados.0.smithi122.stdout:3365: seq_num 1183 ranges {682681=654282,1876551=610740,3053871=470203} 2024-09-15T04:01:15.844 INFO:tasks.rados.rados.0.smithi122.stdout:3365: writing smithi12223906-22 from 682681 to 1336963 tid 1 2024-09-15T04:01:15.850 INFO:tasks.rados.rados.0.smithi122.stdout:3365: writing smithi12223906-22 from 1876551 to 2487291 tid 2 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3365: writing smithi12223906-22 from 3053871 to 3524074 tid 3 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3366: snap_create 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3364: finishing rollback tid 0 to smithi12223906-29 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3364: finishing rollback tid 1 to smithi12223906-29 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3364: finishing rollback tid 2 to smithi12223906-29 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 2419 (ObjNum 1085 snap 355 seq_num 1085) dirty exists 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3365: finishing write tid 1 to smithi12223906-22 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3363: finishing copy_from racing read to smithi12223906-38 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:3363: finishing copy_from to smithi12223906-38 2024-09-15T04:01:15.857 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 2128 (ObjNum 1040 snap 330 seq_num 1040) dirty exists 2024-09-15T04:01:15.859 INFO:tasks.rados.rados.0.smithi122.stdout:3365: finishing write tid 2 to smithi12223906-22 2024-09-15T04:01:15.862 INFO:tasks.rados.rados.0.smithi122.stdout:3365: finishing write tid 3 to smithi12223906-22 2024-09-15T04:01:15.862 INFO:tasks.rados.rados.0.smithi122.stdout:3365: finishing write tid 4 to smithi12223906-22 2024-09-15T04:01:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:3365: finishing write tid 5 to smithi12223906-22 2024-09-15T04:01:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 2735 (ObjNum 1183 snap 389 seq_num 1183) dirty exists 2024-09-15T04:01:15.863 INFO:tasks.rados.rados.0.smithi122.stdout:3365: left oid 22 (ObjNum 1183 snap 389 seq_num 1183) 2024-09-15T04:01:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:3363: done (3 left) 2024-09-15T04:01:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:3364: done (2 left) 2024-09-15T04:01:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:3365: done (1 left) 2024-09-15T04:01:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:3366: done (0 left) 2024-09-15T04:01:16.742 INFO:tasks.rados.rados.0.smithi122.stdout:3367: delete oid 36 current snap is 390 2024-09-15T04:01:16.761 INFO:tasks.rados.rados.0.smithi122.stdout:3367: done (0 left) 2024-09-15T04:01:16.761 INFO:tasks.rados.rados.0.smithi122.stdout:3368: read oid 25 snap -1 2024-09-15T04:01:16.762 INFO:tasks.rados.rados.0.smithi122.stdout:3368: expect (ObjNum 1182 snap 388 seq_num 1182) 2024-09-15T04:01:16.762 INFO:tasks.rados.rados.0.smithi122.stdout:3369: delete oid 6 current snap is 390 2024-09-15T04:01:16.769 INFO:tasks.rados.rados.0.smithi122.stdout:3369: done (1 left) 2024-09-15T04:01:16.769 INFO:tasks.rados.rados.0.smithi122.stdout:3370: rollback oid 38 current snap is 390 2024-09-15T04:01:16.769 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 328 2024-09-15T04:01:16.769 INFO:tasks.rados.rados.0.smithi122.stdout:3371: write oid 42 current snap is 390 2024-09-15T04:01:16.769 INFO:tasks.rados.rados.0.smithi122.stdout:3371: seq_num 1186 ranges {733102=596382,1768350=758067,3136140=704019} 2024-09-15T04:01:16.780 INFO:tasks.rados.rados.0.smithi122.stdout:3371: writing smithi12223906-42 from 733102 to 1329484 tid 1 2024-09-15T04:01:16.787 INFO:tasks.rados.rados.0.smithi122.stdout:3371: writing smithi12223906-42 from 1768350 to 2526417 tid 2 2024-09-15T04:01:16.795 INFO:tasks.rados.rados.0.smithi122.stdout:3371: writing smithi12223906-42 from 3136140 to 3840159 tid 3 2024-09-15T04:01:16.795 INFO:tasks.rados.rados.0.smithi122.stdout:3372: delete oid 36 current snap is 390 2024-09-15T04:01:16.795 INFO:tasks.rados.rados.0.smithi122.stdout:3368: expect (ObjNum 1182 snap 388 seq_num 1182) 2024-09-15T04:01:17.020 INFO:tasks.rados.rados.0.smithi122.stdout:3370: finishing rollback tid 0 to smithi12223906-38 2024-09-15T04:01:17.020 INFO:tasks.rados.rados.0.smithi122.stdout:3370: finishing rollback tid 1 to smithi12223906-38 2024-09-15T04:01:17.020 INFO:tasks.rados.rados.0.smithi122.stdout:3370: finishing rollback tid 2 to smithi12223906-38 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 2131 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: finishing write tid 1 to smithi12223906-42 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: finishing write tid 2 to smithi12223906-42 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: finishing write tid 3 to smithi12223906-42 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: finishing write tid 4 to smithi12223906-42 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: finishing write tid 5 to smithi12223906-42 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 2779 (ObjNum 1186 snap 390 seq_num 1186) dirty exists 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: left oid 42 (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3368: done (3 left) 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3370: done (2 left) 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3371: done (1 left) 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3372: done (0 left) 2024-09-15T04:01:17.021 INFO:tasks.rados.rados.0.smithi122.stdout:3373: read oid 43 snap -1 2024-09-15T04:01:17.022 INFO:tasks.rados.rados.0.smithi122.stdout:3373: expect (ObjNum 1167 snap 383 seq_num 1167) 2024-09-15T04:01:17.022 INFO:tasks.rados.rados.0.smithi122.stdout:3374: read oid 38 snap 378 2024-09-15T04:01:17.022 INFO:tasks.rados.rados.0.smithi122.stdout:3374: expect (ObjNum 1125 snap 371 seq_num 1125) 2024-09-15T04:01:17.022 INFO:tasks.rados.rados.0.smithi122.stdout:3375: snap_remove snap 347 2024-09-15T04:01:17.846 INFO:tasks.rados.rados.0.smithi122.stdout:3375: done (2 left) 2024-09-15T04:01:17.846 INFO:tasks.rados.rados.0.smithi122.stdout:3376: read oid 40 snap -1 2024-09-15T04:01:17.846 INFO:tasks.rados.rados.0.smithi122.stdout:3376: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:01:17.846 INFO:tasks.rados.rados.0.smithi122.stdout:3377: read oid 37 snap 380 2024-09-15T04:01:17.846 INFO:tasks.rados.rados.0.smithi122.stdout:3377: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:01:17.846 INFO:tasks.rados.rados.0.smithi122.stdout:3374: expect (ObjNum 1125 snap 371 seq_num 1125) 2024-09-15T04:01:17.984 INFO:tasks.rados.rados.0.smithi122.stdout:3373: expect (ObjNum 1167 snap 383 seq_num 1167) 2024-09-15T04:01:18.142 INFO:tasks.rados.rados.0.smithi122.stdout:3376: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:01:18.247 INFO:tasks.rados.rados.0.smithi122.stdout:3373: done (3 left) 2024-09-15T04:01:18.247 INFO:tasks.rados.rados.0.smithi122.stdout:3374: done (2 left) 2024-09-15T04:01:18.247 INFO:tasks.rados.rados.0.smithi122.stdout:3376: done (1 left) 2024-09-15T04:01:18.247 INFO:tasks.rados.rados.0.smithi122.stdout:3378: snap_remove snap 343 2024-09-15T04:01:18.853 INFO:tasks.rados.rados.0.smithi122.stdout:3378: done (1 left) 2024-09-15T04:01:18.853 INFO:tasks.rados.rados.0.smithi122.stdout:3379: read oid 26 snap -1 2024-09-15T04:01:18.853 INFO:tasks.rados.rados.0.smithi122.stdout:3379: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:01:18.853 INFO:tasks.rados.rados.0.smithi122.stdout:3380: delete oid 43 current snap is 390 2024-09-15T04:01:18.854 INFO:tasks.rados.rados.0.smithi122.stdout:3377: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:01:18.986 INFO:tasks.rados.rados.0.smithi122.stdout:3379: expect (ObjNum 984 snap 311 seq_num 984) 2024-09-15T04:01:19.167 INFO:tasks.rados.rados.0.smithi122.stdout:3377: done (2 left) 2024-09-15T04:01:19.167 INFO:tasks.rados.rados.0.smithi122.stdout:3379: done (1 left) 2024-09-15T04:01:19.167 INFO:tasks.rados.rados.0.smithi122.stdout:3380: done (0 left) 2024-09-15T04:01:19.168 INFO:tasks.rados.rados.0.smithi122.stdout:3381: snap_create 2024-09-15T04:01:19.862 INFO:tasks.rados.rados.0.smithi122.stdout:3381: done (0 left) 2024-09-15T04:01:19.862 INFO:tasks.rados.rados.0.smithi122.stdout:3382: snap_remove snap 390 2024-09-15T04:01:20.865 INFO:tasks.rados.rados.0.smithi122.stdout:3382: done (0 left) 2024-09-15T04:01:20.865 INFO:tasks.rados.rados.0.smithi122.stdout:3383: snap_create 2024-09-15T04:01:21.878 INFO:tasks.rados.rados.0.smithi122.stdout:3383: done (0 left) 2024-09-15T04:01:21.878 INFO:tasks.rados.rados.0.smithi122.stdout:3384: snap_remove snap 365 2024-09-15T04:01:22.892 INFO:tasks.rados.rados.0.smithi122.stdout:3384: done (0 left) 2024-09-15T04:01:22.892 INFO:tasks.rados.rados.0.smithi122.stdout:3385: write (excl) oid 7 current snap is 392 2024-09-15T04:01:22.892 INFO:tasks.rados.rados.0.smithi122.stdout:3385: seq_num 1189 ranges {783523=422129,1827443=621747,3218409=654188} 2024-09-15T04:01:22.902 INFO:tasks.rados.rados.0.smithi122.stdout:3385: writing smithi12223906-7 from 783523 to 1205652 tid 1 2024-09-15T04:01:22.909 INFO:tasks.rados.rados.0.smithi122.stdout:3385: writing smithi12223906-7 from 1827443 to 2449190 tid 2 2024-09-15T04:01:22.918 INFO:tasks.rados.rados.0.smithi122.stdout:3385: writing smithi12223906-7 from 3218409 to 3872597 tid 3 2024-09-15T04:01:22.918 INFO:tasks.rados.rados.0.smithi122.stdout:3386: write oid 25 current snap is 392 2024-09-15T04:01:22.918 INFO:tasks.rados.rados.0.smithi122.stdout:3385: finishing write tid 1 to smithi12223906-7 2024-09-15T04:01:22.918 INFO:tasks.rados.rados.0.smithi122.stdout:3386: seq_num 1190 ranges {400330=497378,1485925=514405} 2024-09-15T04:01:22.923 INFO:tasks.rados.rados.0.smithi122.stdout:3386: writing smithi12223906-25 from 400330 to 897708 tid 1 2024-09-15T04:01:22.929 INFO:tasks.rados.rados.0.smithi122.stdout:3386: writing smithi12223906-25 from 1485925 to 2000330 tid 2 2024-09-15T04:01:22.929 INFO:tasks.rados.rados.0.smithi122.stdout:3387: read oid 17 snap -1 2024-09-15T04:01:22.930 INFO:tasks.rados.rados.0.smithi122.stdout:3387: expect (ObjNum 1070 snap 348 seq_num 1070) 2024-09-15T04:01:22.930 INFO:tasks.rados.rados.0.smithi122.stdout:3388: write oid 15 current snap is 392 2024-09-15T04:01:22.930 INFO:tasks.rados.rados.0.smithi122.stdout:3385: finishing write tid 2 to smithi12223906-7 2024-09-15T04:01:22.930 INFO:tasks.rados.rados.0.smithi122.stdout:3388: seq_num 1191 ranges {417137=572627,1428054=589083} 2024-09-15T04:01:22.935 INFO:tasks.rados.rados.0.smithi122.stdout:3388: writing smithi12223906-15 from 417137 to 989764 tid 1 2024-09-15T04:01:22.941 INFO:tasks.rados.rados.0.smithi122.stdout:3388: writing smithi12223906-15 from 1428054 to 2017137 tid 2 2024-09-15T04:01:22.941 INFO:tasks.rados.rados.0.smithi122.stdout:3389: read oid 36 snap 355 2024-09-15T04:01:22.941 INFO:tasks.rados.rados.0.smithi122.stdout:3389: expect deleted 2024-09-15T04:01:22.942 INFO:tasks.rados.rados.0.smithi122.stdout:3390: write (excl) oid 27 current snap is 392 2024-09-15T04:01:22.942 INFO:tasks.rados.rados.0.smithi122.stdout:3390: seq_num 1192 ranges {433944=647876,1486536=547408} 2024-09-15T04:01:22.947 INFO:tasks.rados.rados.0.smithi122.stdout:3390: writing smithi12223906-27 from 433944 to 1081820 tid 1 2024-09-15T04:01:22.953 INFO:tasks.rados.rados.0.smithi122.stdout:3390: writing smithi12223906-27 from 1486536 to 2033944 tid 2 2024-09-15T04:01:22.953 INFO:tasks.rados.rados.0.smithi122.stdout:3391: copy_from oid 19 from oid 44 current snap is 392 2024-09-15T04:01:22.953 INFO:tasks.rados.rados.0.smithi122.stdout:3385: finishing write tid 3 to smithi12223906-7 2024-09-15T04:01:22.953 INFO:tasks.rados.rados.0.smithi122.stdout:3385: finishing write tid 4 to smithi12223906-7 2024-09-15T04:01:22.954 INFO:tasks.rados.rados.0.smithi122.stdout:3385: finishing write tid 5 to smithi12223906-7 2024-09-15T04:01:22.954 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 2423 (ObjNum 1189 snap 392 seq_num 1189) dirty exists 2024-09-15T04:01:22.954 INFO:tasks.rados.rados.0.smithi122.stdout:3385: left oid 7 (ObjNum 1189 snap 392 seq_num 1189) 2024-09-15T04:01:22.954 INFO:tasks.rados.rados.0.smithi122.stdout:3385: done (6 left) 2024-09-15T04:01:22.954 INFO:tasks.rados.rados.0.smithi122.stdout:3392: snap_remove snap 330 2024-09-15T04:01:23.914 INFO:tasks.rados.rados.0.smithi122.stdout:3392: done (6 left) 2024-09-15T04:01:23.914 INFO:tasks.rados.rados.0.smithi122.stdout:3393: delete oid 23 current snap is 392 2024-09-15T04:01:23.922 INFO:tasks.rados.rados.0.smithi122.stdout:3386: finishing write tid 1 to smithi12223906-25 2024-09-15T04:01:23.923 INFO:tasks.rados.rados.0.smithi122.stdout:3386: finishing write tid 2 to smithi12223906-25 2024-09-15T04:01:23.923 INFO:tasks.rados.rados.0.smithi122.stdout:3386: finishing write tid 3 to smithi12223906-25 2024-09-15T04:01:23.923 INFO:tasks.rados.rados.0.smithi122.stdout:3386: finishing write tid 4 to smithi12223906-25 2024-09-15T04:01:23.923 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2814 (ObjNum 1190 snap 392 seq_num 1190) dirty exists 2024-09-15T04:01:23.923 INFO:tasks.rados.rados.0.smithi122.stdout:3386: left oid 25 (ObjNum 1190 snap 392 seq_num 1190) 2024-09-15T04:01:23.923 INFO:tasks.rados.rados.0.smithi122.stdout:3387: expect (ObjNum 1070 snap 348 seq_num 1070) 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3388: finishing write tid 1 to smithi12223906-15 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3388: finishing write tid 2 to smithi12223906-15 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3388: finishing write tid 3 to smithi12223906-15 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3388: finishing write tid 4 to smithi12223906-15 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1927 (ObjNum 1191 snap 392 seq_num 1191) dirty exists 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3388: left oid 15 (ObjNum 1191 snap 392 seq_num 1191) 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3390: finishing write tid 1 to smithi12223906-27 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3390: finishing write tid 2 to smithi12223906-27 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3390: finishing write tid 3 to smithi12223906-27 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:3390: finishing write tid 4 to smithi12223906-27 2024-09-15T04:01:24.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 2766 (ObjNum 1192 snap 392 seq_num 1192) dirty exists 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3390: left oid 27 (ObjNum 1192 snap 392 seq_num 1192) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3391: finishing copy_from to smithi12223906-19 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3391: got expected ENOENT (src dne) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3391: finishing copy_from racing read to smithi12223906-19 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3386: done (6 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3387: done (5 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3388: done (4 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3389: done (3 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3390: done (2 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3391: done (1 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3393: done (0 left) 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3394: write oid 9 current snap is 392 2024-09-15T04:01:24.053 INFO:tasks.rados.rados.0.smithi122.stdout:3394: seq_num 1194 ranges {467558=514727,1719853=347705} 2024-09-15T04:01:24.066 INFO:tasks.rados.rados.0.smithi122.stdout:3394: writing smithi12223906-9 from 467558 to 982285 tid 1 2024-09-15T04:01:24.073 INFO:tasks.rados.rados.0.smithi122.stdout:3394: writing smithi12223906-9 from 1719853 to 2067558 tid 2 2024-09-15T04:01:24.073 INFO:tasks.rados.rados.0.smithi122.stdout:3395: copy_from oid 26 from oid 16 current snap is 392 2024-09-15T04:01:24.073 INFO:tasks.rados.rados.0.smithi122.stdout:3396: rollback oid 34 current snap is 392 2024-09-15T04:01:24.073 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 336 2024-09-15T04:01:24.073 INFO:tasks.rados.rados.0.smithi122.stdout:3397: snap_remove snap 375 2024-09-15T04:01:24.922 INFO:tasks.rados.rados.0.smithi122.stdout:3394: finishing write tid 1 to smithi12223906-9 2024-09-15T04:01:24.922 INFO:tasks.rados.rados.0.smithi122.stdout:3394: finishing write tid 2 to smithi12223906-9 2024-09-15T04:01:24.922 INFO:tasks.rados.rados.0.smithi122.stdout:3394: finishing write tid 3 to smithi12223906-9 2024-09-15T04:01:24.922 INFO:tasks.rados.rados.0.smithi122.stdout:3394: finishing write tid 4 to smithi12223906-9 2024-09-15T04:01:24.922 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2472 (ObjNum 1194 snap 392 seq_num 1194) dirty exists 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3394: left oid 9 (ObjNum 1194 snap 392 seq_num 1194) 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3396: finishing rollback tid 0 to smithi12223906-34 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3396: finishing rollback tid 1 to smithi12223906-34 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3396: finishing rollback tid 2 to smithi12223906-34 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 2121 (ObjNum 982 snap 311 seq_num 982) dirty exists 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3395: finishing copy_from racing read to smithi12223906-26 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3395: finishing copy_from to smithi12223906-26 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 1914 (ObjNum 1153 snap 380 seq_num 1153) dirty exists 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3394: done (3 left) 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3395: done (2 left) 2024-09-15T04:01:24.923 INFO:tasks.rados.rados.0.smithi122.stdout:3396: done (1 left) 2024-09-15T04:01:24.924 INFO:tasks.rados.rados.0.smithi122.stdout:3397: done (0 left) 2024-09-15T04:01:24.924 INFO:tasks.rados.rados.0.smithi122.stdout:3398: write (excl) oid 23 current snap is 392 2024-09-15T04:01:24.924 INFO:tasks.rados.rados.0.smithi122.stdout:3398: seq_num 1195 ranges {484365=589976,1661982=422383} 2024-09-15T04:01:24.930 INFO:tasks.rados.rados.0.smithi122.stdout:3398: writing smithi12223906-23 from 484365 to 1074341 tid 1 2024-09-15T04:01:24.937 INFO:tasks.rados.rados.0.smithi122.stdout:3398: writing smithi12223906-23 from 1661982 to 2084365 tid 2 2024-09-15T04:01:24.937 INFO:tasks.rados.rados.0.smithi122.stdout:3399: snap_create 2024-09-15T04:01:24.940 INFO:tasks.rados.rados.0.smithi122.stdout:3398: finishing write tid 1 to smithi12223906-23 2024-09-15T04:01:24.945 INFO:tasks.rados.rados.0.smithi122.stdout:3398: finishing write tid 2 to smithi12223906-23 2024-09-15T04:01:24.945 INFO:tasks.rados.rados.0.smithi122.stdout:3398: finishing write tid 3 to smithi12223906-23 2024-09-15T04:01:24.945 INFO:tasks.rados.rados.0.smithi122.stdout:3398: finishing write tid 4 to smithi12223906-23 2024-09-15T04:01:24.945 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2793 (ObjNum 1195 snap 392 seq_num 1195) dirty exists 2024-09-15T04:01:24.946 INFO:tasks.rados.rados.0.smithi122.stdout:3398: left oid 23 (ObjNum 1195 snap 392 seq_num 1195) 2024-09-15T04:01:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3398: done (1 left) 2024-09-15T04:01:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3399: done (0 left) 2024-09-15T04:01:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3400: write oid 44 current snap is 393 2024-09-15T04:01:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3400: seq_num 1196 ranges {501172=665225,1720464=380708} 2024-09-15T04:01:25.937 INFO:tasks.rados.rados.0.smithi122.stdout:3400: writing smithi12223906-44 from 501172 to 1166397 tid 1 2024-09-15T04:01:25.943 INFO:tasks.rados.rados.0.smithi122.stdout:3400: writing smithi12223906-44 from 1720464 to 2101172 tid 2 2024-09-15T04:01:25.943 INFO:tasks.rados.rados.0.smithi122.stdout:3401: write oid 3 current snap is 393 2024-09-15T04:01:25.943 INFO:tasks.rados.rados.0.smithi122.stdout:3401: seq_num 1197 ranges {517979=740474,1778946=339033} 2024-09-15T04:01:25.950 INFO:tasks.rados.rados.0.smithi122.stdout:3401: writing smithi12223906-3 from 517979 to 1258453 tid 1 2024-09-15T04:01:25.955 INFO:tasks.rados.rados.0.smithi122.stdout:3401: writing smithi12223906-3 from 1778946 to 2117979 tid 2 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3402: rollback oid 9 current snap is 393 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 363 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3400: finishing write tid 1 to smithi12223906-44 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3400: finishing write tid 2 to smithi12223906-44 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3403: read oid 4 snap -1 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3403: expect (ObjNum 1180 snap 387 seq_num 1180) 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3400: finishing write tid 3 to smithi12223906-44 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3400: finishing write tid 4 to smithi12223906-44 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 2796 (ObjNum 1196 snap 393 seq_num 1196) dirty exists 2024-09-15T04:01:25.956 INFO:tasks.rados.rados.0.smithi122.stdout:3400: left oid 44 (ObjNum 1196 snap 393 seq_num 1196) 2024-09-15T04:01:25.957 INFO:tasks.rados.rados.0.smithi122.stdout:3400: done (3 left) 2024-09-15T04:01:25.957 INFO:tasks.rados.rados.0.smithi122.stdout:3404: snap_create 2024-09-15T04:01:25.963 INFO:tasks.rados.rados.0.smithi122.stdout:3402: finishing rollback tid 0 to smithi12223906-9 2024-09-15T04:01:25.964 INFO:tasks.rados.rados.0.smithi122.stdout:3402: finishing rollback tid 1 to smithi12223906-9 2024-09-15T04:01:25.964 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2474 (ObjNum 129 snap 0 seq_num 1842469432) dirty dne 2024-09-15T04:01:25.966 INFO:tasks.rados.rados.0.smithi122.stdout:3401: finishing write tid 1 to smithi12223906-3 2024-09-15T04:01:25.966 INFO:tasks.rados.rados.0.smithi122.stdout:3401: finishing write tid 2 to smithi12223906-3 2024-09-15T04:01:25.966 INFO:tasks.rados.rados.0.smithi122.stdout:3401: finishing write tid 3 to smithi12223906-3 2024-09-15T04:01:25.967 INFO:tasks.rados.rados.0.smithi122.stdout:3401: finishing write tid 4 to smithi12223906-3 2024-09-15T04:01:25.967 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 2840 (ObjNum 1197 snap 393 seq_num 1197) dirty exists 2024-09-15T04:01:25.967 INFO:tasks.rados.rados.0.smithi122.stdout:3401: left oid 3 (ObjNum 1197 snap 393 seq_num 1197) 2024-09-15T04:01:25.975 INFO:tasks.rados.rados.0.smithi122.stdout:3403: expect (ObjNum 1180 snap 387 seq_num 1180) 2024-09-15T04:01:26.746 INFO:tasks.rados.rados.0.smithi122.stdout:3401: done (3 left) 2024-09-15T04:01:26.746 INFO:tasks.rados.rados.0.smithi122.stdout:3402: done (2 left) 2024-09-15T04:01:26.746 INFO:tasks.rados.rados.0.smithi122.stdout:3403: done (1 left) 2024-09-15T04:01:26.746 INFO:tasks.rados.rados.0.smithi122.stdout:3404: done (0 left) 2024-09-15T04:01:26.746 INFO:tasks.rados.rados.0.smithi122.stdout:3405: snap_remove snap 363 2024-09-15T04:01:27.759 INFO:tasks.rados.rados.0.smithi122.stdout:3405: done (0 left) 2024-09-15T04:01:27.760 INFO:tasks.rados.rados.0.smithi122.stdout:3406: snap_create 2024-09-15T04:01:28.760 INFO:tasks.rados.rados.0.smithi122.stdout:3406: done (0 left) 2024-09-15T04:01:28.760 INFO:tasks.rados.rados.0.smithi122.stdout:3407: delete oid 17 current snap is 395 2024-09-15T04:01:28.767 INFO:tasks.rados.rados.0.smithi122.stdout:3407: done (0 left) 2024-09-15T04:01:28.767 INFO:tasks.rados.rados.0.smithi122.stdout:3408: read oid 42 snap -1 2024-09-15T04:01:28.767 INFO:tasks.rados.rados.0.smithi122.stdout:3408: expect (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:01:28.767 INFO:tasks.rados.rados.0.smithi122.stdout:3409: write (excl) oid 23 current snap is 395 2024-09-15T04:01:28.767 INFO:tasks.rados.rados.0.smithi122.stdout:3409: seq_num 1199 ranges {551593=490972,1779557=372036} 2024-09-15T04:01:28.776 INFO:tasks.rados.rados.0.smithi122.stdout:3409: writing smithi12223906-23 from 551593 to 1042565 tid 1 2024-09-15T04:01:28.784 INFO:tasks.rados.rados.0.smithi122.stdout:3409: writing smithi12223906-23 from 1779557 to 2151593 tid 2 2024-09-15T04:01:28.784 INFO:tasks.rados.rados.0.smithi122.stdout:3410: copy_from oid 14 from oid 35 current snap is 395 2024-09-15T04:01:28.784 INFO:tasks.rados.rados.0.smithi122.stdout:3408: expect (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:01:28.974 INFO:tasks.rados.rados.0.smithi122.stdout:3409: finishing write tid 1 to smithi12223906-23 2024-09-15T04:01:28.974 INFO:tasks.rados.rados.0.smithi122.stdout:3409: finishing write tid 2 to smithi12223906-23 2024-09-15T04:01:28.974 INFO:tasks.rados.rados.0.smithi122.stdout:3409: finishing write tid 3 to smithi12223906-23 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3409: finishing write tid 4 to smithi12223906-23 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2812 (ObjNum 1199 snap 395 seq_num 1199) dirty exists 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3409: left oid 23 (ObjNum 1199 snap 395 seq_num 1199) 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3410: finishing copy_from racing read to smithi12223906-14 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3410: finishing copy_from to smithi12223906-14 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 1843 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3408: done (2 left) 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3409: done (1 left) 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3410: done (0 left) 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3411: write (excl) oid 35 current snap is 395 2024-09-15T04:01:28.975 INFO:tasks.rados.rados.0.smithi122.stdout:3411: seq_num 1200 ranges {568400=566221,1838039=330361} 2024-09-15T04:01:28.986 INFO:tasks.rados.rados.0.smithi122.stdout:3411: writing smithi12223906-35 from 568400 to 1134621 tid 1 2024-09-15T04:01:28.992 INFO:tasks.rados.rados.0.smithi122.stdout:3411: writing smithi12223906-35 from 1838039 to 2168400 tid 2 2024-09-15T04:01:28.992 INFO:tasks.rados.rados.0.smithi122.stdout:3412: snap_remove snap 315 2024-09-15T04:01:29.787 INFO:tasks.rados.rados.0.smithi122.stdout:3412: done (1 left) 2024-09-15T04:01:29.787 INFO:tasks.rados.rados.0.smithi122.stdout:3413: delete oid 26 current snap is 395 2024-09-15T04:01:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:3411: finishing write tid 1 to smithi12223906-35 2024-09-15T04:01:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:3411: finishing write tid 2 to smithi12223906-35 2024-09-15T04:01:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:3411: finishing write tid 3 to smithi12223906-35 2024-09-15T04:01:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:3411: finishing write tid 4 to smithi12223906-35 2024-09-15T04:01:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1846 (ObjNum 1200 snap 395 seq_num 1200) dirty exists 2024-09-15T04:01:29.788 INFO:tasks.rados.rados.0.smithi122.stdout:3411: left oid 35 (ObjNum 1200 snap 395 seq_num 1200) 2024-09-15T04:01:29.794 INFO:tasks.rados.rados.0.smithi122.stdout:3411: done (1 left) 2024-09-15T04:01:29.794 INFO:tasks.rados.rados.0.smithi122.stdout:3413: done (0 left) 2024-09-15T04:01:29.794 INFO:tasks.rados.rados.0.smithi122.stdout:3414: copy_from oid 41 from oid 20 current snap is 395 2024-09-15T04:01:29.795 INFO:tasks.rados.rados.0.smithi122.stdout:3415: snap_create 2024-09-15T04:01:29.795 INFO:tasks.rados.rados.0.smithi122.stdout:3414: finishing copy_from to smithi12223906-41 2024-09-15T04:01:29.795 INFO:tasks.rados.rados.0.smithi122.stdout:3414: got expected ENOENT (src dne) 2024-09-15T04:01:29.795 INFO:tasks.rados.rados.0.smithi122.stdout:3414: finishing copy_from racing read to smithi12223906-41 2024-09-15T04:01:30.781 INFO:tasks.rados.rados.0.smithi122.stdout:3414: done (1 left) 2024-09-15T04:01:30.781 INFO:tasks.rados.rados.0.smithi122.stdout:3415: done (0 left) 2024-09-15T04:01:30.781 INFO:tasks.rados.rados.0.smithi122.stdout:3416: rollback oid 34 current snap is 396 2024-09-15T04:01:30.781 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 354 2024-09-15T04:01:30.783 INFO:tasks.rados.rados.0.smithi122.stdout:3417: read oid 14 snap 341 2024-09-15T04:01:30.783 INFO:tasks.rados.rados.0.smithi122.stdout:3417: expect (ObjNum 1036 snap 327 seq_num 1036) 2024-09-15T04:01:30.783 INFO:tasks.rados.rados.0.smithi122.stdout:3418: snap_remove snap 385 2024-09-15T04:01:31.749 INFO:tasks.rados.rados.0.smithi122.stdout:3418: done (2 left) 2024-09-15T04:01:31.749 INFO:tasks.rados.rados.0.smithi122.stdout:3419: write (excl) oid 1 current snap is 396 2024-09-15T04:01:31.749 INFO:tasks.rados.rados.0.smithi122.stdout:3419: seq_num 1202 ranges {602014=433072,1671356=530658} 2024-09-15T04:01:31.757 INFO:tasks.rados.rados.0.smithi122.stdout:3419: writing smithi12223906-1 from 602014 to 1035086 tid 1 2024-09-15T04:01:31.764 INFO:tasks.rados.rados.0.smithi122.stdout:3419: writing smithi12223906-1 from 1671356 to 2202014 tid 2 2024-09-15T04:01:31.766 INFO:tasks.rados.rados.0.smithi122.stdout:3416: finishing rollback tid 0 to smithi12223906-34 2024-09-15T04:01:31.766 INFO:tasks.rados.rados.0.smithi122.stdout:3416: finishing rollback tid 1 to smithi12223906-34 2024-09-15T04:01:31.766 INFO:tasks.rados.rados.0.smithi122.stdout:3416: finishing rollback tid 2 to smithi12223906-34 2024-09-15T04:01:31.766 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 2124 (ObjNum 1062 snap 343 seq_num 1062) dirty exists 2024-09-15T04:01:31.766 INFO:tasks.rados.rados.0.smithi122.stdout:3417: expect (ObjNum 1036 snap 327 seq_num 1036) 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:3419: finishing write tid 1 to smithi12223906-1 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:3419: finishing write tid 2 to smithi12223906-1 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:3419: finishing write tid 3 to smithi12223906-1 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:3419: finishing write tid 4 to smithi12223906-1 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1930 (ObjNum 1202 snap 396 seq_num 1202) dirty exists 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:3419: left oid 1 (ObjNum 1202 snap 396 seq_num 1202) 2024-09-15T04:01:31.964 INFO:tasks.rados.rados.0.smithi122.stdout:3416: done (2 left) 2024-09-15T04:01:31.965 INFO:tasks.rados.rados.0.smithi122.stdout:3417: done (1 left) 2024-09-15T04:01:31.965 INFO:tasks.rados.rados.0.smithi122.stdout:3419: done (0 left) 2024-09-15T04:01:31.965 INFO:tasks.rados.rados.0.smithi122.stdout:3420: snap_remove snap 328 2024-09-15T04:01:32.796 INFO:tasks.rados.rados.0.smithi122.stdout:3420: done (0 left) 2024-09-15T04:01:32.796 INFO:tasks.rados.rados.0.smithi122.stdout:3421: read oid 38 snap -1 2024-09-15T04:01:32.796 INFO:tasks.rados.rados.0.smithi122.stdout:3421: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T04:01:32.796 INFO:tasks.rados.rados.0.smithi122.stdout:3422: snap_remove snap 395 2024-09-15T04:01:33.817 INFO:tasks.rados.rados.0.smithi122.stdout:3422: done (1 left) 2024-09-15T04:01:33.817 INFO:tasks.rados.rados.0.smithi122.stdout:3423: copy_from oid 49 from oid 30 current snap is 396 2024-09-15T04:01:33.817 INFO:tasks.rados.rados.0.smithi122.stdout:3421: expect (ObjNum 868 snap 281 seq_num 868) 2024-09-15T04:01:33.907 INFO:tasks.rados.rados.0.smithi122.stdout:3423: finishing copy_from racing read to smithi12223906-49 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:3423: finishing copy_from to smithi12223906-49 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 1178 (ObjNum 1066 snap 345 seq_num 1066) dirty exists 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:3421: done (1 left) 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:3423: done (0 left) 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:3424: read oid 18 snap -1 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:3424: expect (ObjNum 1177 snap 386 seq_num 1177) 2024-09-15T04:01:33.908 INFO:tasks.rados.rados.0.smithi122.stdout:3425: snap_create 2024-09-15T04:01:33.927 INFO:tasks.rados.rados.0.smithi122.stdout:3424: expect (ObjNum 1177 snap 386 seq_num 1177) 2024-09-15T04:01:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:3424: done (1 left) 2024-09-15T04:01:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:3425: done (0 left) 2024-09-15T04:01:34.826 INFO:tasks.rados.rados.0.smithi122.stdout:3426: snap_create 2024-09-15T04:01:35.836 INFO:tasks.rados.rados.0.smithi122.stdout:3426: done (0 left) 2024-09-15T04:01:35.836 INFO:tasks.rados.rados.0.smithi122.stdout:3427: write (excl) oid 36 current snap is 398 2024-09-15T04:01:35.836 INFO:tasks.rados.rados.0.smithi122.stdout:3427: seq_num 1203 ranges {618821=508321,1613485=531077} 2024-09-15T04:01:35.846 INFO:tasks.rados.rados.0.smithi122.stdout:3427: writing smithi12223906-36 from 618821 to 1127142 tid 1 2024-09-15T04:01:35.852 INFO:tasks.rados.rados.0.smithi122.stdout:3427: writing smithi12223906-36 from 1613485 to 2144562 tid 2 2024-09-15T04:01:35.852 INFO:tasks.rados.rados.0.smithi122.stdout:3428: write oid 19 current snap is 398 2024-09-15T04:01:35.852 INFO:tasks.rados.rados.0.smithi122.stdout:3428: seq_num 1204 ranges {635628=583570,1671967=563661} 2024-09-15T04:01:35.859 INFO:tasks.rados.rados.0.smithi122.stdout:3428: writing smithi12223906-19 from 635628 to 1219198 tid 1 2024-09-15T04:01:35.865 INFO:tasks.rados.rados.0.smithi122.stdout:3428: writing smithi12223906-19 from 1671967 to 2235628 tid 2 2024-09-15T04:01:35.865 INFO:tasks.rados.rados.0.smithi122.stdout:3429: write (excl) oid 24 current snap is 398 2024-09-15T04:01:35.865 INFO:tasks.rados.rados.0.smithi122.stdout:3429: seq_num 1205 ranges {652435=658819,1730449=521986} 2024-09-15T04:01:35.872 INFO:tasks.rados.rados.0.smithi122.stdout:3429: writing smithi12223906-24 from 652435 to 1311254 tid 1 2024-09-15T04:01:35.877 INFO:tasks.rados.rados.0.smithi122.stdout:3429: writing smithi12223906-24 from 1730449 to 2252435 tid 2 2024-09-15T04:01:35.877 INFO:tasks.rados.rados.0.smithi122.stdout:3430: snap_remove snap 378 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:3430: done (3 left) 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:3431: rollback oid 34 current snap is 398 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:3427: finishing write tid 1 to smithi12223906-36 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:3427: finishing write tid 2 to smithi12223906-36 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:3427: finishing write tid 3 to smithi12223906-36 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:3427: finishing write tid 4 to smithi12223906-36 2024-09-15T04:01:36.752 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 2891 (ObjNum 1203 snap 398 seq_num 1203) dirty exists 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3427: left oid 36 (ObjNum 1203 snap 398 seq_num 1203) 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3428: finishing write tid 1 to smithi12223906-19 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3428: finishing write tid 2 to smithi12223906-19 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3428: finishing write tid 3 to smithi12223906-19 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3428: finishing write tid 4 to smithi12223906-19 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 2121 (ObjNum 1204 snap 398 seq_num 1204) dirty exists 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3428: left oid 19 (ObjNum 1204 snap 398 seq_num 1204) 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3429: finishing write tid 1 to smithi12223906-24 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3429: finishing write tid 2 to smithi12223906-24 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3429: finishing write tid 3 to smithi12223906-24 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3429: finishing write tid 4 to smithi12223906-24 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 2895 (ObjNum 1205 snap 398 seq_num 1205) dirty exists 2024-09-15T04:01:36.753 INFO:tasks.rados.rados.0.smithi122.stdout:3429: left oid 24 (ObjNum 1205 snap 398 seq_num 1205) 2024-09-15T04:01:36.754 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 34 to 345 2024-09-15T04:01:36.754 INFO:tasks.rados.rados.0.smithi122.stdout:3427: done (3 left) 2024-09-15T04:01:36.754 INFO:tasks.rados.rados.0.smithi122.stdout:3428: done (2 left) 2024-09-15T04:01:36.754 INFO:tasks.rados.rados.0.smithi122.stdout:3429: done (1 left) 2024-09-15T04:01:36.755 INFO:tasks.rados.rados.0.smithi122.stdout:3432: write (excl) oid 9 current snap is 398 2024-09-15T04:01:36.755 INFO:tasks.rados.rados.0.smithi122.stdout:3432: seq_num 1206 ranges {669242=734068,2188931=80311} 2024-09-15T04:01:36.763 INFO:tasks.rados.rados.0.smithi122.stdout:3432: writing smithi12223906-9 from 669242 to 1403310 tid 1 2024-09-15T04:01:36.769 INFO:tasks.rados.rados.0.smithi122.stdout:3432: writing smithi12223906-9 from 2188931 to 2269242 tid 2 2024-09-15T04:01:36.769 INFO:tasks.rados.rados.0.smithi122.stdout:3433: read oid 28 snap 397 2024-09-15T04:01:36.769 INFO:tasks.rados.rados.0.smithi122.stdout:3433: expect (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:01:36.769 INFO:tasks.rados.rados.0.smithi122.stdout:3434: delete oid 21 current snap is 398 2024-09-15T04:01:36.772 INFO:tasks.rados.rados.0.smithi122.stdout:3431: finishing rollback tid 0 to smithi12223906-34 2024-09-15T04:01:36.774 INFO:tasks.rados.rados.0.smithi122.stdout:3431: finishing rollback tid 1 to smithi12223906-34 2024-09-15T04:01:36.774 INFO:tasks.rados.rados.0.smithi122.stdout:3431: finishing rollback tid 2 to smithi12223906-34 2024-09-15T04:01:36.774 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 2127 (ObjNum 1062 snap 343 seq_num 1062) dirty exists 2024-09-15T04:01:36.775 INFO:tasks.rados.rados.0.smithi122.stdout:3432: finishing write tid 1 to smithi12223906-9 2024-09-15T04:01:36.776 INFO:tasks.rados.rados.0.smithi122.stdout:3432: finishing write tid 2 to smithi12223906-9 2024-09-15T04:01:36.780 INFO:tasks.rados.rados.0.smithi122.stdout:3431: done (3 left) 2024-09-15T04:01:36.780 INFO:tasks.rados.rados.0.smithi122.stdout:3434: done (2 left) 2024-09-15T04:01:36.780 INFO:tasks.rados.rados.0.smithi122.stdout:3435: copy_from oid 34 from oid 41 current snap is 398 2024-09-15T04:01:36.780 INFO:tasks.rados.rados.0.smithi122.stdout:3436: snap_create 2024-09-15T04:01:36.781 INFO:tasks.rados.rados.0.smithi122.stdout:3432: finishing write tid 3 to smithi12223906-9 2024-09-15T04:01:36.781 INFO:tasks.rados.rados.0.smithi122.stdout:3432: finishing write tid 4 to smithi12223906-9 2024-09-15T04:01:36.781 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2477 (ObjNum 1206 snap 398 seq_num 1206) dirty exists 2024-09-15T04:01:36.781 INFO:tasks.rados.rados.0.smithi122.stdout:3432: left oid 9 (ObjNum 1206 snap 398 seq_num 1206) 2024-09-15T04:01:36.782 INFO:tasks.rados.rados.0.smithi122.stdout:3435: finishing copy_from to smithi12223906-34 2024-09-15T04:01:36.782 INFO:tasks.rados.rados.0.smithi122.stdout:3435: got expected ENOENT (src dne) 2024-09-15T04:01:36.782 INFO:tasks.rados.rados.0.smithi122.stdout:3435: finishing copy_from racing read to smithi12223906-34 2024-09-15T04:01:36.784 INFO:tasks.rados.rados.0.smithi122.stdout:3433: expect (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:01:37.762 INFO:tasks.rados.rados.0.smithi122.stdout:3432: done (3 left) 2024-09-15T04:01:37.762 INFO:tasks.rados.rados.0.smithi122.stdout:3433: done (2 left) 2024-09-15T04:01:37.762 INFO:tasks.rados.rados.0.smithi122.stdout:3435: done (1 left) 2024-09-15T04:01:37.762 INFO:tasks.rados.rados.0.smithi122.stdout:3436: done (0 left) 2024-09-15T04:01:37.762 INFO:tasks.rados.rados.0.smithi122.stdout:3437: write oid 39 current snap is 399 2024-09-15T04:01:37.762 INFO:tasks.rados.rados.0.smithi122.stdout:3437: seq_num 1208 ranges {702856=484566,1789542=513314} 2024-09-15T04:01:37.771 INFO:tasks.rados.rados.0.smithi122.stdout:3437: writing smithi12223906-39 from 702856 to 1187422 tid 1 2024-09-15T04:01:37.777 INFO:tasks.rados.rados.0.smithi122.stdout:3437: writing smithi12223906-39 from 1789542 to 2302856 tid 2 2024-09-15T04:01:37.778 INFO:tasks.rados.rados.0.smithi122.stdout:3438: snap_remove snap 374 2024-09-15T04:01:38.775 INFO:tasks.rados.rados.0.smithi122.stdout:3438: done (1 left) 2024-09-15T04:01:38.775 INFO:tasks.rados.rados.0.smithi122.stdout:3439: read oid 12 snap -1 2024-09-15T04:01:38.775 INFO:tasks.rados.rados.0.smithi122.stdout:3439: expect (ObjNum 1172 snap 383 seq_num 1172) 2024-09-15T04:01:38.775 INFO:tasks.rados.rados.0.smithi122.stdout:3440: write oid 7 current snap is 399 2024-09-15T04:01:38.775 INFO:tasks.rados.rados.0.smithi122.stdout:3440: seq_num 1209 ranges {719663=676168,1964377=355286} 2024-09-15T04:01:38.785 INFO:tasks.rados.rados.0.smithi122.stdout:3440: writing smithi12223906-7 from 719663 to 1395831 tid 1 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3440: writing smithi12223906-7 from 1964377 to 2319663 tid 2 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3441: read oid 3 snap -1 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3441: expect (ObjNum 1197 snap 393 seq_num 1197) 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3442: snap_create 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3437: finishing write tid 1 to smithi12223906-39 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3437: finishing write tid 2 to smithi12223906-39 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3437: finishing write tid 3 to smithi12223906-39 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3437: finishing write tid 4 to smithi12223906-39 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2909 (ObjNum 1208 snap 399 seq_num 1208) dirty exists 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3437: left oid 39 (ObjNum 1208 snap 399 seq_num 1208) 2024-09-15T04:01:38.792 INFO:tasks.rados.rados.0.smithi122.stdout:3439: expect (ObjNum 1172 snap 383 seq_num 1172) 2024-09-15T04:01:38.962 INFO:tasks.rados.rados.0.smithi122.stdout:3441: expect (ObjNum 1197 snap 393 seq_num 1197) 2024-09-15T04:01:39.101 INFO:tasks.rados.rados.0.smithi122.stdout:3440: finishing write tid 1 to smithi12223906-7 2024-09-15T04:01:39.101 INFO:tasks.rados.rados.0.smithi122.stdout:3440: finishing write tid 2 to smithi12223906-7 2024-09-15T04:01:39.101 INFO:tasks.rados.rados.0.smithi122.stdout:3440: finishing write tid 3 to smithi12223906-7 2024-09-15T04:01:39.101 INFO:tasks.rados.rados.0.smithi122.stdout:3440: finishing write tid 4 to smithi12223906-7 2024-09-15T04:01:39.101 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 2426 (ObjNum 1209 snap 399 seq_num 1209) dirty exists 2024-09-15T04:01:39.101 INFO:tasks.rados.rados.0.smithi122.stdout:3440: left oid 7 (ObjNum 1209 snap 399 seq_num 1209) 2024-09-15T04:01:39.777 INFO:tasks.rados.rados.0.smithi122.stdout:3437: done (4 left) 2024-09-15T04:01:39.777 INFO:tasks.rados.rados.0.smithi122.stdout:3439: done (3 left) 2024-09-15T04:01:39.778 INFO:tasks.rados.rados.0.smithi122.stdout:3440: done (2 left) 2024-09-15T04:01:39.778 INFO:tasks.rados.rados.0.smithi122.stdout:3441: done (1 left) 2024-09-15T04:01:39.778 INFO:tasks.rados.rados.0.smithi122.stdout:3442: done (0 left) 2024-09-15T04:01:39.778 INFO:tasks.rados.rados.0.smithi122.stdout:3443: snap_remove snap 377 2024-09-15T04:01:40.790 INFO:tasks.rados.rados.0.smithi122.stdout:3443: done (0 left) 2024-09-15T04:01:40.790 INFO:tasks.rados.rados.0.smithi122.stdout:3444: read oid 2 snap -1 2024-09-15T04:01:40.790 INFO:tasks.rados.rados.0.smithi122.stdout:3444: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T04:01:40.791 INFO:tasks.rados.rados.0.smithi122.stdout:3445: snap_remove snap 392 2024-09-15T04:01:41.807 INFO:tasks.rados.rados.0.smithi122.stdout:3445: done (1 left) 2024-09-15T04:01:41.807 INFO:tasks.rados.rados.0.smithi122.stdout:3446: snap_remove snap 302 2024-09-15T04:01:42.829 INFO:tasks.rados.rados.0.smithi122.stdout:3446: done (1 left) 2024-09-15T04:01:42.829 INFO:tasks.rados.rados.0.smithi122.stdout:3447: snap_create 2024-09-15T04:01:42.829 INFO:tasks.rados.rados.0.smithi122.stdout:3444: expect (ObjNum 981 snap 310 seq_num 981) 2024-09-15T04:01:43.822 INFO:tasks.rados.rados.0.smithi122.stdout:3444: done (1 left) 2024-09-15T04:01:43.822 INFO:tasks.rados.rados.0.smithi122.stdout:3447: done (0 left) 2024-09-15T04:01:43.822 INFO:tasks.rados.rados.0.smithi122.stdout:3448: rollback oid 1 current snap is 401 2024-09-15T04:01:43.822 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 1 to 339 2024-09-15T04:01:43.823 INFO:tasks.rados.rados.0.smithi122.stdout:3449: copy_from oid 44 from oid 22 current snap is 401 2024-09-15T04:01:43.823 INFO:tasks.rados.rados.0.smithi122.stdout:3450: snap_remove snap 397 2024-09-15T04:01:44.827 INFO:tasks.rados.rados.0.smithi122.stdout:3450: done (2 left) 2024-09-15T04:01:44.827 INFO:tasks.rados.rados.0.smithi122.stdout:3451: delete oid 24 current snap is 401 2024-09-15T04:01:44.827 INFO:tasks.rados.rados.0.smithi122.stdout:3448: finishing rollback tid 0 to smithi12223906-1 2024-09-15T04:01:44.827 INFO:tasks.rados.rados.0.smithi122.stdout:3448: finishing rollback tid 1 to smithi12223906-1 2024-09-15T04:01:44.827 INFO:tasks.rados.rados.0.smithi122.stdout:3448: finishing rollback tid 2 to smithi12223906-1 2024-09-15T04:01:44.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 1 v 1933 (ObjNum 1011 snap 316 seq_num 1011) dirty exists 2024-09-15T04:01:44.828 INFO:tasks.rados.rados.0.smithi122.stdout:3449: finishing copy_from racing read to smithi12223906-44 2024-09-15T04:01:44.828 INFO:tasks.rados.rados.0.smithi122.stdout:3449: finishing copy_from to smithi12223906-44 2024-09-15T04:01:44.828 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 2735 (ObjNum 1183 snap 389 seq_num 1183) dirty exists 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3448: done (2 left) 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3449: done (1 left) 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3451: done (0 left) 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3452: read oid 19 snap -1 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3452: expect (ObjNum 1204 snap 398 seq_num 1204) 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3453: read oid 31 snap -1 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3453: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3454: write (excl) oid 37 current snap is 401 2024-09-15T04:01:44.837 INFO:tasks.rados.rados.0.smithi122.stdout:3454: seq_num 1211 ranges {753277=426666,1681341=545753} 2024-09-15T04:01:44.844 INFO:tasks.rados.rados.0.smithi122.stdout:3454: writing smithi12223906-37 from 753277 to 1179943 tid 1 2024-09-15T04:01:44.852 INFO:tasks.rados.rados.0.smithi122.stdout:3454: writing smithi12223906-37 from 1681341 to 2227094 tid 2 2024-09-15T04:01:44.852 INFO:tasks.rados.rados.0.smithi122.stdout:3455: delete oid 26 current snap is 401 2024-09-15T04:01:44.852 INFO:tasks.rados.rados.0.smithi122.stdout:3452: expect (ObjNum 1204 snap 398 seq_num 1204) 2024-09-15T04:01:44.949 INFO:tasks.rados.rados.0.smithi122.stdout:3453: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:01:45.134 INFO:tasks.rados.rados.0.smithi122.stdout:3454: finishing write tid 1 to smithi12223906-37 2024-09-15T04:01:45.134 INFO:tasks.rados.rados.0.smithi122.stdout:3454: finishing write tid 2 to smithi12223906-37 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3454: finishing write tid 3 to smithi12223906-37 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3454: finishing write tid 4 to smithi12223906-37 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 2132 (ObjNum 1211 snap 401 seq_num 1211) dirty exists 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3454: left oid 37 (ObjNum 1211 snap 401 seq_num 1211) 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3452: done (3 left) 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3453: done (2 left) 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3454: done (1 left) 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3455: done (0 left) 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3456: write oid 19 current snap is 401 2024-09-15T04:01:45.135 INFO:tasks.rados.rados.0.smithi122.stdout:3456: seq_num 1213 ranges {786891=577164,2081952=304939} 2024-09-15T04:01:45.149 INFO:tasks.rados.rados.0.smithi122.stdout:3456: writing smithi12223906-19 from 786891 to 1364055 tid 1 2024-09-15T04:01:45.155 INFO:tasks.rados.rados.0.smithi122.stdout:3456: writing smithi12223906-19 from 2081952 to 2386891 tid 2 2024-09-15T04:01:45.156 INFO:tasks.rados.rados.0.smithi122.stdout:3457: rollback oid 40 current snap is 401 2024-09-15T04:01:45.156 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 389 2024-09-15T04:01:45.156 INFO:tasks.rados.rados.0.smithi122.stdout:3458: delete oid 9 current snap is 401 2024-09-15T04:01:45.163 INFO:tasks.rados.rados.0.smithi122.stdout:3458: done (2 left) 2024-09-15T04:01:45.163 INFO:tasks.rados.rados.0.smithi122.stdout:3459: read oid 44 snap -1 2024-09-15T04:01:45.163 INFO:tasks.rados.rados.0.smithi122.stdout:3459: expect (ObjNum 1183 snap 389 seq_num 1183) 2024-09-15T04:01:45.163 INFO:tasks.rados.rados.0.smithi122.stdout:3460: rollback oid 33 current snap is 401 2024-09-15T04:01:45.163 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 33 to 388 2024-09-15T04:01:45.163 INFO:tasks.rados.rados.0.smithi122.stdout:3461: read oid 15 snap 342 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:3461: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:3462: rollback oid 9 current snap is 401 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 353 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:3463: rollback oid 39 current snap is 401 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 39 to 335 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:3464: rollback oid 35 current snap is 401 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 35 to 394 2024-09-15T04:01:45.164 INFO:tasks.rados.rados.0.smithi122.stdout:3465: delete oid 13 current snap is 401 2024-09-15T04:01:45.167 INFO:tasks.rados.rados.0.smithi122.stdout:3457: finishing rollback tid 0 to smithi12223906-40 2024-09-15T04:01:45.169 INFO:tasks.rados.rados.0.smithi122.stdout:3461: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T04:01:45.471 INFO:tasks.rados.rados.0.smithi122.stdout:3456: finishing write tid 1 to smithi12223906-19 2024-09-15T04:01:45.471 INFO:tasks.rados.rados.0.smithi122.stdout:3462: finishing rollback tid 1 to smithi12223906-9 2024-09-15T04:01:45.471 INFO:tasks.rados.rados.0.smithi122.stdout:3462: finishing rollback tid 2 to smithi12223906-9 2024-09-15T04:01:45.471 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2480 (ObjNum 1078 snap 353 seq_num 1078) dirty exists 2024-09-15T04:01:45.471 INFO:tasks.rados.rados.0.smithi122.stdout:3464: finishing rollback tid 0 to smithi12223906-35 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3464: finishing rollback tid 1 to smithi12223906-35 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3456: finishing write tid 2 to smithi12223906-19 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3464: finishing rollback tid 2 to smithi12223906-35 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1849 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3457: finishing rollback tid 1 to smithi12223906-40 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3461: done (8 left) 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3462: done (7 left) 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3464: done (6 left) 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3465: done (5 left) 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3466: write (excl) oid 22 current snap is 401 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3457: finishing rollback tid 2 to smithi12223906-40 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2228 (ObjNum 1160 snap 381 seq_num 1160) dirty exists 2024-09-15T04:01:45.472 INFO:tasks.rados.rados.0.smithi122.stdout:3463: finishing rollback tid 0 to smithi12223906-39 2024-09-15T04:01:45.473 INFO:tasks.rados.rados.0.smithi122.stdout:3459: expect (ObjNum 1183 snap 389 seq_num 1183) 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3456: finishing write tid 3 to smithi12223906-19 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3456: finishing write tid 4 to smithi12223906-19 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 2124 (ObjNum 1213 snap 401 seq_num 1213) dirty exists 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3456: left oid 19 (ObjNum 1213 snap 401 seq_num 1213) 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3463: finishing rollback tid 1 to smithi12223906-39 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3460: finishing rollback tid 0 to smithi12223906-33 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3460: finishing rollback tid 1 to smithi12223906-33 2024-09-15T04:01:45.587 INFO:tasks.rados.rados.0.smithi122.stdout:3463: finishing rollback tid 2 to smithi12223906-39 2024-09-15T04:01:45.588 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 2955 (ObjNum 1041 snap 330 seq_num 1041) dirty exists 2024-09-15T04:01:45.588 INFO:tasks.rados.rados.0.smithi122.stdout:3460: finishing rollback tid 2 to smithi12223906-33 2024-09-15T04:01:45.588 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2890 (ObjNum 1173 snap 384 seq_num 1173) dirty exists 2024-09-15T04:01:45.588 INFO:tasks.rados.rados.0.smithi122.stdout:3466: seq_num 1216 ranges {437312=402911,1341045=696749} 2024-09-15T04:01:45.600 INFO:tasks.rados.rados.0.smithi122.stdout:3466: writing smithi12223906-22 from 437312 to 840223 tid 1 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3466: writing smithi12223906-22 from 1341045 to 2037794 tid 2 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3456: done (5 left) 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3457: done (4 left) 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3459: done (3 left) 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3460: done (2 left) 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3463: done (1 left) 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3467: copy_from oid 50 from oid 47 current snap is 401 2024-09-15T04:01:45.609 INFO:tasks.rados.rados.0.smithi122.stdout:3468: snap_create 2024-09-15T04:01:45.619 INFO:tasks.rados.rados.0.smithi122.stdout:3466: finishing write tid 1 to smithi12223906-22 2024-09-15T04:01:45.620 INFO:tasks.rados.rados.0.smithi122.stdout:3466: finishing write tid 2 to smithi12223906-22 2024-09-15T04:01:45.620 INFO:tasks.rados.rados.0.smithi122.stdout:3466: finishing write tid 3 to smithi12223906-22 2024-09-15T04:01:45.620 INFO:tasks.rados.rados.0.smithi122.stdout:3466: finishing write tid 4 to smithi12223906-22 2024-09-15T04:01:45.620 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 2894 (ObjNum 1216 snap 401 seq_num 1216) dirty exists 2024-09-15T04:01:45.620 INFO:tasks.rados.rados.0.smithi122.stdout:3466: left oid 22 (ObjNum 1216 snap 401 seq_num 1216) 2024-09-15T04:01:45.635 INFO:tasks.rados.rados.0.smithi122.stdout:3467: finishing copy_from racing read to smithi12223906-50 2024-09-15T04:01:45.637 INFO:tasks.rados.rados.0.smithi122.stdout:3467: finishing copy_from to smithi12223906-50 2024-09-15T04:01:45.637 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2614 (ObjNum 1147 snap 377 seq_num 1147) dirty exists 2024-09-15T04:01:45.862 INFO:tasks.rados.rados.0.smithi122.stdout:3466: done (2 left) 2024-09-15T04:01:45.862 INFO:tasks.rados.rados.0.smithi122.stdout:3467: done (1 left) 2024-09-15T04:01:45.862 INFO:tasks.rados.rados.0.smithi122.stdout:3468: done (0 left) 2024-09-15T04:01:45.862 INFO:tasks.rados.rados.0.smithi122.stdout:3469: delete oid 12 current snap is 402 2024-09-15T04:01:45.868 INFO:tasks.rados.rados.0.smithi122.stdout:3469: done (0 left) 2024-09-15T04:01:45.868 INFO:tasks.rados.rados.0.smithi122.stdout:3470: write oid 18 current snap is 402 2024-09-15T04:01:45.869 INFO:tasks.rados.rados.0.smithi122.stdout:3470: seq_num 1218 ranges {470926=669762,1574362=700418} 2024-09-15T04:01:45.876 INFO:tasks.rados.rados.0.smithi122.stdout:3470: writing smithi12223906-18 from 470926 to 1140688 tid 1 2024-09-15T04:01:45.883 INFO:tasks.rados.rados.0.smithi122.stdout:3470: writing smithi12223906-18 from 1574362 to 2274780 tid 2 2024-09-15T04:01:45.883 INFO:tasks.rados.rados.0.smithi122.stdout:3471: write oid 26 current snap is 402 2024-09-15T04:01:45.883 INFO:tasks.rados.rados.0.smithi122.stdout:3471: seq_num 1219 ranges {487733=745011,1632844=560429} 2024-09-15T04:01:45.889 INFO:tasks.rados.rados.0.smithi122.stdout:3471: writing smithi12223906-26 from 487733 to 1232744 tid 1 2024-09-15T04:01:45.895 INFO:tasks.rados.rados.0.smithi122.stdout:3471: writing smithi12223906-26 from 1632844 to 2193273 tid 2 2024-09-15T04:01:45.895 INFO:tasks.rados.rados.0.smithi122.stdout:3472: snap_remove snap 398 2024-09-15T04:01:46.759 INFO:tasks.rados.rados.0.smithi122.stdout:3472: done (2 left) 2024-09-15T04:01:46.759 INFO:tasks.rados.rados.0.smithi122.stdout:3473: copy_from oid 46 from oid 21 current snap is 402 2024-09-15T04:01:46.759 INFO:tasks.rados.rados.0.smithi122.stdout:3474: read oid 31 snap -1 2024-09-15T04:01:46.759 INFO:tasks.rados.rados.0.smithi122.stdout:3474: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:01:46.760 INFO:tasks.rados.rados.0.smithi122.stdout:3475: snap_remove snap 352 2024-09-15T04:01:47.775 INFO:tasks.rados.rados.0.smithi122.stdout:3475: done (4 left) 2024-09-15T04:01:47.775 INFO:tasks.rados.rados.0.smithi122.stdout:3476: copy_from oid 45 from oid 8 current snap is 402 2024-09-15T04:01:47.775 INFO:tasks.rados.rados.0.smithi122.stdout:3477: snap_remove snap 387 2024-09-15T04:01:48.778 INFO:tasks.rados.rados.0.smithi122.stdout:3477: done (5 left) 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3478: rollback oid 2 current snap is 402 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 2 to 401 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3479: delete oid 13 current snap is 402 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3470: finishing write tid 1 to smithi12223906-18 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3470: finishing write tid 2 to smithi12223906-18 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3470: finishing write tid 3 to smithi12223906-18 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3470: finishing write tid 4 to smithi12223906-18 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2900 (ObjNum 1218 snap 402 seq_num 1218) dirty exists 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3470: left oid 18 (ObjNum 1218 snap 402 seq_num 1218) 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3471: finishing write tid 1 to smithi12223906-26 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3471: finishing write tid 2 to smithi12223906-26 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3471: finishing write tid 3 to smithi12223906-26 2024-09-15T04:01:48.779 INFO:tasks.rados.rados.0.smithi122.stdout:3471: finishing write tid 4 to smithi12223906-26 2024-09-15T04:01:48.780 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 2135 (ObjNum 1219 snap 402 seq_num 1219) dirty exists 2024-09-15T04:01:48.780 INFO:tasks.rados.rados.0.smithi122.stdout:3471: left oid 26 (ObjNum 1219 snap 402 seq_num 1219) 2024-09-15T04:01:48.780 INFO:tasks.rados.rados.0.smithi122.stdout:3473: finishing copy_from to smithi12223906-46 2024-09-15T04:01:48.780 INFO:tasks.rados.rados.0.smithi122.stdout:3473: got expected ENOENT (src dne) 2024-09-15T04:01:48.780 INFO:tasks.rados.rados.0.smithi122.stdout:3473: finishing copy_from racing read to smithi12223906-46 2024-09-15T04:01:48.780 INFO:tasks.rados.rados.0.smithi122.stdout:3474: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:01:48.966 INFO:tasks.rados.rados.0.smithi122.stdout:3476: finishing copy_from racing read to smithi12223906-45 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3476: finishing copy_from to smithi12223906-45 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 1926 (ObjNum 1164 snap 382 seq_num 1164) dirty exists 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3478: finishing rollback tid 0 to smithi12223906-2 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3478: finishing rollback tid 1 to smithi12223906-2 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3478: finishing rollback tid 2 to smithi12223906-2 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 1181 (ObjNum 981 snap 310 seq_num 981) dirty exists 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3470: done (6 left) 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3471: done (5 left) 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3473: done (4 left) 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3474: done (3 left) 2024-09-15T04:01:48.967 INFO:tasks.rados.rados.0.smithi122.stdout:3476: done (2 left) 2024-09-15T04:01:48.968 INFO:tasks.rados.rados.0.smithi122.stdout:3478: done (1 left) 2024-09-15T04:01:48.968 INFO:tasks.rados.rados.0.smithi122.stdout:3479: done (0 left) 2024-09-15T04:01:48.968 INFO:tasks.rados.rados.0.smithi122.stdout:3480: write (excl) oid 45 current snap is 402 2024-09-15T04:01:48.968 INFO:tasks.rados.rados.0.smithi122.stdout:3480: seq_num 1221 ranges {521347=495509,1633455=564098} 2024-09-15T04:01:48.980 INFO:tasks.rados.rados.0.smithi122.stdout:3480: writing smithi12223906-45 from 521347 to 1016856 tid 1 2024-09-15T04:01:48.986 INFO:tasks.rados.rados.0.smithi122.stdout:3480: writing smithi12223906-45 from 1633455 to 2197553 tid 2 2024-09-15T04:01:48.987 INFO:tasks.rados.rados.0.smithi122.stdout:3481: write oid 4 current snap is 402 2024-09-15T04:01:48.987 INFO:tasks.rados.rados.0.smithi122.stdout:3481: seq_num 1222 ranges {538154=570758,1691937=707756} 2024-09-15T04:01:48.993 INFO:tasks.rados.rados.0.smithi122.stdout:3481: writing smithi12223906-4 from 538154 to 1108912 tid 1 2024-09-15T04:01:49.001 INFO:tasks.rados.rados.0.smithi122.stdout:3481: writing smithi12223906-4 from 1691937 to 2399693 tid 2 2024-09-15T04:01:49.001 INFO:tasks.rados.rados.0.smithi122.stdout:3482: delete oid 15 current snap is 402 2024-09-15T04:01:49.001 INFO:tasks.rados.rados.0.smithi122.stdout:3480: finishing write tid 1 to smithi12223906-45 2024-09-15T04:01:49.001 INFO:tasks.rados.rados.0.smithi122.stdout:3480: finishing write tid 2 to smithi12223906-45 2024-09-15T04:01:49.002 INFO:tasks.rados.rados.0.smithi122.stdout:3480: finishing write tid 3 to smithi12223906-45 2024-09-15T04:01:49.002 INFO:tasks.rados.rados.0.smithi122.stdout:3480: finishing write tid 4 to smithi12223906-45 2024-09-15T04:01:49.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 2127 (ObjNum 1221 snap 402 seq_num 1221) dirty exists 2024-09-15T04:01:49.002 INFO:tasks.rados.rados.0.smithi122.stdout:3480: left oid 45 (ObjNum 1221 snap 402 seq_num 1221) 2024-09-15T04:01:49.005 INFO:tasks.rados.rados.0.smithi122.stdout:3480: done (2 left) 2024-09-15T04:01:49.005 INFO:tasks.rados.rados.0.smithi122.stdout:3482: done (1 left) 2024-09-15T04:01:49.005 INFO:tasks.rados.rados.0.smithi122.stdout:3483: read oid 8 snap 326 2024-09-15T04:01:49.005 INFO:tasks.rados.rados.0.smithi122.stdout:3483: expect (ObjNum 958 snap 304 seq_num 958) 2024-09-15T04:01:49.006 INFO:tasks.rados.rados.0.smithi122.stdout:3484: read oid 5 snap -1 2024-09-15T04:01:49.006 INFO:tasks.rados.rados.0.smithi122.stdout:3484: expect deleted 2024-09-15T04:01:49.006 INFO:tasks.rados.rados.0.smithi122.stdout:3485: write oid 10 current snap is 402 2024-09-15T04:01:49.006 INFO:tasks.rados.rados.0.smithi122.stdout:3485: seq_num 1224 ranges {571768=437609,1808901=711425} 2024-09-15T04:01:49.011 INFO:tasks.rados.rados.0.smithi122.stdout:3485: writing smithi12223906-10 from 571768 to 1009377 tid 1 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3485: writing smithi12223906-10 from 1808901 to 2520326 tid 2 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3486: delete oid 24 current snap is 402 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3481: finishing write tid 1 to smithi12223906-4 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3481: finishing write tid 2 to smithi12223906-4 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3481: finishing write tid 3 to smithi12223906-4 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3481: finishing write tid 4 to smithi12223906-4 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2622 (ObjNum 1222 snap 402 seq_num 1222) dirty exists 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3481: left oid 4 (ObjNum 1222 snap 402 seq_num 1222) 2024-09-15T04:01:49.021 INFO:tasks.rados.rados.0.smithi122.stdout:3483: expect (ObjNum 958 snap 304 seq_num 958) 2024-09-15T04:01:49.092 INFO:tasks.rados.rados.0.smithi122.stdout:3485: finishing write tid 1 to smithi12223906-10 2024-09-15T04:01:49.092 INFO:tasks.rados.rados.0.smithi122.stdout:3485: finishing write tid 2 to smithi12223906-10 2024-09-15T04:01:49.093 INFO:tasks.rados.rados.0.smithi122.stdout:3485: finishing write tid 3 to smithi12223906-10 2024-09-15T04:01:49.093 INFO:tasks.rados.rados.0.smithi122.stdout:3485: finishing write tid 4 to smithi12223906-10 2024-09-15T04:01:49.093 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 2135 (ObjNum 1224 snap 402 seq_num 1224) dirty exists 2024-09-15T04:01:49.093 INFO:tasks.rados.rados.0.smithi122.stdout:3485: left oid 10 (ObjNum 1224 snap 402 seq_num 1224) 2024-09-15T04:01:49.093 INFO:tasks.rados.rados.0.smithi122.stdout:3481: done (4 left) 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3483: done (3 left) 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3484: done (2 left) 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3485: done (1 left) 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3486: done (0 left) 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3487: read oid 46 snap 345 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3487: expect (ObjNum 1053 snap 339 seq_num 1053) 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3488: write oid 18 current snap is 402 2024-09-15T04:01:49.094 INFO:tasks.rados.rados.0.smithi122.stdout:3488: seq_num 1226 ranges {605382=588107,1925865=679517} 2024-09-15T04:01:49.100 INFO:tasks.rados.rados.0.smithi122.stdout:3488: writing smithi12223906-18 from 605382 to 1193489 tid 1 2024-09-15T04:01:49.109 INFO:tasks.rados.rados.0.smithi122.stdout:3488: writing smithi12223906-18 from 1925865 to 2605382 tid 2 2024-09-15T04:01:49.110 INFO:tasks.rados.rados.0.smithi122.stdout:3489: read oid 9 snap -1 2024-09-15T04:01:49.110 INFO:tasks.rados.rados.0.smithi122.stdout:3489: expect (ObjNum 1078 snap 353 seq_num 1078) 2024-09-15T04:01:49.110 INFO:tasks.rados.rados.0.smithi122.stdout:3490: read oid 28 snap -1 2024-09-15T04:01:49.110 INFO:tasks.rados.rados.0.smithi122.stdout:3490: expect (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:01:49.110 INFO:tasks.rados.rados.0.smithi122.stdout:3491: snap_create 2024-09-15T04:01:49.115 INFO:tasks.rados.rados.0.smithi122.stdout:3487: expect (ObjNum 1053 snap 339 seq_num 1053) 2024-09-15T04:01:49.394 INFO:tasks.rados.rados.0.smithi122.stdout:3488: finishing write tid 1 to smithi12223906-18 2024-09-15T04:01:49.394 INFO:tasks.rados.rados.0.smithi122.stdout:3490: expect (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:01:49.591 INFO:tasks.rados.rados.0.smithi122.stdout:3488: finishing write tid 2 to smithi12223906-18 2024-09-15T04:01:49.592 INFO:tasks.rados.rados.0.smithi122.stdout:3488: finishing write tid 3 to smithi12223906-18 2024-09-15T04:01:49.592 INFO:tasks.rados.rados.0.smithi122.stdout:3488: finishing write tid 4 to smithi12223906-18 2024-09-15T04:01:49.592 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 2933 (ObjNum 1226 snap 402 seq_num 1226) dirty exists 2024-09-15T04:01:49.592 INFO:tasks.rados.rados.0.smithi122.stdout:3488: left oid 18 (ObjNum 1226 snap 402 seq_num 1226) 2024-09-15T04:01:49.592 INFO:tasks.rados.rados.0.smithi122.stdout:3489: expect (ObjNum 1078 snap 353 seq_num 1078) 2024-09-15T04:01:49.796 INFO:tasks.rados.rados.0.smithi122.stdout:3487: done (4 left) 2024-09-15T04:01:49.796 INFO:tasks.rados.rados.0.smithi122.stdout:3488: done (3 left) 2024-09-15T04:01:49.796 INFO:tasks.rados.rados.0.smithi122.stdout:3489: done (2 left) 2024-09-15T04:01:49.797 INFO:tasks.rados.rados.0.smithi122.stdout:3490: done (1 left) 2024-09-15T04:01:49.797 INFO:tasks.rados.rados.0.smithi122.stdout:3491: done (0 left) 2024-09-15T04:01:49.797 INFO:tasks.rados.rados.0.smithi122.stdout:3492: snap_create 2024-09-15T04:01:50.800 INFO:tasks.rados.rados.0.smithi122.stdout:3492: done (0 left) 2024-09-15T04:01:50.800 INFO:tasks.rados.rados.0.smithi122.stdout:3493: read oid 16 snap -1 2024-09-15T04:01:50.800 INFO:tasks.rados.rados.0.smithi122.stdout:3493: expect (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:01:50.800 INFO:tasks.rados.rados.0.smithi122.stdout:3494: delete oid 1 current snap is 404 2024-09-15T04:01:50.811 INFO:tasks.rados.rados.0.smithi122.stdout:3493: expect (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:01:51.003 INFO:tasks.rados.rados.0.smithi122.stdout:3493: done (1 left) 2024-09-15T04:01:51.003 INFO:tasks.rados.rados.0.smithi122.stdout:3494: done (0 left) 2024-09-15T04:01:51.003 INFO:tasks.rados.rados.0.smithi122.stdout:3495: write (excl) oid 45 current snap is 404 2024-09-15T04:01:51.004 INFO:tasks.rados.rados.0.smithi122.stdout:3495: seq_num 1228 ranges {638996=738605,1926476=712520} 2024-09-15T04:01:51.021 INFO:tasks.rados.rados.0.smithi122.stdout:3495: writing smithi12223906-45 from 638996 to 1377601 tid 1 2024-09-15T04:01:51.028 INFO:tasks.rados.rados.0.smithi122.stdout:3495: writing smithi12223906-45 from 1926476 to 2638996 tid 2 2024-09-15T04:01:51.029 INFO:tasks.rados.rados.0.smithi122.stdout:3496: write (excl) oid 29 current snap is 404 2024-09-15T04:01:51.029 INFO:tasks.rados.rados.0.smithi122.stdout:3496: seq_num 1229 ranges {655803=413854,1584958=462421} 2024-09-15T04:01:51.034 INFO:tasks.rados.rados.0.smithi122.stdout:3496: writing smithi12223906-29 from 655803 to 1069657 tid 1 2024-09-15T04:01:51.039 INFO:tasks.rados.rados.0.smithi122.stdout:3496: writing smithi12223906-29 from 1584958 to 2047379 tid 2 2024-09-15T04:01:51.040 INFO:tasks.rados.rados.0.smithi122.stdout:3497: snap_remove snap 401 2024-09-15T04:01:51.763 INFO:tasks.rados.rados.0.smithi122.stdout:3497: done (2 left) 2024-09-15T04:01:51.763 INFO:tasks.rados.rados.0.smithi122.stdout:3498: write (excl) oid 25 current snap is 404 2024-09-15T04:01:51.763 INFO:tasks.rados.rados.0.smithi122.stdout:3495: finishing write tid 1 to smithi12223906-45 2024-09-15T04:01:51.763 INFO:tasks.rados.rados.0.smithi122.stdout:3498: seq_num 1230 ranges {672610=489103,1643440=722432} 2024-09-15T04:01:51.769 INFO:tasks.rados.rados.0.smithi122.stdout:3498: writing smithi12223906-25 from 672610 to 1161713 tid 1 2024-09-15T04:01:51.776 INFO:tasks.rados.rados.0.smithi122.stdout:3498: writing smithi12223906-25 from 1643440 to 2365872 tid 2 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3499: write (excl) oid 28 current snap is 404 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3495: finishing write tid 2 to smithi12223906-45 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3495: finishing write tid 3 to smithi12223906-45 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3495: finishing write tid 4 to smithi12223906-45 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 2130 (ObjNum 1228 snap 404 seq_num 1228) dirty exists 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3495: left oid 45 (ObjNum 1228 snap 404 seq_num 1228) 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3496: finishing write tid 1 to smithi12223906-29 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3496: finishing write tid 2 to smithi12223906-29 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3496: finishing write tid 3 to smithi12223906-29 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3496: finishing write tid 4 to smithi12223906-29 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 29 v 2429 (ObjNum 1229 snap 404 seq_num 1229) dirty exists 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3496: left oid 29 (ObjNum 1229 snap 404 seq_num 1229) 2024-09-15T04:01:51.777 INFO:tasks.rados.rados.0.smithi122.stdout:3498: finishing write tid 1 to smithi12223906-25 2024-09-15T04:01:51.778 INFO:tasks.rados.rados.0.smithi122.stdout:3499: seq_num 1231 ranges {689417=564352,1701922=466090} 2024-09-15T04:01:51.783 INFO:tasks.rados.rados.0.smithi122.stdout:3499: writing smithi12223906-28 from 689417 to 1253769 tid 1 2024-09-15T04:01:51.789 INFO:tasks.rados.rados.0.smithi122.stdout:3499: writing smithi12223906-28 from 1701922 to 2168012 tid 2 2024-09-15T04:01:51.789 INFO:tasks.rados.rados.0.smithi122.stdout:3495: done (3 left) 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3496: done (2 left) 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3500: copy_from oid 6 from oid 19 current snap is 404 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3501: rollback oid 32 current snap is 404 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 339 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3498: finishing write tid 2 to smithi12223906-25 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3498: finishing write tid 3 to smithi12223906-25 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3498: finishing write tid 4 to smithi12223906-25 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2988 (ObjNum 1230 snap 404 seq_num 1230) dirty exists 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3498: left oid 25 (ObjNum 1230 snap 404 seq_num 1230) 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3498: done (3 left) 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3502: write oid 24 current snap is 404 2024-09-15T04:01:51.790 INFO:tasks.rados.rados.0.smithi122.stdout:3502: seq_num 1232 ranges {706224=755954,2160404=545820} 2024-09-15T04:01:51.797 INFO:tasks.rados.rados.0.smithi122.stdout:3502: writing smithi12223906-24 from 706224 to 1462178 tid 1 2024-09-15T04:01:51.805 INFO:tasks.rados.rados.0.smithi122.stdout:3502: writing smithi12223906-24 from 2160404 to 2706224 tid 2 2024-09-15T04:01:51.805 INFO:tasks.rados.rados.0.smithi122.stdout:3503: delete oid 23 current snap is 404 2024-09-15T04:01:51.805 INFO:tasks.rados.rados.0.smithi122.stdout:3499: finishing write tid 1 to smithi12223906-28 2024-09-15T04:01:51.805 INFO:tasks.rados.rados.0.smithi122.stdout:3499: finishing write tid 2 to smithi12223906-28 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3499: finishing write tid 3 to smithi12223906-28 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3499: finishing write tid 4 to smithi12223906-28 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 2432 (ObjNum 1231 snap 404 seq_num 1231) dirty exists 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3499: left oid 28 (ObjNum 1231 snap 404 seq_num 1231) 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3500: finishing copy_from racing read to smithi12223906-6 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3501: finishing rollback tid 0 to smithi12223906-32 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3501: finishing rollback tid 1 to smithi12223906-32 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:3501: finishing rollback tid 2 to smithi12223906-32 2024-09-15T04:01:51.806 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 2939 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:01:51.810 INFO:tasks.rados.rados.0.smithi122.stdout:3500: finishing copy_from to smithi12223906-6 2024-09-15T04:01:51.810 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2124 (ObjNum 1213 snap 401 seq_num 1213) dirty exists 2024-09-15T04:01:51.814 INFO:tasks.rados.rados.0.smithi122.stdout:3502: finishing write tid 1 to smithi12223906-24 2024-09-15T04:01:51.814 INFO:tasks.rados.rados.0.smithi122.stdout:3499: done (4 left) 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3500: done (3 left) 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3501: done (2 left) 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3503: done (1 left) 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3504: copy_from oid 2 from oid 37 current snap is 404 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3505: delete oid 6 current snap is 404 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3502: finishing write tid 2 to smithi12223906-24 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3502: finishing write tid 3 to smithi12223906-24 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3502: finishing write tid 4 to smithi12223906-24 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 2991 (ObjNum 1232 snap 404 seq_num 1232) dirty exists 2024-09-15T04:01:51.815 INFO:tasks.rados.rados.0.smithi122.stdout:3502: left oid 24 (ObjNum 1232 snap 404 seq_num 1232) 2024-09-15T04:01:51.817 INFO:tasks.rados.rados.0.smithi122.stdout:3502: done (2 left) 2024-09-15T04:01:51.817 INFO:tasks.rados.rados.0.smithi122.stdout:3505: done (1 left) 2024-09-15T04:01:51.818 INFO:tasks.rados.rados.0.smithi122.stdout:3506: write oid 6 current snap is 404 2024-09-15T04:01:51.818 INFO:tasks.rados.rados.0.smithi122.stdout:3506: seq_num 1235 ranges {756645=581701,1935850=473428} 2024-09-15T04:01:51.827 INFO:tasks.rados.rados.0.smithi122.stdout:3506: writing smithi12223906-6 from 756645 to 1338346 tid 1 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3506: writing smithi12223906-6 from 1935850 to 2409278 tid 2 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3507: read oid 14 snap -1 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3507: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3504: finishing copy_from racing read to smithi12223906-2 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3508: read oid 28 snap -1 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3508: expect (ObjNum 1231 snap 404 seq_num 1231) 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3504: finishing copy_from to smithi12223906-2 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 2132 (ObjNum 1211 snap 401 seq_num 1211) dirty exists 2024-09-15T04:01:51.836 INFO:tasks.rados.rados.0.smithi122.stdout:3506: finishing write tid 1 to smithi12223906-6 2024-09-15T04:01:51.837 INFO:tasks.rados.rados.0.smithi122.stdout:3509: read oid 30 snap -1 2024-09-15T04:01:51.837 INFO:tasks.rados.rados.0.smithi122.stdout:3509: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:01:51.837 INFO:tasks.rados.rados.0.smithi122.stdout:3504: done (4 left) 2024-09-15T04:01:51.837 INFO:tasks.rados.rados.0.smithi122.stdout:3510: delete oid 18 current snap is 404 2024-09-15T04:01:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:3510: done (4 left) 2024-09-15T04:01:51.843 INFO:tasks.rados.rados.0.smithi122.stdout:3511: snap_create 2024-09-15T04:01:51.844 INFO:tasks.rados.rados.0.smithi122.stdout:3508: expect (ObjNum 1231 snap 404 seq_num 1231) 2024-09-15T04:01:52.006 INFO:tasks.rados.rados.0.smithi122.stdout:3506: finishing write tid 2 to smithi12223906-6 2024-09-15T04:01:52.006 INFO:tasks.rados.rados.0.smithi122.stdout:3506: finishing write tid 3 to smithi12223906-6 2024-09-15T04:01:52.006 INFO:tasks.rados.rados.0.smithi122.stdout:3506: finishing write tid 4 to smithi12223906-6 2024-09-15T04:01:52.006 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2484 (ObjNum 1235 snap 404 seq_num 1235) dirty exists 2024-09-15T04:01:52.006 INFO:tasks.rados.rados.0.smithi122.stdout:3506: left oid 6 (ObjNum 1235 snap 404 seq_num 1235) 2024-09-15T04:01:52.006 INFO:tasks.rados.rados.0.smithi122.stdout:3509: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:01:52.130 INFO:tasks.rados.rados.0.smithi122.stdout:3507: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T04:01:52.815 INFO:tasks.rados.rados.0.smithi122.stdout:3506: done (4 left) 2024-09-15T04:01:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:3507: done (3 left) 2024-09-15T04:01:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:3508: done (2 left) 2024-09-15T04:01:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:3509: done (1 left) 2024-09-15T04:01:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:3511: done (0 left) 2024-09-15T04:01:52.816 INFO:tasks.rados.rados.0.smithi122.stdout:3512: delete oid 44 current snap is 405 2024-09-15T04:01:52.822 INFO:tasks.rados.rados.0.smithi122.stdout:3512: done (0 left) 2024-09-15T04:01:52.822 INFO:tasks.rados.rados.0.smithi122.stdout:3513: read oid 39 snap -1 2024-09-15T04:01:52.823 INFO:tasks.rados.rados.0.smithi122.stdout:3513: expect (ObjNum 1041 snap 330 seq_num 1041) 2024-09-15T04:01:52.823 INFO:tasks.rados.rados.0.smithi122.stdout:3514: write oid 2 current snap is 405 2024-09-15T04:01:52.823 INFO:tasks.rados.rados.0.smithi122.stdout:3514: seq_num 1238 ranges {407066=407448,1594943=620755} 2024-09-15T04:01:52.832 INFO:tasks.rados.rados.0.smithi122.stdout:3514: writing smithi12223906-2 from 407066 to 814514 tid 1 2024-09-15T04:01:52.842 INFO:tasks.rados.rados.0.smithi122.stdout:3514: writing smithi12223906-2 from 1594943 to 2215698 tid 2 2024-09-15T04:01:52.842 INFO:tasks.rados.rados.0.smithi122.stdout:3515: snap_remove snap 372 2024-09-15T04:01:53.839 INFO:tasks.rados.rados.0.smithi122.stdout:3515: done (2 left) 2024-09-15T04:01:53.839 INFO:tasks.rados.rados.0.smithi122.stdout:3516: write (excl) oid 8 current snap is 405 2024-09-15T04:01:53.839 INFO:tasks.rados.rados.0.smithi122.stdout:3516: seq_num 1239 ranges {423873=482697,1653425=480766,2691441=132432} 2024-09-15T04:01:53.844 INFO:tasks.rados.rados.0.smithi122.stdout:3516: writing smithi12223906-8 from 423873 to 906570 tid 1 2024-09-15T04:01:53.852 INFO:tasks.rados.rados.0.smithi122.stdout:3516: writing smithi12223906-8 from 1653425 to 2134191 tid 2 2024-09-15T04:01:53.856 INFO:tasks.rados.rados.0.smithi122.stdout:3516: writing smithi12223906-8 from 2691441 to 2823873 tid 3 2024-09-15T04:01:53.856 INFO:tasks.rados.rados.0.smithi122.stdout:3517: snap_create 2024-09-15T04:01:53.856 INFO:tasks.rados.rados.0.smithi122.stdout:3513: expect (ObjNum 1041 snap 330 seq_num 1041) 2024-09-15T04:01:53.972 INFO:tasks.rados.rados.0.smithi122.stdout:3514: finishing write tid 1 to smithi12223906-2 2024-09-15T04:01:53.972 INFO:tasks.rados.rados.0.smithi122.stdout:3514: finishing write tid 2 to smithi12223906-2 2024-09-15T04:01:53.972 INFO:tasks.rados.rados.0.smithi122.stdout:3514: finishing write tid 3 to smithi12223906-2 2024-09-15T04:01:53.972 INFO:tasks.rados.rados.0.smithi122.stdout:3514: finishing write tid 4 to smithi12223906-2 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 2135 (ObjNum 1238 snap 405 seq_num 1238) dirty exists 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3514: left oid 2 (ObjNum 1238 snap 405 seq_num 1238) 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3516: finishing write tid 1 to smithi12223906-8 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3516: finishing write tid 2 to smithi12223906-8 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3516: finishing write tid 3 to smithi12223906-8 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3516: finishing write tid 4 to smithi12223906-8 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3516: finishing write tid 5 to smithi12223906-8 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 2139 (ObjNum 1239 snap 405 seq_num 1239) dirty exists 2024-09-15T04:01:53.973 INFO:tasks.rados.rados.0.smithi122.stdout:3516: left oid 8 (ObjNum 1239 snap 405 seq_num 1239) 2024-09-15T04:01:54.837 INFO:tasks.rados.rados.0.smithi122.stdout:3513: done (3 left) 2024-09-15T04:01:54.837 INFO:tasks.rados.rados.0.smithi122.stdout:3514: done (2 left) 2024-09-15T04:01:54.837 INFO:tasks.rados.rados.0.smithi122.stdout:3516: done (1 left) 2024-09-15T04:01:54.837 INFO:tasks.rados.rados.0.smithi122.stdout:3517: done (0 left) 2024-09-15T04:01:54.837 INFO:tasks.rados.rados.0.smithi122.stdout:3518: write (excl) oid 4 current snap is 406 2024-09-15T04:01:54.837 INFO:tasks.rados.rados.0.smithi122.stdout:3518: seq_num 1240 ranges {440680=674299,1711907=624424} 2024-09-15T04:01:54.846 INFO:tasks.rados.rados.0.smithi122.stdout:3518: writing smithi12223906-4 from 440680 to 1114979 tid 1 2024-09-15T04:01:54.854 INFO:tasks.rados.rados.0.smithi122.stdout:3518: writing smithi12223906-4 from 1711907 to 2336331 tid 2 2024-09-15T04:01:54.854 INFO:tasks.rados.rados.0.smithi122.stdout:3519: delete oid 44 current snap is 406 2024-09-15T04:01:54.859 INFO:tasks.rados.rados.0.smithi122.stdout:3518: finishing write tid 1 to smithi12223906-4 2024-09-15T04:01:54.860 INFO:tasks.rados.rados.0.smithi122.stdout:3519: done (1 left) 2024-09-15T04:01:54.860 INFO:tasks.rados.rados.0.smithi122.stdout:3520: write (excl) oid 12 current snap is 406 2024-09-15T04:01:54.860 INFO:tasks.rados.rados.0.smithi122.stdout:3520: seq_num 1242 ranges {474294=424797,1428871=628093,2657357=216937} 2024-09-15T04:01:54.870 INFO:tasks.rados.rados.0.smithi122.stdout:3520: writing smithi12223906-12 from 474294 to 899091 tid 1 2024-09-15T04:01:54.879 INFO:tasks.rados.rados.0.smithi122.stdout:3520: writing smithi12223906-12 from 1428871 to 2056964 tid 2 2024-09-15T04:01:54.883 INFO:tasks.rados.rados.0.smithi122.stdout:3520: writing smithi12223906-12 from 2657357 to 2874294 tid 3 2024-09-15T04:01:54.884 INFO:tasks.rados.rados.0.smithi122.stdout:3521: snap_remove snap 373 2024-09-15T04:01:55.860 INFO:tasks.rados.rados.0.smithi122.stdout:3518: finishing write tid 2 to smithi12223906-4 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3518: finishing write tid 3 to smithi12223906-4 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3518: finishing write tid 4 to smithi12223906-4 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2625 (ObjNum 1240 snap 406 seq_num 1240) dirty exists 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3518: left oid 4 (ObjNum 1240 snap 406 seq_num 1240) 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3520: finishing write tid 1 to smithi12223906-12 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3520: finishing write tid 2 to smithi12223906-12 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3520: finishing write tid 3 to smithi12223906-12 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3520: finishing write tid 4 to smithi12223906-12 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3520: finishing write tid 5 to smithi12223906-12 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 2132 (ObjNum 1242 snap 406 seq_num 1242) dirty exists 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3520: left oid 12 (ObjNum 1242 snap 406 seq_num 1242) 2024-09-15T04:01:55.861 INFO:tasks.rados.rados.0.smithi122.stdout:3518: done (2 left) 2024-09-15T04:01:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:3520: done (1 left) 2024-09-15T04:01:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:3521: done (0 left) 2024-09-15T04:01:55.862 INFO:tasks.rados.rados.0.smithi122.stdout:3522: snap_create 2024-09-15T04:01:56.760 INFO:tasks.rados.rados.0.smithi122.stdout:3522: done (0 left) 2024-09-15T04:01:56.761 INFO:tasks.rados.rados.0.smithi122.stdout:3523: write (excl) oid 26 current snap is 407 2024-09-15T04:01:56.761 INFO:tasks.rados.rados.0.smithi122.stdout:3523: seq_num 1243 ranges {491101=500046,1487353=488104,2401133=489968} 2024-09-15T04:01:56.769 INFO:tasks.rados.rados.0.smithi122.stdout:3523: writing smithi12223906-26 from 491101 to 991147 tid 1 2024-09-15T04:01:56.774 INFO:tasks.rados.rados.0.smithi122.stdout:3523: writing smithi12223906-26 from 1487353 to 1975457 tid 2 2024-09-15T04:01:56.780 INFO:tasks.rados.rados.0.smithi122.stdout:3523: writing smithi12223906-26 from 2401133 to 2891101 tid 3 2024-09-15T04:01:56.780 INFO:tasks.rados.rados.0.smithi122.stdout:3524: delete oid 27 current snap is 407 2024-09-15T04:01:56.784 INFO:tasks.rados.rados.0.smithi122.stdout:3523: finishing write tid 1 to smithi12223906-26 2024-09-15T04:01:56.786 INFO:tasks.rados.rados.0.smithi122.stdout:3524: done (1 left) 2024-09-15T04:01:56.786 INFO:tasks.rados.rados.0.smithi122.stdout:3525: read oid 6 snap 356 2024-09-15T04:01:56.786 INFO:tasks.rados.rados.0.smithi122.stdout:3525: expect (ObjNum 1093 snap 356 seq_num 1093) 2024-09-15T04:01:56.786 INFO:tasks.rados.rados.0.smithi122.stdout:3526: write oid 4 current snap is 407 2024-09-15T04:01:56.786 INFO:tasks.rados.rados.0.smithi122.stdout:3526: seq_num 1245 ranges {524715=650544,1887964=491773} 2024-09-15T04:01:56.793 INFO:tasks.rados.rados.0.smithi122.stdout:3526: writing smithi12223906-4 from 524715 to 1175259 tid 1 2024-09-15T04:01:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:3526: writing smithi12223906-4 from 1887964 to 2379737 tid 2 2024-09-15T04:01:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:3527: write (excl) oid 25 current snap is 407 2024-09-15T04:01:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:3523: finishing write tid 2 to smithi12223906-26 2024-09-15T04:01:56.801 INFO:tasks.rados.rados.0.smithi122.stdout:3527: seq_num 1246 ranges {541522=725793,1946446=635431} 2024-09-15T04:01:56.808 INFO:tasks.rados.rados.0.smithi122.stdout:3527: writing smithi12223906-25 from 541522 to 1267315 tid 1 2024-09-15T04:01:56.816 INFO:tasks.rados.rados.0.smithi122.stdout:3527: writing smithi12223906-25 from 1946446 to 2581877 tid 2 2024-09-15T04:01:56.816 INFO:tasks.rados.rados.0.smithi122.stdout:3528: snap_remove snap 406 2024-09-15T04:01:57.772 INFO:tasks.rados.rados.0.smithi122.stdout:3528: done (4 left) 2024-09-15T04:01:57.772 INFO:tasks.rados.rados.0.smithi122.stdout:3529: write oid 21 current snap is 407 2024-09-15T04:01:57.772 INFO:tasks.rados.rados.0.smithi122.stdout:3523: finishing write tid 3 to smithi12223906-26 2024-09-15T04:01:57.773 INFO:tasks.rados.rados.0.smithi122.stdout:3529: seq_num 1247 ranges {558329=517395,1721281=779089} 2024-09-15T04:01:57.779 INFO:tasks.rados.rados.0.smithi122.stdout:3529: writing smithi12223906-21 from 558329 to 1075724 tid 1 2024-09-15T04:01:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:3529: writing smithi12223906-21 from 1721281 to 2500370 tid 2 2024-09-15T04:01:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:3530: copy_from oid 48 from oid 27 current snap is 407 2024-09-15T04:01:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:3523: finishing write tid 4 to smithi12223906-26 2024-09-15T04:01:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:3523: finishing write tid 5 to smithi12223906-26 2024-09-15T04:01:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 2143 (ObjNum 1243 snap 407 seq_num 1243) dirty exists 2024-09-15T04:01:57.788 INFO:tasks.rados.rados.0.smithi122.stdout:3523: left oid 26 (ObjNum 1243 snap 407 seq_num 1243) 2024-09-15T04:01:57.789 INFO:tasks.rados.rados.0.smithi122.stdout:3525: expect (ObjNum 1093 snap 356 seq_num 1093) 2024-09-15T04:01:57.945 INFO:tasks.rados.rados.0.smithi122.stdout:3526: finishing write tid 1 to smithi12223906-4 2024-09-15T04:01:57.945 INFO:tasks.rados.rados.0.smithi122.stdout:3526: finishing write tid 2 to smithi12223906-4 2024-09-15T04:01:57.945 INFO:tasks.rados.rados.0.smithi122.stdout:3526: finishing write tid 3 to smithi12223906-4 2024-09-15T04:01:57.945 INFO:tasks.rados.rados.0.smithi122.stdout:3526: finishing write tid 4 to smithi12223906-4 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2628 (ObjNum 1245 snap 407 seq_num 1245) dirty exists 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3526: left oid 4 (ObjNum 1245 snap 407 seq_num 1245) 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3527: finishing write tid 1 to smithi12223906-25 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3527: finishing write tid 2 to smithi12223906-25 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3527: finishing write tid 3 to smithi12223906-25 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3527: finishing write tid 4 to smithi12223906-25 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 3011 (ObjNum 1246 snap 407 seq_num 1246) dirty exists 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3527: left oid 25 (ObjNum 1246 snap 407 seq_num 1246) 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3529: finishing write tid 1 to smithi12223906-21 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3530: finishing copy_from to smithi12223906-48 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3530: got expected ENOENT (src dne) 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3530: finishing copy_from racing read to smithi12223906-48 2024-09-15T04:01:57.946 INFO:tasks.rados.rados.0.smithi122.stdout:3529: finishing write tid 2 to smithi12223906-21 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3529: finishing write tid 3 to smithi12223906-21 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3529: finishing write tid 4 to smithi12223906-21 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 21 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3529: left oid 21 (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3523: done (5 left) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3525: done (4 left) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3526: done (3 left) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3527: done (2 left) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3529: done (1 left) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3530: done (0 left) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3531: read oid 50 snap -1 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3531: expect (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3532: write oid 48 current snap is 407 2024-09-15T04:01:57.947 INFO:tasks.rados.rados.0.smithi122.stdout:3532: seq_num 1248 ranges {575136=592644,1663410=639100,2705542=269594} 2024-09-15T04:01:57.960 INFO:tasks.rados.rados.0.smithi122.stdout:3532: writing smithi12223906-48 from 575136 to 1167780 tid 1 2024-09-15T04:01:57.966 INFO:tasks.rados.rados.0.smithi122.stdout:3532: writing smithi12223906-48 from 1663410 to 2302510 tid 2 2024-09-15T04:01:57.970 INFO:tasks.rados.rados.0.smithi122.stdout:3532: writing smithi12223906-48 from 2705542 to 2975136 tid 3 2024-09-15T04:01:57.970 INFO:tasks.rados.rados.0.smithi122.stdout:3533: snap_create 2024-09-15T04:01:57.970 INFO:tasks.rados.rados.0.smithi122.stdout:3531: expect (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:01:58.165 INFO:tasks.rados.rados.0.smithi122.stdout:3532: finishing write tid 1 to smithi12223906-48 2024-09-15T04:01:58.165 INFO:tasks.rados.rados.0.smithi122.stdout:3532: finishing write tid 2 to smithi12223906-48 2024-09-15T04:01:58.166 INFO:tasks.rados.rados.0.smithi122.stdout:3532: finishing write tid 3 to smithi12223906-48 2024-09-15T04:01:58.166 INFO:tasks.rados.rados.0.smithi122.stdout:3532: finishing write tid 4 to smithi12223906-48 2024-09-15T04:01:58.166 INFO:tasks.rados.rados.0.smithi122.stdout:3532: finishing write tid 5 to smithi12223906-48 2024-09-15T04:01:58.166 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2142 (ObjNum 1248 snap 407 seq_num 1248) dirty exists 2024-09-15T04:01:58.166 INFO:tasks.rados.rados.0.smithi122.stdout:3532: left oid 48 (ObjNum 1248 snap 407 seq_num 1248) 2024-09-15T04:01:58.863 INFO:tasks.rados.rados.0.smithi122.stdout:3531: done (2 left) 2024-09-15T04:01:58.863 INFO:tasks.rados.rados.0.smithi122.stdout:3532: done (1 left) 2024-09-15T04:01:58.863 INFO:tasks.rados.rados.0.smithi122.stdout:3533: done (0 left) 2024-09-15T04:01:58.863 INFO:tasks.rados.rados.0.smithi122.stdout:3534: snap_create 2024-09-15T04:01:59.871 INFO:tasks.rados.rados.0.smithi122.stdout:3534: done (0 left) 2024-09-15T04:01:59.871 INFO:tasks.rados.rados.0.smithi122.stdout:3535: write (excl) oid 9 current snap is 409 2024-09-15T04:01:59.871 INFO:tasks.rados.rados.0.smithi122.stdout:3535: seq_num 1249 ranges {591943=667893,1721892=782758} 2024-09-15T04:01:59.885 INFO:tasks.rados.rados.0.smithi122.stdout:3535: writing smithi12223906-9 from 591943 to 1259836 tid 1 2024-09-15T04:01:59.892 INFO:tasks.rados.rados.0.smithi122.stdout:3535: writing smithi12223906-9 from 1721892 to 2504650 tid 2 2024-09-15T04:01:59.892 INFO:tasks.rados.rados.0.smithi122.stdout:3536: delete oid 2 current snap is 409 2024-09-15T04:01:59.900 INFO:tasks.rados.rados.0.smithi122.stdout:3536: done (1 left) 2024-09-15T04:01:59.900 INFO:tasks.rados.rados.0.smithi122.stdout:3537: delete oid 49 current snap is 409 2024-09-15T04:01:59.901 INFO:tasks.rados.rados.0.smithi122.stdout:3535: finishing write tid 1 to smithi12223906-9 2024-09-15T04:01:59.901 INFO:tasks.rados.rados.0.smithi122.stdout:3535: finishing write tid 2 to smithi12223906-9 2024-09-15T04:01:59.901 INFO:tasks.rados.rados.0.smithi122.stdout:3535: finishing write tid 3 to smithi12223906-9 2024-09-15T04:01:59.902 INFO:tasks.rados.rados.0.smithi122.stdout:3535: finishing write tid 4 to smithi12223906-9 2024-09-15T04:01:59.902 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2487 (ObjNum 1249 snap 409 seq_num 1249) dirty exists 2024-09-15T04:01:59.902 INFO:tasks.rados.rados.0.smithi122.stdout:3535: left oid 9 (ObjNum 1249 snap 409 seq_num 1249) 2024-09-15T04:01:59.907 INFO:tasks.rados.rados.0.smithi122.stdout:3535: done (1 left) 2024-09-15T04:01:59.907 INFO:tasks.rados.rados.0.smithi122.stdout:3537: done (0 left) 2024-09-15T04:01:59.907 INFO:tasks.rados.rados.0.smithi122.stdout:3538: write (excl) oid 28 current snap is 409 2024-09-15T04:01:59.908 INFO:tasks.rados.rados.0.smithi122.stdout:3538: seq_num 1252 ranges {642364=493640,1897338=646438} 2024-09-15T04:01:59.915 INFO:tasks.rados.rados.0.smithi122.stdout:3538: writing smithi12223906-28 from 642364 to 1136004 tid 1 2024-09-15T04:01:59.924 INFO:tasks.rados.rados.0.smithi122.stdout:3538: writing smithi12223906-28 from 1897338 to 2543776 tid 2 2024-09-15T04:01:59.925 INFO:tasks.rados.rados.0.smithi122.stdout:3539: snap_remove snap 403 2024-09-15T04:02:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:3538: finishing write tid 1 to smithi12223906-28 2024-09-15T04:02:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:3538: finishing write tid 2 to smithi12223906-28 2024-09-15T04:02:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:3538: finishing write tid 3 to smithi12223906-28 2024-09-15T04:02:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:3538: finishing write tid 4 to smithi12223906-28 2024-09-15T04:02:00.885 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 2435 (ObjNum 1252 snap 409 seq_num 1252) dirty exists 2024-09-15T04:02:00.886 INFO:tasks.rados.rados.0.smithi122.stdout:3538: left oid 28 (ObjNum 1252 snap 409 seq_num 1252) 2024-09-15T04:02:00.886 INFO:tasks.rados.rados.0.smithi122.stdout:3538: done (1 left) 2024-09-15T04:02:00.886 INFO:tasks.rados.rados.0.smithi122.stdout:3539: done (0 left) 2024-09-15T04:02:00.886 INFO:tasks.rados.rados.0.smithi122.stdout:3540: delete oid 17 current snap is 409 2024-09-15T04:02:00.893 INFO:tasks.rados.rados.0.smithi122.stdout:3540: done (0 left) 2024-09-15T04:02:00.893 INFO:tasks.rados.rados.0.smithi122.stdout:3541: read oid 11 snap -1 2024-09-15T04:02:00.893 INFO:tasks.rados.rados.0.smithi122.stdout:3541: expect (ObjNum 1161 snap 382 seq_num 1161) 2024-09-15T04:02:00.893 INFO:tasks.rados.rados.0.smithi122.stdout:3542: snap_create 2024-09-15T04:02:00.902 INFO:tasks.rados.rados.0.smithi122.stdout:3541: expect (ObjNum 1161 snap 382 seq_num 1161) 2024-09-15T04:02:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:3541: done (1 left) 2024-09-15T04:02:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:3542: done (0 left) 2024-09-15T04:02:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:3543: read oid 30 snap -1 2024-09-15T04:02:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:3543: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:02:01.763 INFO:tasks.rados.rados.0.smithi122.stdout:3544: delete oid 34 current snap is 410 2024-09-15T04:02:01.770 INFO:tasks.rados.rados.0.smithi122.stdout:3544: done (1 left) 2024-09-15T04:02:01.770 INFO:tasks.rados.rados.0.smithi122.stdout:3545: read oid 17 snap 386 2024-09-15T04:02:01.770 INFO:tasks.rados.rados.0.smithi122.stdout:3545: expect (ObjNum 1070 snap 348 seq_num 1070) 2024-09-15T04:02:01.771 INFO:tasks.rados.rados.0.smithi122.stdout:3546: copy_from oid 2 from oid 7 current snap is 410 2024-09-15T04:02:01.771 INFO:tasks.rados.rados.0.smithi122.stdout:3547: write (excl) oid 46 current snap is 410 2024-09-15T04:02:01.771 INFO:tasks.rados.rados.0.smithi122.stdout:3547: seq_num 1255 ranges {692785=435740,1672784=793765,2897503=195282} 2024-09-15T04:02:01.779 INFO:tasks.rados.rados.0.smithi122.stdout:3547: writing smithi12223906-46 from 692785 to 1128525 tid 1 2024-09-15T04:02:01.788 INFO:tasks.rados.rados.0.smithi122.stdout:3547: writing smithi12223906-46 from 1672784 to 2466549 tid 2 2024-09-15T04:02:01.792 INFO:tasks.rados.rados.0.smithi122.stdout:3547: writing smithi12223906-46 from 2897503 to 3092785 tid 3 2024-09-15T04:02:01.792 INFO:tasks.rados.rados.0.smithi122.stdout:3548: copy_from oid 25 from oid 21 current snap is 410 2024-09-15T04:02:01.792 INFO:tasks.rados.rados.0.smithi122.stdout:3543: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:02:01.919 INFO:tasks.rados.rados.0.smithi122.stdout:3547: finishing write tid 1 to smithi12223906-46 2024-09-15T04:02:01.919 INFO:tasks.rados.rados.0.smithi122.stdout:3545: expect (ObjNum 1070 snap 348 seq_num 1070) 2024-09-15T04:02:02.051 INFO:tasks.rados.rados.0.smithi122.stdout:3546: finishing copy_from racing read to smithi12223906-2 2024-09-15T04:02:02.051 INFO:tasks.rados.rados.0.smithi122.stdout:3546: finishing copy_from to smithi12223906-2 2024-09-15T04:02:02.051 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 2426 (ObjNum 1209 snap 399 seq_num 1209) dirty exists 2024-09-15T04:02:02.051 INFO:tasks.rados.rados.0.smithi122.stdout:3547: finishing write tid 2 to smithi12223906-46 2024-09-15T04:02:02.051 INFO:tasks.rados.rados.0.smithi122.stdout:3547: finishing write tid 3 to smithi12223906-46 2024-09-15T04:02:02.051 INFO:tasks.rados.rados.0.smithi122.stdout:3547: finishing write tid 4 to smithi12223906-46 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:3547: finishing write tid 5 to smithi12223906-46 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 2134 (ObjNum 1255 snap 410 seq_num 1255) dirty exists 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:3547: left oid 46 (ObjNum 1255 snap 410 seq_num 1255) 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:3548: finishing copy_from racing read to smithi12223906-25 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:3548: finishing copy_from to smithi12223906-25 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 25 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:3543: done (4 left) 2024-09-15T04:02:02.052 INFO:tasks.rados.rados.0.smithi122.stdout:3545: done (3 left) 2024-09-15T04:02:02.053 INFO:tasks.rados.rados.0.smithi122.stdout:3546: done (2 left) 2024-09-15T04:02:02.053 INFO:tasks.rados.rados.0.smithi122.stdout:3547: done (1 left) 2024-09-15T04:02:02.053 INFO:tasks.rados.rados.0.smithi122.stdout:3548: done (0 left) 2024-09-15T04:02:02.053 INFO:tasks.rados.rados.0.smithi122.stdout:3549: snap_create 2024-09-15T04:02:02.894 INFO:tasks.rados.rados.0.smithi122.stdout:3549: done (0 left) 2024-09-15T04:02:02.894 INFO:tasks.rados.rados.0.smithi122.stdout:3550: read oid 48 snap 369 2024-09-15T04:02:02.894 INFO:tasks.rados.rados.0.smithi122.stdout:3550: expect deleted 2024-09-15T04:02:02.895 INFO:tasks.rados.rados.0.smithi122.stdout:3551: copy_from oid 41 from oid 22 current snap is 411 2024-09-15T04:02:02.895 INFO:tasks.rados.rados.0.smithi122.stdout:3552: read oid 26 snap -1 2024-09-15T04:02:02.895 INFO:tasks.rados.rados.0.smithi122.stdout:3552: expect (ObjNum 1243 snap 407 seq_num 1243) 2024-09-15T04:02:02.895 INFO:tasks.rados.rados.0.smithi122.stdout:3553: read oid 23 snap -1 2024-09-15T04:02:02.895 INFO:tasks.rados.rados.0.smithi122.stdout:3553: expect deleted 2024-09-15T04:02:02.896 INFO:tasks.rados.rados.0.smithi122.stdout:3554: read oid 50 snap -1 2024-09-15T04:02:02.896 INFO:tasks.rados.rados.0.smithi122.stdout:3554: expect (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:02:02.896 INFO:tasks.rados.rados.0.smithi122.stdout:3555: write oid 3 current snap is 411 2024-09-15T04:02:02.896 INFO:tasks.rados.rados.0.smithi122.stdout:3555: seq_num 1256 ranges {709592=510989,1731266=653776,3041279=68313} 2024-09-15T04:02:02.906 INFO:tasks.rados.rados.0.smithi122.stdout:3555: writing smithi12223906-3 from 709592 to 1220581 tid 1 2024-09-15T04:02:02.913 INFO:tasks.rados.rados.0.smithi122.stdout:3555: writing smithi12223906-3 from 1731266 to 2385042 tid 2 2024-09-15T04:02:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3555: writing smithi12223906-3 from 3041279 to 3109592 tid 3 2024-09-15T04:02:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3556: snap_create 2024-09-15T04:02:02.917 INFO:tasks.rados.rados.0.smithi122.stdout:3552: expect (ObjNum 1243 snap 407 seq_num 1243) 2024-09-15T04:02:03.044 INFO:tasks.rados.rados.0.smithi122.stdout:3554: expect (ObjNum 1147 snap 377 seq_num 1147) 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3551: finishing copy_from racing read to smithi12223906-41 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3555: finishing write tid 1 to smithi12223906-3 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3551: finishing copy_from to smithi12223906-41 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2894 (ObjNum 1216 snap 401 seq_num 1216) dirty exists 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3555: finishing write tid 2 to smithi12223906-3 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3555: finishing write tid 3 to smithi12223906-3 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3555: finishing write tid 4 to smithi12223906-3 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3555: finishing write tid 5 to smithi12223906-3 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 3022 (ObjNum 1256 snap 411 seq_num 1256) dirty exists 2024-09-15T04:02:03.233 INFO:tasks.rados.rados.0.smithi122.stdout:3555: left oid 3 (ObjNum 1256 snap 411 seq_num 1256) 2024-09-15T04:02:03.911 INFO:tasks.rados.rados.0.smithi122.stdout:3550: done (6 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3551: done (5 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3552: done (4 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3553: done (3 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3554: done (2 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3555: done (1 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3556: done (0 left) 2024-09-15T04:02:03.912 INFO:tasks.rados.rados.0.smithi122.stdout:3557: delete oid 10 current snap is 412 2024-09-15T04:02:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:3557: done (0 left) 2024-09-15T04:02:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:3558: rollback oid 45 current snap is 412 2024-09-15T04:02:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 400 2024-09-15T04:02:03.920 INFO:tasks.rados.rados.0.smithi122.stdout:3559: snap_create 2024-09-15T04:02:03.929 INFO:tasks.rados.rados.0.smithi122.stdout:3558: finishing rollback tid 0 to smithi12223906-45 2024-09-15T04:02:03.929 INFO:tasks.rados.rados.0.smithi122.stdout:3558: finishing rollback tid 1 to smithi12223906-45 2024-09-15T04:02:03.929 INFO:tasks.rados.rados.0.smithi122.stdout:3558: finishing rollback tid 2 to smithi12223906-45 2024-09-15T04:02:03.930 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 2137 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:02:04.921 INFO:tasks.rados.rados.0.smithi122.stdout:3558: done (1 left) 2024-09-15T04:02:04.921 INFO:tasks.rados.rados.0.smithi122.stdout:3559: done (0 left) 2024-09-15T04:02:04.922 INFO:tasks.rados.rados.0.smithi122.stdout:3560: write (excl) oid 30 current snap is 413 2024-09-15T04:02:04.922 INFO:tasks.rados.rados.0.smithi122.stdout:3560: seq_num 1258 ranges {743206=661487,2131877=541092} 2024-09-15T04:02:04.934 INFO:tasks.rados.rados.0.smithi122.stdout:3560: writing smithi12223906-30 from 743206 to 1404693 tid 1 2024-09-15T04:02:04.942 INFO:tasks.rados.rados.0.smithi122.stdout:3560: writing smithi12223906-30 from 2131877 to 2672969 tid 2 2024-09-15T04:02:04.942 INFO:tasks.rados.rados.0.smithi122.stdout:3561: read oid 9 snap -1 2024-09-15T04:02:04.942 INFO:tasks.rados.rados.0.smithi122.stdout:3561: expect (ObjNum 1249 snap 409 seq_num 1249) 2024-09-15T04:02:04.942 INFO:tasks.rados.rados.0.smithi122.stdout:3562: snap_create 2024-09-15T04:02:04.945 INFO:tasks.rados.rados.0.smithi122.stdout:3560: finishing write tid 1 to smithi12223906-30 2024-09-15T04:02:04.947 INFO:tasks.rados.rados.0.smithi122.stdout:3561: expect (ObjNum 1249 snap 409 seq_num 1249) 2024-09-15T04:02:05.083 INFO:tasks.rados.rados.0.smithi122.stdout:3560: finishing write tid 2 to smithi12223906-30 2024-09-15T04:02:05.084 INFO:tasks.rados.rados.0.smithi122.stdout:3560: finishing write tid 3 to smithi12223906-30 2024-09-15T04:02:05.084 INFO:tasks.rados.rados.0.smithi122.stdout:3560: finishing write tid 4 to smithi12223906-30 2024-09-15T04:02:05.084 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 2897 (ObjNum 1258 snap 413 seq_num 1258) dirty exists 2024-09-15T04:02:05.084 INFO:tasks.rados.rados.0.smithi122.stdout:3560: left oid 30 (ObjNum 1258 snap 413 seq_num 1258) 2024-09-15T04:02:05.934 INFO:tasks.rados.rados.0.smithi122.stdout:3560: done (2 left) 2024-09-15T04:02:05.934 INFO:tasks.rados.rados.0.smithi122.stdout:3561: done (1 left) 2024-09-15T04:02:05.934 INFO:tasks.rados.rados.0.smithi122.stdout:3562: done (0 left) 2024-09-15T04:02:05.934 INFO:tasks.rados.rados.0.smithi122.stdout:3563: snap_create 2024-09-15T04:02:06.765 INFO:tasks.rados.rados.0.smithi122.stdout:3563: done (0 left) 2024-09-15T04:02:06.765 INFO:tasks.rados.rados.0.smithi122.stdout:3564: copy_from oid 19 from oid 2 current snap is 415 2024-09-15T04:02:06.765 INFO:tasks.rados.rados.0.smithi122.stdout:3565: snap_create 2024-09-15T04:02:06.788 INFO:tasks.rados.rados.0.smithi122.stdout:3564: finishing copy_from racing read to smithi12223906-19 2024-09-15T04:02:06.790 INFO:tasks.rados.rados.0.smithi122.stdout:3564: finishing copy_from to smithi12223906-19 2024-09-15T04:02:06.791 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 2426 (ObjNum 1209 snap 399 seq_num 1209) dirty exists 2024-09-15T04:02:07.778 INFO:tasks.rados.rados.0.smithi122.stdout:3564: done (1 left) 2024-09-15T04:02:07.778 INFO:tasks.rados.rados.0.smithi122.stdout:3565: done (0 left) 2024-09-15T04:02:07.778 INFO:tasks.rados.rados.0.smithi122.stdout:3566: copy_from oid 24 from oid 46 current snap is 416 2024-09-15T04:02:07.778 INFO:tasks.rados.rados.0.smithi122.stdout:3567: snap_remove snap 404 2024-09-15T04:02:08.793 INFO:tasks.rados.rados.0.smithi122.stdout:3567: done (1 left) 2024-09-15T04:02:08.793 INFO:tasks.rados.rados.0.smithi122.stdout:3568: copy_from oid 19 from oid 44 current snap is 416 2024-09-15T04:02:08.793 INFO:tasks.rados.rados.0.smithi122.stdout:3566: finishing copy_from racing read to smithi12223906-24 2024-09-15T04:02:08.793 INFO:tasks.rados.rados.0.smithi122.stdout:3566: finishing copy_from to smithi12223906-24 2024-09-15T04:02:08.793 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 2134 (ObjNum 1255 snap 410 seq_num 1255) dirty exists 2024-09-15T04:02:08.794 INFO:tasks.rados.rados.0.smithi122.stdout:3566: done (1 left) 2024-09-15T04:02:08.794 INFO:tasks.rados.rados.0.smithi122.stdout:3569: snap_create 2024-09-15T04:02:08.796 INFO:tasks.rados.rados.0.smithi122.stdout:3568: finishing copy_from to smithi12223906-19 2024-09-15T04:02:08.796 INFO:tasks.rados.rados.0.smithi122.stdout:3568: got expected ENOENT (src dne) 2024-09-15T04:02:08.796 INFO:tasks.rados.rados.0.smithi122.stdout:3568: finishing copy_from racing read to smithi12223906-19 2024-09-15T04:02:09.789 INFO:tasks.rados.rados.0.smithi122.stdout:3568: done (1 left) 2024-09-15T04:02:09.789 INFO:tasks.rados.rados.0.smithi122.stdout:3569: done (0 left) 2024-09-15T04:02:09.789 INFO:tasks.rados.rados.0.smithi122.stdout:3570: rollback oid 50 current snap is 417 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 50 to 382 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3571: read oid 29 snap 389 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3571: expect (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3572: rollback oid 20 current snap is 417 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 366 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3573: read oid 40 snap -1 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3573: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3574: copy_from oid 13 from oid 4 current snap is 417 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3575: read oid 1 snap -1 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3575: expect deleted 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3576: write oid 36 current snap is 417 2024-09-15T04:02:09.790 INFO:tasks.rados.rados.0.smithi122.stdout:3576: seq_num 1259 ranges {760013=736736,2190359=401103} 2024-09-15T04:02:09.801 INFO:tasks.rados.rados.0.smithi122.stdout:3576: writing smithi12223906-36 from 760013 to 1496749 tid 1 2024-09-15T04:02:09.809 INFO:tasks.rados.rados.0.smithi122.stdout:3576: writing smithi12223906-36 from 2190359 to 2591462 tid 2 2024-09-15T04:02:09.809 INFO:tasks.rados.rados.0.smithi122.stdout:3577: write (excl) oid 5 current snap is 417 2024-09-15T04:02:09.810 INFO:tasks.rados.rados.0.smithi122.stdout:3577: seq_num 1260 ranges {776820=411985,1848841=544761,2918265=258555} 2024-09-15T04:02:09.814 INFO:tasks.rados.rados.0.smithi122.stdout:3577: writing smithi12223906-5 from 776820 to 1188805 tid 1 2024-09-15T04:02:09.820 INFO:tasks.rados.rados.0.smithi122.stdout:3577: writing smithi12223906-5 from 1848841 to 2393602 tid 2 2024-09-15T04:02:09.825 INFO:tasks.rados.rados.0.smithi122.stdout:3577: writing smithi12223906-5 from 2918265 to 3176820 tid 3 2024-09-15T04:02:09.825 INFO:tasks.rados.rados.0.smithi122.stdout:3578: snap_create 2024-09-15T04:02:09.825 INFO:tasks.rados.rados.0.smithi122.stdout:3572: finishing rollback tid 1 to smithi12223906-20 2024-09-15T04:02:09.826 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1580 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T04:02:09.826 INFO:tasks.rados.rados.0.smithi122.stdout:3570: finishing rollback tid 0 to smithi12223906-50 2024-09-15T04:02:09.826 INFO:tasks.rados.rados.0.smithi122.stdout:3571: expect (ObjNum 1085 snap 355 seq_num 1085) 2024-09-15T04:02:09.932 INFO:tasks.rados.rados.0.smithi122.stdout:3573: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3570: finishing rollback tid 1 to smithi12223906-50 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3574: finishing copy_from racing read to smithi12223906-13 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3570: finishing rollback tid 2 to smithi12223906-50 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 2986 (ObjNum 1155 snap 380 seq_num 1155) dirty exists 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3574: finishing copy_from to smithi12223906-13 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 2628 (ObjNum 1245 snap 407 seq_num 1245) dirty exists 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3576: finishing write tid 1 to smithi12223906-36 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3576: finishing write tid 2 to smithi12223906-36 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3576: finishing write tid 3 to smithi12223906-36 2024-09-15T04:02:10.044 INFO:tasks.rados.rados.0.smithi122.stdout:3576: finishing write tid 4 to smithi12223906-36 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 3034 (ObjNum 1259 snap 417 seq_num 1259) dirty exists 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3576: left oid 36 (ObjNum 1259 snap 417 seq_num 1259) 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3577: finishing write tid 1 to smithi12223906-5 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3577: finishing write tid 2 to smithi12223906-5 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3577: finishing write tid 3 to smithi12223906-5 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3577: finishing write tid 4 to smithi12223906-5 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3577: finishing write tid 5 to smithi12223906-5 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 2618 (ObjNum 1260 snap 417 seq_num 1260) dirty exists 2024-09-15T04:02:10.045 INFO:tasks.rados.rados.0.smithi122.stdout:3577: left oid 5 (ObjNum 1260 snap 417 seq_num 1260) 2024-09-15T04:02:10.814 INFO:tasks.rados.rados.0.smithi122.stdout:3570: done (8 left) 2024-09-15T04:02:10.814 INFO:tasks.rados.rados.0.smithi122.stdout:3571: done (7 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3572: done (6 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3573: done (5 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3574: done (4 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3575: done (3 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3576: done (2 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3577: done (1 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3578: done (0 left) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3579: read oid 50 snap -1 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3579: expect (ObjNum 1155 snap 380 seq_num 1155) 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3580: copy_from oid 49 from oid 33 current snap is 418 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3581: write (excl) oid 38 current snap is 418 2024-09-15T04:02:10.815 INFO:tasks.rados.rados.0.smithi122.stdout:3581: seq_num 1261 ranges {793627=487234,1790970=404772,2829335=364292} 2024-09-15T04:02:10.832 INFO:tasks.rados.rados.0.smithi122.stdout:3581: writing smithi12223906-38 from 793627 to 1280861 tid 1 2024-09-15T04:02:10.836 INFO:tasks.rados.rados.0.smithi122.stdout:3581: writing smithi12223906-38 from 1790970 to 2195742 tid 2 2024-09-15T04:02:10.842 INFO:tasks.rados.rados.0.smithi122.stdout:3581: writing smithi12223906-38 from 2829335 to 3193627 tid 3 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3582: read oid 34 snap 411 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3582: expect deleted 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3583: read oid 40 snap -1 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3583: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3584: read oid 7 snap -1 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3584: expect (ObjNum 1209 snap 399 seq_num 1209) 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3585: read oid 14 snap 366 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3585: expect deleted 2024-09-15T04:02:10.843 INFO:tasks.rados.rados.0.smithi122.stdout:3586: read oid 37 snap -1 2024-09-15T04:02:10.846 INFO:tasks.rados.rados.0.smithi122.stdout:3586: expect (ObjNum 1211 snap 401 seq_num 1211) 2024-09-15T04:02:10.846 INFO:tasks.rados.rados.0.smithi122.stdout:3587: read oid 16 snap -1 2024-09-15T04:02:10.846 INFO:tasks.rados.rados.0.smithi122.stdout:3587: expect (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:02:10.846 INFO:tasks.rados.rados.0.smithi122.stdout:3588: write (excl) oid 20 current snap is 418 2024-09-15T04:02:10.846 INFO:tasks.rados.rados.0.smithi122.stdout:3588: seq_num 1262 ranges {410434=678836,1565805=548430,2573111=637323} 2024-09-15T04:02:10.849 INFO:tasks.rados.rados.0.smithi122.stdout:3588: writing smithi12223906-20 from 410434 to 1089270 tid 1 2024-09-15T04:02:10.856 INFO:tasks.rados.rados.0.smithi122.stdout:3588: writing smithi12223906-20 from 1565805 to 2114235 tid 2 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3588: writing smithi12223906-20 from 2573111 to 3210434 tid 3 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3589: read oid 13 snap -1 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3589: expect (ObjNum 1245 snap 407 seq_num 1245) 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3590: read oid 10 snap 321 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3590: expect (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3591: read oid 31 snap -1 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3591: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3592: rollback oid 23 current snap is 418 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 407 2024-09-15T04:02:10.864 INFO:tasks.rados.rados.0.smithi122.stdout:3579: expect (ObjNum 1155 snap 380 seq_num 1155) 2024-09-15T04:02:11.064 INFO:tasks.rados.rados.0.smithi122.stdout:3580: finishing copy_from racing read to smithi12223906-49 2024-09-15T04:02:11.064 INFO:tasks.rados.rados.0.smithi122.stdout:3579: done (13 left) 2024-09-15T04:02:11.064 INFO:tasks.rados.rados.0.smithi122.stdout:3593: rollback oid 12 current snap is 418 2024-09-15T04:02:11.064 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 391 2024-09-15T04:02:11.064 INFO:tasks.rados.rados.0.smithi122.stdout:3584: expect (ObjNum 1209 snap 399 seq_num 1209) 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3581: finishing write tid 1 to smithi12223906-38 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3581: finishing write tid 2 to smithi12223906-38 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3581: finishing write tid 3 to smithi12223906-38 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3581: finishing write tid 4 to smithi12223906-38 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3582: done (13 left) 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3584: done (12 left) 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3585: done (11 left) 2024-09-15T04:02:11.170 INFO:tasks.rados.rados.0.smithi122.stdout:3594: rollback oid 11 current snap is 418 2024-09-15T04:02:11.171 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 11 to 399 2024-09-15T04:02:11.171 INFO:tasks.rados.rados.0.smithi122.stdout:3581: finishing write tid 5 to smithi12223906-38 2024-09-15T04:02:11.171 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 2147 (ObjNum 1261 snap 418 seq_num 1261) dirty exists 2024-09-15T04:02:11.171 INFO:tasks.rados.rados.0.smithi122.stdout:3581: left oid 38 (ObjNum 1261 snap 418 seq_num 1261) 2024-09-15T04:02:11.171 INFO:tasks.rados.rados.0.smithi122.stdout:3583: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:11.279 INFO:tasks.rados.rados.0.smithi122.stdout:3586: expect (ObjNum 1211 snap 401 seq_num 1211) 2024-09-15T04:02:11.383 INFO:tasks.rados.rados.0.smithi122.stdout:3580: finishing copy_from to smithi12223906-49 2024-09-15T04:02:11.384 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 2890 (ObjNum 1173 snap 384 seq_num 1173) dirty exists 2024-09-15T04:02:11.384 INFO:tasks.rados.rados.0.smithi122.stdout:3587: expect (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:02:11.565 INFO:tasks.rados.rados.0.smithi122.stdout:3588: finishing write tid 1 to smithi12223906-20 2024-09-15T04:02:11.565 INFO:tasks.rados.rados.0.smithi122.stdout:3588: finishing write tid 2 to smithi12223906-20 2024-09-15T04:02:11.565 INFO:tasks.rados.rados.0.smithi122.stdout:3580: done (11 left) 2024-09-15T04:02:11.565 INFO:tasks.rados.rados.0.smithi122.stdout:3581: done (10 left) 2024-09-15T04:02:11.565 INFO:tasks.rados.rados.0.smithi122.stdout:3583: done (9 left) 2024-09-15T04:02:11.565 INFO:tasks.rados.rados.0.smithi122.stdout:3586: done (8 left) 2024-09-15T04:02:11.566 INFO:tasks.rados.rados.0.smithi122.stdout:3587: done (7 left) 2024-09-15T04:02:11.566 INFO:tasks.rados.rados.0.smithi122.stdout:3595: snap_remove snap 356 2024-09-15T04:02:11.770 INFO:tasks.rados.rados.0.smithi122.stdout:3595: done (7 left) 2024-09-15T04:02:11.770 INFO:tasks.rados.rados.0.smithi122.stdout:3596: read oid 1 snap 389 2024-09-15T04:02:11.770 INFO:tasks.rados.rados.0.smithi122.stdout:3596: expect (ObjNum 1127 snap 371 seq_num 1127) 2024-09-15T04:02:11.770 INFO:tasks.rados.rados.0.smithi122.stdout:3597: delete oid 5 current snap is 418 2024-09-15T04:02:11.770 INFO:tasks.rados.rados.0.smithi122.stdout:3588: finishing write tid 3 to smithi12223906-20 2024-09-15T04:02:11.770 INFO:tasks.rados.rados.0.smithi122.stdout:3588: finishing write tid 4 to smithi12223906-20 2024-09-15T04:02:11.771 INFO:tasks.rados.rados.0.smithi122.stdout:3588: finishing write tid 5 to smithi12223906-20 2024-09-15T04:02:11.771 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1584 (ObjNum 1262 snap 418 seq_num 1262) dirty exists 2024-09-15T04:02:11.771 INFO:tasks.rados.rados.0.smithi122.stdout:3588: left oid 20 (ObjNum 1262 snap 418 seq_num 1262) 2024-09-15T04:02:11.771 INFO:tasks.rados.rados.0.smithi122.stdout:3590: expect (ObjNum 1014 snap 318 seq_num 1014) 2024-09-15T04:02:11.993 INFO:tasks.rados.rados.0.smithi122.stdout:3588: done (8 left) 2024-09-15T04:02:11.994 INFO:tasks.rados.rados.0.smithi122.stdout:3590: done (7 left) 2024-09-15T04:02:11.994 INFO:tasks.rados.rados.0.smithi122.stdout:3597: done (6 left) 2024-09-15T04:02:11.994 INFO:tasks.rados.rados.0.smithi122.stdout:3598: write (excl) oid 2 current snap is 418 2024-09-15T04:02:11.994 INFO:tasks.rados.rados.0.smithi122.stdout:3592: finishing rollback tid 1 to smithi12223906-23 2024-09-15T04:02:11.994 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2941 (ObjNum 65 snap 0 seq_num 1842441008) dirty dne 2024-09-15T04:02:11.994 INFO:tasks.rados.rados.0.smithi122.stdout:3598: seq_num 1264 ranges {444048=429334,1282769=552099,2627957=616091} 2024-09-15T04:02:11.998 INFO:tasks.rados.rados.0.smithi122.stdout:3598: writing smithi12223906-2 from 444048 to 873382 tid 1 2024-09-15T04:02:12.003 INFO:tasks.rados.rados.0.smithi122.stdout:3598: writing smithi12223906-2 from 1282769 to 1834868 tid 2 2024-09-15T04:02:12.012 INFO:tasks.rados.rados.0.smithi122.stdout:3598: writing smithi12223906-2 from 2627957 to 3244048 tid 3 2024-09-15T04:02:12.013 INFO:tasks.rados.rados.0.smithi122.stdout:3592: done (6 left) 2024-09-15T04:02:12.013 INFO:tasks.rados.rados.0.smithi122.stdout:3599: snap_remove snap 413 2024-09-15T04:02:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3599: done (6 left) 2024-09-15T04:02:12.827 INFO:tasks.rados.rados.0.smithi122.stdout:3589: expect (ObjNum 1245 snap 407 seq_num 1245) 2024-09-15T04:02:13.104 INFO:tasks.rados.rados.0.smithi122.stdout:3591: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:02:13.276 INFO:tasks.rados.rados.0.smithi122.stdout:3593: finishing rollback tid 0 to smithi12223906-12 2024-09-15T04:02:13.276 INFO:tasks.rados.rados.0.smithi122.stdout:3593: finishing rollback tid 1 to smithi12223906-12 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:3593: finishing rollback tid 2 to smithi12223906-12 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 2136 (ObjNum 1172 snap 383 seq_num 1172) dirty exists 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:3594: finishing rollback tid 0 to smithi12223906-11 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:3594: finishing rollback tid 1 to smithi12223906-11 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:3594: finishing rollback tid 2 to smithi12223906-11 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1938 (ObjNum 1161 snap 382 seq_num 1161) dirty exists 2024-09-15T04:02:13.277 INFO:tasks.rados.rados.0.smithi122.stdout:3596: expect (ObjNum 1127 snap 371 seq_num 1127) 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3598: finishing write tid 1 to smithi12223906-2 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3598: finishing write tid 2 to smithi12223906-2 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3598: finishing write tid 3 to smithi12223906-2 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3598: finishing write tid 4 to smithi12223906-2 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3598: finishing write tid 5 to smithi12223906-2 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 2894 (ObjNum 1264 snap 418 seq_num 1264) dirty exists 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3598: left oid 2 (ObjNum 1264 snap 418 seq_num 1264) 2024-09-15T04:02:13.393 INFO:tasks.rados.rados.0.smithi122.stdout:3600: read oid 15 snap -1 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3600: expect deleted 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3589: done (6 left) 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3591: done (5 left) 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3593: done (4 left) 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3594: done (3 left) 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3596: done (2 left) 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3598: done (1 left) 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3601: rollback oid 32 current snap is 418 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 358 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3602: copy_from oid 11 from oid 20 current snap is 418 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3603: copy_from oid 33 from oid 7 current snap is 418 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3604: copy_from oid 17 from oid 14 current snap is 418 2024-09-15T04:02:13.394 INFO:tasks.rados.rados.0.smithi122.stdout:3605: read oid 2 snap -1 2024-09-15T04:02:13.395 INFO:tasks.rados.rados.0.smithi122.stdout:3605: expect (ObjNum 1264 snap 418 seq_num 1264) 2024-09-15T04:02:13.395 INFO:tasks.rados.rados.0.smithi122.stdout:3606: delete oid 23 current snap is 418 2024-09-15T04:02:13.399 INFO:tasks.rados.rados.0.smithi122.stdout:3605: expect (ObjNum 1264 snap 418 seq_num 1264) 2024-09-15T04:02:13.452 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-15T04:02:13.453 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-09-15T04:02:13.453 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-15T04:02:13.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-15T04:02:13.456 INFO:tasks.ceph.mon.h:Started 2024-09-15T04:02:13.456 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-09-15T04:02:13.456 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-09-15T04:02:13.456 INFO:tasks.ceph.mon.i:Restarting daemon 2024-09-15T04:02:13.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-09-15T04:02:13.458 INFO:tasks.ceph.mon.i:Started 2024-09-15T04:02:13.459 INFO:tasks.mon_thrash.mon_thrasher:killing mon.g 2024-09-15T04:02:13.459 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.g 2024-09-15T04:02:13.459 INFO:tasks.ceph.mon.g:Restarting daemon 2024-09-15T04:02:13.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-09-15T04:02:13.461 INFO:tasks.ceph.mon.g:Started 2024-09-15T04:02:13.461 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-15T04:02:13.461 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-15T04:02:13.461 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T04:02:13.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T04:02:13.464 INFO:tasks.ceph.mon.b:Started 2024-09-15T04:02:13.464 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T04:02:13.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:02:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.561+0000 7f946a805700 10 monclient: get_monmap_and_config 2024-09-15T04:02:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.561+0000 7f946a805700 10 monclient: build_initial_monmap 2024-09-15T04:02:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.561+0000 7f946a805700 10 monclient: monmap: 2024-09-15T04:02:13.574 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:13.567388+0000 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:13.567388+0000 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:13.575 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 Processor -- start 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- start start 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: init 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: _add_conns ranks=[1,6,2,7,8,5,0,4,3] 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient(hunting): picked mon.noname-f con 0x7f946406d3f0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient(hunting): picked mon.noname-d con 0x7f946412e5e0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient(hunting): picked mon.noname-b con 0x7f946412f0f0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946404ca10 con 0x7f946406d3f0 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946409d300 con 0x7f946412f0f0 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946409d090 con 0x7f946412e5e0 2024-09-15T04:02:13.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient(hunting): authenticate will time out at 1877.996842s 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9469002700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f946412e5e0 0x7f946412e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:34648/0 (socket says 172.21.15.26:34648) 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9469002700 1 -- 172.21.15.26:0/2105581204 learned_addr learned my addr 172.21.15.26:0/2105581204 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546893760 0 0) 0x7f946409d090 con 0x7f946412e5e0 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient(hunting): my global_id is 195072 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9450003570 con 0x7f946412e5e0 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549207077 0 0) 0x7f946404ca10 con 0x7f946406d3f0 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient(hunting): my global_id is 204823 2024-09-15T04:02:13.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946409d090 con 0x7f946406d3f0 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117161279 0 0) 0x7f9450003570 con 0x7f946412e5e0 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 >> v1:172.21.15.26:6790/0 conn(0x7f946412f0f0 legacy=0x7f946413b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 >> v1:172.21.15.122:6789/0 conn(0x7f946406d3f0 legacy=0x7f946409dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: _finish_hunting 0 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 monclient: found mon.noname-d 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946409e6e0 con 0x7f946412e5e0 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: _finish_auth 0 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 5 monclient: authenticate success, global_id 195072 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:13.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f94540017b0 con 0x7f946412e5e0 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: dump: 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:13.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9454001a20 con 0x7f946412e5e0 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:13.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 1 -- 172.21.15.26:0/2105581204 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9454004c30 con 0x7f946412e5e0 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 4 set_mon_vals no callback set 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f9468801700 10 monclient: dump: 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:13.581 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: shutdown 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- 172.21.15.26:0/2105581204 >> v1:172.21.15.26:6792/0 conn(0x7f946412e5e0 legacy=0x7f946412e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- 172.21.15.26:0/2105581204 shutdown_connections 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- 172.21.15.26:0/2105581204 >> 172.21.15.26:0/2105581204 conn(0x7f9464063d90 msgr2=0x7f94640a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- 172.21.15.26:0/2105581204 shutdown_connections 2024-09-15T04:02:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- 172.21.15.26:0/2105581204 wait complete. 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: build_initial_monmap 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 10 monclient: monmap: 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:13.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:13.584 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:13.584 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:13.584 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:13.584 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:13.585 INFO:tasks.rados.rados.0.smithi122.stdout:3600: done (6 left) 2024-09-15T04:02:13.585 INFO:tasks.rados.rados.0.smithi122.stdout:3605: done (5 left) 2024-09-15T04:02:13.585 INFO:tasks.rados.rados.0.smithi122.stdout:3606: done (4 left) 2024-09-15T04:02:13.585 INFO:tasks.rados.rados.0.smithi122.stdout:3607: read oid 40 snap -1 2024-09-15T04:02:13.585 INFO:tasks.rados.rados.0.smithi122.stdout:3607: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3608: delete oid 44 current snap is 418 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3601: finishing rollback tid 0 to smithi12223906-32 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3601: finishing rollback tid 1 to smithi12223906-32 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3602: finishing copy_from racing read to smithi12223906-11 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3604: finishing copy_from racing read to smithi12223906-17 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3601: finishing rollback tid 2 to smithi12223906-32 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 3003 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3604: finishing copy_from to smithi12223906-17 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 1843 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3602: finishing copy_from to smithi12223906-11 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1584 (ObjNum 1262 snap 418 seq_num 1262) dirty exists 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3603: finishing copy_from racing read to smithi12223906-33 2024-09-15T04:02:13.586 INFO:tasks.rados.rados.0.smithi122.stdout:3603: finishing copy_from to smithi12223906-33 2024-09-15T04:02:13.587 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2426 (ObjNum 1209 snap 399 seq_num 1209) dirty exists 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 Processor -- start 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.565+0000 7f946a805700 1 -- start start 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: init 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: _add_conns ranks=[3,8,2,1,0,7,5,6,4] 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient(hunting): picked mon.noname-g con 0x7f946406d3f0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient(hunting): picked mon.noname-e con 0x7f946412e5e0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient(hunting): picked mon.noname-b con 0x7f946412f0f0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:13.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946406dfc0 con 0x7f946412f0f0 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464142960 con 0x7f946406d3f0 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946411f7a0 con 0x7f946412e5e0 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient(hunting): authenticate will time out at 1878.000947s 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9469803700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f946406d3f0 0x7f9464141a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:42938/0 (socket says 172.21.15.26:42938) 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9469803700 1 -- 172.21.15.26:0/1550797063 learned_addr learned my addr 172.21.15.26:0/1550797063 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749864540 0 0) 0x7f946411f7a0 con 0x7f946412e5e0 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): my global_id is 135638 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9448003570 con 0x7f946412e5e0 2024-09-15T04:02:13.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3997786787 0 0) 0x7f9448003570 con 0x7f946412e5e0 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f946411f7a0 con 0x7f946412e5e0 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9454001830 con 0x7f946412e5e0 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): dump: 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3543124838 0 0) 0x7f946411f7a0 con 0x7f946412e5e0 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 >> v1:172.21.15.122:6790/0 conn(0x7f946406d3f0 legacy=0x7f9464141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 >> v1:172.21.15.26:6790/0 conn(0x7f946412f0f0 legacy=0x7f9464149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:02:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 monclient: found mon.e 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946406e420 con 0x7f946412e5e0 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: _finish_auth 0 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9454004f00 con 0x7f946412e5e0 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f94540053b0 con 0x7f946412e5e0 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f945b7fe700 4 set_mon_vals no callback set 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f9459ffb700 10 monclient: dump: 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:13.595 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 5 monclient: authenticate success, global_id 135638 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: _renew_subs 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946411fbb0 con 0x7f946412e5e0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: _renew_subs 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.569+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f946412e030 con 0x7f946412e5e0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f946a805700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f946a805700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f946a805700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946404bbf0 con 0x7f946412e5e0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f945400fd70 con 0x7f946412e5e0 2024-09-15T04:02:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(812..812 src has 435..812) v4 ==== 3955+0+0 (unknown 2951786867 0 0) 0x7f94540440e0 con 0x7f946412e5e0 2024-09-15T04:02:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.573+0000 7f9469803700 10 monclient: get_auth_request con 0x7f9448037600 auth_method 0 2024-09-15T04:02:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.581+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f9454014be0 con 0x7f946412e5e0 2024-09-15T04:02:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.581+0000 7f9459ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.581+0000 7f9459ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:13.597 INFO:tasks.rados.rados.0.smithi122.stdout:3601: done (5 left) 2024-09-15T04:02:13.597 INFO:tasks.rados.rados.0.smithi122.stdout:3602: done (4 left) 2024-09-15T04:02:13.597 INFO:tasks.rados.rados.0.smithi122.stdout:3603: done (3 left) 2024-09-15T04:02:13.597 INFO:tasks.rados.rados.0.smithi122.stdout:3604: done (2 left) 2024-09-15T04:02:13.598 INFO:tasks.rados.rados.0.smithi122.stdout:3608: done (1 left) 2024-09-15T04:02:13.598 INFO:tasks.rados.rados.0.smithi122.stdout:3609: write oid 38 current snap is 418 2024-09-15T04:02:13.598 INFO:tasks.rados.rados.0.smithi122.stdout:3609: seq_num 1267 ranges {494469=655081,1741862=415779,2710226=584243} 2024-09-15T04:02:13.598 INFO:tasks.rados.rados.0.smithi122.stdout:3609: writing smithi12223906-38 from 494469 to 1149550 tid 1 2024-09-15T04:02:13.605 INFO:tasks.rados.rados.0.smithi122.stdout:3609: writing smithi12223906-38 from 1741862 to 2157641 tid 2 2024-09-15T04:02:13.616 INFO:tasks.rados.rados.0.smithi122.stdout:3609: writing smithi12223906-38 from 2710226 to 3294469 tid 3 2024-09-15T04:02:13.616 INFO:tasks.rados.rados.0.smithi122.stdout:3610: snap_create 2024-09-15T04:02:13.617 INFO:tasks.rados.rados.0.smithi122.stdout:3607: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:3609: finishing write tid 1 to smithi12223906-38 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:3609: finishing write tid 2 to smithi12223906-38 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:3609: finishing write tid 3 to smithi12223906-38 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:3609: finishing write tid 4 to smithi12223906-38 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:3609: finishing write tid 5 to smithi12223906-38 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 2151 (ObjNum 1267 snap 418 seq_num 1267) dirty exists 2024-09-15T04:02:13.734 INFO:tasks.rados.rados.0.smithi122.stdout:3609: left oid 38 (ObjNum 1267 snap 418 seq_num 1267) 2024-09-15T04:02:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.817+0000 7f946a805700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:02:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.817+0000 7f946a805700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.817+0000 7f946a805700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:13.817+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9464066de0 con 0x7f946412e5e0 2024-09-15T04:02:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:16.569+0000 7f9458ff9700 10 monclient: tick 2024-09-15T04:02:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:16.569+0000 7f9458ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:16.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:16.569+0000 7f9458ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:17.569+0000 7f9458ff9700 10 monclient: tick 2024-09-15T04:02:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:17.569+0000 7f9458ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.569+0000 7f9458ff9700 10 monclient: tick 2024-09-15T04:02:18.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.569+0000 7f9458ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:18.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.661+0000 7f9459ffb700 1 -- 172.21.15.26:0/1550797063 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 223660630) 0x7f9454014a00 con 0x7f946412e5e0 2024-09-15T04:02:18.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.661+0000 7f9459ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:18.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.661+0000 7f9459ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:18.671 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":110,"quorum":[0,1,4,5,6,8],"quorum_names":["a","f","c","h","d","e"],"quorum_leader_name":"e","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:02:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 >> v1:172.21.15.122:6812/20895 conn(0x7f9448037600 legacy=0x7f9448039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 10 monclient: shutdown 2024-09-15T04:02:18.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 >> v1:172.21.15.26:6793/0 conn(0x7f946412e5e0 legacy=0x7f946412d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 shutdown_connections 2024-09-15T04:02:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 >> 172.21.15.26:0/1550797063 conn(0x7f9464063d90 msgr2=0x7f9464097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 shutdown_connections 2024-09-15T04:02:18.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:18.669+0000 7f946a805700 1 -- 172.21.15.26:0/1550797063 wait complete. 2024-09-15T04:02:19.720 INFO:tasks.rados.rados.0.smithi122.stdout:3607: done (2 left) 2024-09-15T04:02:19.720 INFO:tasks.rados.rados.0.smithi122.stdout:3609: done (1 left) 2024-09-15T04:02:19.721 INFO:tasks.rados.rados.0.smithi122.stdout:3610: done (0 left) 2024-09-15T04:02:19.721 INFO:tasks.rados.rados.0.smithi122.stdout:3611: write (excl) oid 39 current snap is 419 2024-09-15T04:02:19.721 INFO:tasks.rados.rados.0.smithi122.stdout:3611: seq_num 1268 ranges {511276=730330,1800344=559437,3021296=289980} 2024-09-15T04:02:19.731 INFO:tasks.rados.rados.0.smithi122.stdout:3611: writing smithi12223906-39 from 511276 to 1241606 tid 1 2024-09-15T04:02:19.737 INFO:tasks.rados.rados.0.smithi122.stdout:3611: writing smithi12223906-39 from 1800344 to 2359781 tid 2 2024-09-15T04:02:19.743 INFO:tasks.rados.rados.0.smithi122.stdout:3611: writing smithi12223906-39 from 3021296 to 3311276 tid 3 2024-09-15T04:02:19.743 INFO:tasks.rados.rados.0.smithi122.stdout:3612: delete oid 9 current snap is 419 2024-09-15T04:02:19.749 INFO:tasks.rados.rados.0.smithi122.stdout:3612: done (1 left) 2024-09-15T04:02:19.749 INFO:tasks.rados.rados.0.smithi122.stdout:3613: read oid 3 snap -1 2024-09-15T04:02:19.750 INFO:tasks.rados.rados.0.smithi122.stdout:3613: expect (ObjNum 1256 snap 411 seq_num 1256) 2024-09-15T04:02:19.750 INFO:tasks.rados.rados.0.smithi122.stdout:3614: delete oid 4 current snap is 419 2024-09-15T04:02:19.756 INFO:tasks.rados.rados.0.smithi122.stdout:3614: done (2 left) 2024-09-15T04:02:19.756 INFO:tasks.rados.rados.0.smithi122.stdout:3615: write oid 50 current snap is 419 2024-09-15T04:02:19.757 INFO:tasks.rados.rados.0.smithi122.stdout:3615: seq_num 1271 ranges {561697=672430,1975790=706764,3103565=258132} 2024-09-15T04:02:19.766 INFO:tasks.rados.rados.0.smithi122.stdout:3615: writing smithi12223906-50 from 561697 to 1234127 tid 1 2024-09-15T04:02:19.776 INFO:tasks.rados.rados.0.smithi122.stdout:3615: writing smithi12223906-50 from 1975790 to 2682554 tid 2 2024-09-15T04:02:19.780 INFO:tasks.rados.rados.0.smithi122.stdout:3615: writing smithi12223906-50 from 3103565 to 3361697 tid 3 2024-09-15T04:02:19.780 INFO:tasks.rados.rados.0.smithi122.stdout:3616: snap_create 2024-09-15T04:02:19.780 INFO:tasks.rados.rados.0.smithi122.stdout:3611: finishing write tid 1 to smithi12223906-39 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:3611: finishing write tid 2 to smithi12223906-39 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:3611: finishing write tid 3 to smithi12223906-39 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:3611: finishing write tid 4 to smithi12223906-39 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:3611: finishing write tid 5 to smithi12223906-39 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 3051 (ObjNum 1268 snap 419 seq_num 1268) dirty exists 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:3611: left oid 39 (ObjNum 1268 snap 419 seq_num 1268) 2024-09-15T04:02:19.781 INFO:tasks.rados.rados.0.smithi122.stdout:3613: expect (ObjNum 1256 snap 411 seq_num 1256) 2024-09-15T04:02:20.002 INFO:tasks.rados.rados.0.smithi122.stdout:3615: finishing write tid 1 to smithi12223906-50 2024-09-15T04:02:20.002 INFO:tasks.rados.rados.0.smithi122.stdout:3615: finishing write tid 2 to smithi12223906-50 2024-09-15T04:02:20.002 INFO:tasks.rados.rados.0.smithi122.stdout:3615: finishing write tid 3 to smithi12223906-50 2024-09-15T04:02:20.002 INFO:tasks.rados.rados.0.smithi122.stdout:3615: finishing write tid 4 to smithi12223906-50 2024-09-15T04:02:20.002 INFO:tasks.rados.rados.0.smithi122.stdout:3615: finishing write tid 5 to smithi12223906-50 2024-09-15T04:02:20.003 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 3012 (ObjNum 1271 snap 419 seq_num 1271) dirty exists 2024-09-15T04:02:20.003 INFO:tasks.rados.rados.0.smithi122.stdout:3615: left oid 50 (ObjNum 1271 snap 419 seq_num 1271) 2024-09-15T04:02:20.744 INFO:tasks.rados.rados.0.smithi122.stdout:3611: done (3 left) 2024-09-15T04:02:20.744 INFO:tasks.rados.rados.0.smithi122.stdout:3613: done (2 left) 2024-09-15T04:02:20.744 INFO:tasks.rados.rados.0.smithi122.stdout:3615: done (1 left) 2024-09-15T04:02:20.744 INFO:tasks.rados.rados.0.smithi122.stdout:3616: done (0 left) 2024-09-15T04:02:20.744 INFO:tasks.rados.rados.0.smithi122.stdout:3617: snap_remove snap 355 2024-09-15T04:02:21.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:02:21.739 INFO:tasks.rados.rados.0.smithi122.stdout:3617: done (0 left) 2024-09-15T04:02:21.739 INFO:tasks.rados.rados.0.smithi122.stdout:3618: write (excl) oid 26 current snap is 420 2024-09-15T04:02:21.739 INFO:tasks.rados.rados.0.smithi122.stdout:3618: seq_num 1272 ranges {578504=747679,2034272=566775,3130988=247516} 2024-09-15T04:02:21.747 INFO:tasks.rados.rados.0.smithi122.stdout:3618: writing smithi12223906-26 from 578504 to 1326183 tid 1 2024-09-15T04:02:21.755 INFO:tasks.rados.rados.0.smithi122.stdout:3618: writing smithi12223906-26 from 2034272 to 2601047 tid 2 2024-09-15T04:02:21.760 INFO:tasks.rados.rados.0.smithi122.stdout:3618: writing smithi12223906-26 from 3130988 to 3378504 tid 3 2024-09-15T04:02:21.760 INFO:tasks.rados.rados.0.smithi122.stdout:3619: write oid 47 current snap is 420 2024-09-15T04:02:21.761 INFO:tasks.rados.rados.0.smithi122.stdout:3619: seq_num 1273 ranges {595311=422928,1576401=710433,3042058=353253} 2024-09-15T04:02:21.766 INFO:tasks.rados.rados.0.smithi122.stdout:3619: writing smithi12223906-47 from 595311 to 1018239 tid 1 2024-09-15T04:02:21.774 INFO:tasks.rados.rados.0.smithi122.stdout:3619: writing smithi12223906-47 from 1576401 to 2286834 tid 2 2024-09-15T04:02:21.781 INFO:tasks.rados.rados.0.smithi122.stdout:3619: writing smithi12223906-47 from 3042058 to 3395311 tid 3 2024-09-15T04:02:21.781 INFO:tasks.rados.rados.0.smithi122.stdout:3620: snap_create 2024-09-15T04:02:21.781 INFO:tasks.rados.rados.0.smithi122.stdout:3618: finishing write tid 1 to smithi12223906-26 2024-09-15T04:02:21.781 INFO:tasks.rados.rados.0.smithi122.stdout:3618: finishing write tid 2 to smithi12223906-26 2024-09-15T04:02:21.781 INFO:tasks.rados.rados.0.smithi122.stdout:3618: finishing write tid 3 to smithi12223906-26 2024-09-15T04:02:21.782 INFO:tasks.rados.rados.0.smithi122.stdout:3618: finishing write tid 4 to smithi12223906-26 2024-09-15T04:02:21.782 INFO:tasks.rados.rados.0.smithi122.stdout:3618: finishing write tid 5 to smithi12223906-26 2024-09-15T04:02:21.782 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 2155 (ObjNum 1272 snap 420 seq_num 1272) dirty exists 2024-09-15T04:02:21.782 INFO:tasks.rados.rados.0.smithi122.stdout:3618: left oid 26 (ObjNum 1272 snap 420 seq_num 1272) 2024-09-15T04:02:21.782 INFO:tasks.rados.rados.0.smithi122.stdout:3619: finishing write tid 1 to smithi12223906-47 2024-09-15T04:02:21.785 INFO:tasks.rados.rados.0.smithi122.stdout:3619: finishing write tid 2 to smithi12223906-47 2024-09-15T04:02:21.786 INFO:tasks.rados.rados.0.smithi122.stdout:3619: finishing write tid 3 to smithi12223906-47 2024-09-15T04:02:21.786 INFO:tasks.rados.rados.0.smithi122.stdout:3619: finishing write tid 4 to smithi12223906-47 2024-09-15T04:02:21.787 INFO:tasks.rados.rados.0.smithi122.stdout:3619: finishing write tid 5 to smithi12223906-47 2024-09-15T04:02:21.787 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 2623 (ObjNum 1273 snap 420 seq_num 1273) dirty exists 2024-09-15T04:02:21.787 INFO:tasks.rados.rados.0.smithi122.stdout:3619: left oid 47 (ObjNum 1273 snap 420 seq_num 1273) 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient: get_monmap_and_config 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient: build_initial_monmap 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient: monmap: 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:21.821791+0000 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:21.821791+0000 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:21.825 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:21.826 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 1 Processor -- start 2024-09-15T04:02:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 1 -- start start 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient: init 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient: _add_conns ranks=[2,5,8,4,1,0,7,3,6] 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient(hunting): picked mon.noname-b con 0x7fe59c099910 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient(hunting): picked mon.noname-h con 0x7fe59c12ff80 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient(hunting): picked mon.noname-e con 0x7fe59c1303d0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c04ca10 con 0x7fe59c099910 2024-09-15T04:02:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c098be0 con 0x7fe59c12ff80 2024-09-15T04:02:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c099460 con 0x7fe59c1303d0 2024-09-15T04:02:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59bfff700 10 monclient(hunting): authenticate will time out at 1886.251568s 2024-09-15T04:02:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59affd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fe59c099910 0x7fe59c12f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:49698/0 (socket says 172.21.15.26:49698) 2024-09-15T04:02:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.817+0000 7fe59affd700 1 -- 172.21.15.26:0/3546423684 learned_addr learned my addr 172.21.15.26:0/3546423684 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682333529 0 0) 0x7fe59c099460 con 0x7fe59c1303d0 2024-09-15T04:02:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient(hunting): my global_id is 135647 2024-09-15T04:02:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe584003570 con 0x7fe59c1303d0 2024-09-15T04:02:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061134986 0 0) 0x7fe584003570 con 0x7fe59c1303d0 2024-09-15T04:02:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 >> v1:172.21.15.122:6791/0 conn(0x7fe59c12ff80 legacy=0x7fe59c134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 >> v1:172.21.15.26:6790/0 conn(0x7fe59c099910 legacy=0x7fe59c12f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:02:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 monclient: found mon.noname-e 2024-09-15T04:02:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:02:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c099110 con 0x7fe59c1303d0 2024-09-15T04:02:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: _finish_auth 0 2024-09-15T04:02:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe594001b30 con 0x7fe59c1303d0 2024-09-15T04:02:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: dump: 2024-09-15T04:02:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:21.837 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:21.838 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:21.839 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 5 monclient: authenticate success, global_id 135647 2024-09-15T04:02:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe594001de0 con 0x7fe59c1303d0 2024-09-15T04:02:21.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 1 -- 172.21.15.26:0/3546423684 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe5940025c0 con 0x7fe59c1303d0 2024-09-15T04:02:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 4 set_mon_vals no callback set 2024-09-15T04:02:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:21.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe599ffb700 10 monclient: dump: 2024-09-15T04:02:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:21.843 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 10 monclient: shutdown 2024-09-15T04:02:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 1 -- 172.21.15.26:0/3546423684 >> v1:172.21.15.26:6793/0 conn(0x7fe59c1303d0 legacy=0x7fe59c1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 1 -- 172.21.15.26:0/3546423684 shutdown_connections 2024-09-15T04:02:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.821+0000 7fe59bfff700 1 -- 172.21.15.26:0/3546423684 >> 172.21.15.26:0/3546423684 conn(0x7fe59c071860 msgr2=0x7fe59c0a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.825+0000 7fe59bfff700 1 -- 172.21.15.26:0/3546423684 shutdown_connections 2024-09-15T04:02:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.825+0000 7fe59bfff700 1 -- 172.21.15.26:0/3546423684 wait complete. 2024-09-15T04:02:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.825+0000 7fe59bfff700 10 monclient: build_initial_monmap 2024-09-15T04:02:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.825+0000 7fe59bfff700 10 monclient: monmap: 2024-09-15T04:02:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:21.852 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 1 Processor -- start 2024-09-15T04:02:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 1 -- start start 2024-09-15T04:02:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient: init 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient: _add_conns ranks=[5,1,6,7,0,3,8,2,4] 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient(hunting): picked mon.noname-h con 0x7fe59c099910 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient(hunting): picked mon.noname-f con 0x7fe59c12ff80 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient(hunting): picked mon.noname-d con 0x7fe59c1303d0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c098300 con 0x7fe59c12ff80 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c097ea0 con 0x7fe59c099910 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c06cd70 con 0x7fe59c1303d0 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59bfff700 10 monclient(hunting): authenticate will time out at 1886.261297s 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fe59c1303d0 0x7fe59c06b3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55616/0 (socket says 172.21.15.26:55616) 2024-09-15T04:02:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.829+0000 7fe59b7fe700 1 -- 172.21.15.26:0/2409692986 learned_addr learned my addr 172.21.15.26:0/2409692986 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627420968 0 0) 0x7fe59c098300 con 0x7fe59c12ff80 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): my global_id is 204832 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe588003570 con 0x7fe59c12ff80 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249130996 0 0) 0x7fe59c097ea0 con 0x7fe59c099910 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): my global_id is 254102 2024-09-15T04:02:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59c098300 con 0x7fe59c099910 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970463230 0 0) 0x7fe59c06cd70 con 0x7fe59c1303d0 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): my global_id is 195081 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59c097ea0 con 0x7fe59c1303d0 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 836759901 0 0) 0x7fe59c098300 con 0x7fe59c099910 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe59c06cd70 con 0x7fe59c099910 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe58c001990 con 0x7fe59c099910 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): dump: 2024-09-15T04:02:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1306418325 0 0) 0x7fe588003570 con 0x7fe59c12ff80 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe59c098300 con 0x7fe59c12ff80 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe5900019a0 con 0x7fe59c12ff80 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient(hunting): dump: 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:21.858 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3583622795 0 0) 0x7fe59c06cd70 con 0x7fe59c099910 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 >> v1:172.21.15.26:6792/0 conn(0x7fe59c1303d0 legacy=0x7fe59c06b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 >> v1:172.21.15.122:6789/0 conn(0x7fe59c12ff80 legacy=0x7fe59c06acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 monclient: found mon.h 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c06dbc0 con 0x7fe59c099910 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: _finish_auth 0 2024-09-15T04:02:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 5 monclient: authenticate success, global_id 254102 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 monclient: _renew_subs 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c119530 con 0x7fe59c099910 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 monclient: _renew_subs 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c12b420 con 0x7fe59c099910 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe58c0048b0 con 0x7fe59c099910 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe58c004de0 con 0x7fe59c099910 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe5997fa700 4 set_mon_vals no callback set 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 10 monclient: dump: 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe58c004fc0 con 0x7fe59c099910 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(816..816 src has 435..816) v4 ==== 3955+0+0 (unknown 2702332893 0 0) 0x7fe58c043ec0 con 0x7fe59c099910 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59a7fc700 10 monclient: get_auth_request con 0x7fe588037c60 auth_method 0 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.833+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c0050b0 con 0x7fe59c099910 2024-09-15T04:02:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.841+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fe58c0148a0 con 0x7fe59c099910 2024-09-15T04:02:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.841+0000 7fe582ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:21.841+0000 7fe582ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.064+0000 7fe59bfff700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:02:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.064+0000 7fe59bfff700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.064+0000 7fe59bfff700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.064+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 --> v1:172.21.15.122:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe57c0058e0 con 0x7fe59c099910 2024-09-15T04:02:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe582ffd700 1 -- 172.21.15.26:0/2409692986 <== mon.5 v1:172.21.15.122:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 2037425502) 0x7fe58c0146c0 con 0x7fe59c099910 2024-09-15T04:02:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe582ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe582ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:22.075 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":112,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":3,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 >> v1:172.21.15.122:6812/20895 conn(0x7fe588037c60 legacy=0x7fe58803a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 10 monclient: shutdown 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 >> v1:172.21.15.122:6791/0 conn(0x7fe59c099910 legacy=0x7fe59c06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 shutdown_connections 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 >> 172.21.15.26:0/2409692986 conn(0x7fe59c071860 msgr2=0x7fe59c0a0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 shutdown_connections 2024-09-15T04:02:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.068+0000 7fe59bfff700 1 -- 172.21.15.26:0/2409692986 wait complete. 2024-09-15T04:02:22.091 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T04:02:22.091 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T04:02:22.091 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.192+0000 7fd86d28a700 10 monclient: get_monmap_and_config 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.192+0000 7fd86d28a700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.192+0000 7fd86d28a700 10 monclient: monmap: 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:22.199577+0000 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:22.199577+0000 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.203 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.204 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 Processor -- start 2024-09-15T04:02:22.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- start start 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: init 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: _add_conns ranks=[5,1,0,4,7,6,3,2,8] 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient(hunting): picked mon.noname-h con 0x7fd8680a4740 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient(hunting): picked mon.noname-f con 0x7fd86809aca0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient(hunting): picked mon.noname-a con 0x7fd868136f40 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86804ca10 con 0x7fd868136f40 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86809c120 con 0x7fd86809aca0 2024-09-15T04:02:22.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86809d7a0 con 0x7fd8680a4740 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient(hunting): authenticate will time out at 1886.628884s 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86ca89700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd868136f40 0x7fd868139320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40296/0 (socket says 172.21.15.26:40296) 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86ca89700 1 -- 172.21.15.26:0/3957383575 learned_addr learned my addr 172.21.15.26:0/3957383575 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777300057 0 0) 0x7fd86804ca10 con 0x7fd868136f40 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient(hunting): my global_id is 194967 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd850003570 con 0x7fd868136f40 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3143768187 0 0) 0x7fd850003570 con 0x7fd868136f40 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 >> v1:172.21.15.122:6791/0 conn(0x7fd8680a4740 legacy=0x7fd86809dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 >> v1:172.21.15.122:6789/0 conn(0x7fd86809aca0 legacy=0x7fd868135820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 monclient: found mon.noname-a 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86809b510 con 0x7fd868136f40 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd860001b30 con 0x7fd868136f40 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: dump: 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.213 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.214 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 5 monclient: authenticate success, global_id 194967 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd860001de0 con 0x7fd868136f40 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 1 -- 172.21.15.26:0/3957383575 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd860004fb0 con 0x7fd868136f40 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd866ffd700 10 monclient: dump: 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.215 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 4 set_mon_vals no callback set 2024-09-15T04:02:22.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: shutdown 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- 172.21.15.26:0/3957383575 >> v1:172.21.15.26:6789/0 conn(0x7fd868136f40 legacy=0x7fd868139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- 172.21.15.26:0/3957383575 shutdown_connections 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- 172.21.15.26:0/3957383575 >> 172.21.15.26:0/3957383575 conn(0x7fd8680638b0 msgr2=0x7fd8680a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- 172.21.15.26:0/3957383575 shutdown_connections 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 1 -- 172.21.15.26:0/3957383575 wait complete. 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.196+0000 7fd86d28a700 10 monclient: monmap: 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.217 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 Processor -- start 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 -- start start 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: init 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: _add_conns ranks=[6,5,7,3,4,8,2,0,1] 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient(hunting): picked mon.noname-d con 0x7fd86809aca0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient(hunting): picked mon.noname-h con 0x7fd8680a4740 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient(hunting): picked mon.noname-i con 0x7fd868136f40 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86806e6a0 con 0x7fd8680a4740 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86806e2b0 con 0x7fd86809aca0 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd868136880 con 0x7fd868136f40 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient(hunting): authenticate will time out at 1886.633125s 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd867fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd86809aca0 0x7fd86813f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55624/0 (socket says 172.21.15.26:55624) 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd867fff700 1 -- 172.21.15.26:0/3225466820 learned_addr learned my addr 172.21.15.26:0/3225466820 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746852442 0 0) 0x7fd86806e2b0 con 0x7fd86809aca0 2024-09-15T04:02:22.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): my global_id is 195090 2024-09-15T04:02:22.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd854003570 con 0x7fd86809aca0 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552869634 0 0) 0x7fd868136880 con 0x7fd868136f40 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): my global_id is 254104 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd86806e2b0 con 0x7fd868136f40 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986000381 0 0) 0x7fd86806e6a0 con 0x7fd8680a4740 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): my global_id is 254120 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd868136880 con 0x7fd8680a4740 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 531193465 0 0) 0x7fd854003570 con 0x7fd86809aca0 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd86806e6a0 con 0x7fd86809aca0 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd858001b20 con 0x7fd86809aca0 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): dump: 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.226 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:22.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1175391959 0 0) 0x7fd868136880 con 0x7fd8680a4740 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd854003570 con 0x7fd8680a4740 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd85c001a60 con 0x7fd8680a4740 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): dump: 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.228 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3346437908 0 0) 0x7fd86806e2b0 con 0x7fd868136f40 2024-09-15T04:02:22.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd868136880 con 0x7fd868136f40 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd860001af0 con 0x7fd868136f40 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient(hunting): dump: 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.230 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1782072642 0 0) 0x7fd86806e6a0 con 0x7fd86809aca0 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.122:6792/0 conn(0x7fd868136f40 legacy=0x7fd868147bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.122:6791/0 conn(0x7fd8680a4740 legacy=0x7fd868136170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 monclient: found mon.d 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868134960 con 0x7fd86809aca0 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 5 monclient: authenticate success, global_id 195090 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: _renew_subs 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd868132d30 con 0x7fd86809aca0 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: _renew_subs 2024-09-15T04:02:22.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd868140750 con 0x7fd86809aca0 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd858002270 con 0x7fd86809aca0 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd8580027a0 con 0x7fd86809aca0 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd8667fc700 4 set_mon_vals no callback set 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd84ffff700 10 monclient: dump: 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.233 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.234 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.200+0000 7fd86d28a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd858002980 con 0x7fd86809aca0 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd848006590 con 0x7fd848002840 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(816..816 src has 435..816) v4 ==== 3955+0+0 (unknown 2702332893 0 0) 0x7fd858044150 con 0x7fd86809aca0 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733979211 0 0) 0x7fd848006590 con 0x7fd848002840 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient(hunting): my global_id is 194976 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd86806e6a0 con 0x7fd848002840 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd86ca89700 10 monclient: get_auth_request con 0x7fd854038080 auth_method 0 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 212872845 0 0) 0x7fd86806e6a0 con 0x7fd848002840 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd848006590 con 0x7fd848002840 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd85c002230 con 0x7fd848002840 2024-09-15T04:02:22.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient: dump: 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.236 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4206792989 0 0) 0x7fd848006590 con 0x7fd848002840 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd84800a130 con 0x7fd848002840 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd85c002630 con 0x7fd848002840 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.26:6789/0 conn(0x7fd848002840 legacy=0x7fd848004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.204+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.26:6789/0 conn(0x7fd848002840 legacy=0x7fd848004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd86d28a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd848009640 con 0x7fd8480090b0 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990668557 0 0) 0x7fd848009640 con 0x7fd8480090b0 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd84ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd84ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd84ffff700 10 monclient(hunting): my global_id is 194985 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.212+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd848006590 con 0x7fd8480090b0 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3518840586 0 0) 0x7fd848006590 con 0x7fd8480090b0 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd848009640 con 0x7fd8480090b0 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd85c002e10 con 0x7fd8480090b0 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 10 monclient: dump: 2024-09-15T04:02:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2968071084 0 0) 0x7fd848009640 con 0x7fd8480090b0 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd8480098f0 con 0x7fd8480090b0 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 3946796819) 0x7fd85c005060 con 0x7fd8480090b0 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.26:6789/0 conn(0x7fd8480090b0 legacy=0x7fd848008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd84ffff700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.26:6789/0 conn(0x7fd8480090b0 legacy=0x7fd848008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:22.241 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.242 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.243 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:22.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.245 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.247 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.248 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.250 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.251 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.252 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:22.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.258 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.259 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.260 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.261 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.261 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.261 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:22.261 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:22.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.122:6812/20895 conn(0x7fd854038080 legacy=0x7fd85403a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 10 monclient: shutdown 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 >> v1:172.21.15.26:6792/0 conn(0x7fd86809aca0 legacy=0x7fd86813f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86ca89700 1 -- 172.21.15.26:0/3225466820 reap_dead start 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86ca89700 1 -- 172.21.15.26:0/3225466820 reap_dead start 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 shutdown_connections 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 >> 172.21.15.26:0/3225466820 conn(0x7fd8680638b0 msgr2=0x7fd8680a2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 shutdown_connections 2024-09-15T04:02:22.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.216+0000 7fd86d28a700 1 -- 172.21.15.26:0/3225466820 wait complete. 2024-09-15T04:02:22.264 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:22.264 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T04:02:22.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T04:02:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient: get_monmap_and_config 2024-09-15T04:02:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient: monmap: 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:22.377150+0000 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:22.377150+0000 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.381 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.382 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.382 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 1 Processor -- start 2024-09-15T04:02:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 1 -- start start 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient: init 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient: _add_conns ranks=[0,8,6,4,1,3,2,5,7] 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient(hunting): picked mon.noname-a con 0x7f3a78070de0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient(hunting): picked mon.noname-e con 0x7f3a7809bdd0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient(hunting): picked mon.noname-d con 0x7f3a78136b20 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7804ca10 con 0x7f3a78070de0 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a780999f0 con 0x7f3a78136b20 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7809c8b0 con 0x7f3a7809bdd0 2024-09-15T04:02:22.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7db2f700 10 monclient(hunting): authenticate will time out at 1886.807145s 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7d32e700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f3a78136b20 0x7f3a78138f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55640/0 (socket says 172.21.15.26:55640) 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a77fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f3a7809bdd0 0x7f3a7809c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50464/0 (socket says 172.21.15.26:50464) 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a7d32e700 1 -- 172.21.15.26:0/989476453 learned_addr learned my addr 172.21.15.26:0/989476453 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772158036 0 0) 0x7f3a780999f0 con 0x7f3a78136b20 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 10 monclient(hunting): my global_id is 195099 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a60003570 con 0x7f3a78136b20 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727774090 0 0) 0x7f3a7809c8b0 con 0x7f3a7809bdd0 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 10 monclient(hunting): my global_id is 135656 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.372+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a780999f0 con 0x7f3a7809bdd0 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107980610 0 0) 0x7f3a7804ca10 con 0x7f3a78070de0 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient(hunting): my global_id is 194994 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7809c8b0 con 0x7f3a78070de0 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1275156803 0 0) 0x7f3a60003570 con 0x7f3a78136b20 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 >> v1:172.21.15.26:6793/0 conn(0x7f3a7809bdd0 legacy=0x7f3a7809c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 >> v1:172.21.15.26:6789/0 conn(0x7f3a78070de0 legacy=0x7f3a78130d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 monclient: found mon.noname-d 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78098d10 con 0x7f3a78136b20 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 5 monclient: authenticate success, global_id 195099 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a70001b30 con 0x7f3a78136b20 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: dump: 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.388 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3a70001d20 con 0x7f3a78136b20 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 1 -- 172.21.15.26:0/989476453 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a70004c70 con 0x7f3a78136b20 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 4 set_mon_vals no callback set 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a777fe700 10 monclient: dump: 2024-09-15T04:02:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 10 monclient: shutdown 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 -- 172.21.15.26:0/989476453 >> v1:172.21.15.26:6792/0 conn(0x7f3a78136b20 legacy=0x7f3a78138f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 -- 172.21.15.26:0/989476453 shutdown_connections 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 -- 172.21.15.26:0/989476453 >> 172.21.15.26:0/989476453 conn(0x7f3a780a2ea0 msgr2=0x7f3a7809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 -- 172.21.15.26:0/989476453 shutdown_connections 2024-09-15T04:02:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 -- 172.21.15.26:0/989476453 wait complete. 2024-09-15T04:02:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 10 monclient: monmap: 2024-09-15T04:02:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 Processor -- start 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 1 -- start start 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.376+0000 7f3a7db2f700 10 monclient: init 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient: _add_conns ranks=[7,8,6,0,1,5,2,4,3] 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient(hunting): picked mon.noname-i con 0x7f3a78070de0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient(hunting): picked mon.noname-e con 0x7f3a7809bdd0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient(hunting): picked mon.noname-d con 0x7f3a78136b20 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7806cf40 con 0x7f3a78136b20 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7806abe0 con 0x7f3a78070de0 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7806ddd0 con 0x7f3a7809bdd0 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient(hunting): authenticate will time out at 1886.813013s 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a77fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f3a7809bdd0 0x7f3a7806a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50472/0 (socket says 172.21.15.26:50472) 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a77fff700 1 -- 172.21.15.26:0/3354912800 learned_addr learned my addr 172.21.15.26:0/3354912800 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710789723 0 0) 0x7f3a7806ddd0 con 0x7f3a7809bdd0 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): my global_id is 135665 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a64003570 con 0x7f3a7809bdd0 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971054934 0 0) 0x7f3a7806cf40 con 0x7f3a78136b20 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): my global_id is 195108 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7806ddd0 con 0x7f3a78136b20 2024-09-15T04:02:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328305404 0 0) 0x7f3a7806abe0 con 0x7f3a78070de0 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): my global_id is 254113 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7806cf40 con 0x7f3a78070de0 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2518454438 0 0) 0x7f3a7806ddd0 con 0x7f3a78136b20 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3a7806abe0 con 0x7f3a78136b20 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a70001a30 con 0x7f3a78136b20 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2075250775 0 0) 0x7f3a64003570 con 0x7f3a7809bdd0 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3a7806ddd0 con 0x7f3a7809bdd0 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a68001470 con 0x7f3a7809bdd0 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 316022553 0 0) 0x7f3a7806abe0 con 0x7f3a78136b20 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.26:6793/0 conn(0x7f3a7809bdd0 legacy=0x7f3a7806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.122:6792/0 conn(0x7f3a78070de0 legacy=0x7f3a78069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 monclient: found mon.d 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7806bd00 con 0x7f3a78136b20 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 5 monclient: authenticate success, global_id 195108 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient: _renew_subs 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7806cc60 con 0x7f3a78136b20 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient: _renew_subs 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3a70004e60 con 0x7f3a78136b20 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7806d920 con 0x7f3a78136b20 2024-09-15T04:02:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a70005310 con 0x7f3a78136b20 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a76ffd700 4 set_mon_vals no callback set 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 10 monclient: dump: 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.380+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3a7000fcd0 con 0x7f3a78136b20 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a7db2f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3a7806db70 con 0x7f3a7802ae80 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(816..816 src has 435..816) v4 ==== 3955+0+0 (unknown 2702332893 0 0) 0x7f3a700440c0 con 0x7f3a78136b20 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a77fff700 10 monclient: get_auth_request con 0x7f3a64037d40 auth_method 0 2024-09-15T04:02:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854484878 0 0) 0x7f3a7806db70 con 0x7f3a7802ae80 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient(hunting): my global_id is 254099 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7806abe0 con 0x7f3a7802ae80 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1529694879 0 0) 0x7f3a7806abe0 con 0x7f3a7802ae80 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3a7806db70 con 0x7f3a7802ae80 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a6c002310 con 0x7f3a7802ae80 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient: dump: 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1430760166 0 0) 0x7f3a7806db70 con 0x7f3a7802ae80 2024-09-15T04:02:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a7804bbf0 con 0x7f3a7802ae80 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f3a6c002900 con 0x7f3a7802ae80 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.26:6790/0 conn(0x7f3a7802ae80 legacy=0x7f3a7805e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.384+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.26:6790/0 conn(0x7f3a7802ae80 legacy=0x7f3a7805e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.392+0000 7f3a7db2f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.392+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3a7806fac0 con 0x7f3a781356c0 2024-09-15T04:02:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250205329 0 0) 0x7f3a7806fac0 con 0x7f3a781356c0 2024-09-15T04:02:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient(hunting): my global_id is 254108 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a7806db70 con 0x7f3a781356c0 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3290875079 0 0) 0x7f3a7806db70 con 0x7f3a781356c0 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3a7806fac0 con 0x7f3a781356c0 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3a6c002d80 con 0x7f3a781356c0 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient: dump: 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3199567797 0 0) 0x7f3a7806fac0 con 0x7f3a781356c0 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3a7806b150 con 0x7f3a781356c0 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 1889932643) 0x7f3a6c0054b0 con 0x7f3a781356c0 2024-09-15T04:02:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.26:6790/0 conn(0x7f3a781356c0 legacy=0x7f3a78131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a74ff9700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.26:6790/0 conn(0x7f3a781356c0 legacy=0x7f3a78131d60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:22.419 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:22.420 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:22.421 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:22.422 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:22.423 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.424 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:22.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.427 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.432 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.433 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.434 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.438 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:22.439 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.122:6812/20895 conn(0x7f3a64037d40 legacy=0x7f3a6403a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 10 monclient: shutdown 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 >> v1:172.21.15.26:6792/0 conn(0x7f3a78136b20 legacy=0x7f3a78147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7d32e700 1 -- 172.21.15.26:0/3354912800 reap_dead start 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7d32e700 1 -- 172.21.15.26:0/3354912800 reap_dead start 2024-09-15T04:02:22.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 shutdown_connections 2024-09-15T04:02:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 >> 172.21.15.26:0/3354912800 conn(0x7f3a780a2ea0 msgr2=0x7f3a78136050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 shutdown_connections 2024-09-15T04:02:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.396+0000 7f3a7db2f700 1 -- 172.21.15.26:0/3354912800 wait complete. 2024-09-15T04:02:22.442 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:22.442 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T04:02:22.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:02:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient: get_monmap_and_config 2024-09-15T04:02:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient: monmap: 2024-09-15T04:02:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.559 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:22.555982+0000 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:22.555982+0000 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.560 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 1 Processor -- start 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 1 -- start start 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient: init 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient: _add_conns ranks=[3,7,5,1,8,2,0,4,6] 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient(hunting): picked mon.noname-g con 0x7f9308132b40 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient(hunting): picked mon.noname-i con 0x7f930809faa0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient(hunting): picked mon.noname-h con 0x7f9308138d40 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930804ca10 con 0x7f9308132b40 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930809d5d0 con 0x7f9308138d40 2024-09-15T04:02:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9308133620 con 0x7f930809faa0 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f930d634700 10 monclient(hunting): authenticate will time out at 1886.986826s 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f9307fff700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f9308132b40 0x7f9308132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:51200/0 (socket says 172.21.15.26:51200) 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f9307fff700 1 -- 172.21.15.26:0/2724183832 learned_addr learned my addr 172.21.15.26:0/2724183832 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079121839 0 0) 0x7f930804ca10 con 0x7f9308132b40 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): my global_id is 254100 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92f4003570 con 0x7f9308132b40 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373608479 0 0) 0x7f9308133620 con 0x7f930809faa0 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): my global_id is 254122 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f930804ca10 con 0x7f930809faa0 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781529309 0 0) 0x7f930809d5d0 con 0x7f9308138d40 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 10 monclient(hunting): my global_id is 254129 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.552+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9308133620 con 0x7f9308138d40 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457061306 0 0) 0x7f92f4003570 con 0x7f9308132b40 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 >> v1:172.21.15.122:6792/0 conn(0x7f930809faa0 legacy=0x7f930809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 >> v1:172.21.15.122:6791/0 conn(0x7f9308138d40 legacy=0x7f930813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 monclient: found mon.noname-g 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93080a0cc0 con 0x7f9308132b40 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f80017b0 con 0x7f9308132b40 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 5 monclient: authenticate success, global_id 254100 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T04:02:22.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: dump: 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f92f80019a0 con 0x7f9308132b40 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 1 -- 172.21.15.26:0/2724183832 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f80048f0 con 0x7f9308132b40 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 4 set_mon_vals no callback set 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f93077fe700 10 monclient: dump: 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.570 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 10 monclient: shutdown 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 1 -- 172.21.15.26:0/2724183832 >> v1:172.21.15.122:6790/0 conn(0x7f9308132b40 legacy=0x7f9308132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 1 -- 172.21.15.26:0/2724183832 shutdown_connections 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 1 -- 172.21.15.26:0/2724183832 >> 172.21.15.26:0/2724183832 conn(0x7f930806e370 msgr2=0x7f93080a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 1 -- 172.21.15.26:0/2724183832 shutdown_connections 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 1 -- 172.21.15.26:0/2724183832 wait complete. 2024-09-15T04:02:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.556+0000 7f930d634700 10 monclient: monmap: 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 1 Processor -- start 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 1 -- start start 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient: init 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient: _add_conns ranks=[3,5,4,1,6,7,8,0,2] 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient(hunting): picked mon.noname-g con 0x7f930809faa0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient(hunting): picked mon.noname-h con 0x7f9308132b40 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient(hunting): picked mon.noname-c con 0x7f9308138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930809d010 con 0x7f930809faa0 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9308135040 con 0x7f9308138d40 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930809ccc0 con 0x7f9308132b40 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930d634700 10 monclient(hunting): authenticate will time out at 1886.993814s 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9307fff700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7f930809faa0 0x7f930809a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:51202/0 (socket says 172.21.15.26:51202) 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f930ce33700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f9308138d40 0x7f930814a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40952/0 (socket says 172.21.15.26:40952) 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9307fff700 1 -- 172.21.15.26:0/3260708586 learned_addr learned my addr 172.21.15.26:0/3260708586 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327540621 0 0) 0x7f9308135040 con 0x7f9308138d40 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): my global_id is 204826 2024-09-15T04:02:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e8003570 con 0x7f9308138d40 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418934739 0 0) 0x7f93080a0cc0 con 0x7f930809faa0 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): my global_id is 254109 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9308135040 con 0x7f930809faa0 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149597708 0 0) 0x7f930809ccc0 con 0x7f9308132b40 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): my global_id is 254138 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93080a0cc0 con 0x7f9308132b40 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1178227588 0 0) 0x7f92e8003570 con 0x7f9308138d40 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f930809ccc0 con 0x7f9308138d40 2024-09-15T04:02:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9300001d40 con 0x7f9308138d40 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3249983738 0 0) 0x7f93080a0cc0 con 0x7f9308132b40 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92e8003570 con 0x7f9308132b40 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2339249423 0 0) 0x7f9308135040 con 0x7f930809faa0 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f93080a0cc0 con 0x7f930809faa0 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f0001a50 con 0x7f9308132b40 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f8001770 con 0x7f930809faa0 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.585 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2312568887 0 0) 0x7f92e8003570 con 0x7f9308132b40 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.26:6791/0 conn(0x7f9308138d40 legacy=0x7f930814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.560+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.122:6790/0 conn(0x7f930809faa0 legacy=0x7f930809a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 monclient: found mon.h 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930809c820 con 0x7f9308132b40 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 5 monclient: authenticate success, global_id 254138 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 10 monclient: _renew_subs 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9308134a80 con 0x7f9308132b40 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 10 monclient: _renew_subs 2024-09-15T04:02:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 10 monclient: _send_mon_message to mon.h at v1:172.21.15.122:6791/0 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.122:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f930809bed0 con 0x7f9308132b40 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f92f0001f50 con 0x7f9308132b40 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f0002400 con 0x7f9308132b40 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9306ffd700 4 set_mon_vals no callback set 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: dump: 2024-09-15T04:02:22.588 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.589 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f92f0002560 con 0x7f9308132b40 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f92ec006590 con 0x7f92ec002840 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.5 v1:172.21.15.122:6791/0 8 ==== osd_map(816..816 src has 435..816) v4 ==== 3955+0+0 (unknown 2702332893 0 0) 0x7f92f0041340 con 0x7f9308132b40 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506525976 0 0) 0x7f92ec006590 con 0x7f92ec002840 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient(hunting): my global_id is 204835 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e8003570 con 0x7f92ec002840 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f930ce33700 10 monclient: get_auth_request con 0x7f92e8038200 auth_method 0 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 153184580 0 0) 0x7f92e8003570 con 0x7f92ec002840 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92ec006590 con 0x7f92ec002840 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f80011b0 con 0x7f92ec002840 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: dump: 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 124611110 0 0) 0x7f92ec006590 con 0x7f92ec002840 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92ec00a130 con 0x7f92ec002840 2024-09-15T04:02:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f92f8007530 con 0x7f92ec002840 2024-09-15T04:02:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.26:6791/0 conn(0x7f92ec002840 legacy=0x7f92ec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.564+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.26:6791/0 conn(0x7f92ec002840 legacy=0x7f92ec004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f92ec007500 con 0x7f92ec0068b0 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938457676 0 0) 0x7f92ec007500 con 0x7f92ec0068b0 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient(hunting): my global_id is 204844 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92ec006590 con 0x7f92ec0068b0 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2100437955 0 0) 0x7f92ec006590 con 0x7f92ec0068b0 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92ec007500 con 0x7f92ec0068b0 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f92f8005d20 con 0x7f92ec0068b0 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient: dump: 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3366479528 0 0) 0x7f92ec007500 con 0x7f92ec0068b0 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f92ec008760 con 0x7f92ec0068b0 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 2777252759) 0x7f92f8007250 con 0x7f92ec0068b0 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.26:6791/0 conn(0x7f92ec0068b0 legacy=0x7f92ec006d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f9304ff9700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.26:6791/0 conn(0x7f92ec0068b0 legacy=0x7f92ec006d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 3, 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.122:6812/20895 conn(0x7f92e8038200 legacy=0x7f92e803a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 10 monclient: shutdown 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 >> v1:172.21.15.122:6791/0 conn(0x7f9308132b40 legacy=0x7f930809acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930ce33700 1 -- 172.21.15.26:0/3260708586 reap_dead start 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930ce33700 1 -- 172.21.15.26:0/3260708586 reap_dead start 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 shutdown_connections 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 >> 172.21.15.26:0/3260708586 conn(0x7f930806e370 msgr2=0x7f93080a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 shutdown_connections 2024-09-15T04:02:22.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.576+0000 7f930d634700 1 -- 172.21.15.26:0/3260708586 wait complete. 2024-09-15T04:02:22.620 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:22.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T04:02:22.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient: get_monmap_and_config 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient: monmap: 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:22.725625+0000 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:22.725625+0000 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.730 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 1 Processor -- start 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 1 -- start start 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient: init 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient: _add_conns ranks=[3,5,4,8,0,2,7,6,1] 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient(hunting): picked mon.noname-g con 0x7f0ff009bb40 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient(hunting): picked mon.noname-h con 0x7f0ff009ced0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient(hunting): picked mon.noname-c con 0x7f0ff0098a70 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff004ca10 con 0x7f0ff009bb40 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff009cc70 con 0x7f0ff0098a70 2024-09-15T04:02:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff009a760 con 0x7f0ff009ced0 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0ff4c12700 10 monclient(hunting): authenticate will time out at 1887.155310s 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0feed9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f0ff0098a70 0x7f0ff013f2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40978/0 (socket says 172.21.15.26:40978) 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.720+0000 7f0feed9d700 1 -- 172.21.15.26:0/3008483795 learned_addr learned my addr 172.21.15.26:0/3008483795 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903561121 0 0) 0x7f0ff009cc70 con 0x7f0ff0098a70 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): my global_id is 204853 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd8003570 con 0x7f0ff0098a70 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521407875 0 0) 0x7f0ff009a760 con 0x7f0ff009ced0 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): my global_id is 254147 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff009cc70 con 0x7f0ff009ced0 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509246818 0 0) 0x7f0ff004ca10 con 0x7f0ff009bb40 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient(hunting): my global_id is 254118 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff009a760 con 0x7f0ff009bb40 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239359295 0 0) 0x7f0fd8003570 con 0x7f0ff0098a70 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 >> v1:172.21.15.122:6791/0 conn(0x7f0ff009ced0 legacy=0x7f0ff009a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 >> v1:172.21.15.122:6790/0 conn(0x7f0ff009bb40 legacy=0x7f0ff009bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 monclient: found mon.noname-c 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff009e290 con 0x7f0ff0098a70 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe80017b0 con 0x7f0ff0098a70 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 5 monclient: authenticate success, global_id 204853 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: dump: 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0fe8001a90 con 0x7f0ff0098a70 2024-09-15T04:02:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 1 -- 172.21.15.26:0/3008483795 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe8004c30 con 0x7f0ff0098a70 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 4 set_mon_vals no callback set 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0fed59a700 10 monclient: dump: 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 10 monclient: shutdown 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 -- 172.21.15.26:0/3008483795 >> v1:172.21.15.26:6791/0 conn(0x7f0ff0098a70 legacy=0x7f0ff013f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 -- 172.21.15.26:0/3008483795 shutdown_connections 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 -- 172.21.15.26:0/3008483795 >> 172.21.15.26:0/3008483795 conn(0x7f0ff0063f30 msgr2=0x7f0ff00a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 -- 172.21.15.26:0/3008483795 shutdown_connections 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 -- 172.21.15.26:0/3008483795 wait complete. 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 10 monclient: monmap: 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 Processor -- start 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 1 -- start start 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.724+0000 7f0ff4c12700 10 monclient: init 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient: _add_conns ranks=[3,8,2,6,1,4,7,0,5] 2024-09-15T04:02:22.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient(hunting): picked mon.noname-g con 0x7f0ff0098a70 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient(hunting): picked mon.noname-e con 0x7f0ff009bb40 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient(hunting): picked mon.noname-b con 0x7f0ff009ced0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff013fef0 con 0x7f0ff009ced0 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0138e00 con 0x7f0ff0098a70 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff006e490 con 0x7f0ff009bb40 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient(hunting): authenticate will time out at 1887.160261s 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fedd9b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f0ff009bb40 0x7f0ff0137fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50484/0 (socket says 172.21.15.26:50484) 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fedd9b700 1 -- 172.21.15.26:0/4091882917 learned_addr learned my addr 172.21.15.26:0/4091882917 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273631068 0 0) 0x7f0ff013fef0 con 0x7f0ff009ced0 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): my global_id is 254117 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd0003570 con 0x7f0ff009ced0 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673666805 0 0) 0x7f0ff006e490 con 0x7f0ff009bb40 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): my global_id is 135674 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff013fef0 con 0x7f0ff009bb40 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975210101 0 0) 0x7f0ff0138e00 con 0x7f0ff0098a70 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): my global_id is 254127 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff006e490 con 0x7f0ff0098a70 2024-09-15T04:02:22.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3894126868 0 0) 0x7f0fd0003570 con 0x7f0ff009ced0 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ff0138e00 con 0x7f0ff009ced0 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe8004b70 con 0x7f0ff009ced0 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.750 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4048119213 0 0) 0x7f0ff013fef0 con 0x7f0ff009bb40 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fd0003570 con 0x7f0ff009bb40 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe0001d40 con 0x7f0ff009bb40 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:22.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 650990816 0 0) 0x7f0ff006e490 con 0x7f0ff0098a70 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0ff013fef0 con 0x7f0ff0098a70 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe40019e0 con 0x7f0ff0098a70 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.753 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 97878246 0 0) 0x7f0ff0138e00 con 0x7f0ff009ced0 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.26:6793/0 conn(0x7f0ff009bb40 legacy=0x7f0ff0137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.122:6790/0 conn(0x7f0ff0098a70 legacy=0x7f0ff013f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 monclient: found mon.b 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff013e880 con 0x7f0ff009ced0 2024-09-15T04:02:22.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 5 monclient: authenticate success, global_id 254117 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient: _renew_subs 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff013eb60 con 0x7f0ff009ced0 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient: _renew_subs 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff014f580 con 0x7f0ff009ced0 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0fe8001860 con 0x7f0ff009ced0 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fecd99700 4 set_mon_vals no callback set 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe8001230 con 0x7f0ff009ced0 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient: dump: 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.760 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453211798 0 0) 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:02:22.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 10 monclient(hunting): my global_id is 195117 2024-09-15T04:02:22.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff0138e00 con 0x7f0fd4002840 2024-09-15T04:02:22.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.728+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0fe800fb10 con 0x7f0ff009ced0 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(816..816 src has 435..816) v4 ==== 3955+0+0 (unknown 2702332893 0 0) 0x7f0fe8044000 con 0x7f0ff009ced0 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1573120661 0 0) 0x7f0ff0138e00 con 0x7f0fd4002840 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe40020b0 con 0x7f0fd4002840 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 10 monclient: dump: 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fedd9b700 10 monclient: get_auth_request con 0x7f0fd0037f40 auth_method 0 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1575546990 0 0) 0x7f0fd4006590 con 0x7f0fd4002840 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fd400a130 con 0x7f0fd4002840 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0fe4002530 con 0x7f0fd4002840 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.26:6792/0 conn(0x7f0fd4002840 legacy=0x7f0fd4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.732+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.26:6792/0 conn(0x7f0fd4002840 legacy=0x7f0fd4004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.744+0000 7f0ff4c12700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.744+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0fd4007520 con 0x7f0fd40068d0 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.748+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075017521 0 0) 0x7f0fd4007520 con 0x7f0fd40068d0 2024-09-15T04:02:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.748+0000 7f0fde7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.748+0000 7f0fde7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.748+0000 7f0fde7fc700 10 monclient(hunting): my global_id is 195126 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.748+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd4006590 con 0x7f0fd40068d0 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 850278007 0 0) 0x7f0fd4006590 con 0x7f0fd40068d0 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0fd4007520 con 0x7f0fd40068d0 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0fe4002d90 con 0x7f0fd40068d0 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 10 monclient: dump: 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 195964474 0 0) 0x7f0fd4007520 con 0x7f0fd40068d0 2024-09-15T04:02:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0fd4008760 con 0x7f0fd40068d0 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 2264040235) 0x7f0fe40051d0 con 0x7f0fd40068d0 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.26:6792/0 conn(0x7f0fd40068d0 legacy=0x7f0fd4006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0fde7fc700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.26:6792/0 conn(0x7f0fd40068d0 legacy=0x7f0fd4006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:22.774 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.775 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:22.776 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:22.777 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:22.778 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.779 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:22.781 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.782 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.784 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.787 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.788 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.789 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.790 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.791 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.792 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.793 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.794 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:22.794 INFO:tasks.rados.rados.0.smithi122.stdout:3618: done (2 left) 2024-09-15T04:02:22.795 INFO:tasks.rados.rados.0.smithi122.stdout:3619: done (1 left) 2024-09-15T04:02:22.795 INFO:tasks.rados.rados.0.smithi122.stdout:3620: done (0 left) 2024-09-15T04:02:22.795 INFO:tasks.rados.rados.0.smithi122.stdout:3621: delete oid 4 current snap is 421 2024-09-15T04:02:22.795 INFO:tasks.rados.rados.0.smithi122.stdout:3621: done (0 left) 2024-09-15T04:02:22.795 INFO:tasks.rados.rados.0.smithi122.stdout:3622: snap_remove snap 354 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.122:6812/20895 conn(0x7f0fd0037f40 legacy=0x7f0fd003a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 10 monclient: shutdown 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 >> v1:172.21.15.26:6790/0 conn(0x7f0ff009ced0 legacy=0x7f0ff01386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0feed9d700 1 -- 172.21.15.26:0/4091882917 reap_dead start 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0feed9d700 1 -- 172.21.15.26:0/4091882917 reap_dead start 2024-09-15T04:02:22.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 shutdown_connections 2024-09-15T04:02:22.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 >> 172.21.15.26:0/4091882917 conn(0x7f0ff0063f30 msgr2=0x7f0ff00a0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 shutdown_connections 2024-09-15T04:02:22.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.752+0000 7f0ff4c12700 1 -- 172.21.15.26:0/4091882917 wait complete. 2024-09-15T04:02:22.797 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:22.797 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T04:02:22.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient: get_monmap_and_config 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient: monmap: 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:22.904763+0000 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:22.904763+0000 2024-09-15T04:02:22.908 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.909 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 1 Processor -- start 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 1 -- start start 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient: init 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient: _add_conns ranks=[4,2,8,6,0,1,3,7,5] 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient(hunting): picked mon.noname-c con 0x7fd26809bfb0 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient(hunting): picked mon.noname-b con 0x7fd268130ff0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient(hunting): picked mon.noname-e con 0x7fd2681313c0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26804ca10 con 0x7fd268130ff0 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26809f600 con 0x7fd26809bfb0 2024-09-15T04:02:22.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26809c380 con 0x7fd2681313c0 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26f1cb700 10 monclient(hunting): authenticate will time out at 1887.335770s 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26d9c8700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fd268130ff0 0x7fd268135c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:49732/0 (socket says 172.21.15.26:49732) 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26d9c8700 1 -- 172.21.15.26:0/3043728337 learned_addr learned my addr 172.21.15.26:0/3043728337 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.900+0000 7fd26e1c9700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd26809bfb0 0x7fd2681308e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40982/0 (socket says 172.21.15.26:40982) 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187616793 0 0) 0x7fd26809c380 con 0x7fd2681313c0 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): my global_id is 135683 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd254003570 con 0x7fd2681313c0 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821850230 0 0) 0x7fd26804ca10 con 0x7fd268130ff0 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): my global_id is 254126 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd26809c380 con 0x7fd268130ff0 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150977499 0 0) 0x7fd26809f600 con 0x7fd26809bfb0 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient(hunting): my global_id is 204862 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd26804ca10 con 0x7fd26809bfb0 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600782215 0 0) 0x7fd254003570 con 0x7fd2681313c0 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 >> v1:172.21.15.26:6791/0 conn(0x7fd26809bfb0 legacy=0x7fd2681308e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 >> v1:172.21.15.26:6790/0 conn(0x7fd268130ff0 legacy=0x7fd268135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 monclient: found mon.noname-e 2024-09-15T04:02:22.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268098e00 con 0x7fd2681313c0 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd260001b30 con 0x7fd2681313c0 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: dump: 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.917 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 5 monclient: authenticate success, global_id 135683 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:22.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd260001da0 con 0x7fd2681313c0 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 4 set_mon_vals no callback set 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 1 -- 172.21.15.26:0/3043728337 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd2600022d0 con 0x7fd2681313c0 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d1c7700 10 monclient: dump: 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.919 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: shutdown 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- 172.21.15.26:0/3043728337 >> v1:172.21.15.26:6793/0 conn(0x7fd2681313c0 legacy=0x7fd268139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- 172.21.15.26:0/3043728337 shutdown_connections 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- 172.21.15.26:0/3043728337 >> 172.21.15.26:0/3043728337 conn(0x7fd2680a2ea0 msgr2=0x7fd26809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- 172.21.15.26:0/3043728337 shutdown_connections 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- 172.21.15.26:0/3043728337 wait complete. 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: build_initial_monmap 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: monmap: 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:22.921 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 Processor -- start 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- start start 2024-09-15T04:02:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: init 2024-09-15T04:02:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient: _add_conns ranks=[4,0,8,7,2,5,3,1,6] 2024-09-15T04:02:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient(hunting): picked mon.noname-c con 0x7fd26809bfb0 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:22.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient(hunting): picked mon.noname-a con 0x7fd268130ff0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient(hunting): picked mon.noname-e con 0x7fd2681313c0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26809f800 con 0x7fd268130ff0 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd268142880 con 0x7fd26809bfb0 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26812dec0 con 0x7fd2681313c0 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26f1cb700 10 monclient(hunting): authenticate will time out at 1887.339664s 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d9c8700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd268130ff0 0x7fd268141a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40340/0 (socket says 172.21.15.26:40340) 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.904+0000 7fd26d9c8700 1 -- 172.21.15.26:0/1568986743 learned_addr learned my addr 172.21.15.26:0/1568986743 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825756234 0 0) 0x7fd26809f800 con 0x7fd268130ff0 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): my global_id is 195003 2024-09-15T04:02:22.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd24c003570 con 0x7fd268130ff0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488184589 0 0) 0x7fd268142880 con 0x7fd26809bfb0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): my global_id is 204871 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd26809f800 con 0x7fd26809bfb0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593272700 0 0) 0x7fd26812dec0 con 0x7fd2681313c0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): my global_id is 135692 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd268142880 con 0x7fd2681313c0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 843865828 0 0) 0x7fd268142880 con 0x7fd2681313c0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd26812dec0 con 0x7fd2681313c0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd260001af0 con 0x7fd2681313c0 2024-09-15T04:02:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.930 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4016238192 0 0) 0x7fd24c003570 con 0x7fd268130ff0 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd268142880 con 0x7fd268130ff0 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2262497672 0 0) 0x7fd26809f800 con 0x7fd26809bfb0 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd24c003570 con 0x7fd26809bfb0 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd258001510 con 0x7fd268130ff0 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:22.931 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.932 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd2640019e0 con 0x7fd26809bfb0 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.933 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2983684457 0 0) 0x7fd26812dec0 con 0x7fd2681313c0 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6791/0 conn(0x7fd26809bfb0 legacy=0x7fd26812d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6789/0 conn(0x7fd268130ff0 legacy=0x7fd268141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:22.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 monclient: found mon.e 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268098050 con 0x7fd2681313c0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 5 monclient: authenticate success, global_id 135692 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 10 monclient: _renew_subs 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd268097b40 con 0x7fd2681313c0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 10 monclient: _renew_subs 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26812ba70 con 0x7fd2681313c0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd260001440 con 0x7fd2681313c0 2024-09-15T04:02:22.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd260002af0 con 0x7fd2681313c0 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26c9c6700 4 set_mon_vals no callback set 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient: dump: 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.936 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd26000d4b0 con 0x7fd2681313c0 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd250006590 con 0x7fd250002840 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(817..817 src has 435..817) v4 ==== 3895+0+0 (unknown 3142528763 0 0) 0x7fd260041920 con 0x7fd2681313c0 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516320431 0 0) 0x7fd250006590 con 0x7fd250002840 2024-09-15T04:02:22.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 10 monclient(hunting): my global_id is 135701 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd26812dec0 con 0x7fd250002840 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.908+0000 7fd26d9c8700 10 monclient: get_auth_request con 0x7fd24c038180 auth_method 0 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 572776832 0 0) 0x7fd26812dec0 con 0x7fd250002840 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd250006590 con 0x7fd250002840 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd264001d20 con 0x7fd250002840 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 10 monclient: dump: 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.938 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1861267966 0 0) 0x7fd250006590 con 0x7fd250002840 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd25000a130 con 0x7fd250002840 2024-09-15T04:02:22.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fd264005020 con 0x7fd250002840 2024-09-15T04:02:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6793/0 conn(0x7fd250002840 legacy=0x7fd250004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.912+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6793/0 conn(0x7fd250002840 legacy=0x7fd250004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd26f1cb700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:22.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fd250009640 con 0x7fd2500090b0 2024-09-15T04:02:22.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376715185 0 0) 0x7fd250009640 con 0x7fd2500090b0 2024-09-15T04:02:22.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:22.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 10 monclient(hunting): my global_id is 135710 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd250006590 con 0x7fd2500090b0 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1734912366 0 0) 0x7fd250006590 con 0x7fd2500090b0 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd250009640 con 0x7fd2500090b0 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd264005870 con 0x7fd2500090b0 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 10 monclient: dump: 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:22.942 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4244773979 0 0) 0x7fd250009640 con 0x7fd2500090b0 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.920+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd2500098f0 con 0x7fd2500090b0 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7571 (unknown 0 0 44694986) 0x7fd264005c40 con 0x7fd2500090b0 2024-09-15T04:02:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd25e7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd25e7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6793/0 conn(0x7fd2500090b0 legacy=0x7fd250008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd25e7fc700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6793/0 conn(0x7fd2500090b0 legacy=0x7fd250008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:22.944 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:22.945 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.946 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:22.947 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:22.948 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:22.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:22.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.957 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.958 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.962 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:22.963 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:22.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:22.964 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:22.964 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.122:6812/20895 conn(0x7fd24c038180 legacy=0x7fd24c03a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 10 monclient: shutdown 2024-09-15T04:02:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 >> v1:172.21.15.26:6793/0 conn(0x7fd2681313c0 legacy=0x7fd268142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26e9ca700 1 -- 172.21.15.26:0/1568986743 reap_dead start 2024-09-15T04:02:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26e9ca700 1 -- 172.21.15.26:0/1568986743 reap_dead start 2024-09-15T04:02:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 shutdown_connections 2024-09-15T04:02:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 >> 172.21.15.26:0/1568986743 conn(0x7fd2680a2ea0 msgr2=0x7fd2680669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 shutdown_connections 2024-09-15T04:02:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:22.924+0000 7fd26f1cb700 1 -- 172.21.15.26:0/1568986743 wait complete. 2024-09-15T04:02:22.966 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:22.966 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T04:02:22.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.076+0000 7f7e9ddc3700 10 monclient: get_monmap_and_config 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.076+0000 7f7e9ddc3700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.076+0000 7f7e9ddc3700 10 monclient: monmap: 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:23.082760+0000 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:23.082760+0000 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.087 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.076+0000 7f7e9ddc3700 1 Processor -- start 2024-09-15T04:02:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.076+0000 7f7e9ddc3700 1 -- start start 2024-09-15T04:02:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.076+0000 7f7e9ddc3700 10 monclient: init 2024-09-15T04:02:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient: _add_conns ranks=[1,4,5,6,8,0,2,3,7] 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient(hunting): picked mon.noname-f con 0x7f7e98064350 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient(hunting): picked mon.noname-c con 0x7f7e98132d20 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient(hunting): picked mon.noname-h con 0x7f7e981395b0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9804ca10 con 0x7f7e98064350 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e980a1e40 con 0x7f7e98132d20 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98133150 con 0x7f7e981395b0 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient(hunting): authenticate will time out at 1887.512426s 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e96ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f7e98132d20 0x7f7e98137e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41006/0 (socket says 172.21.15.26:41006) 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e96ffd700 1 -- 172.21.15.26:0/371923743 learned_addr learned my addr 172.21.15.26:0/371923743 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635981339 0 0) 0x7f7e980a1e40 con 0x7f7e98132d20 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): my global_id is 204880 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e80003570 con 0x7f7e98132d20 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619045768 0 0) 0x7f7e9804ca10 con 0x7f7e98064350 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): my global_id is 204850 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e980a1e40 con 0x7f7e98064350 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036196037 0 0) 0x7f7e98133150 con 0x7f7e981395b0 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient(hunting): my global_id is 254156 2024-09-15T04:02:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e9804ca10 con 0x7f7e981395b0 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3310781301 0 0) 0x7f7e80003570 con 0x7f7e98132d20 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 >> v1:172.21.15.122:6791/0 conn(0x7f7e981395b0 legacy=0x7f7e9813b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 >> v1:172.21.15.122:6789/0 conn(0x7f7e98064350 legacy=0x7f7e9806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 monclient: found mon.noname-c 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980a2120 con 0x7f7e98132d20 2024-09-15T04:02:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 5 monclient: authenticate success, global_id 204880 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e8c0017b0 con 0x7f7e98132d20 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: dump: 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e8c001a90 con 0x7f7e98132d20 2024-09-15T04:02:23.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 4 set_mon_vals no callback set 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 1 -- 172.21.15.26:0/371923743 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e8c004c30 con 0x7f7e98132d20 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e967fc700 10 monclient: dump: 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient: shutdown 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/371923743 >> v1:172.21.15.26:6791/0 conn(0x7f7e98132d20 legacy=0x7f7e98137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/371923743 shutdown_connections 2024-09-15T04:02:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/371923743 >> 172.21.15.26:0/371923743 conn(0x7f7e9809f3d0 msgr2=0x7f7e9809f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/371923743 shutdown_connections 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/371923743 wait complete. 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.080+0000 7f7e9ddc3700 10 monclient: monmap: 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 Processor -- start 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 -- start start 2024-09-15T04:02:23.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: init 2024-09-15T04:02:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: _add_conns ranks=[8,7,4,6,3,5,0,1,2] 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient(hunting): picked mon.noname-e con 0x7f7e98064350 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient(hunting): picked mon.noname-i con 0x7f7e981395b0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient(hunting): picked mon.noname-c con 0x7f7e9806d740 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98142ae0 con 0x7f7e9806d740 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e98136510 con 0x7f7e981395b0 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9806dbc0 con 0x7f7e98064350 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient(hunting): authenticate will time out at 1887.517222s 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e977fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f7e98064350 0x7f7e98138000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50534/0 (socket says 172.21.15.26:50534) 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e977fe700 1 -- 172.21.15.26:0/274589717 learned_addr learned my addr 172.21.15.26:0/274589717 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e97fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f7e9806d740 0x7f7e9814a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41020/0 (socket says 172.21.15.26:41020) 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971140280 0 0) 0x7f7e9806dbc0 con 0x7f7e98064350 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): my global_id is 135719 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e84003570 con 0x7f7e98064350 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935074 0 0) 0x7f7e98142ae0 con 0x7f7e9806d740 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): my global_id is 204889 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e9806dbc0 con 0x7f7e9806d740 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223671962 0 0) 0x7f7e98136510 con 0x7f7e981395b0 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): my global_id is 254131 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e98142ae0 con 0x7f7e981395b0 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3938679695 0 0) 0x7f7e9806dbc0 con 0x7f7e9806d740 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e98136510 con 0x7f7e9806d740 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e90001d40 con 0x7f7e9806d740 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): dump: 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3166215813 0 0) 0x7f7e84003570 con 0x7f7e98064350 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e9806dbc0 con 0x7f7e98064350 2024-09-15T04:02:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e88001b20 con 0x7f7e98064350 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient(hunting): dump: 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4118247319 0 0) 0x7f7e98142ae0 con 0x7f7e981395b0 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e84003570 con 0x7f7e981395b0 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1387710255 0 0) 0x7f7e98136510 con 0x7f7e9806d740 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.26:6793/0 conn(0x7f7e98064350 legacy=0x7f7e98138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.122:6792/0 conn(0x7f7e981395b0 legacy=0x7f7e98146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 monclient: found mon.c 2024-09-15T04:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98135340 con 0x7f7e9806d740 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 5 monclient: authenticate success, global_id 204889 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: _renew_subs 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e98136230 con 0x7f7e9806d740 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: _renew_subs 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9812d160 con 0x7f7e9806d740 2024-09-15T04:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f7e90003110 con 0x7f7e9806d740 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e900026e0 con 0x7f7e9806d740 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e95ffb700 4 set_mon_vals no callback set 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e7f7fe700 10 monclient: dump: 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.084+0000 7f7e9ddc3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f7e900028c0 con 0x7f7e9806d740 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7e78006590 con 0x7f7e78002840 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(817..817 src has 435..817) v4 ==== 3895+0+0 (unknown 3142528763 0 0) 0x7f7e900441a0 con 0x7f7e9806d740 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e96ffd700 10 monclient: get_auth_request con 0x7f7e84037f30 auth_method 0 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080095925 0 0) 0x7f7e78006590 con 0x7f7e78002840 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient(hunting): my global_id is 204859 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e98136510 con 0x7f7e78002840 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1247621465 0 0) 0x7f7e98136510 con 0x7f7e78002840 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e78006590 con 0x7f7e78002840 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e880025f0 con 0x7f7e78002840 2024-09-15T04:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient: dump: 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3513717875 0 0) 0x7f7e78006590 con 0x7f7e78002840 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e7800a130 con 0x7f7e78002840 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f7e88002b90 con 0x7f7e78002840 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.122:6789/0 conn(0x7f7e78002840 legacy=0x7f7e78004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.088+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.122:6789/0 conn(0x7f7e78002840 legacy=0x7f7e78004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.096+0000 7f7e9ddc3700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.096+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f7e78007520 con 0x7f7e780068d0 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116641492 0 0) 0x7f7e78007520 con 0x7f7e780068d0 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient(hunting): my global_id is 204868 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e78006590 con 0x7f7e780068d0 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2163653993 0 0) 0x7f7e78006590 con 0x7f7e780068d0 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e78007520 con 0x7f7e780068d0 2024-09-15T04:02:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7e88005d40 con 0x7f7e780068d0 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient: dump: 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4101471162 0 0) 0x7f7e78007520 con 0x7f7e780068d0 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7e78008760 con 0x7f7e780068d0 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7739 (unknown 0 0 3288151313) 0x7f7e88007e50 con 0x7f7e780068d0 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.122:6789/0 conn(0x7f7e780068d0 legacy=0x7f7e78006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e7f7fe700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.122:6789/0 conn(0x7f7e780068d0 legacy=0x7f7e78006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:23.123 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:23.124 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.125 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:23.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.128 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.132 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:23.133 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.136 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.137 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.139 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:23.140 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:23.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:23.142 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.144 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:23.144 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:23.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.122:6812/20895 conn(0x7f7e84037f30 legacy=0x7f7e8403a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 10 monclient: shutdown 2024-09-15T04:02:23.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 >> v1:172.21.15.26:6791/0 conn(0x7f7e9806d740 legacy=0x7f7e9814a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e97fff700 1 -- 172.21.15.26:0/274589717 reap_dead start 2024-09-15T04:02:23.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e97fff700 1 -- 172.21.15.26:0/274589717 reap_dead start 2024-09-15T04:02:23.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 shutdown_connections 2024-09-15T04:02:23.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 >> 172.21.15.26:0/274589717 conn(0x7f7e9809f3d0 msgr2=0x7f7e98131c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 shutdown_connections 2024-09-15T04:02:23.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.100+0000 7f7e9ddc3700 1 -- 172.21.15.26:0/274589717 wait complete. 2024-09-15T04:02:23.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:23.146 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T04:02:23.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T04:02:23.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient: get_monmap_and_config 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient: monmap: 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:23.261538+0000 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:23.261538+0000 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.265 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.266 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 1 Processor -- start 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 1 -- start start 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient: init 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient: _add_conns ranks=[1,0,4,7,6,3,8,5,2] 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient(hunting): picked mon.noname-f con 0x7f6450064390 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient(hunting): picked mon.noname-a con 0x7f645006c580 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient(hunting): picked mon.noname-c con 0x7f6450138cb0 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645004ca10 con 0x7f645006c580 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645006bc80 con 0x7f6450064390 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645006d2b0 con 0x7f6450138cb0 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64559ae700 10 monclient(hunting): authenticate will time out at 1887.690926s 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f645006c580 0x7f645006af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40342/0 (socket says 172.21.15.26:40342) 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644ffff700 1 -- 172.21.15.26:0/2801835763 learned_addr learned my addr 172.21.15.26:0/2801835763 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f64551ad700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f6450138cb0 0x7f645013b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41036/0 (socket says 172.21.15.26:41036) 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247824230 0 0) 0x7f645006d2b0 con 0x7f6450138cb0 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): my global_id is 204898 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6430003570 con 0x7f6450138cb0 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517305384 0 0) 0x7f645004ca10 con 0x7f645006c580 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): my global_id is 195012 2024-09-15T04:02:23.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645006d2b0 con 0x7f645006c580 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781618535 0 0) 0x7f645006bc80 con 0x7f6450064390 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 10 monclient(hunting): my global_id is 204877 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.256+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645004ca10 con 0x7f6450064390 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814913072 0 0) 0x7f6430003570 con 0x7f6450138cb0 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 >> v1:172.21.15.122:6789/0 conn(0x7f6450064390 legacy=0x7f645006a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 >> v1:172.21.15.26:6789/0 conn(0x7f645006c580 legacy=0x7f645006af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 monclient: found mon.noname-c 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645006b630 con 0x7f6450138cb0 2024-09-15T04:02:23.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64400016e0 con 0x7f6450138cb0 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 5 monclient: authenticate success, global_id 204898 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: dump: 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.273 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6440001950 con 0x7f6450138cb0 2024-09-15T04:02:23.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 4 set_mon_vals no callback set 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 1 -- 172.21.15.26:0/2801835763 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6440004830 con 0x7f6450138cb0 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f644f7fe700 10 monclient: dump: 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.275 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 10 monclient: shutdown 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 1 -- 172.21.15.26:0/2801835763 >> v1:172.21.15.26:6791/0 conn(0x7f6450138cb0 legacy=0x7f645013b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 1 -- 172.21.15.26:0/2801835763 shutdown_connections 2024-09-15T04:02:23.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 1 -- 172.21.15.26:0/2801835763 >> 172.21.15.26:0/2801835763 conn(0x7f6450063340 msgr2=0x7f6450063740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 1 -- 172.21.15.26:0/2801835763 shutdown_connections 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 1 -- 172.21.15.26:0/2801835763 wait complete. 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 10 monclient: monmap: 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.278 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.278 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.260+0000 7f64559ae700 1 Processor -- start 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 1 -- start start 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: init 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: _add_conns ranks=[3,1,0,4,2,5,6,7,8] 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient(hunting): picked mon.noname-g con 0x7f6450064390 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient(hunting): picked mon.noname-f con 0x7f645006c580 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient(hunting): picked mon.noname-a con 0x7f6450138cb0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64501352b0 con 0x7f6450138cb0 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645012cf50 con 0x7f645006c580 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645006cad0 con 0x7f6450064390 2024-09-15T04:02:23.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient(hunting): authenticate will time out at 1887.697139s 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64551ad700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6450138cb0 0x7f645014a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40356/0 (socket says 172.21.15.26:40356) 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64551ad700 1 -- 172.21.15.26:0/4165095808 learned_addr learned my addr 172.21.15.26:0/4165095808 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420013382 0 0) 0x7f64501352b0 con 0x7f6450138cb0 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): my global_id is 195021 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6434003570 con 0x7f6450138cb0 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167747102 0 0) 0x7f645012cf50 con 0x7f645006c580 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): my global_id is 204886 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64501352b0 con 0x7f645006c580 2024-09-15T04:02:23.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067406133 0 0) 0x7f645006cad0 con 0x7f6450064390 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): my global_id is 254136 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645012cf50 con 0x7f6450064390 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 630216135 0 0) 0x7f6434003570 con 0x7f6450138cb0 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f645006cad0 con 0x7f6450138cb0 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f64400016a0 con 0x7f6450138cb0 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): dump: 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.285 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.286 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2858445611 0 0) 0x7f64501352b0 con 0x7f645006c580 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6434003570 con 0x7f645006c580 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6438001890 con 0x7f645006c580 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient(hunting): dump: 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.287 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1921909280 0 0) 0x7f645006cad0 con 0x7f6450138cb0 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6790/0 conn(0x7f6450064390 legacy=0x7f64501339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6789/0 conn(0x7f645006c580 legacy=0x7f64501340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 monclient: found mon.a 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64501347f0 con 0x7f6450138cb0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f644cff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 5 monclient: authenticate success, global_id 195021 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: _renew_subs 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6450134d70 con 0x7f6450138cb0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: _renew_subs 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.264+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450137c80 con 0x7f6450138cb0 2024-09-15T04:02:23.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f64559ae700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f645004bbf0 con 0x7f6450135900 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6440004c90 con 0x7f6450138cb0 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6440005140 con 0x7f6450138cb0 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: dump: 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644e7fc700 4 set_mon_vals no callback set 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f644000fb00 con 0x7f6450138cb0 2024-09-15T04:02:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803440128 0 0) 0x7f645004bbf0 con 0x7f6450135900 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient(hunting): my global_id is 254145 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645006cad0 con 0x7f6450135900 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(817..817 src has 435..817) v4 ==== 3895+0+0 (unknown 3142528763 0 0) 0x7f6440044050 con 0x7f6450138cb0 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644ffff700 10 monclient: get_auth_request con 0x7f6434037bf0 auth_method 0 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 387975499 0 0) 0x7f645006cad0 con 0x7f6450135900 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f645004bbf0 con 0x7f6450135900 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6448002170 con 0x7f6450135900 2024-09-15T04:02:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: dump: 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1982133153 0 0) 0x7f645004bbf0 con 0x7f6450135900 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6450066de0 con 0x7f6450135900 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6448004410 con 0x7f6450135900 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.268+0000 7f644cff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.272+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6790/0 conn(0x7f6450135900 legacy=0x7f64501324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.272+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6790/0 conn(0x7f6450135900 legacy=0x7f64501324f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f64559ae700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6450137520 con 0x7f645009f820 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453152129 0 0) 0x7f6450137520 con 0x7f645009f820 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 10 monclient(hunting): my global_id is 254154 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645004bbf0 con 0x7f645009f820 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1035144302 0 0) 0x7f645004bbf0 con 0x7f645009f820 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6450137520 con 0x7f645009f820 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6448002bb0 con 0x7f645009f820 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 10 monclient: dump: 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3079512827 0 0) 0x7f6450137520 con 0x7f645009f820 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.280+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f645009fbf0 con 0x7f645009f820 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 1641231691) 0x7f6448004120 con 0x7f645009f820 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f644cff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f644cff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6790/0 conn(0x7f645009f820 legacy=0x7f645002ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f644cff9700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6790/0 conn(0x7f645009f820 legacy=0x7f645002ad10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.298 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:23.299 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.300 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.301 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.303 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:23.304 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:23.308 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.309 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:23.312 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.313 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.316 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.317 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.318 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.318 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:23.318 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.122:6812/20895 conn(0x7f6434037bf0 legacy=0x7f643403a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 10 monclient: shutdown 2024-09-15T04:02:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 >> v1:172.21.15.26:6789/0 conn(0x7f6450138cb0 legacy=0x7f645014a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64551ad700 1 -- 172.21.15.26:0/4165095808 reap_dead start 2024-09-15T04:02:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 shutdown_connections 2024-09-15T04:02:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 >> 172.21.15.26:0/4165095808 conn(0x7f6450063340 msgr2=0x7f64500a0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 shutdown_connections 2024-09-15T04:02:23.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.284+0000 7f64559ae700 1 -- 172.21.15.26:0/4165095808 wait complete. 2024-09-15T04:02:23.320 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:23.320 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T04:02:23.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient: get_monmap_and_config 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient: monmap: 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:23.432463+0000 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:23.432463+0000 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.437 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 1 Processor -- start 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 1 -- start start 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient: init 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient: _add_conns ranks=[3,1,5,4,8,6,7,0,2] 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient(hunting): picked mon.noname-g con 0x7fb21c0990c0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient(hunting): picked mon.noname-f con 0x7fb21c0a1310 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient(hunting): picked mon.noname-h con 0x7fb21c1335a0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c04ca10 con 0x7fb21c0a1310 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c0a1b20 con 0x7fb21c0990c0 2024-09-15T04:02:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c0a10b0 con 0x7fb21c1335a0 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb220fc0700 10 monclient(hunting): authenticate will time out at 1887.862877s 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb219d9b700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb21c0a1310 0x7fb21c099490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:58366/0 (socket says 172.21.15.26:58366) 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb219d9b700 1 -- 172.21.15.26:0/3372781925 learned_addr learned my addr 172.21.15.26:0/3372781925 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21ad9d700 1 --1- 172.21.15.26:0/3372781925 >> v1:172.21.15.122:6791/0 conn(0x7fb21c1335a0 0x7fb21c13afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:59624/0 (socket says 172.21.15.26:59624) 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21a59c700 1 --1- 172.21.15.26:0/3372781925 >> v1:172.21.15.122:6790/0 conn(0x7fb21c0990c0 0x7fb21c132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:51244/0 (socket says 172.21.15.26:51244) 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857060130 0 0) 0x7fb21c04ca10 con 0x7fb21c0a1310 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): my global_id is 204895 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb208003570 con 0x7fb21c0a1310 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839507817 0 0) 0x7fb21c0a10b0 con 0x7fb21c1335a0 2024-09-15T04:02:23.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): my global_id is 254165 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c04ca10 con 0x7fb21c1335a0 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024700973 0 0) 0x7fb21c0a1b20 con 0x7fb21c0990c0 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 10 monclient(hunting): my global_id is 254163 2024-09-15T04:02:23.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.428+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c0a10b0 con 0x7fb21c0990c0 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201379044 0 0) 0x7fb21c04ca10 con 0x7fb21c1335a0 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 >> v1:172.21.15.122:6790/0 conn(0x7fb21c0990c0 legacy=0x7fb21c132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 >> v1:172.21.15.122:6789/0 conn(0x7fb21c0a1310 legacy=0x7fb21c099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 monclient: found mon.noname-h 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.122:6791/0 2024-09-15T04:02:23.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 --> v1:172.21.15.122:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c099ba0 con 0x7fb21c1335a0 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 5 monclient: authenticate success, global_id 254165 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb214001b10 con 0x7fb21c1335a0 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: dump: 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.5 v1:172.21.15.122:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb214001d30 con 0x7fb21c1335a0 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 1 -- 172.21.15.26:0/3372781925 <== mon.5 v1:172.21.15.122:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb214004ef0 con 0x7fb21c1335a0 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 4 set_mon_vals no callback set 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb21959a700 10 monclient: dump: 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.450 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 10 monclient: shutdown 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 -- 172.21.15.26:0/3372781925 >> v1:172.21.15.122:6791/0 conn(0x7fb21c1335a0 legacy=0x7fb21c13afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 -- 172.21.15.26:0/3372781925 shutdown_connections 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 -- 172.21.15.26:0/3372781925 >> 172.21.15.26:0/3372781925 conn(0x7fb21c06e370 msgr2=0x7fb21c0a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 -- 172.21.15.26:0/3372781925 shutdown_connections 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 -- 172.21.15.26:0/3372781925 wait complete. 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 10 monclient: monmap: 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 Processor -- start 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 1 -- start start 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.432+0000 7fb220fc0700 10 monclient: init 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient: _add_conns ranks=[7,4,3,1,2,6,0,5,8] 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient(hunting): picked mon.noname-i con 0x7fb21c0990c0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient(hunting): picked mon.noname-c con 0x7fb21c0a1310 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient(hunting): picked mon.noname-g con 0x7fb21c1335a0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c0a1ce0 con 0x7fb21c1335a0 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c09a010 con 0x7fb21c0a1310 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c120770 con 0x7fb21c0990c0 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient(hunting): authenticate will time out at 1887.868155s 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb219d9b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fb21c0a1310 0x7fb21c1470b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41038/0 (socket says 172.21.15.26:41038) 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb219d9b700 1 -- 172.21.15.26:0/3911750664 learned_addr learned my addr 172.21.15.26:0/3911750664 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209382210 0 0) 0x7fb21c09a010 con 0x7fb21c0a1310 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): my global_id is 204907 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc003570 con 0x7fb21c0a1310 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176802226 0 0) 0x7fb21c0a1ce0 con 0x7fb21c1335a0 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): my global_id is 254172 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c09a010 con 0x7fb21c1335a0 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760895838 0 0) 0x7fb21c120770 con 0x7fb21c0990c0 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): my global_id is 254140 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c0a1ce0 con 0x7fb21c0990c0 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3898444137 0 0) 0x7fb1fc003570 con 0x7fb21c0a1310 2024-09-15T04:02:23.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb21c120770 con 0x7fb21c0a1310 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb20c001a40 con 0x7fb21c0a1310 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): dump: 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.456 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3393351280 0 0) 0x7fb21c09a010 con 0x7fb21c1335a0 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb1fc003570 con 0x7fb21c1335a0 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb214001a10 con 0x7fb21c1335a0 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): dump: 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.458 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2566809521 0 0) 0x7fb21c0a1ce0 con 0x7fb21c0990c0 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb21c09a010 con 0x7fb21c0990c0 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb204001a60 con 0x7fb21c0990c0 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient(hunting): dump: 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.459 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3681793756 0 0) 0x7fb21c120770 con 0x7fb21c0a1310 2024-09-15T04:02:23.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6792/0 conn(0x7fb21c0990c0 legacy=0x7fb21c142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6790/0 conn(0x7fb21c1335a0 legacy=0x7fb21c14a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 monclient: found mon.c 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c11ff90 con 0x7fb21c0a1310 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 5 monclient: authenticate success, global_id 204907 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient: _renew_subs 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c121a50 con 0x7fb21c0a1310 2024-09-15T04:02:23.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient: _renew_subs 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c122cd0 con 0x7fb21c0a1310 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb20c004ba0 con 0x7fb21c0a1310 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb20c0050d0 con 0x7fb21c0a1310 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb218d99700 4 set_mon_vals no callback set 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 10 monclient: dump: 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.462 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb200006590 con 0x7fb200002840 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.436+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb20c0052b0 con 0x7fb21c0a1310 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(817..817 src has 435..817) v4 ==== 3895+0+0 (unknown 3142528763 0 0) 0x7fb20c044050 con 0x7fb21c0a1310 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548583263 0 0) 0x7fb200006590 con 0x7fb200002840 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient(hunting): my global_id is 254174 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c120770 con 0x7fb200002840 2024-09-15T04:02:23.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb21ad9d700 10 monclient: get_auth_request con 0x7fb1fc038090 auth_method 0 2024-09-15T04:02:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 392175536 0 0) 0x7fb21c120770 con 0x7fb200002840 2024-09-15T04:02:23.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb200006590 con 0x7fb200002840 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb204004b70 con 0x7fb200002840 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient: dump: 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.467 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3605650650 0 0) 0x7fb200006590 con 0x7fb200002840 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb20000a130 con 0x7fb200002840 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fb204005860 con 0x7fb200002840 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:23.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6791/0 conn(0x7fb200002840 legacy=0x7fb200004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.440+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6791/0 conn(0x7fb200002840 legacy=0x7fb200004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb220fc0700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fb200007520 con 0x7fb2000068d0 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151418809 0 0) 0x7fb200007520 con 0x7fb2000068d0 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 10 monclient(hunting): my global_id is 254183 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb200006590 con 0x7fb2000068d0 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4247544552 0 0) 0x7fb200006590 con 0x7fb2000068d0 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb200007520 con 0x7fb2000068d0 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2040054c0 con 0x7fb2000068d0 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:23.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 10 monclient: dump: 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4282003249 0 0) 0x7fb200007520 con 0x7fb2000068d0 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.452+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb200008760 con 0x7fb2000068d0 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 527310182) 0x7fb204005b10 con 0x7fb2000068d0 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb2127fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb2127fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6791/0 conn(0x7fb2000068d0 legacy=0x7fb200006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb2127fc700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6791/0 conn(0x7fb2000068d0 legacy=0x7fb200006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.471 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:23.472 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.473 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.474 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:23.475 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.476 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.478 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.479 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.480 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:23.481 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.482 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.484 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.485 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.486 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.489 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.490 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.491 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:23.491 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.122:6812/20895 conn(0x7fb1fc038090 legacy=0x7fb1fc03a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 10 monclient: shutdown 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 >> v1:172.21.15.26:6791/0 conn(0x7fb21c0a1310 legacy=0x7fb21c1470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb21ad9d700 1 -- 172.21.15.26:0/3911750664 reap_dead start 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb21ad9d700 1 -- 172.21.15.26:0/3911750664 reap_dead start 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 shutdown_connections 2024-09-15T04:02:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 >> 172.21.15.26:0/3911750664 conn(0x7fb21c06e370 msgr2=0x7fb21c0a4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 shutdown_connections 2024-09-15T04:02:23.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.456+0000 7fb220fc0700 1 -- 172.21.15.26:0/3911750664 wait complete. 2024-09-15T04:02:23.493 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:23.493 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T04:02:23.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient: get_monmap_and_config 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient: monmap: 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:23.610436+0000 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:23.610436+0000 2024-09-15T04:02:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 1 Processor -- start 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 1 -- start start 2024-09-15T04:02:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient: init 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient: _add_conns ranks=[3,0,4,1,7,8,2,6,5] 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient(hunting): picked mon.noname-g con 0x7fa87c0a6880 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient(hunting): picked mon.noname-a con 0x7fa87c09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient(hunting): picked mon.noname-c con 0x7fa87c138d30 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c04ca10 con 0x7fa87c09faa0 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c0a0580 con 0x7fa87c0a6880 2024-09-15T04:02:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c133780 con 0x7fa87c138d30 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87bfff700 10 monclient(hunting): authenticate will time out at 1888.039803s 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.604+0000 7fa87a7fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa87c09faa0 0x7fa87c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40372/0 (socket says 172.21.15.26:40372) 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87a7fc700 1 -- 172.21.15.26:0/2402366138 learned_addr learned my addr 172.21.15.26:0/2402366138 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986601959 0 0) 0x7fa87c04ca10 con 0x7fa87c09faa0 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): my global_id is 195030 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa864003570 con 0x7fa87c09faa0 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294615652 0 0) 0x7fa87c133780 con 0x7fa87c138d30 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): my global_id is 204916 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c04ca10 con 0x7fa87c138d30 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313820369 0 0) 0x7fa870001110 con 0x7fa87c0a6880 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient(hunting): my global_id is 254181 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c133780 con 0x7fa87c0a6880 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470081660 0 0) 0x7fa864003570 con 0x7fa87c09faa0 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 >> v1:172.21.15.26:6791/0 conn(0x7fa87c138d30 legacy=0x7fa87c13b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 >> v1:172.21.15.122:6790/0 conn(0x7fa87c0a6880 legacy=0x7fa87c133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 monclient: found mon.noname-a 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0a6c50 con 0x7fa87c09faa0 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa86c0017b0 con 0x7fa87c09faa0 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: dump: 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 5 monclient: authenticate success, global_id 195030 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa86c001a90 con 0x7fa87c09faa0 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 4 set_mon_vals no callback set 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 1 -- 172.21.15.26:0/2402366138 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa86c004c30 con 0x7fa87c09faa0 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa879ffb700 10 monclient: dump: 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.624 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 10 monclient: shutdown 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 -- 172.21.15.26:0/2402366138 >> v1:172.21.15.26:6789/0 conn(0x7fa87c09faa0 legacy=0x7fa87c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 -- 172.21.15.26:0/2402366138 shutdown_connections 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 -- 172.21.15.26:0/2402366138 >> 172.21.15.26:0/2402366138 conn(0x7fa87c06e2a0 msgr2=0x7fa87c0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 -- 172.21.15.26:0/2402366138 shutdown_connections 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 -- 172.21.15.26:0/2402366138 wait complete. 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 10 monclient: monmap: 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.626 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 Processor -- start 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 1 -- start start 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.608+0000 7fa87bfff700 10 monclient: init 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient: _add_conns ranks=[4,8,5,0,3,6,1,2,7] 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient(hunting): picked mon.noname-c con 0x7fa87c09faa0 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient(hunting): picked mon.noname-e con 0x7fa87c0a6880 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient(hunting): picked mon.noname-h con 0x7fa87c138d30 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c0a18c0 con 0x7fa87c09faa0 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c09cf10 con 0x7fa87c138d30 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c1363a0 con 0x7fa87c0a6880 2024-09-15T04:02:23.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient(hunting): authenticate will time out at 1888.044818s 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87a7fc700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fa87c0a6880 0x7fa87c135c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50546/0 (socket says 172.21.15.26:50546) 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87a7fc700 1 -- 172.21.15.26:0/2577615441 learned_addr learned my addr 172.21.15.26:0/2577615441 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792596928 0 0) 0x7fa87c1363a0 con 0x7fa87c0a6880 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): my global_id is 135728 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa868003570 con 0x7fa87c0a6880 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900526456 0 0) 0x7fa87c0a18c0 con 0x7fa87c09faa0 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): my global_id is 204925 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c1363a0 con 0x7fa87c09faa0 2024-09-15T04:02:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659900960 0 0) 0x7fa87c09cf10 con 0x7fa87c138d30 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): my global_id is 254192 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c0a18c0 con 0x7fa87c138d30 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2047944692 0 0) 0x7fa868003570 con 0x7fa87c0a6880 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa87c09cf10 con 0x7fa87c0a6880 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa86c004b70 con 0x7fa87c0a6880 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): dump: 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2552890187 0 0) 0x7fa87c1363a0 con 0x7fa87c09faa0 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa868003570 con 0x7fa87c09faa0 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa870001bf0 con 0x7fa87c09faa0 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): dump: 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.636 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1364476650 0 0) 0x7fa87c0a18c0 con 0x7fa87c138d30 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa87c1363a0 con 0x7fa87c138d30 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa874001c80 con 0x7fa87c138d30 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient(hunting): dump: 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 785976770 0 0) 0x7fa87c09cf10 con 0x7fa87c0a6880 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.122:6791/0 conn(0x7fa87c138d30 legacy=0x7fa87c14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.26:6791/0 conn(0x7fa87c09faa0 legacy=0x7fa87c135580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 monclient: found mon.e 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c1375e0 con 0x7fa87c0a6880 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 5 monclient: authenticate success, global_id 135728 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient: _renew_subs 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c1370d0 con 0x7fa87c0a6880 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient: _renew_subs 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c11d470 con 0x7fa87c0a6880 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa86c001860 con 0x7fa87c0a6880 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa86c0011d0 con 0x7fa87c0a6880 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa8797fa700 4 set_mon_vals no callback set 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa862ffd700 10 monclient: dump: 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.612+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa85c006590 con 0x7fa85c002840 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa86c00faa0 con 0x7fa87c0a6880 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(817..817 src has 435..817) v4 ==== 3895+0+0 (unknown 3142528763 0 0) 0x7fa86c043fc0 con 0x7fa87c0a6880 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867519875 0 0) 0x7fa85c006590 con 0x7fa85c002840 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient(hunting): my global_id is 254149 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa87c09cf10 con 0x7fa85c002840 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa87b7fe700 10 monclient: get_auth_request con 0x7fa868038080 auth_method 0 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 593759824 0 0) 0x7fa87c09cf10 con 0x7fa85c002840 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa85c006590 con 0x7fa85c002840 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa870002670 con 0x7fa85c002840 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient: dump: 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 399958991 0 0) 0x7fa85c006590 con 0x7fa85c002840 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa85c00a130 con 0x7fa85c002840 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa870002ae0 con 0x7fa85c002840 2024-09-15T04:02:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.122:6792/0 conn(0x7fa85c002840 legacy=0x7fa85c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.616+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.122:6792/0 conn(0x7fa85c002840 legacy=0x7fa85c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa87bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa85c009640 con 0x7fa85c0090b0 2024-09-15T04:02:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182369784 0 0) 0x7fa85c009640 con 0x7fa85c0090b0 2024-09-15T04:02:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 10 monclient(hunting): my global_id is 254158 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa85c006590 con 0x7fa85c0090b0 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1147925586 0 0) 0x7fa85c006590 con 0x7fa85c0090b0 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa85c009640 con 0x7fa85c0090b0 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa870003530 con 0x7fa85c0090b0 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 10 monclient: dump: 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.648 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3564717072 0 0) 0x7fa85c009640 con 0x7fa85c0090b0 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.628+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa85c0098f0 con 0x7fa85c0090b0 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 852021063) 0x7fa870003840 con 0x7fa85c0090b0 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa862ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa862ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.122:6792/0 conn(0x7fa85c0090b0 legacy=0x7fa85c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa862ffd700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.122:6792/0 conn(0x7fa85c0090b0 legacy=0x7fa85c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:23.649 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.650 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 4, 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.651 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.652 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:23.653 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:23.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.656 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:23.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.658 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.659 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.660 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.662 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:23.665 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:23.666 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:23.667 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:23.668 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.122:6812/20895 conn(0x7fa868038080 legacy=0x7fa86803a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 10 monclient: shutdown 2024-09-15T04:02:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 >> v1:172.21.15.26:6793/0 conn(0x7fa87c0a6880 legacy=0x7fa87c135c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87b7fe700 1 -- 172.21.15.26:0/2577615441 reap_dead start 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87b7fe700 1 -- 172.21.15.26:0/2577615441 reap_dead start 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 shutdown_connections 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 >> 172.21.15.26:0/2577615441 conn(0x7fa87c06e2a0 msgr2=0x7fa87c067880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 shutdown_connections 2024-09-15T04:02:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.632+0000 7fa87bfff700 1 -- 172.21.15.26:0/2577615441 wait complete. 2024-09-15T04:02:23.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:23.670 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T04:02:23.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T04:02:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient: get_monmap_and_config 2024-09-15T04:02:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient: monmap: 2024-09-15T04:02:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.799 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:23.796057+0000 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:23.796057+0000 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.800 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.801 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 1 Processor -- start 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 1 -- start start 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient: init 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient: _add_conns ranks=[3,5,7,1,0,6,8,4,2] 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient(hunting): picked mon.noname-g con 0x7fdde00a6a80 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient(hunting): picked mon.noname-h con 0x7fdde009fb80 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient(hunting): picked mon.noname-i con 0x7fdde0138d30 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde004ca10 con 0x7fdde00a6a80 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00a0f50 con 0x7fdde009fb80 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde0133680 con 0x7fdde0138d30 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde726f700 10 monclient(hunting): authenticate will time out at 1888.226002s 2024-09-15T04:02:23.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde5a6c700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7fdde009fb80 0x7fdde009ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:59664/0 (socket says 172.21.15.26:59664) 2024-09-15T04:02:23.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.792+0000 7fdde5a6c700 1 -- 172.21.15.26:0/3556307149 learned_addr learned my addr 172.21.15.26:0/3556307149 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261345989 0 0) 0x7fdde00a0f50 con 0x7fdde009fb80 2024-09-15T04:02:23.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient(hunting): my global_id is 254201 2024-09-15T04:02:23.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc003570 con 0x7fdde009fb80 2024-09-15T04:02:23.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507521502 0 0) 0x7fdde004ca10 con 0x7fdde00a6a80 2024-09-15T04:02:23.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient(hunting): my global_id is 254190 2024-09-15T04:02:23.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde00a0f50 con 0x7fdde00a6a80 2024-09-15T04:02:23.806 INFO:tasks.rados.rados.0.smithi122.stdout:3622: done (0 left) 2024-09-15T04:02:23.806 INFO:tasks.rados.rados.0.smithi122.stdout:3623: copy_from oid 46 from oid 26 current snap is 421 2024-09-15T04:02:23.807 INFO:tasks.rados.rados.0.smithi122.stdout:3624: write oid 18 current snap is 421 2024-09-15T04:02:23.807 INFO:tasks.rados.rados.0.smithi122.stdout:3624: seq_num 1275 ranges {628925=573426,1693365=714102,3096904=332021} 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2109171414 0 0) 0x7fdde00a0f50 con 0x7fdde00a6a80 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 >> v1:172.21.15.122:6792/0 conn(0x7fdde0138d30 legacy=0x7fdde013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 >> v1:172.21.15.122:6791/0 conn(0x7fdde009fb80 legacy=0x7fdde009ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 monclient: found mon.noname-g 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.122:6790/0 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00a6e50 con 0x7fdde00a6a80 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddd8001b10 con 0x7fdde00a6a80 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde526b700 10 monclient: dump: 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.809 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde726f700 5 monclient: authenticate success, global_id 254190 2024-09-15T04:02:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.796+0000 7fdde726f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 <== mon.3 v1:172.21.15.122:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fddd8001e30 con 0x7fdde00a6a80 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde526b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde526b700 1 -- 172.21.15.26:0/3556307149 <== mon.3 v1:172.21.15.122:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddd80025c0 con 0x7fdde00a6a80 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde726f700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde726f700 4 set_mon_vals no callback set 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde526b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde526b700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde526b700 10 monclient: dump: 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.813 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde726f700 10 monclient: shutdown 2024-09-15T04:02:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde726f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.800+0000 7fdde726f700 1 -- 172.21.15.26:0/3556307149 >> v1:172.21.15.122:6790/0 conn(0x7fdde00a6a80 legacy=0x7fdde0132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- 172.21.15.26:0/3556307149 shutdown_connections 2024-09-15T04:02:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- 172.21.15.26:0/3556307149 >> 172.21.15.26:0/3556307149 conn(0x7fdde006e380 msgr2=0x7fdde00a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- 172.21.15.26:0/3556307149 shutdown_connections 2024-09-15T04:02:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- 172.21.15.26:0/3556307149 wait complete. 2024-09-15T04:02:23.815 INFO:tasks.rados.rados.0.smithi122.stdout:3624: writing smithi12223906-18 from 628925 to 1202351 tid 1 2024-09-15T04:02:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: build_initial_monmap 2024-09-15T04:02:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: monmap: 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:23.818 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 Processor -- start 2024-09-15T04:02:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- start start 2024-09-15T04:02:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: init 2024-09-15T04:02:23.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: _add_conns ranks=[6,3,1,4,7,5,0,8,2] 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient(hunting): picked mon.noname-d con 0x7fdde009fb80 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient(hunting): picked mon.noname-g con 0x7fdde00a6a80 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient(hunting): picked mon.noname-f con 0x7fdde0138d30 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00a1920 con 0x7fdde0138d30 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde009cfc0 con 0x7fdde00a6a80 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde0069bd0 con 0x7fdde009fb80 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient(hunting): authenticate will time out at 1888.242436s 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde626d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fdde009fb80 0x7fdde01463a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55680/0 (socket says 172.21.15.26:55680) 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde626d700 1 -- 172.21.15.26:0/214324161 learned_addr learned my addr 172.21.15.26:0/214324161 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868456509 0 0) 0x7fdde0069bd0 con 0x7fdde009fb80 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): my global_id is 195135 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc4003570 con 0x7fdde009fb80 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559053506 0 0) 0x7fdde00a1920 con 0x7fdde0138d30 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): my global_id is 204904 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde0069bd0 con 0x7fdde0138d30 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612092324 0 0) 0x7fdde009cfc0 con 0x7fdde00a6a80 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:23.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): my global_id is 254199 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde00a1920 con 0x7fdde00a6a80 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3055287319 0 0) 0x7fddc4003570 con 0x7fdde009fb80 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdde009cfc0 con 0x7fdde009fb80 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddd80022e0 con 0x7fdde009fb80 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): dump: 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.825 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 294656541 0 0) 0x7fdde00a1920 con 0x7fdde00a6a80 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddc4003570 con 0x7fdde00a6a80 2024-09-15T04:02:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 942655683 0 0) 0x7fdde0069bd0 con 0x7fdde0138d30 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdde00a1920 con 0x7fdde0138d30 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fdddc001a00 con 0x7fdde00a6a80 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): dump: 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.827 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddd00017d0 con 0x7fdde0138d30 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient(hunting): dump: 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.828 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3126465947 0 0) 0x7fdde009cfc0 con 0x7fdde009fb80 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 >> v1:172.21.15.122:6790/0 conn(0x7fdde00a6a80 legacy=0x7fdde0146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 >> v1:172.21.15.122:6789/0 conn(0x7fdde0138d30 legacy=0x7fdde014e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 monclient: found mon.d 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde009b340 con 0x7fdde009fb80 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fddd67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 5 monclient: authenticate success, global_id 195135 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: _renew_subs 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:23.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde00698f0 con 0x7fdde009fb80 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: _renew_subs 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.808+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde01328a0 con 0x7fdde009fb80 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fddd8002570 con 0x7fdde009fb80 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddd8002f30 con 0x7fdde009fb80 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 10 monclient: dump: 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:23.831 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fdde726f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fdde726f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fdde726f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:23.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde0069e10 con 0x7fdde009fb80 2024-09-15T04:02:23.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fddd8003090 con 0x7fdde009fb80 2024-09-15T04:02:23.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fdde4a6a700 4 set_mon_vals no callback set 2024-09-15T04:02:23.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(818..818 src has 435..818) v4 ==== 3939+0+0 (unknown 3603286327 0 0) 0x7fddd80444c0 con 0x7fdde009fb80 2024-09-15T04:02:23.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.812+0000 7fdde5a6c700 10 monclient: get_auth_request con 0x7fddc4038260 auth_method 0 2024-09-15T04:02:23.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.816+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fddd8014fd0 con 0x7fdde009fb80 2024-09-15T04:02:23.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.816+0000 7fddd67fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:23.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:23.816+0000 7fddd67fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:23.834 INFO:tasks.rados.rados.0.smithi122.stdout:3624: writing smithi12223906-18 from 1693365 to 2407467 tid 2 2024-09-15T04:02:23.834 INFO:tasks.rados.rados.0.smithi122.stdout:3624: writing smithi12223906-18 from 3096904 to 3428925 tid 3 2024-09-15T04:02:23.834 INFO:tasks.rados.rados.0.smithi122.stdout:3625: snap_remove snap 402 2024-09-15T04:02:24.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.040+0000 7fdde726f700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T04:02:24.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.040+0000 7fdde726f700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T04:02:24.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.040+0000 7fdde726f700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:24.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.040+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fdde004bbf0 con 0x7fdde009fb80 2024-09-15T04:02:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fddd67fc700 1 -- 172.21.15.26:0/214324161 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7fddd8014fd0 con 0x7fdde009fb80 2024-09-15T04:02:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fddd67fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T04:02:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fddd67fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:24.053 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.053 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.053 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=30,config=2,health=11,kv=8,logm=49} crc {auth=4185348528,config=3140514009,health=3318728087,kv=2519098299,logm=918546673}) 2024-09-15T04:02:24.053 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.053 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.053 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=3633363978,config=3140514009,health=3234334190,kv=2519098299,logm=317144568}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=28,config=2,health=12,kv=8,logm=50} crc {auth=2803909902,config=3140514009,health=3234334190,kv=2519098299,logm=884494167}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=30,config=2,health=12,kv=7,logm=49} crc {auth=2298218179,config=3140514009,health=3234334190,kv=1247634456,logm=1593415923}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=29,config=2,health=10,kv=8,logm=51} crc {auth=1344189196,config=3140514009,health=987857472,kv=2519098299,logm=2947826483}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=29,config=2,health=11,kv=8,logm=50} crc {auth=2497107157,config=3140514009,health=2885029788,kv=2519098299,logm=2158340900}) 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.054 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.055 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=30,config=2,health=12,kv=8,logm=48} crc {auth=364120777,config=3140514009,health=3234334190,kv=2519098299,logm=4115945313}) 2024-09-15T04:02:24.055 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.055 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=28,config=2,health=12,kv=7,logm=51} crc {auth=1234301984,config=3140514009,health=3234334190,kv=2519098299,logm=2661074052}) 2024-09-15T04:02:24.055 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.044+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=29,config=2,health=12,kv=8,logm=49} crc {auth=56153372,config=3140514009,health=3234334190,kv=2519098299,logm=3396345881}) 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 >> v1:172.21.15.122:6812/20895 conn(0x7fddc4038260 legacy=0x7fddc403a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fdde726f700 10 monclient: shutdown 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fdde726f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.044+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 >> v1:172.21.15.26:6792/0 conn(0x7fdde009fb80 legacy=0x7fdde01463a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.048+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 shutdown_connections 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.048+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 >> 172.21.15.26:0/214324161 conn(0x7fdde006e380 msgr2=0x7fdde0068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.048+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 shutdown_connections 2024-09-15T04:02:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:24.048+0000 7fdde726f700 1 -- 172.21.15.26:0/214324161 wait complete. 2024-09-15T04:02:24.059 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.059 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.059 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3638568117}) 2024-09-15T04:02:24.059 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.059 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.059 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1031639426}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1308476496}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=564150924}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2188211754}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2618024014}) 2024-09-15T04:02:24.060 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.061 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.061 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4139281392}) 2024-09-15T04:02:24.061 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.061 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2337296107}) 2024-09-15T04:02:24.061 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.048+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3309962634}) 2024-09-15T04:02:24.062 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.062 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2169953596}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2922470113}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3884020618}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1230200968}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.063 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1378509853}) 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4009760536}) 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2796417734}) 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=493195134}) 2024-09-15T04:02:24.064 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.052+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=696005280}) 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2227945540}) 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1875281629}) 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.066 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2598695067}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=156100328}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2287355981}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3702557497}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3786535632}) 2024-09-15T04:02:24.067 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.068 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=31731714}) 2024-09-15T04:02:24.068 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.056+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2750875057}) 2024-09-15T04:02:24.069 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2124842958}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2678193625}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=162417338}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.070 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2409846371}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3719516865}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2282916549}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2151365457}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126742508}) 2024-09-15T04:02:24.071 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3642059889}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2180932012}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=407743518}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1004861710}) 2024-09-15T04:02:24.073 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4014592067}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=756175582}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1063087332}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1464852348}) 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.074 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924064843}) 2024-09-15T04:02:24.075 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.060+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2336169601}) 2024-09-15T04:02:24.075 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T04:02:24.102 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=14997991}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=151045169}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3973486751}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2051158797}) 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.103 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3189825421}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1158299570}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955258946}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=810043236}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.064+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2822545416}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.104 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=421226679}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3395742193}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3433372976}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=75869254}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1414867852}) 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.105 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3655718543}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3794032579}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2682932201}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2213247080}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2520460113}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1161227683}) 2024-09-15T04:02:24.106 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3060276355}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1389430455}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1469440881}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1357950452}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.107 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=84714163}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4260202086}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.068+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2483303162}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3802921367}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1420868672}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1530952509}) 2024-09-15T04:02:24.108 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3964997962}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2803289095}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2809991096}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4014736249}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3319626636}) 2024-09-15T04:02:24.109 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1966909081}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2130349172}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2632530}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908383813}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1797756397}) 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.110 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2458120572}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=867462485}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2082684794}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4245999597}) 2024-09-15T04:02:24.111 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.072+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3923489680}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2189387374}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2008662523}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=515882359}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1899818736}) 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.112 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1191836937}) 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1731540069}) 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=919885567}) 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.113 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1519871285}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1400120721}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=694412593}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2874142394}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3559553121}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3920838416}) 2024-09-15T04:02:24.114 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2595905548}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3500729225}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=159143224}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2197150049}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.076+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2729894624}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.115 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2643859070}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4246968779}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2801217707}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1585531488}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=307919787}) 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.116 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3872854294}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3033573284}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2911921468}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=931232340}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1991309099}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3072444966}) 2024-09-15T04:02:24.117 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1433100384}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3408359851}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=85940705}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1376552447}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.118 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3695384239}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1248801350}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.080+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=803705325}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3818692045}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=633467272}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=536777561}) 2024-09-15T04:02:24.119 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3994234228}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1454684014}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3762083151}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085157808}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3652409871}) 2024-09-15T04:02:24.120 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3927894510}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3092273141}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2929189621}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=177066336}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4271756815}) 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.121 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=423142147}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457949168}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=70893226}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3893117086}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.084+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=120501848}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.122 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=560788537}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4242432436}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3104289675}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1464027645}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2584245024}) 2024-09-15T04:02:24.123 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=114328187}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1309943145}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1940775871}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=856078819}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=693698355}) 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.124 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=756776894}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2388130292}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2843859475}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=993925895}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.088+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.125 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2860249257}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1346770497}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3935149306}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2584035865}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3416044399}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1302068079}) 2024-09-15T04:02:24.126 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1778515797}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3176270178}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1722568542}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=283894606}) 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.127 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.128 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2126222588}) 2024-09-15T04:02:24.128 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.128 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1402219321}) 2024-09-15T04:02:24.128 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.092+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=515589049}) 2024-09-15T04:02:24.211 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.211 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2685973584}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3144462391}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2107396144}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4292803480}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.212 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=375539244}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2261444223}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1889142697}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2947091563}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1094976609}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2971784710}) 2024-09-15T04:02:24.213 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2692257305}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1763944932}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1570915852}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=698507617}) 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.214 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=834222477}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1141370589}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2282950148}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.096+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1739795262}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2407949660}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131420962}) 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.215 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=222125387}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1253165458}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3665562558}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=633474867}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.216 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=641818270}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=933512743}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.100+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3625887896}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3831045743}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=995516860}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1503013638}) 2024-09-15T04:02:24.217 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=624564222}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=556654416}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2782783378}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3098693242}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=118730547}) 2024-09-15T04:02:24.218 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4011512791}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4035608715}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4153206429}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2441722707}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=427062331}) 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.219 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=449867640}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3978390404}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695472651}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2751964219}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.104+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=858583806}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.220 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1176814328}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2031887298}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1969556833}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4166799370}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1106993163}) 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.221 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2031435253}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1111852802}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=868851361}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1278429154}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1242648649}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3676817281}) 2024-09-15T04:02:24.222 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1063113019}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1633472311}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=371076174}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=948280333}) 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.223 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3552732727}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2318363859}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.108+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1459917385}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=707494619}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1424738167}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.224 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1661017448}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2615849544}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3641360946}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3863687664}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4184922716}) 2024-09-15T04:02:24.225 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2834153477}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=47229509}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1521397886}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4358388}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3080486289}) 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.226 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=198408043}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3792085948}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1481929178}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1684335972}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1623913922}) 2024-09-15T04:02:24.227 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.112+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3591645769}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=715114011}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2877786228}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=96402366}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3068001447}) 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.228 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3834501296}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3332819752}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=292893334}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=906565832}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1339550114}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=527491094}) 2024-09-15T04:02:24.229 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1143547548}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2377997613}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2532402430}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2605998663}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2017919254}) 2024-09-15T04:02:24.230 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1670612384}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1737593453}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.116+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2851852677}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2346984021}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4278515675}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.231 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1715014649}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3728056377}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3003810762}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=96210350}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.232 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=326866814}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2531661737}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=611911757}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1081538683}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3853493478}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.233 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2945475720}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.120+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2527760466}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4068880013}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2135675969}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=476385182}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=191049252}) 2024-09-15T04:02:24.234 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1947249804}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1142493020}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1958329827}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3658804486}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=605874659}) 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.235 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4062653590}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2144792181}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3689426871}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3648237372}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.124+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3451340654}) 2024-09-15T04:02:24.236 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2323459160}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=404729646}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1912716402}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2639654101}) 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.237 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1728410131}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1002555304}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2735731879}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=941820113}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1848906755}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.238 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3423464403}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3840275520}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1892543406}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3869903286}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3160121221}) 2024-09-15T04:02:24.239 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1740509461}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1169455615}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=250560264}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.128+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3162079310}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3599364826}) 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.240 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=433197245}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=429433048}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1710642980}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4185043480}) 2024-09-15T04:02:24.241 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1387920516}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1774393797}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4242914087}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=397761726}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1407389424}) 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.242 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2772203898}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3429296220}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3202366662}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3413648499}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.243 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4092212588}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3596322757}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3788860094}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.132+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3586750042}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3769151984}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.244 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2544342812}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4110714856}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1618687108}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1491905554}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=172182335}) 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.245 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1676337411}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1012765636}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4052943564}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3044272118}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3837312077}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.246 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1044734838}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1534136358}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3221308713}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1258081876}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.247 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1979820885}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=592034663}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.136+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1649759080}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2087476587}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1504495844}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1971955243}) 2024-09-15T04:02:24.248 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2973087424}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2965926483}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4152234554}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3230984902}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2233707478}) 2024-09-15T04:02:24.249 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3461221242}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.140+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=230193383}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=796973461}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1552873091}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2819612610}) 2024-09-15T04:02:24.250 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1091379419}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1707213525}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=635395542}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1032800502}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1574594816}) 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.251 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3740782892}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2799963807}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3852608461}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=168375877}) 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.252 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2705131049}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3555902463}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1977823331}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1676814581}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2023066179}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.253 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1320171365}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=152297911}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=312967032}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.144+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1848354341}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=123128337}) 2024-09-15T04:02:24.254 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2077064803}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2695645978}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4031239044}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228715229}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1150881976}) 2024-09-15T04:02:24.255 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1933057166}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=724658225}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3084034859}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4226976631}) 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.256 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=475776312}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3934848302}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2149052299}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3273513623}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3365692416}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.257 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2213067938}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3546819594}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=83188180}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=968674589}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=372600719}) 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.258 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2151001701}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=342325434}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.148+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2683235036}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2543174}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=295357493}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.259 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3890382331}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1346826410}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3297823576}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3854506855}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=44184087}) 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.260 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1774059833}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2854084579}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=453081819}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59808153}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3677067091}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.261 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3069072751}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2846551069}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2068394087}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3413837635}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=982921432}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.152+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1751952950}) 2024-09-15T04:02:24.262 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=267502509}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2630278773}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1007275937}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=947213217}) 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.263 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3241369609}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2699419708}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2108870905}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3574992557}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1406828752}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.264 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3803902441}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3855668335}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4261905157}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.156+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3091851252}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1104610317}) 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.265 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2293645047}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3635580347}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2671538948}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=127723382}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2036685210}) 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.266 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=120471883}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4154188855}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2049609968}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2892437170}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2779674832}) 2024-09-15T04:02:24.267 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3720241859}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=742091361}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.160+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3797509760}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2770230995}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.268 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=318626756}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4112961230}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=105008187}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1201230679}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3824285085}) 2024-09-15T04:02:24.269 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=853981100}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3970872936}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2347064405}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1871582905}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2466216584}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.270 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=756136330}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2303633290}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1469971351}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2585663122}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=947485014}) 2024-09-15T04:02:24.271 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1791245428}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.164+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2389591007}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1971250902}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2511928916}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=90180020}) 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.272 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3582862678}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4044531665}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1741504157}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=941489195}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500678214}) 2024-09-15T04:02:24.273 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.168+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1787233099}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4273054859}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2049880456}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2793935315}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1684929980}) 2024-09-15T04:02:24.274 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=309589237}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1825804670}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=595778994}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3140529620}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4282045470}) 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.275 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=729854052}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2270503322}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3231524996}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4181990072}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2153122058}) 2024-09-15T04:02:24.276 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3720461696}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4050703651}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1458503800}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.172+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2632733764}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=304653966}) 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.277 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1032635949}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2156166301}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=42291880}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1030979061}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1495807384}) 2024-09-15T04:02:24.278 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3303544788}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1489373322}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4067271766}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4059616767}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=353122487}) 2024-09-15T04:02:24.279 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1314046015}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=642051711}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3796360134}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=360912014}) 2024-09-15T04:02:24.280 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3123012865}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1850739561}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.176+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1992870513}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4103269550}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2832921299}) 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.281 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3844697259}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2846677327}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3036194400}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1420927558}) 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.282 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1639211081}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=951125761}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3548772256}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=542800661}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=40421406}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.283 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=117398157}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.180+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3678496867}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2020061433}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2022911986}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=532563116}) 2024-09-15T04:02:24.284 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4213134302}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3218768699}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1663133879}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2279533344}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1369013954}) 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.285 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=48711401}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2987513648}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=561905288}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3785850272}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596274565}) 2024-09-15T04:02:24.286 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.184+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2463739144}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=742721741}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3616370617}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2489280407}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3406912994}) 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.287 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1088473355}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=503166285}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2275752906}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2357865840}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1973384520}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.288 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3051387250}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3753169350}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1154847855}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=291195559}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3796123811}) 2024-09-15T04:02:24.289 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3670235405}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1971847351}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2367146101}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.188+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1571619678}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3736346053}) 2024-09-15T04:02:24.290 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2904370812}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3099181381}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3769168764}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=758858937}) 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.291 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2450742330}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3095554151}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1465230834}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.192+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3930178731}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3729013636}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4051611741}) 2024-09-15T04:02:24.292 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2180705246}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=788822668}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3590986666}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4191072489}) 2024-09-15T04:02:24.293 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2265073106}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3260756615}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2235999196}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=647324041}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1516553931}) 2024-09-15T04:02:24.294 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3103254909}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2602376315}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2029443802}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1348904267}) 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.295 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2956876764}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2448088419}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.196+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3343893173}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3381419478}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2738511165}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.296 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3356213615}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1393089864}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2873044715}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=83641479}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=125294568}) 2024-09-15T04:02:24.297 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2848711096}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1502867044}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1611398603}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=42355720}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3797945191}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.298 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2664702640}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3991345697}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1622337584}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3927901573}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1043621361}) 2024-09-15T04:02:24.299 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.200+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3594856209}) 2024-09-15T04:02:24.353 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.353 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.353 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4244611264}) 2024-09-15T04:02:24.353 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.353 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693325037}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2203261525}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2291380884}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2062705041}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.354 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3357303132}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2676852031}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1477583943}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.204+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2824755114}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2218918709}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4275889529}) 2024-09-15T04:02:24.355 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1271798769}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1113343377}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2276286829}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=784581138}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.356 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3306010743}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3095124532}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1773678292}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1180352660}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2419960309}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2758145711}) 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.357 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3751708107}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3109045822}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=212414732}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2309201654}) 2024-09-15T04:02:24.358 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=759878534}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.208+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2811201939}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=865408434}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3725806829}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1539128320}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.359 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=644389817}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3496742355}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1209921886}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3710918937}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=378731817}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1419714858}) 2024-09-15T04:02:24.360 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3578560258}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=298283204}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2813382509}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2378203041}) 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.361 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3050478379}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3698552487}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3240524596}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=72918555}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.212+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1619777156}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.362 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2258941803}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3418820206}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2819228192}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3247133674}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4001346544}) 2024-09-15T04:02:24.363 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=55143974}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1059115707}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1195008413}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2737803938}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2534617263}) 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.364 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1938110452}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3749436379}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=861037919}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1097712805}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.365 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1591390938}) 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2172711971}) 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1434986876}) 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.216+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=322589078}) 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.366 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1432328025}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3546823825}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1400815098}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3951999583}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.368 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2842357656}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1765671990}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2620267538}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2946519307}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.220+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=51061391}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2771254140}) 2024-09-15T04:02:24.369 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4150935155}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3758004510}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=514645788}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=244655087}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.370 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3018989951}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3955476033}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=419602820}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=775941436}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2513171318}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2688249020}) 2024-09-15T04:02:24.371 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2535104459}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4257537558}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.224+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4107643515}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2308250793}) 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.372 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=27356183}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2597737430}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3007343380}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3049915985}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=505670011}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2707858656}) 2024-09-15T04:02:24.373 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1843547577}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2293781909}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2457537518}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3770992675}) 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.374 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1099872285}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4217769111}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.228+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3749479739}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=129522370}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=186910626}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.375 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1911181518}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2994204942}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4101307619}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2328967179}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2647100217}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2427952634}) 2024-09-15T04:02:24.376 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3475034409}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1443055443}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3984174804}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1000880766}) 2024-09-15T04:02:24.377 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=593895239}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2255800755}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3346948023}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2463022527}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.232+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=48724710}) 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.378 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2999420667}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1392808966}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1989503494}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=298233196}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.379 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3297857521}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3363565181}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1305595051}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834288908}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.236+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3433969322}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.380 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917729975}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1626767966}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1481587123}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=592031067}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=572882555}) 2024-09-15T04:02:24.381 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017902093}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2632257530}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=248600955}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2233984417}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3804557694}) 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.382 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=742412692}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1387378302}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1047286653}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.240+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2237685447}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.383 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4115473334}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=506979452}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1256395324}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=282151309}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=159889320}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3424091315}) 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.384 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2698032518}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3731514328}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2851411829}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3929962847}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.385 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=114474187}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3133865865}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.244+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1592492981}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2934051965}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3033913774}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=362564560}) 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.386 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2765757736}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3848641803}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2347768061}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1598013578}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=126427971}) 2024-09-15T04:02:24.387 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=455829455}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2839921406}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2203322024}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2060617331}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=153251665}) 2024-09-15T04:02:24.388 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3950164673}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.248+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3446931952}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3476479138}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2894695687}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.252+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1230244849}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.389 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=3,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=32} crc {logm=1676807223,mdsmap=1531151339,mgr=2380166094,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=4179472217,mon_config_key=2633562880,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3547268808,osd_snap=526053274}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_snap=30} crc {logm=2282231538,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=4179472217,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_snap=4220625810}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=35,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=29} crc {logm=1529807393,mdsmap=1531151339,mgr=436553390,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=952528607,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3547268808,osd_snap=4248846565}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=30} crc {logm=2399891482,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=3572388546,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3547268808,osd_snap=2147493607}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=29} crc {logm=2337136591,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=952528607,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=3547268808,osd_snap=1240411294}) 2024-09-15T04:02:24.390 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=4,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1218359444,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=952528607,mon_config_key=1595646671,monmap=3140514009,osd_metadata=3876164839,osd_pg_creating=3547268809,osd_snap=912615988}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=31} crc {logm=1291433212,mdsmap=2394902515,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2585871464,mon_config_key=733955277,monmap=2173287178,osd_metadata=2994937890,osd_pg_creating=3547268808,osd_snap=747753832}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=5,monmap=3,osd_metadata=5,osd_pg_creating=1,osd_snap=32} crc {logm=1505824853,mdsmap=3140514009,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=692337739,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_pg_creating=3547268808,osd_snap=1839261543}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=32,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=33} crc {logm=2454246942,mdsmap=1531151339,mgr=2132010928,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=952528607,mon_config_key=733955277,monmap=3140514009,osd_metadata=2994937890,osd_pg_creating=3547268808,osd_snap=855970954}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=4230603116}) 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.391 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=2445732111}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=3537778225}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=100} crc {osd_snap=2744744270}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=100} crc {osd_snap=113879589}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.392 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=100} crc {osd_snap=3151524835}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=1892491864}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=1989895629}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.256+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=1922932476}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=1793138698,osdmap=2380657581}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=38,osdmap=62} crc {osd_snap=435525972,osdmap=1992123327}) 2024-09-15T04:02:24.393 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=3163085261,osdmap=89618144}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=38,osdmap=62} crc {osd_snap=4257232048,osdmap=3073017687}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=3290758776,osdmap=995522541}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=1485228676,osdmap=4006919611}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.394 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=34,osdmap=66} crc {osd_snap=4271745421,osdmap=2535015472}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=36,osdmap=64} crc {osd_snap=1342691500,osdmap=2945097980}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=37,osdmap=63} crc {osd_snap=116623771,osdmap=1366721341}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=576794038}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3279638483}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2252439128}) 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.395 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1823652332}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3880203055}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3979941781}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2672421976}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3301382585}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.260+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1065099653}) 2024-09-15T04:02:24.396 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=49075931}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3915988627}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2884660046}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1072387653}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.397 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=907248803}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=344344607}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1019079278}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3634895251}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.264+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1646735339}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2693702450}) 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.398 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=450563690}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=116437342}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3681874905}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1558074188}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.399 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2583743185}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1145729465}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2473682458}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.268+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2277336715}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1595727244}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4227710115}) 2024-09-15T04:02:24.400 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=392199845}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3506703803}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1185222954}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=790124254}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.401 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=133753613}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3030301516}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.272+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3344786935}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=96} crc {osdmap=1248617761}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=3831403503}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.402 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=98} crc {osdmap=882927079}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=95} crc {osdmap=3795400727}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=98} crc {osdmap=636011579}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=95} crc {osdmap=3957599433}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=95} crc {osdmap=2623473172}) 2024-09-15T04:02:24.403 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:02:24.404 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=95} crc {osdmap=616792785}) 2024-09-15T04:02:24.404 INFO:tasks.ceph.mon.e.smithi026.stderr:2024-09-15T04:02:24.276+0000 7f67db141700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=95} crc {osdmap=553397371}) 2024-09-15T04:02:24.905 INFO:tasks.rados.rados.0.smithi122.stdout:3625: done (2 left) 2024-09-15T04:02:24.905 INFO:tasks.rados.rados.0.smithi122.stdout:3626: snap_remove snap 321 2024-09-15T04:02:25.077 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T04:02:25.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:02:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: get_monmap_and_config 2024-09-15T04:02:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: monmap: 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:25.188284+0000 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:25.188284+0000 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.211 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 1 Processor -- start 2024-09-15T04:02:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 1 -- start start 2024-09-15T04:02:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: init 2024-09-15T04:02:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: _add_conns ranks=[8,5,4,7,0,3,2,1,6] 2024-09-15T04:02:25.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient(hunting): picked mon.noname-e con 0x7f27640a6a80 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient(hunting): picked mon.noname-h con 0x7f276409fb80 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient(hunting): picked mon.noname-c con 0x7f2764138d30 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276404ca10 con 0x7f2764138d30 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27640a0f50 con 0x7f276409fb80 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2764133680 con 0x7f27640a6a80 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient(hunting): authenticate will time out at 1889.617668s 2024-09-15T04:02:25.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2762d9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f2764138d30 0x7f276413b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41052/0 (socket says 172.21.15.26:41052) 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2762d9d700 1 -- 172.21.15.26:0/2168991279 learned_addr learned my addr 172.21.15.26:0/2168991279 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685248215 0 0) 0x7f276404ca10 con 0x7f2764138d30 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): my global_id is 204934 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2750003570 con 0x7f2764138d30 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631390503 0 0) 0x7f2764133680 con 0x7f27640a6a80 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): my global_id is 135737 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f276404ca10 con 0x7f27640a6a80 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385124005 0 0) 0x7f27640a0f50 con 0x7f276409fb80 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient(hunting): my global_id is 254210 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2764133680 con 0x7f276409fb80 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143088488 0 0) 0x7f2750003570 con 0x7f2764138d30 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 >> v1:172.21.15.26:6793/0 conn(0x7f27640a6a80 legacy=0x7f2764132f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 >> v1:172.21.15.122:6791/0 conn(0x7f276409fb80 legacy=0x7f276409ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 monclient: found mon.noname-c 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640a6e50 con 0x7f2764138d30 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f275c001b30 con 0x7f2764138d30 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 5 monclient: authenticate success, global_id 204934 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:25.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: dump: 2024-09-15T04:02:25.218 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.218 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.218 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.218 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.219 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.219 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.219 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.220 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.220 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.220 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.220 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.220 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.221 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.221 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.221 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:25.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f275c001e10 con 0x7f2764138d30 2024-09-15T04:02:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 1 -- 172.21.15.26:0/2168991279 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f275c004fb0 con 0x7f2764138d30 2024-09-15T04:02:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 4 set_mon_vals no callback set 2024-09-15T04:02:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f2761d9b700 10 monclient: dump: 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.223 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 10 monclient: shutdown 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.184+0000 7f276359e700 1 -- 172.21.15.26:0/2168991279 >> v1:172.21.15.26:6791/0 conn(0x7f2764138d30 legacy=0x7f276413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- 172.21.15.26:0/2168991279 shutdown_connections 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- 172.21.15.26:0/2168991279 >> 172.21.15.26:0/2168991279 conn(0x7f276406e380 msgr2=0x7f27640a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- 172.21.15.26:0/2168991279 shutdown_connections 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- 172.21.15.26:0/2168991279 wait complete. 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient: monmap: 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.225 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 Processor -- start 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- start start 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient: init 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient: _add_conns ranks=[4,7,0,1,8,2,5,6,3] 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient(hunting): picked mon.noname-c con 0x7f276409fb80 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:25.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient(hunting): picked mon.noname-i con 0x7f27640a6a80 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient(hunting): picked mon.noname-a con 0x7f2764138d30 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27640a1920 con 0x7f2764138d30 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276409cfc0 con 0x7f276409fb80 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276409be40 con 0x7f27640a6a80 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276359e700 10 monclient(hunting): authenticate will time out at 1889.622392s 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276259c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f276409fb80 0x7f2764099990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41062/0 (socket says 172.21.15.26:41062) 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f276259c700 1 -- 172.21.15.26:0/2019056065 learned_addr learned my addr 172.21.15.26:0/2019056065 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478596117 0 0) 0x7f276409cfc0 con 0x7f276409fb80 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): my global_id is 204943 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2744003570 con 0x7f276409fb80 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480981818 0 0) 0x7f27640a1920 con 0x7f2764138d30 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): my global_id is 195039 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f276409cfc0 con 0x7f2764138d30 2024-09-15T04:02:25.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2475180409 0 0) 0x7f2744003570 con 0x7f276409fb80 2024-09-15T04:02:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27640a1920 con 0x7f276409fb80 2024-09-15T04:02:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356896032 0 0) 0x7f276409be40 con 0x7f27640a6a80 2024-09-15T04:02:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): my global_id is 254167 2024-09-15T04:02:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2744003570 con 0x7f27640a6a80 2024-09-15T04:02:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f274c001b20 con 0x7f276409fb80 2024-09-15T04:02:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:25.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.231 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.232 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.233 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 59120464 0 0) 0x7f276409cfc0 con 0x7f2764138d30 2024-09-15T04:02:25.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f276409be40 con 0x7f2764138d30 2024-09-15T04:02:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f275c004ef0 con 0x7f2764138d30 2024-09-15T04:02:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.234 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.235 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.236 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.237 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2577243979 0 0) 0x7f27640a1920 con 0x7f276409fb80 2024-09-15T04:02:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 >> v1:172.21.15.122:6792/0 conn(0x7f27640a6a80 legacy=0x7f276409a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 >> v1:172.21.15.26:6789/0 conn(0x7f2764138d30 legacy=0x7f276414a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 monclient: found mon.c 2024-09-15T04:02:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.188+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276411c970 con 0x7f276409fb80 2024-09-15T04:02:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 5 monclient: authenticate success, global_id 204943 2024-09-15T04:02:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 monclient: _renew_subs 2024-09-15T04:02:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276409b270 con 0x7f276409fb80 2024-09-15T04:02:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 monclient: _renew_subs 2024-09-15T04:02:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f276409a7b0 con 0x7f276409fb80 2024-09-15T04:02:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276411cd90 con 0x7f276409fb80 2024-09-15T04:02:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f274c001ff0 con 0x7f276409fb80 2024-09-15T04:02:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f274c0024a0 con 0x7f276409fb80 2024-09-15T04:02:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f276159a700 4 set_mon_vals no callback set 2024-09-15T04:02:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 10 monclient: dump: 2024-09-15T04:02:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.244 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.245 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f274c002600 con 0x7f276409fb80 2024-09-15T04:02:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(819..819 src has 435..819) v4 ==== 3955+0+0 (unknown 4209169018 0 0) 0x7f274c043ba0 con 0x7f276409fb80 2024-09-15T04:02:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.192+0000 7f275bfff700 10 monclient: get_auth_request con 0x7f2744037ea0 auth_method 0 2024-09-15T04:02:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.196+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f274c002be0 con 0x7f276409fb80 2024-09-15T04:02:25.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.196+0000 7f275a7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.196+0000 7f275a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:25.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f276359e700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:02:25.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f276359e700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:25.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f276359e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f276411cd90 con 0x7f276409fb80 2024-09-15T04:02:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f275a7fc700 1 -- 172.21.15.26:0/2019056065 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 3956013070) 0x7f276411cd90 con 0x7f276409fb80 2024-09-15T04:02:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f275a7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:25.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.420+0000 7f275a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:25.430 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":112,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"e","quorum_age":6,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:02:25.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 >> v1:172.21.15.122:6812/20895 conn(0x7f2744037ea0 legacy=0x7f274403a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 10 monclient: shutdown 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 >> v1:172.21.15.26:6791/0 conn(0x7f276409fb80 legacy=0x7f2764099990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 shutdown_connections 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 >> 172.21.15.26:0/2019056065 conn(0x7f276406e380 msgr2=0x7f276409db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 shutdown_connections 2024-09-15T04:02:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.424+0000 7f276359e700 1 -- 172.21.15.26:0/2019056065 wait complete. 2024-09-15T04:02:25.444 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T04:02:25.444 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-15T04:02:25.444 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T04:02:25.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.548+0000 7ffa7db6f700 10 monclient: get_monmap_and_config 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.548+0000 7ffa7db6f700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.548+0000 7ffa7db6f700 10 monclient: monmap: 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:25.554772+0000 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:25.554772+0000 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.558 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.559 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.548+0000 7ffa7db6f700 1 Processor -- start 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.548+0000 7ffa7db6f700 1 -- start start 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.548+0000 7ffa7db6f700 10 monclient: init 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient: _add_conns ranks=[0,6,1,8,5,7,3,4,2] 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient(hunting): picked mon.noname-a con 0x7ffa7809e440 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient(hunting): picked mon.noname-d con 0x7ffa7812ffc0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient(hunting): picked mon.noname-f con 0x7ffa78130390 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7804ca10 con 0x7ffa7809e440 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7809a0f0 con 0x7ffa78130390 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7809e810 con 0x7ffa7812ffc0 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient(hunting): authenticate will time out at 1889.984676s 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa6ffff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ffa7812ffc0 0x7ffa78134be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55684/0 (socket says 172.21.15.26:55684) 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa6ffff700 1 -- 172.21.15.26:0/1223923170 learned_addr learned my addr 172.21.15.26:0/1223923170 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526648001 0 0) 0x7ffa7809e810 con 0x7ffa7812ffc0 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): my global_id is 195144 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa64003570 con 0x7ffa7812ffc0 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255061036 0 0) 0x7ffa7804ca10 con 0x7ffa7809e440 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): my global_id is 195048 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa7809e810 con 0x7ffa7809e440 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380132888 0 0) 0x7ffa7809a0f0 con 0x7ffa78130390 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient(hunting): my global_id is 204913 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa7804ca10 con 0x7ffa78130390 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1965814524 0 0) 0x7ffa64003570 con 0x7ffa7812ffc0 2024-09-15T04:02:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 >> v1:172.21.15.122:6789/0 conn(0x7ffa78130390 legacy=0x7ffa78138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 >> v1:172.21.15.26:6789/0 conn(0x7ffa7809e440 legacy=0x7ffa7812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 monclient: found mon.noname-d 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7809c2a0 con 0x7ffa7812ffc0 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa680017d0 con 0x7ffa7812ffc0 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 5 monclient: authenticate success, global_id 195144 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: dump: 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.567 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa680019f0 con 0x7ffa7812ffc0 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 4 set_mon_vals no callback set 2024-09-15T04:02:25.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 1 -- 172.21.15.26:0/1223923170 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa68001be0 con 0x7ffa7812ffc0 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa76ffd700 10 monclient: dump: 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.569 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 10 monclient: shutdown 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- 172.21.15.26:0/1223923170 >> v1:172.21.15.26:6792/0 conn(0x7ffa7812ffc0 legacy=0x7ffa78134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- 172.21.15.26:0/1223923170 shutdown_connections 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- 172.21.15.26:0/1223923170 >> 172.21.15.26:0/1223923170 conn(0x7ffa78063f30 msgr2=0x7ffa780a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- 172.21.15.26:0/1223923170 shutdown_connections 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.552+0000 7ffa7db6f700 1 -- 172.21.15.26:0/1223923170 wait complete. 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: monmap: 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.571 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.572 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.572 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 Processor -- start 2024-09-15T04:02:25.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 -- start start 2024-09-15T04:02:25.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: init 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: _add_conns ranks=[3,0,5,8,4,7,6,2,1] 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient(hunting): picked mon.noname-g con 0x7ffa7809e440 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient(hunting): picked mon.noname-a con 0x7ffa7812ffc0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient(hunting): picked mon.noname-h con 0x7ffa78130390 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa78099720 con 0x7ffa7812ffc0 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa780994f0 con 0x7ffa7809e440 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7811dc20 con 0x7ffa78130390 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient(hunting): authenticate will time out at 1889.989669s 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6ffff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffa7812ffc0 0x7ffa78142bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40398/0 (socket says 172.21.15.26:40398) 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6ffff700 1 -- 172.21.15.26:0/975410927 learned_addr learned my addr 172.21.15.26:0/975410927 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167442975 0 0) 0x7ffa78099720 con 0x7ffa7812ffc0 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): my global_id is 195057 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58003570 con 0x7ffa7812ffc0 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3914575691 0 0) 0x7ffa58003570 con 0x7ffa7812ffc0 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa78099720 con 0x7ffa7812ffc0 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa68001b60 con 0x7ffa7812ffc0 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): dump: 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715448104 0 0) 0x7ffa780994f0 con 0x7ffa7809e440 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): my global_id is 254208 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa58003570 con 0x7ffa7809e440 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473851972 0 0) 0x7ffa7811dc20 con 0x7ffa78130390 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient(hunting): my global_id is 254219 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa780994f0 con 0x7ffa78130390 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3195404049 0 0) 0x7ffa78099720 con 0x7ffa7812ffc0 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.122:6791/0 conn(0x7ffa78130390 legacy=0x7ffa78146300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.122:6790/0 conn(0x7ffa7809e440 legacy=0x7ffa7812e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 monclient: found mon.a 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7811d940 con 0x7ffa7812ffc0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 5 monclient: authenticate success, global_id 195057 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: _renew_subs 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7811be80 con 0x7ffa7812ffc0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: _renew_subs 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7812ef10 con 0x7ffa7812ffc0 2024-09-15T04:02:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffa68004e90 con 0x7ffa7812ffc0 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa680053c0 con 0x7ffa7812ffc0 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa6f7fe700 10 monclient: dump: 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.581 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa767fc700 4 set_mon_vals no callback set 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.556+0000 7ffa7db6f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffa5c006590 con 0x7ffa5c002840 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffa6800fe00 con 0x7ffa7812ffc0 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(819..819 src has 435..819) v4 ==== 3955+0+0 (unknown 4209169018 0 0) 0x7ffa68044150 con 0x7ffa7812ffc0 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708572077 0 0) 0x7ffa5c006590 con 0x7ffa5c002840 2024-09-15T04:02:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient(hunting): my global_id is 195066 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa78099720 con 0x7ffa5c002840 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa77fff700 10 monclient: get_auth_request con 0x7ffa58037710 auth_method 0 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3299633497 0 0) 0x7ffa78099720 con 0x7ffa5c002840 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa5c006590 con 0x7ffa5c002840 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa60001650 con 0x7ffa5c002840 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient: dump: 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2887204084 0 0) 0x7ffa5c006590 con 0x7ffa5c002840 2024-09-15T04:02:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa5c00a130 con 0x7ffa5c002840 2024-09-15T04:02:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ffa600020a0 con 0x7ffa5c002840 2024-09-15T04:02:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.26:6789/0 conn(0x7ffa5c002840 legacy=0x7ffa5c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.560+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.26:6789/0 conn(0x7ffa5c002840 legacy=0x7ffa5c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa7db6f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffa5c007520 con 0x7ffa5c0068d0 2024-09-15T04:02:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439100113 0 0) 0x7ffa5c007520 con 0x7ffa5c0068d0 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 10 monclient(hunting): my global_id is 195075 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5c006590 con 0x7ffa5c0068d0 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 999197270 0 0) 0x7ffa5c006590 con 0x7ffa5c0068d0 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffa5c007520 con 0x7ffa5c0068d0 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffa600028d0 con 0x7ffa5c0068d0 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.568+0000 7ffa6f7fe700 10 monclient: dump: 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.588 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1044698524 0 0) 0x7ffa5c007520 con 0x7ffa5c0068d0 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffa5c008760 con 0x7ffa5c0068d0 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 2839160808) 0x7ffa60002cc0 con 0x7ffa5c0068d0 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.26:6789/0 conn(0x7ffa5c0068d0 legacy=0x7ffa5c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa6f7fe700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.26:6789/0 conn(0x7ffa5c0068d0 legacy=0x7ffa5c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:25.589 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:25.590 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 6, 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:25.593 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.594 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.595 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.597 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.598 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:25.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:25.603 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:25.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.609 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:25.609 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.122:6812/20895 conn(0x7ffa58037710 legacy=0x7ffa58039bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 10 monclient: shutdown 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 >> v1:172.21.15.26:6789/0 conn(0x7ffa7812ffc0 legacy=0x7ffa78142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa77fff700 1 -- 172.21.15.26:0/975410927 reap_dead start 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa77fff700 1 -- 172.21.15.26:0/975410927 reap_dead start 2024-09-15T04:02:25.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 shutdown_connections 2024-09-15T04:02:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 >> 172.21.15.26:0/975410927 conn(0x7ffa78063f30 msgr2=0x7ffa7809ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 shutdown_connections 2024-09-15T04:02:25.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.572+0000 7ffa7db6f700 1 -- 172.21.15.26:0/975410927 wait complete. 2024-09-15T04:02:25.611 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:25.611 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T04:02:25.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T04:02:25.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient: get_monmap_and_config 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient: monmap: 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:25.724455+0000 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:25.724455+0000 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.729 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 1 Processor -- start 2024-09-15T04:02:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 1 -- start start 2024-09-15T04:02:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient: init 2024-09-15T04:02:25.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient: _add_conns ranks=[1,7,4,6,0,3,5,8,2] 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient(hunting): picked mon.noname-f con 0x7f203c132b40 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient(hunting): picked mon.noname-i con 0x7f203c09faa0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f203c138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c04ca10 con 0x7f203c132b40 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c09d5d0 con 0x7f203c138d40 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c133620 con 0x7f203c09faa0 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203bfff700 10 monclient(hunting): authenticate will time out at 1890.154637s 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203b7fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f203c138d40 0x7f203c13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41072/0 (socket says 172.21.15.26:41072) 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203b7fe700 1 -- 172.21.15.26:0/3483898304 learned_addr learned my addr 172.21.15.26:0/3483898304 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762055663 0 0) 0x7f203c133620 con 0x7f203c09faa0 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): my global_id is 254176 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c003570 con 0x7f203c09faa0 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834881299 0 0) 0x7f203c09d5d0 con 0x7f203c138d40 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): my global_id is 204952 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c133620 con 0x7f203c138d40 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968063597 0 0) 0x7f203c04ca10 con 0x7f203c132b40 2024-09-15T04:02:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 10 monclient(hunting): my global_id is 204922 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.720+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c09d5d0 con 0x7f203c132b40 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927532819 0 0) 0x7f203c133620 con 0x7f203c138d40 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 >> v1:172.21.15.122:6792/0 conn(0x7f203c09faa0 legacy=0x7f203c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 >> v1:172.21.15.122:6789/0 conn(0x7f203c132b40 legacy=0x7f203c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 monclient: found mon.noname-c 2024-09-15T04:02:25.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0a0cc0 con 0x7f203c138d40 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2034001b10 con 0x7f203c138d40 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: dump: 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.736 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 5 monclient: authenticate success, global_id 204952 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:25.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2034001dc0 con 0x7f203c138d40 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 4 set_mon_vals no callback set 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 1 -- 172.21.15.26:0/3483898304 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2034001fb0 con 0x7f203c138d40 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203a7fc700 10 monclient: dump: 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 10 monclient: shutdown 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 -- 172.21.15.26:0/3483898304 >> v1:172.21.15.26:6791/0 conn(0x7f203c138d40 legacy=0x7f203c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 -- 172.21.15.26:0/3483898304 shutdown_connections 2024-09-15T04:02:25.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 -- 172.21.15.26:0/3483898304 >> 172.21.15.26:0/3483898304 conn(0x7f203c06e370 msgr2=0x7f203c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 -- 172.21.15.26:0/3483898304 shutdown_connections 2024-09-15T04:02:25.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 -- 172.21.15.26:0/3483898304 wait complete. 2024-09-15T04:02:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 10 monclient: monmap: 2024-09-15T04:02:25.744 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.745 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 Processor -- start 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 1 -- start start 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.724+0000 7f203bfff700 10 monclient: init 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient: _add_conns ranks=[0,4,1,7,8,2,5,3,6] 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient(hunting): picked mon.noname-a con 0x7f203c09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient(hunting): picked mon.noname-c con 0x7f203c132b40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient(hunting): picked mon.noname-f con 0x7f203c138d40 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c09cfa0 con 0x7f203c09faa0 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c09b4d0 con 0x7f203c138d40 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c1326c0 con 0x7f203c132b40 2024-09-15T04:02:25.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient(hunting): authenticate will time out at 1890.160873s 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f203c09faa0 0x7f203c099610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40424/0 (socket says 172.21.15.26:40424) 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2033fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f203c132b40 0x7f203c099d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41080/0 (socket says 172.21.15.26:41080) 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203affd700 1 -- 172.21.15.26:0/2479420163 learned_addr learned my addr 172.21.15.26:0/2479420163 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547529506 0 0) 0x7f203c1326c0 con 0x7f203c132b40 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): my global_id is 204961 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2020003570 con 0x7f203c132b40 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727647945 0 0) 0x7f203c09cfa0 con 0x7f203c09faa0 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): my global_id is 195084 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c1326c0 con 0x7f203c09faa0 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446732769 0 0) 0x7f203c09b4d0 con 0x7f203c138d40 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): my global_id is 204931 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c09cfa0 con 0x7f203c138d40 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 527564977 0 0) 0x7f2020003570 con 0x7f203c132b40 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f203c09b4d0 con 0x7f203c132b40 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2024001a80 con 0x7f203c132b40 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): dump: 2024-09-15T04:02:25.748 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.749 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2995354773 0 0) 0x7f203c09b4d0 con 0x7f203c132b40 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.122:6789/0 conn(0x7f203c138d40 legacy=0x7f203c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.26:6789/0 conn(0x7f203c09faa0 legacy=0x7f203c099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 monclient: found mon.c 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c11cb70 con 0x7f203c132b40 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 5 monclient: authenticate success, global_id 204961 2024-09-15T04:02:25.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient: _renew_subs 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c09afc0 con 0x7f203c132b40 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient: _renew_subs 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c09bb90 con 0x7f203c132b40 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2024002d90 con 0x7f203c132b40 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2024002350 con 0x7f203c132b40 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2039ffb700 4 set_mon_vals no callback set 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f2032ffd700 10 monclient: dump: 2024-09-15T04:02:25.751 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.752 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.728+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2014006590 con 0x7f2014002840 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302668592 0 0) 0x7f2014006590 con 0x7f2014002840 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient(hunting): my global_id is 254135 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f203c09b4d0 con 0x7f2014002840 2024-09-15T04:02:25.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2024002530 con 0x7f203c132b40 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(819..819 src has 435..819) v4 ==== 3955+0+0 (unknown 4209169018 0 0) 0x7f2024043f40 con 0x7f203c132b40 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3283521435 0 0) 0x7f203c09b4d0 con 0x7f2014002840 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2014006590 con 0x7f2014002840 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f202c002a00 con 0x7f2014002840 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient: dump: 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f203b7fe700 10 monclient: get_auth_request con 0x7f2020037550 auth_method 0 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1067621081 0 0) 0x7f2014006590 con 0x7f2014002840 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f201400a130 con 0x7f2014002840 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f202c005810 con 0x7f2014002840 2024-09-15T04:02:25.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:25.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.26:6790/0 conn(0x7f2014002840 legacy=0x7f2014004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.732+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.26:6790/0 conn(0x7f2014002840 legacy=0x7f2014004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2014007520 con 0x7f20140068d0 2024-09-15T04:02:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401509033 0 0) 0x7f2014007520 con 0x7f20140068d0 2024-09-15T04:02:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient(hunting): my global_id is 254144 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2014006590 con 0x7f20140068d0 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1220520475 0 0) 0x7f2014006590 con 0x7f20140068d0 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2014007520 con 0x7f20140068d0 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f202c0019f0 con 0x7f20140068d0 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient: dump: 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.759 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3274022295 0 0) 0x7f2014007520 con 0x7f20140068d0 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2014008760 con 0x7f20140068d0 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3785434248) 0x7f202c005aa0 con 0x7f20140068d0 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.26:6790/0 conn(0x7f20140068d0 legacy=0x7f2014006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f2032ffd700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.26:6790/0 conn(0x7f20140068d0 legacy=0x7f2014006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:25.760 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.761 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.762 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:25.764 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.765 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:25.766 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:25.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.768 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:25.769 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.770 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.773 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:25.774 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.775 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.779 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.780 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:25.780 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.122:6812/20895 conn(0x7f2020037550 legacy=0x7f2020039a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 10 monclient: shutdown 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 >> v1:172.21.15.26:6791/0 conn(0x7f203c132b40 legacy=0x7f203c099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203b7fe700 1 -- 172.21.15.26:0/2479420163 reap_dead start 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203b7fe700 1 -- 172.21.15.26:0/2479420163 reap_dead start 2024-09-15T04:02:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 shutdown_connections 2024-09-15T04:02:25.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 >> 172.21.15.26:0/2479420163 conn(0x7f203c06e370 msgr2=0x7f203c09e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 shutdown_connections 2024-09-15T04:02:25.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.740+0000 7f203bfff700 1 -- 172.21.15.26:0/2479420163 wait complete. 2024-09-15T04:02:25.782 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:25.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T04:02:25.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.888+0000 7f49db708700 10 monclient: get_monmap_and_config 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.888+0000 7f49db708700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.888+0000 7f49db708700 10 monclient: monmap: 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:25.894634+0000 2024-09-15T04:02:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:25.894634+0000 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.899 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 1 Processor -- start 2024-09-15T04:02:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 1 -- start start 2024-09-15T04:02:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient: init 2024-09-15T04:02:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient: _add_conns ranks=[3,7,8,1,2,6,0,4,5] 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient(hunting): picked mon.noname-g con 0x7f49d409d6b0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient(hunting): picked mon.noname-i con 0x7f49d409bcb0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient(hunting): picked mon.noname-e con 0x7f49d4137020 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d404ca10 con 0x7f49d409d6b0 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d40a0670 con 0x7f49d409bcb0 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d409d060 con 0x7f49d4137020 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 10 monclient(hunting): authenticate will time out at 1890.325339s 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49d9ca5700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f49d4137020 0x7f49d4139400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50558/0 (socket says 172.21.15.26:50558) 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49d9ca5700 1 -- 172.21.15.26:0/1632131477 learned_addr learned my addr 172.21.15.26:0/1632131477 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572603920 0 0) 0x7f49d40a0670 con 0x7f49d409bcb0 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): my global_id is 254185 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c0003570 con 0x7f49d409bcb0 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680329240 0 0) 0x7f49d404ca10 con 0x7f49d409d6b0 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): my global_id is 254217 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d40a0670 con 0x7f49d409d6b0 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947526135 0 0) 0x7f49d409d060 con 0x7f49d4137020 2024-09-15T04:02:25.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient(hunting): my global_id is 135746 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d404ca10 con 0x7f49d4137020 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1674696857 0 0) 0x7f49d404ca10 con 0x7f49d4137020 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 >> v1:172.21.15.122:6792/0 conn(0x7f49d409bcb0 legacy=0x7f49d4135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 >> v1:172.21.15.122:6790/0 conn(0x7f49d409d6b0 legacy=0x7f49d409aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 monclient: found mon.noname-e 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40a08b0 con 0x7f49d4137020 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f49cc001a50 con 0x7f49d4137020 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49c7fff700 10 monclient: dump: 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 5 monclient: authenticate success, global_id 135746 2024-09-15T04:02:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.892+0000 7f49db708700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49cc001cc0 con 0x7f49d4137020 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49c7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49c7fff700 1 -- 172.21.15.26:0/1632131477 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f49cc004c70 con 0x7f49d4137020 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49c7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49c7fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49c7fff700 10 monclient: dump: 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 4 set_mon_vals no callback set 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 10 monclient: shutdown 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 1 -- 172.21.15.26:0/1632131477 >> v1:172.21.15.26:6793/0 conn(0x7f49d4137020 legacy=0x7f49d4139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 1 -- 172.21.15.26:0/1632131477 shutdown_connections 2024-09-15T04:02:25.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 1 -- 172.21.15.26:0/1632131477 >> 172.21.15.26:0/1632131477 conn(0x7f49d4063a10 msgr2=0x7f49d40a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.896+0000 7f49db708700 1 -- 172.21.15.26:0/1632131477 shutdown_connections 2024-09-15T04:02:25.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 1 -- 172.21.15.26:0/1632131477 wait complete. 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 10 monclient: build_initial_monmap 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 10 monclient: monmap: 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 1 Processor -- start 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 1 -- start start 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 10 monclient: init 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 10 monclient: _add_conns ranks=[8,1,5,3,6,7,0,4,2] 2024-09-15T04:02:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.900+0000 7f49db708700 10 monclient(hunting): picked mon.noname-e con 0x7f49d409bcb0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 10 monclient(hunting): picked mon.noname-f con 0x7f49d409d6b0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 10 monclient(hunting): picked mon.noname-h con 0x7f49d4137020 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d411d650 con 0x7f49d409d6b0 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d4148400 con 0x7f49d4137020 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d411d370 con 0x7f49d409bcb0 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49db708700 10 monclient(hunting): authenticate will time out at 1890.336130s 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49d94a4700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f49d409bcb0 0x7f49d414e660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50574/0 (socket says 172.21.15.26:50574) 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49d94a4700 1 -- 172.21.15.26:0/4267452116 learned_addr learned my addr 172.21.15.26:0/4267452116 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133958945 0 0) 0x7f49d411d370 con 0x7f49d409bcb0 2024-09-15T04:02:25.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): my global_id is 135755 2024-09-15T04:02:25.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b8003570 con 0x7f49d409bcb0 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3912076430 0 0) 0x7f49b8003570 con 0x7f49d409bcb0 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d411d370 con 0x7f49d409bcb0 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f49c8001b40 con 0x7f49d409bcb0 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): dump: 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.904+0000 7f49c57fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:25.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3662578922 0 0) 0x7f49d411d370 con 0x7f49d409bcb0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.122:6791/0 conn(0x7f49d4137020 legacy=0x7f49d41567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.122:6789/0 conn(0x7f49d409d6b0 legacy=0x7f49d414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: _finish_hunting 0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 monclient: found mon.e 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d411d930 con 0x7f49d409bcb0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: _finish_auth 0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f49c80021b0 con 0x7f49d409bcb0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f49c80026e0 con 0x7f49d409bcb0 2024-09-15T04:02:25.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c57fa700 10 monclient: dump: 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.920 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49c77fe700 4 set_mon_vals no callback set 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 5 monclient: authenticate success, global_id 135755 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 10 monclient: _renew_subs 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d411b950 con 0x7f49d409bcb0 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 10 monclient: _renew_subs 2024-09-15T04:02:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:02:25.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d411c620 con 0x7f49d409bcb0 2024-09-15T04:02:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.908+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49d404bbf0 con 0x7f49d406daa0 2024-09-15T04:02:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.912+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f49c8002950 con 0x7f49d409bcb0 2024-09-15T04:02:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.912+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(820..820 src has 435..820) v4 ==== 3971+0+0 (unknown 3862651140 0 0) 0x7f49c8043f70 con 0x7f49d409bcb0 2024-09-15T04:02:25.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.912+0000 7f49d9ca5700 10 monclient: get_auth_request con 0x7f49b8037520 auth_method 0 2024-09-15T04:02:25.923 INFO:tasks.rados.rados.0.smithi122.stdout:3626: done (2 left) 2024-09-15T04:02:25.923 INFO:tasks.rados.rados.0.smithi122.stdout:3627: read oid 1 snap -1 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3627: expect deleted 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3628: delete oid 45 current snap is 421 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3623: finishing copy_from racing read to smithi12223906-46 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3623: finishing copy_from to smithi12223906-46 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 2155 (ObjNum 1272 snap 420 seq_num 1272) dirty exists 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3624: finishing write tid 1 to smithi12223906-18 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3624: finishing write tid 2 to smithi12223906-18 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3624: finishing write tid 3 to smithi12223906-18 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3624: finishing write tid 4 to smithi12223906-18 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3624: finishing write tid 5 to smithi12223906-18 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3038 (ObjNum 1275 snap 421 seq_num 1275) dirty exists 2024-09-15T04:02:25.924 INFO:tasks.rados.rados.0.smithi122.stdout:3624: left oid 18 (ObjNum 1275 snap 421 seq_num 1275) 2024-09-15T04:02:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3623: done (3 left) 2024-09-15T04:02:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3624: done (2 left) 2024-09-15T04:02:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3627: done (1 left) 2024-09-15T04:02:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3628: done (0 left) 2024-09-15T04:02:25.929 INFO:tasks.rados.rados.0.smithi122.stdout:3629: snap_create 2024-09-15T04:02:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 21471761 0 0) 0x7f49d404bbf0 con 0x7f49d406daa0 2024-09-15T04:02:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient(hunting): my global_id is 204970 2024-09-15T04:02:25.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d411d370 con 0x7f49d406daa0 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2274430826 0 0) 0x7f49d411d370 con 0x7f49d406daa0 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d404bbf0 con 0x7f49d406daa0 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f49d0001990 con 0x7f49d406daa0 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient: dump: 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.933 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 858424525 0 0) 0x7f49d404bbf0 con 0x7f49d406daa0 2024-09-15T04:02:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49d406fac0 con 0x7f49d406daa0 2024-09-15T04:02:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f49d0001f70 con 0x7f49d406daa0 2024-09-15T04:02:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.26:6791/0 conn(0x7f49d406daa0 legacy=0x7f49d4068480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.924+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.26:6791/0 conn(0x7f49d406daa0 legacy=0x7f49d4068480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:25.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49db708700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:25.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f49d411db20 con 0x7f49d402ad10 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608384670 0 0) 0x7f49d411db20 con 0x7f49d402ad10 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient(hunting): my global_id is 204979 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d404bbf0 con 0x7f49d402ad10 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2759433412 0 0) 0x7f49d404bbf0 con 0x7f49d402ad10 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f49d411db20 con 0x7f49d402ad10 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f49d00027d0 con 0x7f49d402ad10 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:25.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient: dump: 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:25.944 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3589414541 0 0) 0x7f49d411db20 con 0x7f49d402ad10 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f49d40716c0 con 0x7f49d402ad10 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 881767036) 0x7f49d0002c40 con 0x7f49d402ad10 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.26:6791/0 conn(0x7f49d402ad10 legacy=0x7f49d4068b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.932+0000 7f49c57fa700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.26:6791/0 conn(0x7f49d402ad10 legacy=0x7f49d4068b90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:25.946 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.947 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.948 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:25.949 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:25.950 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.952 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.955 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:25.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.957 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.958 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.960 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:25.961 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:25.963 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:25.964 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:25.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:25.966 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:25.966 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.122:6812/20895 conn(0x7f49b8037520 legacy=0x7f49b80399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 10 monclient: shutdown 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 >> v1:172.21.15.26:6793/0 conn(0x7f49d409bcb0 legacy=0x7f49d414e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49d9ca5700 1 -- 172.21.15.26:0/4267452116 reap_dead start 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49d9ca5700 1 -- 172.21.15.26:0/4267452116 reap_dead start 2024-09-15T04:02:25.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 shutdown_connections 2024-09-15T04:02:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 >> 172.21.15.26:0/4267452116 conn(0x7f49d4063a10 msgr2=0x7f49d41367c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 shutdown_connections 2024-09-15T04:02:25.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:25.936+0000 7f49db708700 1 -- 172.21.15.26:0/4267452116 wait complete. 2024-09-15T04:02:25.968 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:25.968 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T04:02:25.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:02:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient: get_monmap_and_config 2024-09-15T04:02:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient: monmap: 2024-09-15T04:02:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:26.077700+0000 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:26.077700+0000 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.082 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 1 Processor -- start 2024-09-15T04:02:26.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 1 -- start start 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient: init 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient: _add_conns ranks=[1,0,2,6,5,3,8,4,7] 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient(hunting): picked mon.noname-f con 0x7ff8f80982f0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient(hunting): picked mon.noname-a con 0x7ff8f809b460 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient(hunting): picked mon.noname-b con 0x7ff8f809b830 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f804ca10 con 0x7ff8f809b460 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f8099590 con 0x7ff8f80982f0 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80986c0 con 0x7ff8f809b830 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8fd269700 10 monclient(hunting): authenticate will time out at 1890.507165s 2024-09-15T04:02:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f67fc700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8f809b460 0x7ff8f809d100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40436/0 (socket says 172.21.15.26:40436) 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f67fc700 1 -- 172.21.15.26:0/2845229414 learned_addr learned my addr 172.21.15.26:0/2845229414 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f77fe700 1 --1- 172.21.15.26:0/2845229414 >> v1:172.21.15.26:6790/0 conn(0x7ff8f809b830 0x7ff8f8137820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:49760/0 (socket says 172.21.15.26:49760) 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658217291 0 0) 0x7ff8f804ca10 con 0x7ff8f809b460 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 10 monclient(hunting): my global_id is 195093 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e0004680 con 0x7ff8f809b460 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097402657 0 0) 0x7ff8f80986c0 con 0x7ff8f809b830 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 10 monclient(hunting): my global_id is 254153 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.072+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f804ca10 con 0x7ff8f809b830 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237527258 0 0) 0x7ff8f8099590 con 0x7ff8f80982f0 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient(hunting): my global_id is 204940 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f80986c0 con 0x7ff8f80982f0 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3197670640 0 0) 0x7ff8e0004680 con 0x7ff8f809b460 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 >> v1:172.21.15.26:6790/0 conn(0x7ff8f809b830 legacy=0x7ff8f8137820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 >> v1:172.21.15.122:6789/0 conn(0x7ff8f80982f0 legacy=0x7ff8f809c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 monclient: found mon.noname-a 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:02:26.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f8099870 con 0x7ff8f809b460 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 5 monclient: authenticate success, global_id 195093 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8e80017d0 con 0x7ff8f809b460 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: dump: 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.090 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8e8001ab0 con 0x7ff8f809b460 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 1 -- 172.21.15.26:0/2845229414 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8e8004c30 con 0x7ff8f809b460 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 4 set_mon_vals no callback set 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8f5ffb700 10 monclient: dump: 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.092 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 10 monclient: shutdown 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 -- 172.21.15.26:0/2845229414 >> v1:172.21.15.26:6789/0 conn(0x7ff8f809b460 legacy=0x7ff8f809d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 -- 172.21.15.26:0/2845229414 shutdown_connections 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 -- 172.21.15.26:0/2845229414 >> 172.21.15.26:0/2845229414 conn(0x7ff8f8063f30 msgr2=0x7ff8f80a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 -- 172.21.15.26:0/2845229414 shutdown_connections 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 -- 172.21.15.26:0/2845229414 wait complete. 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 10 monclient: monmap: 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:26.094 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 Processor -- start 2024-09-15T04:02:26.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 1 -- start start 2024-09-15T04:02:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.076+0000 7ff8fd269700 10 monclient: init 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient: _add_conns ranks=[4,6,7,3,0,2,5,1,8] 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient(hunting): picked mon.noname-c con 0x7ff8f80982f0 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient(hunting): picked mon.noname-d con 0x7ff8f809b460 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient(hunting): picked mon.noname-i con 0x7ff8f809b830 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f811f310 con 0x7ff8f80982f0 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f813a100 con 0x7ff8f809b460 2024-09-15T04:02:26.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f811ed50 con 0x7ff8f809b830 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8f67fc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ff8f809b460 0x7ff8f8142c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55692/0 (socket says 172.21.15.26:55692) 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8f67fc700 1 -- 172.21.15.26:0/1659139513 learned_addr learned my addr 172.21.15.26:0/1659139513 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8f6ffd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7ff8f80982f0 0x7ff8f81399d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41102/0 (socket says 172.21.15.26:41102) 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient(hunting): authenticate will time out at 1890.512964s 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765557560 0 0) 0x7ff8f813a100 con 0x7ff8f809b460 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): my global_id is 195153 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e4003570 con 0x7ff8f809b460 2024-09-15T04:02:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703213990 0 0) 0x7ff8f811f310 con 0x7ff8f80982f0 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): my global_id is 204988 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f813a100 con 0x7ff8f80982f0 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068595225 0 0) 0x7ff8f811ed50 con 0x7ff8f809b830 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): my global_id is 254194 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f811f310 con 0x7ff8f809b830 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3230991647 0 0) 0x7ff8e4003570 con 0x7ff8f809b460 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8f811ed50 con 0x7ff8f809b460 2024-09-15T04:02:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8e8004b70 con 0x7ff8f809b460 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): dump: 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3025864293 0 0) 0x7ff8f813a100 con 0x7ff8f80982f0 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8e4003570 con 0x7ff8f80982f0 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8ec001a60 con 0x7ff8f80982f0 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): dump: 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2281250260 0 0) 0x7ff8f811f310 con 0x7ff8f809b830 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8f813a100 con 0x7ff8f809b830 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8f0001d00 con 0x7ff8f809b830 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient(hunting): dump: 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3212994125 0 0) 0x7ff8f811ed50 con 0x7ff8f809b460 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.122:6792/0 conn(0x7ff8f809b830 legacy=0x7ff8f8146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.26:6791/0 conn(0x7ff8f80982f0 legacy=0x7ff8f81399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 monclient: found mon.d 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f811c8a0 con 0x7ff8f809b460 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 5 monclient: authenticate success, global_id 195153 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient: _renew_subs 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f811ea70 con 0x7ff8f809b460 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient: _renew_subs 2024-09-15T04:02:26.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f811dba0 con 0x7ff8f809b460 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff8e8004f00 con 0x7ff8f809b460 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8e80053b0 con 0x7ff8f809b460 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8f57fa700 4 set_mon_vals no callback set 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8deffd700 10 monclient: dump: 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.110 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.080+0000 7ff8fd269700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff8d8006590 con 0x7ff8d8002840 2024-09-15T04:02:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff8e800fd70 con 0x7ff8f809b460 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(820..820 src has 435..820) v4 ==== 3971+0+0 (unknown 3862651140 0 0) 0x7ff8e8044180 con 0x7ff8f809b460 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822991449 0 0) 0x7ff8d8006590 con 0x7ff8d8002840 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient(hunting): my global_id is 195162 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f811ed50 con 0x7ff8d8002840 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1613625181 0 0) 0x7ff8f811ed50 con 0x7ff8d8002840 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8d8006590 con 0x7ff8d8002840 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8f77fe700 10 monclient: get_auth_request con 0x7ff8e40380d0 auth_method 0 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8ec002230 con 0x7ff8d8002840 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient: dump: 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3205907813 0 0) 0x7ff8d8006590 con 0x7ff8d8002840 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8d800a130 con 0x7ff8d8002840 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ff8ec005810 con 0x7ff8d8002840 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.26:6792/0 conn(0x7ff8d8002840 legacy=0x7ff8d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.084+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.26:6792/0 conn(0x7ff8d8002840 legacy=0x7ff8d8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ff8d8007520 con 0x7ff8d80068d0 2024-09-15T04:02:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514065500 0 0) 0x7ff8d8007520 con 0x7ff8d80068d0 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient(hunting): my global_id is 195171 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d8006590 con 0x7ff8d80068d0 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 254873042 0 0) 0x7ff8d8006590 con 0x7ff8d80068d0 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8d8007520 con 0x7ff8d80068d0 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff8ec002c60 con 0x7ff8d80068d0 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient: dump: 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4085043601 0 0) 0x7ff8d8007520 con 0x7ff8d80068d0 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff8d8008760 con 0x7ff8d80068d0 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 1146454317) 0x7ff8ec007b20 con 0x7ff8d80068d0 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.26:6792/0 conn(0x7ff8d80068d0 legacy=0x7ff8d8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8deffd700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.26:6792/0 conn(0x7ff8d80068d0 legacy=0x7ff8d8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.118 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:26.119 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.120 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.121 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:26.122 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.123 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:26.124 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.126 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.127 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.128 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.130 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:26.131 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.133 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.134 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.136 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.137 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:26.138 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.122:6812/20895 conn(0x7ff8e40380d0 legacy=0x7ff8e403a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 10 monclient: shutdown 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 >> v1:172.21.15.26:6792/0 conn(0x7ff8f809b460 legacy=0x7ff8f8142c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8f77fe700 1 -- 172.21.15.26:0/1659139513 reap_dead start 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8f77fe700 1 -- 172.21.15.26:0/1659139513 reap_dead start 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 shutdown_connections 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 >> 172.21.15.26:0/1659139513 conn(0x7ff8f8063f30 msgr2=0x7ff8f809a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 shutdown_connections 2024-09-15T04:02:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.096+0000 7ff8fd269700 1 -- 172.21.15.26:0/1659139513 wait complete. 2024-09-15T04:02:26.141 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:26.141 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T04:02:26.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T04:02:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient: get_monmap_and_config 2024-09-15T04:02:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient: monmap: 2024-09-15T04:02:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:26.254116+0000 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:26.254116+0000 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.259 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 1 Processor -- start 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 1 -- start start 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient: init 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient: _add_conns ranks=[4,3,7,0,5,2,1,6,8] 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient(hunting): picked mon.noname-c con 0x7f11b8070fb0 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient(hunting): picked mon.noname-g con 0x7f11b809de00 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient(hunting): picked mon.noname-i con 0x7f11b809e8e0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b804ca10 con 0x7f11b809de00 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b809b120 con 0x7f11b8070fb0 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b809b400 con 0x7f11b809e8e0 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.248+0000 7f11bc93a700 10 monclient(hunting): authenticate will time out at 1890.683882s 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b6d9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f11b8070fb0 0x7f11b809c1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:41104/0 (socket says 172.21.15.26:41104) 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b6d9d700 1 -- 172.21.15.26:0/1270038711 learned_addr learned my addr 172.21.15.26:0/1270038711 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808926867 0 0) 0x7f11b809b120 con 0x7f11b8070fb0 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): my global_id is 204997 2024-09-15T04:02:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a4003570 con 0x7f11b8070fb0 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520912872 0 0) 0x7f11b809b400 con 0x7f11b809e8e0 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): my global_id is 254203 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b809b120 con 0x7f11b809e8e0 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386679991 0 0) 0x7f11b804ca10 con 0x7f11b809de00 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient(hunting): my global_id is 254226 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b809b400 con 0x7f11b809de00 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 894657794 0 0) 0x7f11a4003570 con 0x7f11b8070fb0 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 >> v1:172.21.15.122:6792/0 conn(0x7f11b809e8e0 legacy=0x7f11b8137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 >> v1:172.21.15.122:6790/0 conn(0x7f11b809de00 legacy=0x7f11b809e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 monclient: found mon.noname-c 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b813c490 con 0x7f11b8070fb0 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a80017b0 con 0x7f11b8070fb0 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 5 monclient: authenticate success, global_id 204997 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: dump: 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f11a8001a20 con 0x7f11b8070fb0 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 1 -- 172.21.15.26:0/1270038711 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a8004c30 con 0x7f11b8070fb0 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 4 set_mon_vals no callback set 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11b659c700 10 monclient: dump: 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.268 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 10 monclient: shutdown 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 1 -- 172.21.15.26:0/1270038711 >> v1:172.21.15.26:6791/0 conn(0x7f11b8070fb0 legacy=0x7f11b809c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 1 -- 172.21.15.26:0/1270038711 shutdown_connections 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 1 -- 172.21.15.26:0/1270038711 >> 172.21.15.26:0/1270038711 conn(0x7f11b80a2ea0 msgr2=0x7f11b809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 1 -- 172.21.15.26:0/1270038711 shutdown_connections 2024-09-15T04:02:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.252+0000 7f11bc93a700 1 -- 172.21.15.26:0/1270038711 wait complete. 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient: monmap: 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 1 Processor -- start 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 1 -- start start 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient: init 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient: _add_conns ranks=[7,1,5,8,6,0,4,3,2] 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient(hunting): picked mon.noname-i con 0x7f11b809de00 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient(hunting): picked mon.noname-f con 0x7f11b809e8e0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient(hunting): picked mon.noname-h con 0x7f11b806cda0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b806e230 con 0x7f11b809e8e0 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b806d1e0 con 0x7f11b806cda0 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b812ce30 con 0x7f11b809de00 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11bc93a700 10 monclient(hunting): authenticate will time out at 1890.690430s 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11b6d9d700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f11b809de00 0x7f11b812c720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:55914/0 (socket says 172.21.15.26:55914) 2024-09-15T04:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11b759e700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f11b806cda0 0x7f11b8145fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:59686/0 (socket says 172.21.15.26:59686) 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11b6d9d700 1 -- 172.21.15.26:0/1914884884 learned_addr learned my addr 172.21.15.26:0/1914884884 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11ae59c700 1 --1- 172.21.15.26:0/1914884884 >> v1:172.21.15.122:6789/0 conn(0x7f11b809e8e0 0x7f11b8142870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:58448/0 (socket says 172.21.15.26:58448) 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240477834 0 0) 0x7f11b809b120 con 0x7f11b806cda0 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): my global_id is 254228 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1198003570 con 0x7f11b806cda0 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374995702 0 0) 0x7f11b806e230 con 0x7f11b809e8e0 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): my global_id is 204949 2024-09-15T04:02:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b809b120 con 0x7f11b809e8e0 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293121813 0 0) 0x7f11b812ce30 con 0x7f11b809de00 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 10 monclient(hunting): my global_id is 254212 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b806e230 con 0x7f11b809de00 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.256+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3101585723 0 0) 0x7f11b806e230 con 0x7f11b809de00 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11b812ce30 con 0x7f11b809de00 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a80017b0 con 0x7f11b809de00 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): dump: 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.280 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.281 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3410111873 0 0) 0x7f1198003570 con 0x7f11b806cda0 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11b806e230 con 0x7f11b806cda0 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2525915757 0 0) 0x7f11b809b120 con 0x7f11b809e8e0 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1198003570 con 0x7f11b809e8e0 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11b0001c80 con 0x7f11b806cda0 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:26.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): dump: 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.283 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a00019e0 con 0x7f11b809e8e0 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): dump: 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.284 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2952781484 0 0) 0x7f11b812ce30 con 0x7f11b809de00 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.122:6791/0 conn(0x7f11b806cda0 legacy=0x7f11b8145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.122:6789/0 conn(0x7f11b809e8e0 legacy=0x7f11b8142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 monclient: found mon.i 2024-09-15T04:02:26.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b809f910 con 0x7f11b809de00 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 5 monclient: authenticate success, global_id 254212 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 10 monclient: _renew_subs 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b8127920 con 0x7f11b809de00 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 10 monclient: _renew_subs 2024-09-15T04:02:26.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b813cb30 con 0x7f11b809de00 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f11a8004c30 con 0x7f11b809de00 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a8001210 con 0x7f11b809de00 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: dump: 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.288 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11b5d9b700 4 set_mon_vals no callback set 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f119c006590 con 0x7f119c002840 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644252572 0 0) 0x7f119c006590 con 0x7f119c002840 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient(hunting): my global_id is 135764 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b812ce30 con 0x7f119c002840 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f11a800faa0 con 0x7f11b809de00 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(820..820 src has 435..820) v4 ==== 3971+0+0 (unknown 3862651140 0 0) 0x7f11a8044110 con 0x7f11b809de00 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 510242177 0 0) 0x7f11b812ce30 con 0x7f119c002840 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f119c006590 con 0x7f119c002840 2024-09-15T04:02:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a00020f0 con 0x7f119c002840 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 10 monclient: dump: 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.290 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11b759e700 10 monclient: get_auth_request con 0x7f1198038040 auth_method 0 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3959046218 0 0) 0x7f119c006590 con 0x7f119c002840 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.260+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f119c00a130 con 0x7f119c002840 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.264+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f11a0005810 con 0x7f119c002840 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.264+0000 7f11af7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.264+0000 7f11af7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.264+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.26:6793/0 conn(0x7f119c002840 legacy=0x7f119c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.264+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.26:6793/0 conn(0x7f119c002840 legacy=0x7f119c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.268+0000 7f11bc93a700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.268+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f119c007520 con 0x7f119c0068d0 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361940865 0 0) 0x7f119c007520 con 0x7f119c0068d0 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient(hunting): my global_id is 135773 2024-09-15T04:02:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f119c006590 con 0x7f119c0068d0 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1385797364 0 0) 0x7f119c006590 con 0x7f119c0068d0 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f119c007520 con 0x7f119c0068d0 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11a0002b60 con 0x7f119c0068d0 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient: dump: 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.295 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3064237420 0 0) 0x7f119c007520 con 0x7f119c0068d0 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f119c008760 con 0x7f119c0068d0 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7571 (unknown 0 0 3222208460) 0x7f11a0007da0 con 0x7f119c0068d0 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.26:6793/0 conn(0x7f119c0068d0 legacy=0x7f119c006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11af7fe700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.26:6793/0 conn(0x7f119c0068d0 legacy=0x7f119c006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.296 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:26.297 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.298 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.299 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:26.300 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.301 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.302 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.303 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:26.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.303 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:26.303 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.303 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:26.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:26.304 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.304 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.305 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.306 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.308 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.309 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.313 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.315 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.317 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:26.318 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.122:6812/20895 conn(0x7f1198038040 legacy=0x7f119803a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 10 monclient: shutdown 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 >> v1:172.21.15.122:6792/0 conn(0x7f11b809de00 legacy=0x7f11b812c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11b759e700 1 -- 172.21.15.26:0/1914884884 reap_dead start 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 shutdown_connections 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 >> 172.21.15.26:0/1914884884 conn(0x7f11b80a2ea0 msgr2=0x7f11b812fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 shutdown_connections 2024-09-15T04:02:26.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.272+0000 7f11bc93a700 1 -- 172.21.15.26:0/1914884884 wait complete. 2024-09-15T04:02:26.321 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:26.321 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T04:02:26.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient: get_monmap_and_config 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient: monmap: 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:26.433849+0000 2024-09-15T04:02:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:26.433849+0000 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.438 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 1 Processor -- start 2024-09-15T04:02:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 1 -- start start 2024-09-15T04:02:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient: init 2024-09-15T04:02:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient: _add_conns ranks=[6,3,8,0,7,1,4,2,5] 2024-09-15T04:02:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.428+0000 7f57349d2700 10 monclient(hunting): picked mon.noname-d con 0x7f5730132b40 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient(hunting): picked mon.noname-g con 0x7f573009faa0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient(hunting): picked mon.noname-e con 0x7f5730138d40 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573004ca10 con 0x7f573009faa0 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573009d5d0 con 0x7f5730132b40 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5730133620 con 0x7f5730138d40 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient(hunting): authenticate will time out at 1890.864176s 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572f7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5730132b40 0x7f5730132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55712/0 (socket says 172.21.15.26:55712) 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572ffff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5730138d40 0x7f573013b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:50590/0 (socket says 172.21.15.26:50590) 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572f7fe700 1 -- 172.21.15.26:0/2887252189 learned_addr learned my addr 172.21.15.26:0/2887252189 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277475010 0 0) 0x7f5730133620 con 0x7f5730138d40 2024-09-15T04:02:26.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): my global_id is 135782 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c003570 con 0x7f5730138d40 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665035575 0 0) 0x7f573009d5d0 con 0x7f5730132b40 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): my global_id is 195180 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5730133620 con 0x7f5730132b40 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043179294 0 0) 0x7f573004ca10 con 0x7f573009faa0 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient(hunting): my global_id is 254235 2024-09-15T04:02:26.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f573009d5d0 con 0x7f573009faa0 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043509311 0 0) 0x7f571c003570 con 0x7f5730138d40 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 >> v1:172.21.15.26:6792/0 conn(0x7f5730132b40 legacy=0x7f5730132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 >> v1:172.21.15.122:6790/0 conn(0x7f573009faa0 legacy=0x7f573009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 monclient: found mon.noname-e 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57300a0cc0 con 0x7f5730138d40 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 5 monclient: authenticate success, global_id 135782 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5728001b30 con 0x7f5730138d40 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: dump: 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5728001da0 con 0x7f5730138d40 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 1 -- 172.21.15.26:0/2887252189 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5728004ef0 con 0x7f5730138d40 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 4 set_mon_vals no callback set 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f572effd700 10 monclient: dump: 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.448 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.448 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient: shutdown 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- 172.21.15.26:0/2887252189 >> v1:172.21.15.26:6793/0 conn(0x7f5730138d40 legacy=0x7f573013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- 172.21.15.26:0/2887252189 shutdown_connections 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- 172.21.15.26:0/2887252189 >> 172.21.15.26:0/2887252189 conn(0x7f573006e370 msgr2=0x7f57300a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- 172.21.15.26:0/2887252189 shutdown_connections 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 1 -- 172.21.15.26:0/2887252189 wait complete. 2024-09-15T04:02:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.432+0000 7f57349d2700 10 monclient: monmap: 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.454 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 Processor -- start 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 -- start start 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: init 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: _add_conns ranks=[6,3,4,8,0,5,2,7,1] 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient(hunting): picked mon.noname-d con 0x7f573009faa0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient(hunting): picked mon.noname-g con 0x7f5730132b40 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient(hunting): picked mon.noname-c con 0x7f5730138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573009cfa0 con 0x7f5730132b40 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573011c990 con 0x7f5730138d40 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5730142fc0 con 0x7f573009faa0 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient(hunting): authenticate will time out at 1890.869458s 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f572f7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f573009faa0 0x7f57301421a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55716/0 (socket says 172.21.15.26:55716) 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f572f7fe700 1 -- 172.21.15.26:0/463859544 learned_addr learned my addr 172.21.15.26:0/463859544 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352202717 0 0) 0x7f5730142fc0 con 0x7f573009faa0 2024-09-15T04:02:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): my global_id is 195189 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5710003570 con 0x7f573009faa0 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494647673 0 0) 0x7f573011c990 con 0x7f5730138d40 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): my global_id is 205006 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5730142fc0 con 0x7f5730138d40 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563695406 0 0) 0x7f573009cfa0 con 0x7f5730132b40 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): my global_id is 254244 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f573011c990 con 0x7f5730132b40 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1790473710 0 0) 0x7f5730142fc0 con 0x7f5730138d40 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f573009cfa0 con 0x7f5730138d40 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5728004f30 con 0x7f5730138d40 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): dump: 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4127255265 0 0) 0x7f5710003570 con 0x7f573009faa0 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5730142fc0 con 0x7f573009faa0 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5720001aa0 con 0x7f573009faa0 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient(hunting): dump: 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3486822246 0 0) 0x7f573009cfa0 con 0x7f5730138d40 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.26:6792/0 conn(0x7f573009faa0 legacy=0x7f57301421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.122:6790/0 conn(0x7f5730132b40 legacy=0x7f57301428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 monclient: found mon.c 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573009b670 con 0x7f5730138d40 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 5 monclient: authenticate success, global_id 205006 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: _renew_subs 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573011c640 con 0x7f5730138d40 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: _renew_subs 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f573009cc20 con 0x7f5730138d40 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5728001bd0 con 0x7f5730138d40 2024-09-15T04:02:26.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f57280015b0 con 0x7f5730138d40 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f572e7fc700 4 set_mon_vals no callback set 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.436+0000 7f5727fff700 10 monclient: dump: 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.463 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f572800fd40 con 0x7f5730138d40 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f57349d2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5714006590 con 0x7f5714002840 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(820..820 src has 435..820) v4 ==== 3971+0+0 (unknown 3862651140 0 0) 0x7f5728044340 con 0x7f5730138d40 2024-09-15T04:02:26.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5726ffd700 10 monclient: get_auth_request con 0x7f5710037ca0 auth_method 0 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796659259 0 0) 0x7f5714006590 con 0x7f5714002840 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient(hunting): my global_id is 204958 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f573009cfa0 con 0x7f5714002840 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4204936360 0 0) 0x7f573009cfa0 con 0x7f5714002840 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5714006590 con 0x7f5714002840 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5720004bb0 con 0x7f5714002840 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient: dump: 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.465 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.466 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1328240868 0 0) 0x7f5714006590 con 0x7f5714002840 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f571400a130 con 0x7f5714002840 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f5720005920 con 0x7f5714002840 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.122:6789/0 conn(0x7f5714002840 legacy=0x7f5714004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.440+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.122:6789/0 conn(0x7f5714002840 legacy=0x7f5714004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f57349d2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f5714007520 con 0x7f57140068d0 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827177054 0 0) 0x7f5714007520 con 0x7f57140068d0 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 10 monclient(hunting): my global_id is 204967 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5714006590 con 0x7f57140068d0 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3523923498 0 0) 0x7f5714006590 con 0x7f57140068d0 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5714007520 con 0x7f57140068d0 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5720005660 con 0x7f57140068d0 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 10 monclient: dump: 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.470 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3109449291 0 0) 0x7f5714007520 con 0x7f57140068d0 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.452+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5714008760 con 0x7f57140068d0 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7739 (unknown 0 0 3941277634) 0x7f5720007e50 con 0x7f57140068d0 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f5727fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f5727fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.122:6789/0 conn(0x7f57140068d0 legacy=0x7f5714006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f5727fff700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.122:6789/0 conn(0x7f57140068d0 legacy=0x7f5714006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.471 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:26.472 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.473 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.474 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:26.475 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.476 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.477 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.479 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.480 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.482 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.483 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.484 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.485 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:26.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.489 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:26.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.491 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:26.492 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.122:6812/20895 conn(0x7f5710037ca0 legacy=0x7f571003a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 10 monclient: shutdown 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 >> v1:172.21.15.26:6791/0 conn(0x7f5730138d40 legacy=0x7f573014a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f572ffff700 1 -- 172.21.15.26:0/463859544 reap_dead start 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f572ffff700 1 -- 172.21.15.26:0/463859544 reap_dead start 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 shutdown_connections 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 >> 172.21.15.26:0/463859544 conn(0x7f573006e370 msgr2=0x7f57300a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 shutdown_connections 2024-09-15T04:02:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.456+0000 7f57349d2700 1 -- 172.21.15.26:0/463859544 wait complete. 2024-09-15T04:02:26.494 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:26.494 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T04:02:26.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T04:02:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.600+0000 7fbb7c533700 10 monclient: get_monmap_and_config 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.600+0000 7fbb7c533700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.600+0000 7fbb7c533700 10 monclient: monmap: 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:26.607379+0000 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:26.607379+0000 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.612 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.612 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.612 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.612 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.612 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.612 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 Processor -- start 2024-09-15T04:02:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- start start 2024-09-15T04:02:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: init 2024-09-15T04:02:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: _add_conns ranks=[0,2,8,6,1,3,4,7,5] 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient(hunting): picked mon.noname-a con 0x7fbb740990c0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient(hunting): picked mon.noname-b con 0x7fbb740a1310 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient(hunting): picked mon.noname-e con 0x7fbb741335a0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7404ca10 con 0x7fbb740990c0 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740a1b20 con 0x7fbb740a1310 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740a10b0 con 0x7fbb741335a0 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient(hunting): authenticate will time out at 1891.036885s 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7a2cf700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb740990c0 0x7fbb74132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47110/0 (socket says 172.21.15.26:47110) 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7a2cf700 1 -- 172.21.15.26:0/2451543092 learned_addr learned my addr 172.21.15.26:0/2451543092 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7aad0700 1 --1- 172.21.15.26:0/2451543092 >> v1:172.21.15.26:6793/0 conn(0x7fbb741335a0 0x7fbb7413afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37766/0 (socket says 172.21.15.26:37766) 2024-09-15T04:02:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537215173 0 0) 0x7fbb7404ca10 con 0x7fbb740990c0 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): my global_id is 195102 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb60003570 con 0x7fbb740990c0 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153607249 0 0) 0x7fbb740a1b20 con 0x7fbb740a1310 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): my global_id is 254162 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7404ca10 con 0x7fbb740a1310 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279091376 0 0) 0x7fbb740a10b0 con 0x7fbb741335a0 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient(hunting): my global_id is 135791 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb740a1b20 con 0x7fbb741335a0 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1016819146 0 0) 0x7fbb60003570 con 0x7fbb740990c0 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 >> v1:172.21.15.26:6793/0 conn(0x7fbb741335a0 legacy=0x7fbb7413afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 >> v1:172.21.15.26:6790/0 conn(0x7fbb740a1310 legacy=0x7fbb74099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 monclient: found mon.noname-a 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74099ba0 con 0x7fbb740990c0 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 5 monclient: authenticate success, global_id 195102 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb700017b0 con 0x7fbb740990c0 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: dump: 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb70001a90 con 0x7fbb740990c0 2024-09-15T04:02:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 1 -- 172.21.15.26:0/2451543092 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb70001c80 con 0x7fbb740990c0 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 4 set_mon_vals no callback set 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb792cd700 10 monclient: dump: 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: shutdown 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- 172.21.15.26:0/2451543092 >> v1:172.21.15.26:6789/0 conn(0x7fbb740990c0 legacy=0x7fbb74132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- 172.21.15.26:0/2451543092 shutdown_connections 2024-09-15T04:02:26.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- 172.21.15.26:0/2451543092 >> 172.21.15.26:0/2451543092 conn(0x7fbb7406e370 msgr2=0x7fbb740a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- 172.21.15.26:0/2451543092 shutdown_connections 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 1 -- 172.21.15.26:0/2451543092 wait complete. 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.604+0000 7fbb7c533700 10 monclient: monmap: 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.623 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 1 Processor -- start 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 1 -- start start 2024-09-15T04:02:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient: init 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient: _add_conns ranks=[7,1,3,6,8,2,5,4,0] 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient(hunting): picked mon.noname-i con 0x7fbb740990c0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient(hunting): picked mon.noname-f con 0x7fbb740a1310 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient(hunting): picked mon.noname-g con 0x7fbb741335a0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740a1ce0 con 0x7fbb740a1310 2024-09-15T04:02:26.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7409a010 con 0x7fbb741335a0 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb741201a0 con 0x7fbb740990c0 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7c533700 10 monclient(hunting): authenticate will time out at 1891.041967s 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb79ace700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fbb740a1310 0x7fbb74147030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:45820/0 (socket says 172.21.15.26:45820) 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7aad0700 1 --1- >> v1:172.21.15.122:6790/0 conn(0x7fbb741335a0 0x7fbb7414a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6790/0 says I am v1:172.21.15.26:44762/0 (socket says 172.21.15.26:44762) 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb79ace700 1 -- 172.21.15.26:0/2850126284 learned_addr learned my addr 172.21.15.26:0/2850126284 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb7a2cf700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fbb740990c0 0x7fbb74131b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:38518/0 (socket says 172.21.15.26:38518) 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352912123 0 0) 0x7fbb7409a010 con 0x7fbb741335a0 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): my global_id is 254253 2024-09-15T04:02:26.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb58003570 con 0x7fbb741335a0 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136356945 0 0) 0x7fbb740a1ce0 con 0x7fbb740a1310 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): my global_id is 204976 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7409a010 con 0x7fbb740a1310 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469449088 0 0) 0x7fbb741201a0 con 0x7fbb740990c0 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): my global_id is 254221 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb740a1ce0 con 0x7fbb740990c0 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2034544611 0 0) 0x7fbb7409a010 con 0x7fbb740a1310 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb741201a0 con 0x7fbb740a1310 2024-09-15T04:02:26.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1723620568 0 0) 0x7fbb58003570 con 0x7fbb741335a0 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb7409a010 con 0x7fbb741335a0 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb64001b20 con 0x7fbb740a1310 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.633 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:26.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb6c001cc0 con 0x7fbb741335a0 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.635 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2784508136 0 0) 0x7fbb740a1ce0 con 0x7fbb740990c0 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb58003570 con 0x7fbb740990c0 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb70001770 con 0x7fbb740990c0 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:26.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4169640053 0 0) 0x7fbb741201a0 con 0x7fbb740a1310 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6792/0 conn(0x7fbb740990c0 legacy=0x7fbb74131b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6790/0 conn(0x7fbb741335a0 legacy=0x7fbb7414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 1 monclient: found mon.f 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.608+0000 7fbb6a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb741208f0 con 0x7fbb740a1310 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 5 monclient: authenticate success, global_id 204976 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 10 monclient: _renew_subs 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb74121cd0 con 0x7fbb740a1310 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 10 monclient: _renew_subs 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb741341e0 con 0x7fbb740a1310 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb64002080 con 0x7fbb740a1310 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb64002530 con 0x7fbb740a1310 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb78acc700 4 set_mon_vals no callback set 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: dump: 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb5c006590 con 0x7fbb5c002840 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbb64002710 con 0x7fbb740a1310 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(820..820 src has 435..820) v4 ==== 3971+0+0 (unknown 3862651140 0 0) 0x7fbb64043ee0 con 0x7fbb740a1310 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654381774 0 0) 0x7fbb5c006590 con 0x7fbb5c002840 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient(hunting): my global_id is 254262 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb741201a0 con 0x7fbb5c002840 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb7aad0700 10 monclient: get_auth_request con 0x7fbb58037ff0 auth_method 0 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 969334166 0 0) 0x7fbb741201a0 con 0x7fbb5c002840 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb5c006590 con 0x7fbb5c002840 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb700010a0 con 0x7fbb5c002840 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: dump: 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.642 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1560595227 0 0) 0x7fbb5c006590 con 0x7fbb5c002840 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb5c00a130 con 0x7fbb5c002840 2024-09-15T04:02:26.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbb700051d0 con 0x7fbb5c002840 2024-09-15T04:02:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6790/0 conn(0x7fbb5c002840 legacy=0x7fbb5c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.612+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6790/0 conn(0x7fbb5c002840 legacy=0x7fbb5c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb7c533700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb5c009640 con 0x7fbb5c0090b0 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297819304 0 0) 0x7fbb5c009640 con 0x7fbb5c0090b0 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient(hunting): my global_id is 254271 2024-09-15T04:02:26.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c006590 con 0x7fbb5c0090b0 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2484110259 0 0) 0x7fbb5c006590 con 0x7fbb5c0090b0 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb5c009640 con 0x7fbb5c0090b0 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb70005aa0 con 0x7fbb5c0090b0 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient: dump: 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.646 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4190427425 0 0) 0x7fbb5c009640 con 0x7fbb5c0090b0 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb5c0098f0 con 0x7fbb5c0090b0 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 1282047096) 0x7fbb700081c0 con 0x7fbb5c0090b0 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:26.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6790/0 conn(0x7fbb5c0090b0 legacy=0x7fbb5c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.628+0000 7fbb6a7fc700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6790/0 conn(0x7fbb5c0090b0 legacy=0x7fbb5c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 7, 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:26.649 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.650 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:26.651 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:26.652 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.653 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.654 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.655 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:26.656 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.657 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.658 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:26.659 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.660 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.661 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.662 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.663 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.664 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:26.665 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.666 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.667 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:26.668 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6812/20895 conn(0x7fbb58037ff0 legacy=0x7fbb5803a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 10 monclient: shutdown 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 >> v1:172.21.15.122:6789/0 conn(0x7fbb740a1310 legacy=0x7fbb74147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7aad0700 1 -- 172.21.15.26:0/2850126284 reap_dead start 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7aad0700 1 -- 172.21.15.26:0/2850126284 reap_dead start 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 shutdown_connections 2024-09-15T04:02:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 >> 172.21.15.26:0/2850126284 conn(0x7fbb7406e370 msgr2=0x7fbb7409f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 shutdown_connections 2024-09-15T04:02:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.632+0000 7fbb7c533700 1 -- 172.21.15.26:0/2850126284 wait complete. 2024-09-15T04:02:26.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:26.670 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T04:02:26.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: get_monmap_and_config 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: monmap: 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:26.780762+0000 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:26.780762+0000 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.785 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 Processor -- start 2024-09-15T04:02:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- start start 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: init 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: _add_conns ranks=[8,0,3,2,1,6,5,7,4] 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient(hunting): picked mon.noname-e con 0x7ffb5012fd20 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient(hunting): picked mon.noname-a con 0x7ffb501300f0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient(hunting): picked mon.noname-g con 0x7ffb50130520 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5004ca10 con 0x7ffb501300f0 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb500a2830 con 0x7ffb50130520 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb50097b70 con 0x7ffb5012fd20 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient(hunting): authenticate will time out at 1891.210157s 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb55d22700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7ffb5012fd20 0x7ffb50133270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:37780/0 (socket says 172.21.15.26:37780) 2024-09-15T04:02:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb55d22700 1 -- 172.21.15.26:0/1241958353 learned_addr learned my addr 172.21.15.26:0/1241958353 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146430818 0 0) 0x7ffb50097b70 con 0x7ffb5012fd20 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient(hunting): my global_id is 135800 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb3c003570 con 0x7ffb5012fd20 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991015903 0 0) 0x7ffb5004ca10 con 0x7ffb501300f0 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient(hunting): my global_id is 195111 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb50097b70 con 0x7ffb501300f0 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1606332596 0 0) 0x7ffb3c003570 con 0x7ffb5012fd20 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 >> v1:172.21.15.122:6790/0 conn(0x7ffb50130520 legacy=0x7ffb5013b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 >> v1:172.21.15.26:6789/0 conn(0x7ffb501300f0 legacy=0x7ffb50137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 monclient: found mon.noname-e 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50133980 con 0x7ffb5012fd20 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 5 monclient: authenticate success, global_id 135800 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb4c0017b0 con 0x7ffb5012fd20 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:02:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: dump: 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffb4c0019e0 con 0x7ffb5012fd20 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 1 -- 172.21.15.26:0/1241958353 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb4c004c30 con 0x7ffb5012fd20 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 4 set_mon_vals no callback set 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb54d20700 10 monclient: dump: 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.796 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 10 monclient: shutdown 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- 172.21.15.26:0/1241958353 >> v1:172.21.15.26:6793/0 conn(0x7ffb5012fd20 legacy=0x7ffb50133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- 172.21.15.26:0/1241958353 shutdown_connections 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- 172.21.15.26:0/1241958353 >> 172.21.15.26:0/1241958353 conn(0x7ffb5006e410 msgr2=0x7ffb500a6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- 172.21.15.26:0/1241958353 shutdown_connections 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.776+0000 7ffb57f86700 1 -- 172.21.15.26:0/1241958353 wait complete. 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: monmap: 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 Processor -- start 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- start start 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: init 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: _add_conns ranks=[0,5,1,3,6,4,8,7,2] 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient(hunting): picked mon.noname-a con 0x7ffb5012fd20 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient(hunting): picked mon.noname-h con 0x7ffb501300f0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient(hunting): picked mon.noname-f con 0x7ffb50130520 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb501340b0 con 0x7ffb5012fd20 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb500a1b50 con 0x7ffb50130520 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5011f390 con 0x7ffb501300f0 2024-09-15T04:02:26.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient(hunting): authenticate will time out at 1891.213199s 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb55d22700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffb5012fd20 0x7ffb5012e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47132/0 (socket says 172.21.15.26:47132) 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb55d22700 1 -- 172.21.15.26:0/3247369470 learned_addr learned my addr 172.21.15.26:0/3247369470 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952877330 0 0) 0x7ffb501340b0 con 0x7ffb5012fd20 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): my global_id is 195120 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb34003570 con 0x7ffb5012fd20 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4041373828 0 0) 0x7ffb34003570 con 0x7ffb5012fd20 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb501340b0 con 0x7ffb5012fd20 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb4c0017f0 con 0x7ffb5012fd20 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): dump: 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.801 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891114164 0 0) 0x7ffb5011f390 con 0x7ffb501300f0 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): my global_id is 254237 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb34003570 con 0x7ffb501300f0 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028558308 0 0) 0x7ffb500a1b50 con 0x7ffb50130520 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient(hunting): my global_id is 204985 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb5011f390 con 0x7ffb50130520 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 20540549 0 0) 0x7ffb501340b0 con 0x7ffb5012fd20 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6791/0 conn(0x7ffb501300f0 legacy=0x7ffb50146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6789/0 conn(0x7ffb50130520 legacy=0x7ffb5014a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 monclient: found mon.a 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5011db90 con 0x7ffb5012fd20 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 5 monclient: authenticate success, global_id 195120 2024-09-15T04:02:26.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: _renew_subs 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5011eaf0 con 0x7ffb5012fd20 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: _renew_subs 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5009c520 con 0x7ffb5012fd20 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffb4c004f40 con 0x7ffb5012fd20 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb4c0011d0 con 0x7ffb5012fd20 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 10 monclient: dump: 2024-09-15T04:02:26.804 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.805 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.806 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb47fff700 4 set_mon_vals no callback set 2024-09-15T04:02:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffb30006590 con 0x7ffb30002840 2024-09-15T04:02:26.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.780+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffb4c00fc50 con 0x7ffb5012fd20 2024-09-15T04:02:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(820..820 src has 435..820) v4 ==== 3971+0+0 (unknown 3862651140 0 0) 0x7ffb4c044290 con 0x7ffb5012fd20 2024-09-15T04:02:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156376442 0 0) 0x7ffb30006590 con 0x7ffb30002840 2024-09-15T04:02:26.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient(hunting): my global_id is 254246 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb501340b0 con 0x7ffb30002840 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb56523700 10 monclient: get_auth_request con 0x7ffb340375f0 auth_method 0 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1516390210 0 0) 0x7ffb501340b0 con 0x7ffb30002840 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb30006590 con 0x7ffb30002840 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb400018c0 con 0x7ffb30002840 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient: dump: 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.809 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.810 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2988577888 0 0) 0x7ffb30006590 con 0x7ffb30002840 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb3000a130 con 0x7ffb30002840 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7ffb40001f40 con 0x7ffb30002840 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6791/0 conn(0x7ffb30002840 legacy=0x7ffb30004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.784+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6791/0 conn(0x7ffb30002840 legacy=0x7ffb30004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb57f86700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7ffb30007520 con 0x7ffb300068d0 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 603112814 0 0) 0x7ffb30007520 con 0x7ffb300068d0 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 10 monclient(hunting): my global_id is 254255 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb30006590 con 0x7ffb300068d0 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3368518991 0 0) 0x7ffb30006590 con 0x7ffb300068d0 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb30007520 con 0x7ffb300068d0 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffb40002770 con 0x7ffb300068d0 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 10 monclient: dump: 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.812 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 851291372 0 0) 0x7ffb30007520 con 0x7ffb300068d0 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.792+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffb30008760 con 0x7ffb300068d0 2024-09-15T04:02:26.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 2844473770) 0x7ffb40002910 con 0x7ffb300068d0 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb45ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb45ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6791/0 conn(0x7ffb300068d0 legacy=0x7ffb30006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb45ffb700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6791/0 conn(0x7ffb300068d0 legacy=0x7ffb30006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:26.814 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 8, 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.815 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:26.816 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:26.817 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:26.818 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.819 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.820 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:26.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.823 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.828 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.829 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:26.830 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.831 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.832 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:26.833 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.122:6812/20895 conn(0x7ffb340375f0 legacy=0x7ffb34039aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 10 monclient: shutdown 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 >> v1:172.21.15.26:6789/0 conn(0x7ffb5012fd20 legacy=0x7ffb5012e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb56523700 1 -- 172.21.15.26:0/3247369470 reap_dead start 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb56523700 1 -- 172.21.15.26:0/3247369470 reap_dead start 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 shutdown_connections 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 >> 172.21.15.26:0/3247369470 conn(0x7ffb5006e410 msgr2=0x7ffb500a4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 shutdown_connections 2024-09-15T04:02:26.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.796+0000 7ffb57f86700 1 -- 172.21.15.26:0/3247369470 wait complete. 2024-09-15T04:02:26.836 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:26.836 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T04:02:26.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T04:02:26.943 INFO:tasks.rados.rados.0.smithi122.stdout:3629: done (0 left) 2024-09-15T04:02:26.943 INFO:tasks.rados.rados.0.smithi122.stdout:3630: snap_remove snap 421 2024-09-15T04:02:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: get_monmap_and_config 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: monmap: 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:26.944744+0000 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:26.944744+0000 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.949 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 Processor -- start 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- start start 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: init 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: _add_conns ranks=[6,1,4,5,2,3,0,7,8] 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient(hunting): picked mon.noname-d con 0x7f120809d6b0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient(hunting): picked mon.noname-f con 0x7f120809bcb0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient(hunting): picked mon.noname-c con 0x7f1208137020 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120804ca10 con 0x7f120809bcb0 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12080a0670 con 0x7f1208137020 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120809d060 con 0x7f120809d6b0 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient(hunting): authenticate will time out at 1891.374133s 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120e3ea700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f120809d6b0 0x7f120809aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55798/0 (socket says 172.21.15.26:55798) 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120e3ea700 1 -- 172.21.15.26:0/3589296435 learned_addr learned my addr 172.21.15.26:0/3589296435 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563987636 0 0) 0x7f120809d060 con 0x7f120809d6b0 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): my global_id is 195198 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120dbe9700 1 -- 172.21.15.26:0/3589296435 >> v1:172.21.15.122:6789/0 conn(0x7f120809bcb0 legacy=0x7f1208135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120dbe9700 1 -- 172.21.15.26:0/3589296435 >> v1:172.21.15.122:6789/0 conn(0x7f120809bcb0 legacy=0x7f1208135900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11f4003570 con 0x7f120809d6b0 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120dbe9700 1 --1- 172.21.15.26:0/3589296435 >> v1:172.21.15.122:6789/0 conn(0x7f120809bcb0 0x7f1208135900 :-1 s=OPENED pgs=450 cs=1 l=1).handle_message read tag failed 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120dbe9700 1 --1- 172.21.15.26:0/3589296435 >> v1:172.21.15.122:6789/0 conn(0x7f120809bcb0 0x7f1208135900 :-1 s=OPENED pgs=450 cs=1 l=1).fault on lossy channel, failing 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.122:6789/0 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27775360 0 0) 0x7f12080a0670 con 0x7f1208137020 2024-09-15T04:02:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient(hunting): my global_id is 205015 2024-09-15T04:02:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120809d060 con 0x7f1208137020 2024-09-15T04:02:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675583028 0 0) 0x7f11f4003570 con 0x7f120809d6b0 2024-09-15T04:02:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 >> v1:172.21.15.26:6791/0 conn(0x7f1208137020 legacy=0x7f1208139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 >> v1:172.21.15.122:6789/0 conn(0x7f120809bcb0 legacy=0x7f1208135900 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 monclient: found mon.noname-d 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080a08b0 con 0x7f120809d6b0 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f12040017b0 con 0x7f120809d6b0 2024-09-15T04:02:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 5 monclient: authenticate success, global_id 195198 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: dump: 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.962 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1204001a20 con 0x7f120809d6b0 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 1 -- 172.21.15.26:0/3589296435 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1204004c30 con 0x7f120809d6b0 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 4 set_mon_vals no callback set 2024-09-15T04:02:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f120d3e8700 10 monclient: dump: 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 10 monclient: shutdown 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- 172.21.15.26:0/3589296435 >> v1:172.21.15.26:6792/0 conn(0x7f120809d6b0 legacy=0x7f120809aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- 172.21.15.26:0/3589296435 shutdown_connections 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- 172.21.15.26:0/3589296435 >> 172.21.15.26:0/3589296435 conn(0x7f1208063a10 msgr2=0x7f12080a4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- 172.21.15.26:0/3589296435 shutdown_connections 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.940+0000 7f121064e700 1 -- 172.21.15.26:0/3589296435 wait complete. 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: build_initial_monmap 2024-09-15T04:02:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: monmap: 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:26.966 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 Processor -- start 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- start start 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: init 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: _add_conns ranks=[5,2,6,4,7,3,8,0,1] 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient(hunting): picked mon.noname-h con 0x7f120809bcb0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient(hunting): picked mon.noname-b con 0x7f120809d6b0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient(hunting): picked mon.noname-d con 0x7f1208137020 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120811f070 con 0x7f120809d6b0 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120806e730 con 0x7f120809bcb0 2024-09-15T04:02:26.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120806e1b0 con 0x7f1208137020 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient(hunting): authenticate will time out at 1891.377322s 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f120dbe9700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f120809d6b0 0x7f1208144b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:49930/0 (socket says 172.21.15.26:49930) 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f120ebeb700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f1208137020 0x7f1208148250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:55812/0 (socket says 172.21.15.26:55812) 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f120dbe9700 1 -- 172.21.15.26:0/4149844393 learned_addr learned my addr 172.21.15.26:0/4149844393 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307805003 0 0) 0x7f120806e1b0 con 0x7f1208137020 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): my global_id is 195207 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ec003570 con 0x7f1208137020 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241098463 0 0) 0x7f120811f070 con 0x7f120809d6b0 2024-09-15T04:02:26.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): my global_id is 254171 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120806e1b0 con 0x7f120809d6b0 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610236272 0 0) 0x7f120806e730 con 0x7f120809bcb0 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): my global_id is 254264 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120811f070 con 0x7f120809bcb0 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2509494338 0 0) 0x7f120806e1b0 con 0x7f120809d6b0 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f120806e730 con 0x7f120809d6b0 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11f8001b80 con 0x7f120809d6b0 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4036801579 0 0) 0x7f11ec003570 con 0x7f1208137020 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f120806e1b0 con 0x7f1208137020 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1200001cc0 con 0x7f1208137020 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.971 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 817308098 0 0) 0x7f120811f070 con 0x7f120809bcb0 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11ec003570 con 0x7f120809bcb0 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1204001770 con 0x7f120809bcb0 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient(hunting): dump: 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.973 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2170102164 0 0) 0x7f120806e730 con 0x7f120809d6b0 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.26:6792/0 conn(0x7f1208137020 legacy=0x7f1208148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.122:6791/0 conn(0x7f120809bcb0 legacy=0x7f120813ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 monclient: found mon.b 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208120620 con 0x7f120809d6b0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 5 monclient: authenticate success, global_id 254171 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: _renew_subs 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1208140620 con 0x7f120809d6b0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: _renew_subs 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:26.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1208136b60 con 0x7f120809d6b0 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f11f8002210 con 0x7f120809d6b0 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11f8002740 con 0x7f120809d6b0 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f120cbe7700 4 set_mon_vals no callback set 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 10 monclient: dump: 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.976 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f11f8002920 con 0x7f120809d6b0 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f11f0006590 con 0x7f11f0002840 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.944+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(821..821 src has 435..821) v4 ==== 3943+0+0 (unknown 4168553605 0 0) 0x7f11f8044140 con 0x7f120809d6b0 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f120ebeb700 10 monclient: get_auth_request con 0x7f11ec038170 auth_method 0 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831196108 0 0) 0x7f11f0006590 con 0x7f11f0002840 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient(hunting): my global_id is 254230 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120806e730 con 0x7f11f0002840 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1550517615 0 0) 0x7f120806e730 con 0x7f11f0002840 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11f0006590 con 0x7f11f0002840 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1204004bb0 con 0x7f11f0002840 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:26.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient: dump: 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3810698883 0 0) 0x7f11f0006590 con 0x7f11f0002840 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11f000a130 con 0x7f11f0002840 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1204006ff0 con 0x7f11f0002840 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.122:6792/0 conn(0x7f11f0002840 legacy=0x7f11f0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.948+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.122:6792/0 conn(0x7f11f0002840 legacy=0x7f11f0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.956+0000 7f121064e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.956+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f11f0009640 con 0x7f11f00090b0 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463095406 0 0) 0x7f11f0009640 con 0x7f11f00090b0 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient(hunting): my global_id is 254239 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11f0006590 con 0x7f11f00090b0 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4037942940 0 0) 0x7f11f0006590 con 0x7f11f00090b0 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11f0009640 con 0x7f11f00090b0 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f12040057d0 con 0x7f11f00090b0 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient: dump: 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:26.983 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3713827322 0 0) 0x7f11f0009640 con 0x7f11f00090b0 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f11f00098f0 con 0x7f11f00090b0 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 2217601931) 0x7f1204006be0 con 0x7f11f00090b0 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.122:6792/0 conn(0x7f11f00090b0 legacy=0x7f11f0008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f11fe7fc700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.122:6792/0 conn(0x7f11f00090b0 legacy=0x7f11f0008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 112, 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:26.985 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 8, 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.986 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.987 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:26.988 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:26.989 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.990 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:26.991 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.993 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.995 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.996 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.997 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:26.998 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:26.999 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:27.000 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:27.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:27.002 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:27.003 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:27.004 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.122:6812/20895 conn(0x7f11ec038170 legacy=0x7f11ec03a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 10 monclient: shutdown 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 >> v1:172.21.15.26:6790/0 conn(0x7f120809d6b0 legacy=0x7f1208144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f120ebeb700 1 -- 172.21.15.26:0/4149844393 reap_dead start 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f120ebeb700 1 -- 172.21.15.26:0/4149844393 reap_dead start 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 shutdown_connections 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 >> 172.21.15.26:0/4149844393 conn(0x7f1208063a10 msgr2=0x7f120806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 shutdown_connections 2024-09-15T04:02:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:26.960+0000 7f121064e700 1 -- 172.21.15.26:0/4149844393 wait complete. 2024-09-15T04:02:27.007 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:02:27.007 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['h', 'e', 'f'] 2024-09-15T04:02:27.007 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['g'] 2024-09-15T04:02:27.007 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.h 2024-09-15T04:02:27.007 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-15T04:02:27.008 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-15T04:02:27.008 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:02:27.026 INFO:tasks.ceph.mon.h:Stopped 2024-09-15T04:02:27.026 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-09-15T04:02:27.026 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T04:02:27.026 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-15T04:02:27.026 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:02:27.051 INFO:tasks.ceph.mon.e:Stopped 2024-09-15T04:02:27.052 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.f 2024-09-15T04:02:27.052 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T04:02:27.052 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T04:02:27.052 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:02:27.091 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T04:02:27.092 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-09-15T04:02:27.092 INFO:tasks.ceph.mon.g:Sent signal 19 2024-09-15T04:02:27.092 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T04:02:42.026 INFO:tasks.rados.rados.0.smithi122.stdout:3630: done (0 left) 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:3631: rollback oid 8 current snap is 422 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 8 to 341 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:3632: read oid 21 snap 417 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:3632: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:3633: read oid 40 snap -1 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:3633: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:42.027 INFO:tasks.rados.rados.0.smithi122.stdout:3634: delete oid 5 current snap is 422 2024-09-15T04:02:42.030 INFO:tasks.rados.rados.0.smithi122.stdout:3634: done (3 left) 2024-09-15T04:02:42.030 INFO:tasks.rados.rados.0.smithi122.stdout:3635: write (excl) oid 3 current snap is 422 2024-09-15T04:02:42.030 INFO:tasks.rados.rados.0.smithi122.stdout:3635: seq_num 1278 ranges {679346=515526,1868811=577782,3179173=300173} 2024-09-15T04:02:42.039 INFO:tasks.rados.rados.0.smithi122.stdout:3635: writing smithi12223906-3 from 679346 to 1194872 tid 1 2024-09-15T04:02:42.046 INFO:tasks.rados.rados.0.smithi122.stdout:3635: writing smithi12223906-3 from 1868811 to 2446593 tid 2 2024-09-15T04:02:42.052 INFO:tasks.rados.rados.0.smithi122.stdout:3635: writing smithi12223906-3 from 3179173 to 3479346 tid 3 2024-09-15T04:02:42.052 INFO:tasks.rados.rados.0.smithi122.stdout:3636: read oid 30 snap 411 2024-09-15T04:02:42.052 INFO:tasks.rados.rados.0.smithi122.stdout:3636: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:02:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:3637: snap_create 2024-09-15T04:02:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:3631: finishing rollback tid 0 to smithi12223906-8 2024-09-15T04:02:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:3631: finishing rollback tid 1 to smithi12223906-8 2024-09-15T04:02:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:3631: finishing rollback tid 2 to smithi12223906-8 2024-09-15T04:02:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 2158 (ObjNum 958 snap 304 seq_num 958) dirty exists 2024-09-15T04:02:42.053 INFO:tasks.rados.rados.0.smithi122.stdout:3632: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:02:42.093 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-09-15T04:02:42.094 INFO:tasks.ceph.mon.g:Sent signal 18 2024-09-15T04:02:42.094 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 6 2024-09-15T04:02:42.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:02:42.155 INFO:tasks.rados.rados.0.smithi122.stdout:3633: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.200+0000 7f3ace70d700 10 monclient: get_monmap_and_config 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.200+0000 7f3ace70d700 10 monclient: build_initial_monmap 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.200+0000 7f3ace70d700 10 monclient: monmap: 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:42.207729+0000 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:42.207729+0000 2024-09-15T04:02:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:42.212 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 1 Processor -- start 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 1 -- start start 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient: init 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient: _add_conns ranks=[5,7,8,4,2,6,0,3,1] 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient(hunting): picked mon.noname-h con 0x7f3ac80a6a80 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient(hunting): picked mon.noname-i con 0x7f3ac809fb80 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient(hunting): picked mon.noname-e con 0x7f3ac8138d30 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac804ca10 con 0x7f3ac80a6a80 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80a0f50 con 0x7f3ac809fb80 2024-09-15T04:02:42.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac8133680 con 0x7f3ac8138d30 2024-09-15T04:02:42.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:42.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3ace70d700 10 monclient(hunting): authenticate will time out at 1906.637161s 2024-09-15T04:02:42.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3acd70b700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f3ac80a6a80 legacy=0x7f3ac8132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:42.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3acdf0c700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f3ac8138d30 legacy=0x7f3ac813b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:42.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3accf0a700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f3ac809fb80 0x7f3ac809ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:40582/0 (socket says 172.21.15.26:40582) 2024-09-15T04:02:42.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.204+0000 7f3accf0a700 1 -- 172.21.15.26:0/3946525545 learned_addr learned my addr 172.21.15.26:0/3946525545 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:42.268 INFO:tasks.rados.rados.0.smithi122.stdout:3635: finishing write tid 1 to smithi12223906-3 2024-09-15T04:02:42.268 INFO:tasks.rados.rados.0.smithi122.stdout:3635: finishing write tid 2 to smithi12223906-3 2024-09-15T04:02:42.268 INFO:tasks.rados.rados.0.smithi122.stdout:3636: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:02:42.390 INFO:tasks.rados.rados.0.smithi122.stdout:3635: finishing write tid 3 to smithi12223906-3 2024-09-15T04:02:42.390 INFO:tasks.rados.rados.0.smithi122.stdout:3635: finishing write tid 4 to smithi12223906-3 2024-09-15T04:02:42.390 INFO:tasks.rados.rados.0.smithi122.stdout:3635: finishing write tid 5 to smithi12223906-3 2024-09-15T04:02:42.391 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 3090 (ObjNum 1278 snap 422 seq_num 1278) dirty exists 2024-09-15T04:02:42.391 INFO:tasks.rados.rados.0.smithi122.stdout:3635: left oid 3 (ObjNum 1278 snap 422 seq_num 1278) 2024-09-15T04:02:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.408+0000 7f3acdf0c700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.26:6793/0 conn(0x7f3ac8138d30 legacy=0x7f3ac813b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.408+0000 7f3acd70b700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.122:6791/0 conn(0x7f3ac80a6a80 legacy=0x7f3ac8132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.812+0000 7f3acdf0c700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.26:6793/0 conn(0x7f3ac8138d30 legacy=0x7f3ac813b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:42.812+0000 7f3acd70b700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.122:6791/0 conn(0x7f3ac80a6a80 legacy=0x7f3ac8132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:43.616+0000 7f3acdf0c700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.26:6793/0 conn(0x7f3ac8138d30 legacy=0x7f3ac813b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:43.616+0000 7f3acd70b700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.122:6791/0 conn(0x7f3ac80a6a80 legacy=0x7f3ac8132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient(hunting): tick 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 monclient(hunting): continuing hunt 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient(hunting): _reopen_session rank -1 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.26:6793/0 conn(0x7f3ac8138d30 legacy=0x7f3ac813b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.122:6792/0 conn(0x7f3ac809fb80 legacy=0x7f3ac809ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.122:6791/0 conn(0x7f3ac80a6a80 legacy=0x7f3ac8132f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:45.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient: _add_conns ranks=[3,0,2,1,6,4] 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient(hunting): picked mon.noname-g con 0x7f3aac0037f0 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient(hunting): picked mon.noname-a con 0x7f3aac007470 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient(hunting): picked mon.noname-b con 0x7f3aac00b1d0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 -- 172.21.15.26:0/3946525545 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80a6e50 con 0x7f3aac007470 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 -- 172.21.15.26:0/3946525545 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac804ca10 con 0x7f3aac00b1d0 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 1 -- 172.21.15.26:0/3946525545 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac8133680 con 0x7f3aac0037f0 2024-09-15T04:02:45.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:45.204+0000 7f3abeffd700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922125471 0 0) 0x7f3ac80a6e50 con 0x7f3aac007470 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient(hunting): my global_id is 195129 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab4003570 con 0x7f3aac007470 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696042509 0 0) 0x7f3ab4003570 con 0x7f3aac007470 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.122:6790/0 conn(0x7f3aac0037f0 legacy=0x7f3aac005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.26:6790/0 conn(0x7f3aac00b1d0 legacy=0x7f3aac00d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 monclient: found mon.noname-a 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3acdf0c700 1 -- 172.21.15.26:0/3946525545 reap_dead start 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac012050 con 0x7f3aac007470 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3ab8001730 con 0x7f3aac007470 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: dump: 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 5 monclient: authenticate success, global_id 195129 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3ab8001980 con 0x7f3aac007470 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 1 -- 172.21.15.26:0/3946525545 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3ab80048f0 con 0x7f3aac007470 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 4 set_mon_vals no callback set 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3abffff700 10 monclient: dump: 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.114 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.115 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 10 monclient: shutdown 2024-09-15T04:02:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 1 -- 172.21.15.26:0/3946525545 >> v1:172.21.15.26:6789/0 conn(0x7f3aac007470 legacy=0x7f3aac009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 1 -- 172.21.15.26:0/3946525545 shutdown_connections 2024-09-15T04:02:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.100+0000 7f3ace70d700 1 -- 172.21.15.26:0/3946525545 >> 172.21.15.26:0/3946525545 conn(0x7f3ac806e380 msgr2=0x7f3ac80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 -- 172.21.15.26:0/3946525545 shutdown_connections 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 -- 172.21.15.26:0/3946525545 wait complete. 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient: monmap: 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.119 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 Processor -- start 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 -- start start 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient: init 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient: _add_conns ranks=[5,7,1,2,0,3,8,6,4] 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient(hunting): picked mon.noname-h con 0x7f3aac007470 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient(hunting): picked mon.noname-i con 0x7f3ac809a260 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient(hunting): picked mon.noname-f con 0x7f3ac81481d0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac811dc00 con 0x7f3ac81481d0 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80a1920 con 0x7f3aac007470 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac809cfc0 con 0x7f3ac809a260 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3acd70b700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f3aac007470 legacy=0x7f3ac8099b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3ace70d700 10 monclient(hunting): authenticate will time out at 1911.538441s 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3acdf0c700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f3ac81481d0 legacy=0x7f3ac814a5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3accf0a700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f3ac809a260 0x7f3ac809a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:35598/0 (socket says 172.21.15.26:35598) 2024-09-15T04:02:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.104+0000 7f3accf0a700 1 -- 172.21.15.26:0/1484806383 learned_addr learned my addr 172.21.15.26:0/1484806383 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249611353 0 0) 0x7f3ac809cfc0 con 0x7f3ac809a260 2024-09-15T04:02:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): my global_id is 254248 2024-09-15T04:02:47.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aac009050 con 0x7f3ac809a260 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3356530554 0 0) 0x7f3aac009050 con 0x7f3ac809a260 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3ac809cfc0 con 0x7f3ac809a260 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3ab8001630 con 0x7f3ac809a260 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): dump: 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.136 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2125405467 0 0) 0x7f3ac809cfc0 con 0x7f3ac809a260 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 >> v1:172.21.15.122:6791/0 conn(0x7f3aac007470 legacy=0x7f3ac8099b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 >> v1:172.21.15.122:6789/0 conn(0x7f3ac81481d0 legacy=0x7f3ac814a5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 monclient: found mon.i 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac811cfa0 con 0x7f3ac809a260 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 5 monclient: authenticate success, global_id 254248 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 10 monclient: _renew_subs 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac809b8d0 con 0x7f3ac809a260 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 10 monclient: _renew_subs 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac809b3c0 con 0x7f3ac809a260 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3ab8004dd0 con 0x7f3ac809a260 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3ab8005280 con 0x7f3ac809a260 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abeffd700 4 set_mon_vals no callback set 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.124+0000 7f3abd7fa700 10 monclient: dump: 2024-09-15T04:02:47.139 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.140 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3ace70d700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f3ab800fc40 con 0x7f3ac809a260 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3ace70d700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3ace70d700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa80050b0 con 0x7f3ac809a260 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(822..822 src has 435..822) v4 ==== 3987+0+0 (unknown 3271769100 0 0) 0x7f3ab8043fd0 con 0x7f3ac809a260 2024-09-15T04:02:47.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.128+0000 7f3acd70b700 10 monclient: get_auth_request con 0x7f3aac038800 auth_method 0 2024-09-15T04:02:47.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.132+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f3ab8014ed0 con 0x7f3ac809a260 2024-09-15T04:02:47.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.132+0000 7f3abd7fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.132+0000 7f3abd7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:47.179 INFO:tasks.rados.rados.0.smithi122.stdout:3631: done (5 left) 2024-09-15T04:02:47.179 INFO:tasks.rados.rados.0.smithi122.stdout:3632: done (4 left) 2024-09-15T04:02:47.180 INFO:tasks.rados.rados.0.smithi122.stdout:3633: done (3 left) 2024-09-15T04:02:47.180 INFO:tasks.rados.rados.0.smithi122.stdout:3635: done (2 left) 2024-09-15T04:02:47.180 INFO:tasks.rados.rados.0.smithi122.stdout:3636: done (1 left) 2024-09-15T04:02:47.180 INFO:tasks.rados.rados.0.smithi122.stdout:3637: done (0 left) 2024-09-15T04:02:47.180 INFO:tasks.rados.rados.0.smithi122.stdout:3638: snap_create 2024-09-15T04:02:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3ace70d700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:02:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3ace70d700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:47.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3ace70d700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:02:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3aa80058e0 con 0x7f3ac809a260 2024-09-15T04:02:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3abd7fa700 1 -- 172.21.15.26:0/1484806383 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2683 (unknown 254746657 0 1200200276) 0x7f3ab80148d0 con 0x7f3ac809a260 2024-09-15T04:02:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3abd7fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:02:47.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.368+0000 7f3abd7fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:47.377 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":116,"quorum":[0,2,3,4,6,7],"quorum_names":["a","b","g","c","d","i"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:02:47.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 >> v1:172.21.15.122:6812/20895 conn(0x7f3aac038800 legacy=0x7f3aac03acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 10 monclient: shutdown 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 >> v1:172.21.15.122:6792/0 conn(0x7f3ac809a260 legacy=0x7f3ac809a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 shutdown_connections 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 >> 172.21.15.26:0/1484806383 conn(0x7f3ac806e380 msgr2=0x7f3ac80a4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 shutdown_connections 2024-09-15T04:02:47.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.372+0000 7f3ace70d700 1 -- 172.21.15.26:0/1484806383 wait complete. 2024-09-15T04:02:47.392 INFO:tasks.mon_thrash.ceph_manager:quorum is size 6 2024-09-15T04:02:47.392 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon a 2024-09-15T04:02:47.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient: get_monmap_and_config 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient: monmap: 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:47.496903+0000 2024-09-15T04:02:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:47.496903+0000 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.501 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 1 Processor -- start 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 1 -- start start 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient: init 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient: _add_conns ranks=[4,1,5,2,8,3,6,0,7] 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient(hunting): picked mon.noname-c con 0x7f311c0a6880 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient(hunting): picked mon.noname-f con 0x7f311c09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient(hunting): picked mon.noname-h con 0x7f311c138d30 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c04ca10 con 0x7f311c09faa0 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0a0580 con 0x7f311c0a6880 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c133780 con 0x7f311c138d30 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3120ab9700 10 monclient(hunting): authenticate will time out at 1911.927103s 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311b7fe700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f311c0a6880 0x7f311c133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:45858/0 (socket says 172.21.15.26:45858) 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311b7fe700 1 -- 172.21.15.26:0/2051250890 learned_addr learned my addr 172.21.15.26:0/2051250890 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f3112ffd700 1 -- 172.21.15.26:0/2051250890 >> v1:172.21.15.122:6789/0 conn(0x7f311c09faa0 legacy=0x7f311c09fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311bfff700 1 -- 172.21.15.26:0/2051250890 >> v1:172.21.15.122:6791/0 conn(0x7f311c138d30 legacy=0x7f311c13b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806502627 0 0) 0x7f310c001110 con 0x7f311c0a6880 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311affd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311affd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311affd700 10 monclient(hunting): my global_id is 205024 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.492+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3108003570 con 0x7f311c0a6880 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3776295776 0 0) 0x7f3108003570 con 0x7f311c0a6880 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 >> v1:172.21.15.122:6791/0 conn(0x7f311c138d30 legacy=0x7f311c13b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 >> v1:172.21.15.122:6789/0 conn(0x7f311c09faa0 legacy=0x7f311c09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 monclient: found mon.noname-c 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0a6c50 con 0x7f311c0a6880 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 5 monclient: authenticate success, global_id 205024 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f310c001940 con 0x7f311c0a6880 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: dump: 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.508 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f310c001c60 con 0x7f311c0a6880 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 1 -- 172.21.15.26:0/2051250890 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f310c004c30 con 0x7f311c0a6880 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 4 set_mon_vals no callback set 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311affd700 10 monclient: dump: 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: shutdown 2024-09-15T04:02:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- 172.21.15.26:0/2051250890 >> v1:172.21.15.26:6791/0 conn(0x7f311c0a6880 legacy=0x7f311c133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- 172.21.15.26:0/2051250890 shutdown_connections 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- 172.21.15.26:0/2051250890 >> 172.21.15.26:0/2051250890 conn(0x7f311c06e2a0 msgr2=0x7f311c0a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- 172.21.15.26:0/2051250890 shutdown_connections 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- 172.21.15.26:0/2051250890 wait complete. 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: monmap: 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 Processor -- start 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- start start 2024-09-15T04:02:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: init 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient: _add_conns ranks=[5,6,4,3,7,2,1,0,8] 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient(hunting): picked mon.noname-h con 0x7f311c09faa0 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient(hunting): picked mon.noname-d con 0x7f311c0a6880 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient(hunting): picked mon.noname-c con 0x7f311c138d30 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c09a620 con 0x7f311c138d30 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c09ab70 con 0x7f311c09faa0 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c098880 con 0x7f311c0a6880 2024-09-15T04:02:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3120ab9700 10 monclient(hunting): authenticate will time out at 1911.930959s 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311b7fe700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f311c09faa0 legacy=0x7f311c097a60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311bfff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f311c138d30 0x7f311c14a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:45872/0 (socket says 172.21.15.26:45872) 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3112ffd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f311c0a6880 0x7f311c098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60796/0 (socket says 172.21.15.26:60796) 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f311bfff700 1 -- 172.21.15.26:0/231212546 learned_addr learned my addr 172.21.15.26:0/231212546 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010074949 0 0) 0x7f311c09a620 con 0x7f311c138d30 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): my global_id is 205033 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30fc003570 con 0x7f311c138d30 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233249080 0 0) 0x7f311c098880 con 0x7f311c0a6880 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): my global_id is 195216 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f311c09a620 con 0x7f311c0a6880 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4083961744 0 0) 0x7f311c09a620 con 0x7f311c0a6880 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f311c098880 con 0x7f311c0a6880 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3104004340 con 0x7f311c0a6880 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): dump: 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3933996797 0 0) 0x7f30fc003570 con 0x7f311c138d30 2024-09-15T04:02:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.496+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f311c09a620 con 0x7f311c138d30 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f31140045b0 con 0x7f311c138d30 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient(hunting): dump: 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 485651664 0 0) 0x7f311c098880 con 0x7f311c0a6880 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.122:6791/0 conn(0x7f311c09faa0 legacy=0x7f311c097a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.26:6791/0 conn(0x7f311c138d30 legacy=0x7f311c14a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 monclient: found mon.d 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c09a890 con 0x7f311c0a6880 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 5 monclient: authenticate success, global_id 195216 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 10 monclient: _renew_subs 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c09ae50 con 0x7f311c0a6880 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 10 monclient: _renew_subs 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c09b9a0 con 0x7f311c0a6880 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f31040049d0 con 0x7f311c0a6880 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f3104004f00 con 0x7f311c0a6880 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3119ffb700 4 set_mon_vals no callback set 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: dump: 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3100006590 con 0x7f3100002840 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f31040050e0 con 0x7f311c0a6880 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(823..823 src has 435..823) v4 ==== 3959+0+0 (unknown 664380144 0 0) 0x7f3104043ec0 con 0x7f311c0a6880 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858232030 0 0) 0x7f3100006590 con 0x7f3100002840 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient(hunting): my global_id is 195138 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f311c098880 con 0x7f3100002840 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f311bfff700 10 monclient: get_auth_request con 0x7f30fc037b60 auth_method 0 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 578798222 0 0) 0x7f311c098880 con 0x7f3100002840 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3100006590 con 0x7f3100002840 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f310c004c30 con 0x7f3100002840 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 10 monclient: dump: 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3388853885 0 0) 0x7f3100006590 con 0x7f3100002840 2024-09-15T04:02:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.500+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f310000a130 con 0x7f3100002840 2024-09-15T04:02:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.504+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f310c001800 con 0x7f3100002840 2024-09-15T04:02:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.504+0000 7f3113fff700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.504+0000 7f3113fff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.504+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.26:6789/0 conn(0x7f3100002840 legacy=0x7f3100004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.504+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.26:6789/0 conn(0x7f3100002840 legacy=0x7f3100004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3120ab9700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f3100007520 con 0x7f31000068d0 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039668786 0 0) 0x7f3100007520 con 0x7f31000068d0 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 10 monclient(hunting): my global_id is 195147 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3100006590 con 0x7f31000068d0 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1085261188 0 0) 0x7f3100006590 con 0x7f31000068d0 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3100007520 con 0x7f31000068d0 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f310c001a90 con 0x7f31000068d0 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 10 monclient: dump: 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3414332679 0 0) 0x7f3100007520 con 0x7f31000068d0 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.512+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3100008760 con 0x7f31000068d0 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7536 (unknown 0 0 2965030014) 0x7f310c0075f0 con 0x7f31000068d0 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3113fff700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3113fff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.26:6789/0 conn(0x7f31000068d0 legacy=0x7f3100006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3113fff700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.26:6789/0 conn(0x7f31000068d0 legacy=0x7f3100006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:47.533 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 116, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: 7 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:47.534 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:47.535 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:47.536 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:47.537 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.538 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:47.539 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.540 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.541 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.542 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:47.543 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:47.544 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:47.545 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.546 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.547 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.550 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.550 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.550 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.550 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:47.550 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.552 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.552 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:47.552 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:47.552 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.552 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.554 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:47.555 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:47.555 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.555 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.555 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.555 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.556 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.556 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:47.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:47.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.556 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.557 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.558 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:47.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.559 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.559 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:47.559 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:47.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.122:6812/20895 conn(0x7f30fc037b60 legacy=0x7f30fc03a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 10 monclient: shutdown 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 >> v1:172.21.15.26:6792/0 conn(0x7f311c0a6880 legacy=0x7f311c098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f311bfff700 1 -- 172.21.15.26:0/231212546 reap_dead start 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f311bfff700 1 -- 172.21.15.26:0/231212546 reap_dead start 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 shutdown_connections 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 >> 172.21.15.26:0/231212546 conn(0x7f311c06e2a0 msgr2=0x7f311c12eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 shutdown_connections 2024-09-15T04:02:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.516+0000 7f3120ab9700 1 -- 172.21.15.26:0/231212546 wait complete. 2024-09-15T04:02:47.564 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T04:02:47.564 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon b 2024-09-15T04:02:47.564 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T04:02:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient: get_monmap_and_config 2024-09-15T04:02:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient: monmap: 2024-09-15T04:02:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:47.672895+0000 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:47.672895+0000 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.678 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 1 Processor -- start 2024-09-15T04:02:47.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 1 -- start start 2024-09-15T04:02:47.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient: init 2024-09-15T04:02:47.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient: _add_conns ranks=[4,1,2,7,5,3,8,0,6] 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient(hunting): picked mon.noname-c con 0x7f6fec132b40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient(hunting): picked mon.noname-f con 0x7f6fec09faa0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient(hunting): picked mon.noname-b con 0x7f6fec138d40 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec04ca10 con 0x7f6fec09faa0 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec09d5d0 con 0x7f6fec138d40 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec133620 con 0x7f6fec132b40 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6feb59e700 10 monclient(hunting): authenticate will time out at 1912.103550s 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6fe3fff700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f6fec09faa0 legacy=0x7f6fec09fe70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6fead9d700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6fec138d40 0x7f6fec13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48916/0 (socket says 172.21.15.26:48916) 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6fead9d700 1 -- 172.21.15.26:0/389483548 learned_addr learned my addr 172.21.15.26:0/389483548 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.668+0000 7f6fea59c700 1 --1- 172.21.15.26:0/389483548 >> v1:172.21.15.26:6791/0 conn(0x7f6fec132b40 0x7f6fec132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:45886/0 (socket says 172.21.15.26:45886) 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102308788 0 0) 0x7f6fec09d5d0 con 0x7f6fec138d40 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient(hunting): my global_id is 254180 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd8003570 con 0x7f6fec138d40 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637994207 0 0) 0x7f6fec133620 con 0x7f6fec132b40 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient(hunting): my global_id is 205042 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec09d5d0 con 0x7f6fec132b40 2024-09-15T04:02:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813642588 0 0) 0x7f6fd8003570 con 0x7f6fec138d40 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 >> v1:172.21.15.26:6791/0 conn(0x7f6fec132b40 legacy=0x7f6fec132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 >> v1:172.21.15.122:6789/0 conn(0x7f6fec09faa0 legacy=0x7f6fec09fe70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 monclient: found mon.noname-b 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0a0cc0 con 0x7f6fec138d40 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6fe4001b30 con 0x7f6fec138d40 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 5 monclient: authenticate success, global_id 254180 2024-09-15T04:02:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: dump: 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fe4001da0 con 0x7f6fec138d40 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 1 -- 172.21.15.26:0/389483548 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6fe4004fb0 con 0x7f6fec138d40 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 4 set_mon_vals no callback set 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6fe9d9b700 10 monclient: dump: 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 10 monclient: shutdown 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 -- 172.21.15.26:0/389483548 >> v1:172.21.15.26:6790/0 conn(0x7f6fec138d40 legacy=0x7f6fec13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 -- 172.21.15.26:0/389483548 shutdown_connections 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 -- 172.21.15.26:0/389483548 >> 172.21.15.26:0/389483548 conn(0x7f6fec06e370 msgr2=0x7f6fec0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 -- 172.21.15.26:0/389483548 shutdown_connections 2024-09-15T04:02:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 -- 172.21.15.26:0/389483548 wait complete. 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 10 monclient: monmap: 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 Processor -- start 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 1 -- start start 2024-09-15T04:02:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.672+0000 7f6feb59e700 10 monclient: init 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient: _add_conns ranks=[8,5,4,1,7,3,6,2,0] 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient(hunting): picked mon.noname-e con 0x7f6fec09faa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient(hunting): picked mon.noname-h con 0x7f6fec132b40 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient(hunting): picked mon.noname-c con 0x7f6fec138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec1386f0 con 0x7f6fec138d40 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec09a1d0 con 0x7f6fec132b40 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fea59c700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f6fec09faa0 legacy=0x7f6fec098a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec1324c0 con 0x7f6fec09faa0 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient(hunting): authenticate will time out at 1912.108684s 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe3fff700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f6fec132b40 legacy=0x7f6fec099140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fead9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f6fec138d40 0x7f6fec14a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:45890/0 (socket says 172.21.15.26:45890) 2024-09-15T04:02:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fead9d700 1 -- 172.21.15.26:0/4194756060 learned_addr learned my addr 172.21.15.26:0/4194756060 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742721694 0 0) 0x7f6fec1386f0 con 0x7f6fec138d40 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): my global_id is 205051 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fcc003570 con 0x7f6fec138d40 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 384123298 0 0) 0x7f6fcc003570 con 0x7f6fec138d40 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fec1386f0 con 0x7f6fec138d40 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6fe4001bb0 con 0x7f6fec138d40 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): dump: 2024-09-15T04:02:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4200102044 0 0) 0x7f6fec1386f0 con 0x7f6fec138d40 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.26:6793/0 conn(0x7f6fec09faa0 legacy=0x7f6fec098a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.122:6791/0 conn(0x7f6fec132b40 legacy=0x7f6fec099140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 monclient: found mon.c 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec099850 con 0x7f6fec138d40 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 5 monclient: authenticate success, global_id 205051 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient: _renew_subs 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec09cfa0 con 0x7f6fec138d40 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient: _renew_subs 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec09b790 con 0x7f6fec138d40 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6fe40051e0 con 0x7f6fec138d40 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6fe4005690 con 0x7f6fec138d40 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe959a700 4 set_mon_vals no callback set 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 10 monclient: dump: 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6fd0006590 con 0x7f6fd0002840 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.676+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6fe4010050 con 0x7f6fec138d40 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(823..823 src has 435..823) v4 ==== 3959+0+0 (unknown 664380144 0 0) 0x7f6fe40444b0 con 0x7f6fec138d40 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4193755333 0 0) 0x7f6fd0006590 con 0x7f6fd0002840 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient(hunting): my global_id is 254189 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec1386f0 con 0x7f6fd0002840 2024-09-15T04:02:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe3fff700 10 monclient: get_auth_request con 0x7f6fcc037450 auth_method 0 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2991780857 0 0) 0x7f6fec1386f0 con 0x7f6fd0002840 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fd0006590 con 0x7f6fd0002840 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6fdc001d10 con 0x7f6fd0002840 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient: dump: 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2737748018 0 0) 0x7f6fd0006590 con 0x7f6fd0002840 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fd000a130 con 0x7f6fd0002840 2024-09-15T04:02:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6fdc0011d0 con 0x7f6fd0002840 2024-09-15T04:02:47.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:47.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.26:6790/0 conn(0x7f6fd0002840 legacy=0x7f6fd0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.680+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.26:6790/0 conn(0x7f6fd0002840 legacy=0x7f6fd0004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:47.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6feb59e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6fd0007520 con 0x7f6fd00068d0 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963633375 0 0) 0x7f6fd0007520 con 0x7f6fd00068d0 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6fe27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6fe27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6fe27fc700 10 monclient(hunting): my global_id is 254198 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.688+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd0006590 con 0x7f6fd00068d0 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 917523563 0 0) 0x7f6fd0006590 con 0x7f6fd00068d0 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6fd0007520 con 0x7f6fd00068d0 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6fdc004f00 con 0x7f6fd00068d0 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:47.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 10 monclient: dump: 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.707 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2706934960 0 0) 0x7f6fd0007520 con 0x7f6fd00068d0 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6fd0008760 con 0x7f6fd00068d0 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7536 (unknown 0 0 759260833) 0x7f6fdc0072b0 con 0x7f6fd00068d0 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.26:6790/0 conn(0x7f6fd00068d0 legacy=0x7f6fd0006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fe27fc700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.26:6790/0 conn(0x7f6fd00068d0 legacy=0x7f6fd0006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:47.708 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 116, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: 7 2024-09-15T04:02:47.709 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.710 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.711 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:47.712 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.713 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.714 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.714 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.714 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:47.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.714 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.715 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:47.716 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.717 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:47.719 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:47.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.721 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:47.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.725 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.726 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.727 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.728 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:47.729 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.122:6812/20895 conn(0x7f6fcc037450 legacy=0x7f6fcc039900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 10 monclient: shutdown 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 >> v1:172.21.15.26:6791/0 conn(0x7f6fec138d40 legacy=0x7f6fec14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fead9d700 1 -- 172.21.15.26:0/4194756060 reap_dead start 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6fead9d700 1 -- 172.21.15.26:0/4194756060 reap_dead start 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 shutdown_connections 2024-09-15T04:02:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 >> 172.21.15.26:0/4194756060 conn(0x7f6fec06e370 msgr2=0x7f6fec12ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 shutdown_connections 2024-09-15T04:02:47.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.692+0000 7f6feb59e700 1 -- 172.21.15.26:0/4194756060 wait complete. 2024-09-15T04:02:47.732 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T04:02:47.732 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon c 2024-09-15T04:02:47.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: get_monmap_and_config 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: monmap: 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:47.848705+0000 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:47.848705+0000 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.852 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.853 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 1 Processor -- start 2024-09-15T04:02:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 1 -- start start 2024-09-15T04:02:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: init 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: _add_conns ranks=[0,8,7,4,5,6,3,2,1] 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient(hunting): picked mon.noname-a con 0x7fbb400a4740 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient(hunting): picked mon.noname-e con 0x7fbb4009aca0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient(hunting): picked mon.noname-i con 0x7fbb40136f40 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4004ca10 con 0x7fbb400a4740 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4009c120 con 0x7fbb40136f40 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4009d7a0 con 0x7fbb4009aca0 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient(hunting): authenticate will time out at 1912.278211s 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3effd700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7fbb4009aca0 legacy=0x7fbb40135820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3f7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb400a4740 0x7fbb4009dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54242/0 (socket says 172.21.15.26:54242) 2024-09-15T04:02:47.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3f7fe700 1 -- 172.21.15.26:0/846260322 learned_addr learned my addr 172.21.15.26:0/846260322 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822637684 0 0) 0x7fbb4004ca10 con 0x7fbb400a4740 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient(hunting): my global_id is 195156 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb28003570 con 0x7fbb400a4740 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131349902 0 0) 0x7fbb4009c120 con 0x7fbb40136f40 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient(hunting): my global_id is 254257 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4004ca10 con 0x7fbb40136f40 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4267822681 0 0) 0x7fbb28003570 con 0x7fbb400a4740 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 >> v1:172.21.15.26:6793/0 conn(0x7fbb4009aca0 legacy=0x7fbb40135820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 >> v1:172.21.15.122:6792/0 conn(0x7fbb40136f40 legacy=0x7fbb40139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 monclient: found mon.noname-a 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4009b510 con 0x7fbb400a4740 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb340017b0 con 0x7fbb400a4740 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 5 monclient: authenticate success, global_id 195156 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: dump: 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb34001a20 con 0x7fbb400a4740 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 1 -- 172.21.15.26:0/846260322 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb34004c30 con 0x7fbb400a4740 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb3e7fc700 10 monclient: dump: 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.844+0000 7fbb44b34700 4 set_mon_vals no callback set 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 10 monclient: shutdown 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 -- 172.21.15.26:0/846260322 >> v1:172.21.15.26:6789/0 conn(0x7fbb400a4740 legacy=0x7fbb4009dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 -- 172.21.15.26:0/846260322 shutdown_connections 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 -- 172.21.15.26:0/846260322 >> 172.21.15.26:0/846260322 conn(0x7fbb400638b0 msgr2=0x7fbb400a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 -- 172.21.15.26:0/846260322 shutdown_connections 2024-09-15T04:02:47.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 -- 172.21.15.26:0/846260322 wait complete. 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 10 monclient: build_initial_monmap 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 10 monclient: monmap: 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:47.869 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 Processor -- start 2024-09-15T04:02:47.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 1 -- start start 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.848+0000 7fbb44b34700 10 monclient: init 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient: _add_conns ranks=[0,8,4,1,5,3,7,6,2] 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient(hunting): picked mon.noname-a con 0x7fbb4009aca0 addr v1:172.21.15.26:6789/0 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient(hunting): picked mon.noname-e con 0x7fbb400a4740 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient(hunting): picked mon.noname-c con 0x7fbb40136f40 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4006e2b0 con 0x7fbb4009aca0 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb401329f0 con 0x7fbb40136f40 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb3effd700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7fbb400a4740 legacy=0x7fbb4006b760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4006be70 con 0x7fbb400a4740 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient(hunting): authenticate will time out at 1912.284522s 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb3f7fe700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb4009aca0 0x7fbb4006b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54244/0 (socket says 172.21.15.26:54244) 2024-09-15T04:02:47.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb3ffff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fbb40136f40 0x7fbb4014a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:45898/0 (socket says 172.21.15.26:45898) 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb3f7fe700 1 -- 172.21.15.26:0/1475768070 learned_addr learned my addr 172.21.15.26:0/1475768070 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349894297 0 0) 0x7fbb401329f0 con 0x7fbb40136f40 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): my global_id is 205060 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb2c003570 con 0x7fbb40136f40 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037551184 0 0) 0x7fbb4006e2b0 con 0x7fbb4009aca0 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): my global_id is 195165 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb401329f0 con 0x7fbb4009aca0 2024-09-15T04:02:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1537099341 0 0) 0x7fbb2c003570 con 0x7fbb40136f40 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb4006e2b0 con 0x7fbb40136f40 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb38001980 con 0x7fbb40136f40 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): dump: 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.873 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1118638371 0 0) 0x7fbb401329f0 con 0x7fbb4009aca0 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb2c003570 con 0x7fbb4009aca0 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb34001770 con 0x7fbb4009aca0 2024-09-15T04:02:47.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient(hunting): dump: 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3479829881 0 0) 0x7fbb4006e2b0 con 0x7fbb40136f40 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6793/0 conn(0x7fbb400a4740 legacy=0x7fbb4006b760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6789/0 conn(0x7fbb4009aca0 legacy=0x7fbb4006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 monclient: found mon.c 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4006c600 con 0x7fbb40136f40 2024-09-15T04:02:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 5 monclient: authenticate success, global_id 205060 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient: _renew_subs 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4011eaa0 con 0x7fbb40136f40 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient: _renew_subs 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:02:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4006c0f0 con 0x7fbb40136f40 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbb38002cd0 con 0x7fbb40136f40 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb38002250 con 0x7fbb40136f40 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb3dffb700 4 set_mon_vals no callback set 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 10 monclient: dump: 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.852+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbb38002430 con 0x7fbb40136f40 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb44b34700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(823..823 src has 435..823) v4 ==== 3959+0+0 (unknown 664380144 0 0) 0x7fbb38043e20 con 0x7fbb40136f40 2024-09-15T04:02:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb20006590 con 0x7fbb20002840 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb3f7fe700 10 monclient: get_auth_request con 0x7fbb2c037b70 auth_method 0 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989165283 0 0) 0x7fbb20006590 con 0x7fbb20002840 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient(hunting): my global_id is 205069 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4006e2b0 con 0x7fbb20002840 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1617663755 0 0) 0x7fbb4006e2b0 con 0x7fbb20002840 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb20006590 con 0x7fbb20002840 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb300044d0 con 0x7fbb20002840 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient: dump: 2024-09-15T04:02:47.880 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.881 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3249890554 0 0) 0x7fbb20006590 con 0x7fbb20002840 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb2000a130 con 0x7fbb20002840 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fbb30006880 con 0x7fbb20002840 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6791/0 conn(0x7fbb20002840 legacy=0x7fbb20004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.856+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6791/0 conn(0x7fbb20002840 legacy=0x7fbb20004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fbb20009640 con 0x7fbb200090b0 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582582159 0 0) 0x7fbb20009640 con 0x7fbb200090b0 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient(hunting): my global_id is 205078 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb20006590 con 0x7fbb200090b0 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1910122449 0 0) 0x7fbb20006590 con 0x7fbb200090b0 2024-09-15T04:02:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbb20009640 con 0x7fbb200090b0 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbb30005030 con 0x7fbb200090b0 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient: dump: 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4243761516 0 0) 0x7fbb20009640 con 0x7fbb200090b0 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb200098f0 con 0x7fbb200090b0 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7366 (unknown 0 0 3408041050) 0x7fbb30006600 con 0x7fbb200090b0 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6791/0 conn(0x7fbb200090b0 legacy=0x7fbb20008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb277fe700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6791/0 conn(0x7fbb200090b0 legacy=0x7fbb20008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 116, 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: 7 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.889 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:47.890 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:47.891 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.892 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:47.893 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:47.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.903 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.904 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.905 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:47.906 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.122:6812/20895 conn(0x7fbb2c037b70 legacy=0x7fbb2c03a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 10 monclient: shutdown 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 >> v1:172.21.15.26:6791/0 conn(0x7fbb40136f40 legacy=0x7fbb4014a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb3ffff700 1 -- 172.21.15.26:0/1475768070 reap_dead start 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb3ffff700 1 -- 172.21.15.26:0/1475768070 reap_dead start 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 shutdown_connections 2024-09-15T04:02:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.864+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 >> 172.21.15.26:0/1475768070 conn(0x7fbb400638b0 msgr2=0x7fbb40136280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.868+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 shutdown_connections 2024-09-15T04:02:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:47.868+0000 7fbb44b34700 1 -- 172.21.15.26:0/1475768070 wait complete. 2024-09-15T04:02:47.909 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T04:02:47.909 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon d 2024-09-15T04:02:47.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:02:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient: get_monmap_and_config 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient: build_initial_monmap 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient: monmap: 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:48.026352+0000 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:48.026352+0000 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:48.031 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 1 Processor -- start 2024-09-15T04:02:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 1 -- start start 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient: init 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient: _add_conns ranks=[8,1,2,0,5,6,4,7,3] 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient(hunting): picked mon.noname-e con 0x7f2654132c90 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient(hunting): picked mon.noname-f con 0x7f2654133060 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient(hunting): picked mon.noname-b con 0x7f2654133470 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265404ca10 con 0x7f2654133060 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2654068f90 con 0x7f2654133470 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26540a1430 con 0x7f2654132c90 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265abe2700 10 monclient(hunting): authenticate will time out at 1912.455789s 2024-09-15T04:02:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f265897e700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f2654132c90 legacy=0x7f265409ff60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.020+0000 7f2653fff700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f2654133060 legacy=0x7f2654137fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265917f700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f2654133470 0x7f265413b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48958/0 (socket says 172.21.15.26:48958) 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265917f700 1 -- 172.21.15.26:0/2247748387 learned_addr learned my addr 172.21.15.26:0/2247748387 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152325534 0 0) 0x7f2654068f90 con 0x7f2654133470 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient(hunting): my global_id is 254207 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f263c003570 con 0x7f2654133470 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006072170 0 0) 0x7f263c003570 con 0x7f2654133470 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 >> v1:172.21.15.26:6793/0 conn(0x7f2654132c90 legacy=0x7f265409ff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 >> v1:172.21.15.122:6789/0 conn(0x7f2654133060 legacy=0x7f2654137fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 monclient: found mon.noname-b 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265409da90 con 0x7f2654133470 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f264c001b30 con 0x7f2654133470 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 5 monclient: authenticate success, global_id 254207 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: dump: 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.036 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f264c001e50 con 0x7f2654133470 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 1 -- 172.21.15.26:0/2247748387 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f264c004fb0 con 0x7f2654133470 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 4 set_mon_vals no callback set 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f26537fe700 10 monclient: dump: 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 10 monclient: shutdown 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 1 -- 172.21.15.26:0/2247748387 >> v1:172.21.15.26:6790/0 conn(0x7f2654133470 legacy=0x7f265413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 1 -- 172.21.15.26:0/2247748387 shutdown_connections 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 1 -- 172.21.15.26:0/2247748387 >> 172.21.15.26:0/2247748387 conn(0x7f265406e510 msgr2=0x7f26540a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 1 -- 172.21.15.26:0/2247748387 shutdown_connections 2024-09-15T04:02:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 1 -- 172.21.15.26:0/2247748387 wait complete. 2024-09-15T04:02:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 10 monclient: build_initial_monmap 2024-09-15T04:02:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.024+0000 7f265abe2700 10 monclient: monmap: 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 1 Processor -- start 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 1 -- start start 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient: init 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient: _add_conns ranks=[3,7,1,6,2,0,5,8,4] 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient(hunting): picked mon.noname-g con 0x7f2654132c90 addr v1:172.21.15.122:6790/0 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient(hunting): picked mon.noname-i con 0x7f2654133060 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient(hunting): picked mon.noname-f con 0x7f2654133470 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26540a1e90 con 0x7f2654133470 2024-09-15T04:02:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265411fec0 con 0x7f2654132c90 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265411fbe0 con 0x7f2654133060 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265abe2700 10 monclient(hunting): authenticate will time out at 1912.462253s 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f265917f700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f2654133470 legacy=0x7f2654149e40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2653fff700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7f2654133060 0x7f265411d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:35608/0 (socket says 172.21.15.26:35608) 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2653fff700 1 -- 172.21.15.26:0/1420110737 learned_addr learned my addr 172.21.15.26:0/1420110737 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559539762 0 0) 0x7f265411fec0 con 0x7f2654132c90 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): my global_id is 254289 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2640003570 con 0x7f2654132c90 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833045449 0 0) 0x7f265411fbe0 con 0x7f2654133060 2024-09-15T04:02:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): my global_id is 254266 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f265411fec0 con 0x7f2654133060 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 859359977 0 0) 0x7f2640003570 con 0x7f2654132c90 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f265411fbe0 con 0x7f2654132c90 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26480042c0 con 0x7f2654132c90 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3025073460 0 0) 0x7f265411fec0 con 0x7f2654133060 2024-09-15T04:02:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2640003570 con 0x7f2654133060 2024-09-15T04:02:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26440042b0 con 0x7f2654133060 2024-09-15T04:02:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.028+0000 7f2650ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.055 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.055 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.055 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.055 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1238445889 0 0) 0x7f265411fbe0 con 0x7f2654132c90 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.122:6792/0 conn(0x7f2654133060 legacy=0x7f265411d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.122:6789/0 conn(0x7f2654133470 legacy=0x7f2654149e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 monclient: found mon.g 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265409d120 con 0x7f2654132c90 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: _finish_auth 0 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 5 monclient: authenticate success, global_id 254289 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 10 monclient: _renew_subs 2024-09-15T04:02:48.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265411f900 con 0x7f2654132c90 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 10 monclient: _renew_subs 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 10 monclient: _send_mon_message to mon.g at v1:172.21.15.122:6790/0 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.122:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f265411ea10 con 0x7f2654132c90 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f26480044e0 con 0x7f2654132c90 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2648004a10 con 0x7f2654132c90 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: dump: 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.058 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2652ffd700 4 set_mon_vals no callback set 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2634006590 con 0x7f2634002840 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2648004b70 con 0x7f2654132c90 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.3 v1:172.21.15.122:6790/0 8 ==== osd_map(823..823 src has 435..823) v4 ==== 3959+0+0 (unknown 664380144 0 0) 0x7f2648043d80 con 0x7f2654132c90 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640848813 0 0) 0x7f2634006590 con 0x7f2634002840 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient(hunting): my global_id is 195225 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f265411fbe0 con 0x7f2634002840 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f265917f700 10 monclient: get_auth_request con 0x7f2640037b40 auth_method 0 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2646325919 0 0) 0x7f265411fbe0 con 0x7f2634002840 2024-09-15T04:02:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2634006590 con 0x7f2634002840 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2644004a80 con 0x7f2634002840 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 10 monclient: dump: 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 417025971 0 0) 0x7f2634006590 con 0x7f2634002840 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.032+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f263400a130 con 0x7f2634002840 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.036+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f2644005030 con 0x7f2634002840 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.036+0000 7f2650ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.036+0000 7f2650ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.036+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.26:6792/0 conn(0x7f2634002840 legacy=0x7f2634004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.036+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.26:6792/0 conn(0x7f2634002840 legacy=0x7f2634004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f265abe2700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f2634007520 con 0x7f26340068d0 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350710995 0 0) 0x7f2634007520 con 0x7f26340068d0 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 10 monclient(hunting): my global_id is 195234 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2634006590 con 0x7f26340068d0 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3775875741 0 0) 0x7f2634006590 con 0x7f26340068d0 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2634007520 con 0x7f26340068d0 2024-09-15T04:02:48.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f26440058e0 con 0x7f26340068d0 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 10 monclient: dump: 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1259690586 0 0) 0x7f2634007520 con 0x7f26340068d0 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.044+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2634008760 con 0x7f26340068d0 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7538 (unknown 0 0 1504269045) 0x7f2644005b80 con 0x7f26340068d0 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f2650ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f2650ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.26:6792/0 conn(0x7f26340068d0 legacy=0x7f2634006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f2650ff9700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.26:6792/0 conn(0x7f26340068d0 legacy=0x7f2634006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 116, 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:48.066 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: 7 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 0, 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.067 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.068 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:48.069 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.070 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.073 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:48.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.078 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:48.080 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.081 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:48.082 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.083 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.084 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:48.085 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.122:6812/20895 conn(0x7f2640037b40 legacy=0x7f2640039ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 10 monclient: shutdown 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 >> v1:172.21.15.122:6790/0 conn(0x7f2654132c90 legacy=0x7f265411ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265917f700 1 -- 172.21.15.26:0/1420110737 reap_dead start 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 shutdown_connections 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 >> 172.21.15.26:0/1420110737 conn(0x7f265406e510 msgr2=0x7f265409e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 shutdown_connections 2024-09-15T04:02:48.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.048+0000 7f265abe2700 1 -- 172.21.15.26:0/1420110737 wait complete. 2024-09-15T04:02:48.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T04:02:48.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon g 2024-09-15T04:02:48.088 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T04:02:48.194 INFO:tasks.rados.rados.0.smithi122.stdout:3638: done (0 left) 2024-09-15T04:02:48.194 INFO:tasks.rados.rados.0.smithi122.stdout:3639: write (excl) oid 38 current snap is 424 2024-09-15T04:02:48.194 INFO:tasks.rados.rados.0.smithi122.stdout:3639: seq_num 1279 ranges {696153=590775,1927293=721440,3206596=289557} 2024-09-15T04:02:48.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: get_monmap_and_config 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: build_initial_monmap 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: monmap: 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:48.196335+0000 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:48.196335+0000 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:48.201 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.208 INFO:tasks.rados.rados.0.smithi122.stdout:3639: writing smithi12223906-38 from 696153 to 1286928 tid 1 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 Processor -- start 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- start start 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: init 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: _add_conns ranks=[2,5,7,6,1,3,4,0,8] 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient(hunting): picked mon.noname-b con 0x7f41d00990c0 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient(hunting): picked mon.noname-h con 0x7f41d00a1310 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient(hunting): picked mon.noname-i con 0x7f41d01335a0 addr v1:172.21.15.122:6792/0 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d004ca10 con 0x7f41d00990c0 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d00a1b20 con 0x7f41d00a1310 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d00a10b0 con 0x7f41d01335a0 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient(hunting): authenticate will time out at 1912.625986s 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d59f7700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f41d00a1310 legacy=0x7f41d0099490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d61f8700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f41d00990c0 0x7f41d0132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48972/0 (socket says 172.21.15.26:48972) 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d61f8700 1 -- 172.21.15.26:0/3313853291 learned_addr learned my addr 172.21.15.26:0/3313853291 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831728532 0 0) 0x7f41d004ca10 con 0x7f41d00990c0 2024-09-15T04:02:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient(hunting): my global_id is 254216 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41bc003570 con 0x7f41d00990c0 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3143460696 0 0) 0x7f41bc003570 con 0x7f41d00990c0 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 >> v1:172.21.15.122:6792/0 conn(0x7f41d01335a0 legacy=0x7f41d013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 >> v1:172.21.15.122:6791/0 conn(0x7f41d00a1310 legacy=0x7f41d0099490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: _finish_hunting 0 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 monclient: found mon.noname-b 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0099ba0 con 0x7f41d00990c0 2024-09-15T04:02:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: _finish_auth 0 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 5 monclient: authenticate success, global_id 254216 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41cc0017b0 con 0x7f41d00990c0 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: dump: 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f41cc001ad0 con 0x7f41d00990c0 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 1 -- 172.21.15.26:0/3313853291 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41cc004c30 con 0x7f41d00990c0 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 4 set_mon_vals no callback set 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d51f6700 10 monclient: dump: 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 10 monclient: shutdown 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- 172.21.15.26:0/3313853291 >> v1:172.21.15.26:6790/0 conn(0x7f41d00990c0 legacy=0x7f41d0132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- 172.21.15.26:0/3313853291 shutdown_connections 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- 172.21.15.26:0/3313853291 >> 172.21.15.26:0/3313853291 conn(0x7f41d006e370 msgr2=0x7f41d00a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- 172.21.15.26:0/3313853291 shutdown_connections 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.192+0000 7f41d845c700 1 -- 172.21.15.26:0/3313853291 wait complete. 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: build_initial_monmap 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: monmap: 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:48.218 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 Processor -- start 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 -- start start 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: init 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: _add_conns ranks=[6,4,1,5,3,2,8,0,7] 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient(hunting): picked mon.noname-d con 0x7f41d00990c0 addr v1:172.21.15.26:6792/0 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient(hunting): picked mon.noname-c con 0x7f41d00a1310 addr v1:172.21.15.26:6791/0 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient(hunting): picked mon.noname-f con 0x7f41d01335a0 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d00a1ce0 con 0x7f41d01335a0 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d009a010 con 0x7f41d00a1310 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d012c8d0 con 0x7f41d00990c0 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient(hunting): authenticate will time out at 1912.629773s 2024-09-15T04:02:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d61f8700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f41d00990c0 0x7f41d0132180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:60814/0 (socket says 172.21.15.26:60814) 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d59f7700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f41d00a1310 0x7f41d012bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:45934/0 (socket says 172.21.15.26:45934) 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d61f8700 1 -- 172.21.15.26:0/256612001 learned_addr learned my addr 172.21.15.26:0/256612001 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d69f9700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6789/0 conn(0x7f41d01335a0 legacy=0x7f41d012c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206355777 0 0) 0x7f41d009a010 con 0x7f41d00a1310 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): my global_id is 205087 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b4003570 con 0x7f41d00a1310 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040509136 0 0) 0x7f41d012c8d0 con 0x7f41d00990c0 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): my global_id is 195243 2024-09-15T04:02:48.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d009a010 con 0x7f41d00990c0 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2228073861 0 0) 0x7f41b4003570 con 0x7f41d00a1310 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41d012c8d0 con 0x7f41d00a1310 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41c00042f0 con 0x7f41d00a1310 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): dump: 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.222 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4076381745 0 0) 0x7f41d009a010 con 0x7f41d00990c0 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41b4003570 con 0x7f41d00990c0 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41cc001770 con 0x7f41d00990c0 2024-09-15T04:02:48.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient(hunting): dump: 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1465933561 0 0) 0x7f41b4003570 con 0x7f41d00990c0 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.26:6791/0 conn(0x7f41d00a1310 legacy=0x7f41d012bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6789/0 conn(0x7f41d01335a0 legacy=0x7f41d012c1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: _finish_hunting 0 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 monclient: found mon.d 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d012f6c0 con 0x7f41d00990c0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: _finish_auth 0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 5 monclient: authenticate success, global_id 195243 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: _renew_subs 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d01311f0 con 0x7f41d00990c0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: _renew_subs 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d014aeb0 con 0x7f41d00990c0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f41cc004c00 con 0x7f41d00990c0 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41cc005240 con 0x7f41d00990c0 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d49f5700 4 set_mon_vals no callback set 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41c67fc700 10 monclient: dump: 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.227 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.196+0000 7f41d845c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f41cc00fc00 con 0x7f41d00990c0 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f41b8006590 con 0x7f41b8002840 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(824..824 src has 435..824) v4 ==== 3959+0+0 (unknown 1112471630 0 0) 0x7f41cc0440e0 con 0x7f41d00990c0 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217622976 0 0) 0x7f41b8006590 con 0x7f41b8002840 2024-09-15T04:02:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41d69f9700 10 monclient: get_auth_request con 0x7f41b4037a90 auth_method 0 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient(hunting): my global_id is 254298 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b4003570 con 0x7f41b8002840 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2063300298 0 0) 0x7f41b4003570 con 0x7f41b8002840 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41b8006590 con 0x7f41b8002840 2024-09-15T04:02:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41c0004b90 con 0x7f41b8002840 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient: dump: 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3650390079 0 0) 0x7f41b8006590 con 0x7f41b8002840 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41b800a130 con 0x7f41b8002840 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f41c0006e90 con 0x7f41b8002840 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6790/0 conn(0x7f41b8002840 legacy=0x7f41b8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.200+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6790/0 conn(0x7f41b8002840 legacy=0x7f41b8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.208+0000 7f41d845c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f41b8007520 con 0x7f41b80068d0 2024-09-15T04:02:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678755164 0 0) 0x7f41b8007520 con 0x7f41b80068d0 2024-09-15T04:02:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient(hunting): my global_id is 254307 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b8006590 con 0x7f41b80068d0 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3117400996 0 0) 0x7f41b8006590 con 0x7f41b80068d0 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f41b8007520 con 0x7f41b80068d0 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f41c0005820 con 0x7f41b80068d0 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient: dump: 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1431234336 0 0) 0x7f41b8007520 con 0x7f41b80068d0 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f41b8008760 con 0x7f41b80068d0 2024-09-15T04:02:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7536 (unknown 0 0 2681164880) 0x7f41c0005b40 con 0x7f41b80068d0 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6790/0 conn(0x7f41b80068d0 legacy=0x7f41b8006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41c67fc700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6790/0 conn(0x7f41b80068d0 legacy=0x7f41b8006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:48.237 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 116, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: 7 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:48.238 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:48.239 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:48.240 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:48.241 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:48.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.243 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.244 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:48.245 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.246 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.247 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:48.248 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.249 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.249 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.249 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:48.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.250 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.253 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.256 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:48.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:02:48.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:48.260 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3639: writing smithi12223906-38 from 1927293 to 2648733 tid 2 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3639: writing smithi12223906-38 from 3206596 to 3496153 tid 3 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3640: rollback oid 36 current snap is 424 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 36 to 417 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3639: finishing write tid 1 to smithi12223906-38 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3641: rollback oid 3 current snap is 424 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 3 to 342 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3642: delete oid 29 current snap is 424 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3639: finishing write tid 2 to smithi12223906-38 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3639: finishing write tid 3 to smithi12223906-38 2024-09-15T04:02:48.263 INFO:tasks.rados.rados.0.smithi122.stdout:3639: finishing write tid 4 to smithi12223906-38 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3639: finishing write tid 5 to smithi12223906-38 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 2162 (ObjNum 1279 snap 424 seq_num 1279) dirty exists 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3639: left oid 38 (ObjNum 1279 snap 424 seq_num 1279) 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3639: done (3 left) 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3642: done (2 left) 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3643: write oid 13 current snap is 424 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3643: seq_num 1281 ranges {729767=741273,1927904=725109,3145089=384678} 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3643: writing smithi12223906-13 from 729767 to 1471040 tid 1 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3643: writing smithi12223906-13 from 1927904 to 2653013 tid 2 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3643: writing smithi12223906-13 from 3145089 to 3529767 tid 3 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3644: copy_from oid 30 from oid 19 current snap is 424 2024-09-15T04:02:48.264 INFO:tasks.rados.rados.0.smithi122.stdout:3640: finishing rollback tid 0 to smithi12223906-36 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3640: finishing rollback tid 1 to smithi12223906-36 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3640: finishing rollback tid 2 to smithi12223906-36 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 36 v 3094 (ObjNum 1259 snap 417 seq_num 1259) dirty exists 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3641: finishing rollback tid 0 to smithi12223906-3 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3641: finishing rollback tid 1 to smithi12223906-3 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3641: finishing rollback tid 2 to smithi12223906-3 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 3098 (ObjNum 973 snap 307 seq_num 973) dirty exists 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3643: finishing write tid 1 to smithi12223906-13 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3643: finishing write tid 2 to smithi12223906-13 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3640: done (3 left) 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3641: done (2 left) 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3645: write oid 9 current snap is 424 2024-09-15T04:02:48.265 INFO:tasks.rados.rados.0.smithi122.stdout:3645: seq_num 1282 ranges {746574=416522,1586386=468767,2772512=629544} 2024-09-15T04:02:48.266 INFO:tasks.rados.rados.0.smithi122.stdout:3645: writing smithi12223906-9 from 746574 to 1163096 tid 1 2024-09-15T04:02:48.266 INFO:tasks.rados.rados.0.smithi122.stdout:3645: writing smithi12223906-9 from 1586386 to 2055153 tid 2 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.122:6812/20895 conn(0x7f41b4037a90 legacy=0x7f41b4039f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 10 monclient: shutdown 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 >> v1:172.21.15.26:6792/0 conn(0x7f41d00990c0 legacy=0x7f41d0132180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d69f9700 1 -- 172.21.15.26:0/256612001 reap_dead start 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d69f9700 1 -- 172.21.15.26:0/256612001 reap_dead start 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 shutdown_connections 2024-09-15T04:02:48.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 >> 172.21.15.26:0/256612001 conn(0x7f41d006e370 msgr2=0x7f41d009c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 shutdown_connections 2024-09-15T04:02:48.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.212+0000 7f41d845c700 1 -- 172.21.15.26:0/256612001 wait complete. 2024-09-15T04:02:48.267 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T04:02:48.267 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 6 for mon i 2024-09-15T04:02:48.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T04:02:48.268 INFO:tasks.rados.rados.0.smithi122.stdout:3645: writing smithi12223906-9 from 2772512 to 3402056 tid 3 2024-09-15T04:02:48.268 INFO:tasks.rados.rados.0.smithi122.stdout:3646: snap_remove snap 384 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.368+0000 7f0aad45c700 10 monclient: get_monmap_and_config 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.368+0000 7f0aad45c700 10 monclient: build_initial_monmap 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.368+0000 7f0aad45c700 10 monclient: monmap: 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:02:48.373754+0000 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:02:48.373754+0000 2024-09-15T04:02:48.377 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:48.378 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.368+0000 7f0aad45c700 1 Processor -- start 2024-09-15T04:02:48.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.368+0000 7f0aad45c700 1 -- start start 2024-09-15T04:02:48.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.368+0000 7f0aad45c700 10 monclient: init 2024-09-15T04:02:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient: _add_conns ranks=[8,2,5,3,4,0,6,7,1] 2024-09-15T04:02:48.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient(hunting): picked mon.noname-e con 0x7f0aa8070f80 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient(hunting): picked mon.noname-b con 0x7f0aa8098910 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient(hunting): picked mon.noname-h con 0x7f0aa8098d40 addr v1:172.21.15.122:6791/0 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa804ca10 con 0x7f0aa8098910 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa8096440 con 0x7f0aa8098d40 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa7fff700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f0aa8070f80 legacy=0x7f0aa8132db0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa809f700 con 0x7f0aa8070f80 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient(hunting): authenticate will time out at 1912.805003s 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aacc5b700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7f0aa8098d40 legacy=0x7f0aa813b450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0a9f7fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0aa8098910 0x7f0aa8137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48980/0 (socket says 172.21.15.26:48980) 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0a9f7fe700 1 -- 172.21.15.26:0/1230883411 learned_addr learned my addr 172.21.15.26:0/1230883411 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015369383 0 0) 0x7f0aa804ca10 con 0x7f0aa8098910 2024-09-15T04:02:48.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient(hunting): my global_id is 254225 2024-09-15T04:02:48.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a94003570 con 0x7f0aa8098910 2024-09-15T04:02:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347403394 0 0) 0x7f0a94003570 con 0x7f0aa8098910 2024-09-15T04:02:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 >> v1:172.21.15.26:6793/0 conn(0x7f0aa8070f80 legacy=0x7f0aa8132db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 >> v1:172.21.15.122:6791/0 conn(0x7f0aa8098d40 legacy=0x7f0aa813b450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:02:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: _finish_hunting 0 2024-09-15T04:02:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 monclient: found mon.noname-b 2024-09-15T04:02:48.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa806e620 con 0x7f0aa8098910 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: _finish_auth 0 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 5 monclient: authenticate success, global_id 254225 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a900017b0 con 0x7f0aa8098910 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: dump: 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a90001ad0 con 0x7f0aa8098910 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient: get_monmap_and_config success 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 1 -- 172.21.15.26:0/1230883411 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a90004c30 con 0x7f0aa8098910 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 4 set_mon_vals no callback set 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aa77fe700 10 monclient: dump: 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 10 monclient: shutdown 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:48.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- 172.21.15.26:0/1230883411 >> v1:172.21.15.26:6790/0 conn(0x7f0aa8098910 legacy=0x7f0aa8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- 172.21.15.26:0/1230883411 shutdown_connections 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- 172.21.15.26:0/1230883411 >> 172.21.15.26:0/1230883411 conn(0x7f0aa80a2ea0 msgr2=0x7f0aa809eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- 172.21.15.26:0/1230883411 shutdown_connections 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.372+0000 7f0aad45c700 1 -- 172.21.15.26:0/1230883411 wait complete. 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: build_initial_monmap 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: monmap: 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:02:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 Processor -- start 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 -- start start 2024-09-15T04:02:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: init 2024-09-15T04:02:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: _reopen_session rank -1 2024-09-15T04:02:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: _add_conns ranks=[2,1,8,0,4,3,5,7,6] 2024-09-15T04:02:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient(hunting): picked mon.noname-b con 0x7f0aa8070f80 addr v1:172.21.15.26:6790/0 2024-09-15T04:02:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient(hunting): picked mon.noname-f con 0x7f0aa8098d40 addr v1:172.21.15.122:6789/0 2024-09-15T04:02:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient(hunting): picked mon.noname-e con 0x7f0aa812b2b0 addr v1:172.21.15.26:6793/0 2024-09-15T04:02:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa812db90 con 0x7f0aa8098d40 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa812d1a0 con 0x7f0aa8070f80 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa812c610 con 0x7f0aa812b2b0 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient(hunting): _renew_subs 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient(hunting): authenticate will time out at 1912.810412s 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aacc5b700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7f0aa812b2b0 legacy=0x7f0aa814a4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0a9f7fe700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7f0aa8098d40 legacy=0x7f0aa8146d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa7fff700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f0aa8070f80 0x7f0aa8141d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:48988/0 (socket says 172.21.15.26:48988) 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa7fff700 1 -- 172.21.15.26:0/3859292871 learned_addr learned my addr 172.21.15.26:0/3859292871 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536863561 0 0) 0x7f0aa812d1a0 con 0x7f0aa8070f80 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): my global_id is 254234 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a88003570 con 0x7f0aa8070f80 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3716730196 0 0) 0x7f0a88003570 con 0x7f0aa8070f80 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0aa812d1a0 con 0x7f0aa8070f80 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a98004350 con 0x7f0aa8070f80 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): dump: 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1352090151 0 0) 0x7f0aa812d1a0 con 0x7f0aa8070f80 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.26:6793/0 conn(0x7f0aa812b2b0 legacy=0x7f0aa814a4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.122:6789/0 conn(0x7f0aa8098d40 legacy=0x7f0aa8146d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 monclient: found mon.b 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa812b9a0 con 0x7f0aa8070f80 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: _finish_auth 0 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 5 monclient: authenticate success, global_id 254234 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: _renew_subs 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa812cbe0 con 0x7f0aa8070f80 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: _renew_subs 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:02:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa812a640 con 0x7f0aa8070f80 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0a98004670 con 0x7f0aa8070f80 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa6ffd700 4 set_mon_vals no callback set 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a98004ba0 con 0x7f0aa8070f80 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.376+0000 7f0aa4ff9700 10 monclient: dump: 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aad45c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0a8c006590 con 0x7f0a8c002840 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0a98004d80 con 0x7f0aa8070f80 2024-09-15T04:02:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(824..824 src has 435..824) v4 ==== 3959+0+0 (unknown 1112471630 0 0) 0x7f0a98043fe0 con 0x7f0aa8070f80 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aacc5b700 10 monclient: get_auth_request con 0x7f0a88037450 auth_method 0 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358144319 0 0) 0x7f0a8c006590 con 0x7f0a8c002840 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 10 monclient(hunting): my global_id is 254284 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa812d1a0 con 0x7f0a8c002840 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1541485483 0 0) 0x7f0aa812d1a0 con 0x7f0a8c002840 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a8c006590 con 0x7f0a8c002840 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a90004b70 con 0x7f0a8c002840 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 10 monclient: dump: 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1908606649 0 0) 0x7f0a8c006590 con 0x7f0a8c002840 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.380+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a8c00a130 con 0x7f0a8c002840 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.384+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0a90001520 con 0x7f0a8c002840 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.384+0000 7f0aa4ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.384+0000 7f0aa4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.384+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.122:6792/0 conn(0x7f0a8c002840 legacy=0x7f0a8c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.384+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.122:6792/0 conn(0x7f0a8c002840 legacy=0x7f0a8c004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aad45c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0a8c009640 con 0x7f0a8c0090b0 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592621459 0 0) 0x7f0a8c009640 con 0x7f0a8c0090b0 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 10 monclient(hunting): my global_id is 254293 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a8c006590 con 0x7f0a8c0090b0 2024-09-15T04:02:48.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1179531990 0 0) 0x7f0a8c006590 con 0x7f0a8c0090b0 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0a8c009640 con 0x7f0a8c0090b0 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0a90005340 con 0x7f0a8c0090b0 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 10 monclient: dump: 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:02:48.407 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3825786272 0 0) 0x7f0a8c009640 con 0x7f0a8c0090b0 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.392+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a8c0098f0 con 0x7f0a8c0090b0 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7366 (unknown 0 0 3463178432) 0x7f0a90007540 con 0x7f0a8c0090b0 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aa4ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aa4ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:02:48.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.122:6792/0 conn(0x7f0a8c0090b0 legacy=0x7f0a8c008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aa4ff9700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.122:6792/0 conn(0x7f0a8c0090b0 legacy=0x7f0a8c008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 116, 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:02:48.409 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: 7 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.410 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.411 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:48.412 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:02:48.413 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:02:48.416 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:02:48.416 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:02:48.416 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:02:48.416 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:02:48.416 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:02:48.417 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.418 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:02:48.420 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:02:48.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:02:48.425 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:02:48.426 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:02:48.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.428 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.429 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:02:48.431 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:02:48.432 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:02:48.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:02:48.432 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:02:48.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:02:48.432 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:02:48.432 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:02:48.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.122:6812/20895 conn(0x7f0a88037450 legacy=0x7f0a88039900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 10 monclient: shutdown 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 >> v1:172.21.15.26:6790/0 conn(0x7f0aa8070f80 legacy=0x7f0aa8141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aacc5b700 1 -- 172.21.15.26:0/3859292871 reap_dead start 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aacc5b700 1 -- 172.21.15.26:0/3859292871 reap_dead start 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 shutdown_connections 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 >> 172.21.15.26:0/3859292871 conn(0x7f0aa80a2ea0 msgr2=0x7f0aa80669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 shutdown_connections 2024-09-15T04:02:48.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:02:48.396+0000 7f0aad45c700 1 -- 172.21.15.26:0/3859292871 wait complete. 2024-09-15T04:02:48.435 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 6 2024-09-15T04:02:48.435 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-09-15T04:02:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:3646: done (3 left) 2024-09-15T04:02:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:3647: write oid 6 current snap is 424 2024-09-15T04:02:49.210 INFO:tasks.rados.rados.0.smithi122.stdout:3643: finishing write tid 3 to smithi12223906-13 2024-09-15T04:02:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:3643: finishing write tid 4 to smithi12223906-13 2024-09-15T04:02:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:3643: finishing write tid 5 to smithi12223906-13 2024-09-15T04:02:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 2633 (ObjNum 1281 snap 424 seq_num 1281) dirty exists 2024-09-15T04:02:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:3643: left oid 13 (ObjNum 1281 snap 424 seq_num 1281) 2024-09-15T04:02:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:3645: finishing write tid 1 to smithi12223906-9 2024-09-15T04:02:49.211 INFO:tasks.rados.rados.0.smithi122.stdout:3645: finishing write tid 2 to smithi12223906-9 2024-09-15T04:02:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:3644: finishing copy_from racing read to smithi12223906-30 2024-09-15T04:02:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:3644: finishing copy_from to smithi12223906-30 2024-09-15T04:02:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 2426 (ObjNum 1209 snap 399 seq_num 1209) dirty exists 2024-09-15T04:02:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:3645: finishing write tid 3 to smithi12223906-9 2024-09-15T04:02:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:3645: finishing write tid 4 to smithi12223906-9 2024-09-15T04:02:49.212 INFO:tasks.rados.rados.0.smithi122.stdout:3645: finishing write tid 5 to smithi12223906-9 2024-09-15T04:02:49.213 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2492 (ObjNum 1282 snap 424 seq_num 1282) dirty exists 2024-09-15T04:02:49.213 INFO:tasks.rados.rados.0.smithi122.stdout:3645: left oid 9 (ObjNum 1282 snap 424 seq_num 1282) 2024-09-15T04:02:49.213 INFO:tasks.rados.rados.0.smithi122.stdout:3647: seq_num 1283 ranges {763381=491771,2044868=728778,3199935=363446} 2024-09-15T04:02:49.218 INFO:tasks.rados.rados.0.smithi122.stdout:3647: writing smithi12223906-6 from 763381 to 1255152 tid 1 2024-09-15T04:02:49.227 INFO:tasks.rados.rados.0.smithi122.stdout:3647: writing smithi12223906-6 from 2044868 to 2773646 tid 2 2024-09-15T04:02:49.232 INFO:tasks.rados.rados.0.smithi122.stdout:3647: writing smithi12223906-6 from 3199935 to 3563381 tid 3 2024-09-15T04:02:49.232 INFO:tasks.rados.rados.0.smithi122.stdout:3643: done (3 left) 2024-09-15T04:02:49.232 INFO:tasks.rados.rados.0.smithi122.stdout:3644: done (2 left) 2024-09-15T04:02:49.232 INFO:tasks.rados.rados.0.smithi122.stdout:3645: done (1 left) 2024-09-15T04:02:49.232 INFO:tasks.rados.rados.0.smithi122.stdout:3648: write (excl) oid 2 current snap is 424 2024-09-15T04:02:49.232 INFO:tasks.rados.rados.0.smithi122.stdout:3648: seq_num 1284 ranges {780188=567020,2103350=472436,3227358=352830} 2024-09-15T04:02:49.239 INFO:tasks.rados.rados.0.smithi122.stdout:3648: writing smithi12223906-2 from 780188 to 1347208 tid 1 2024-09-15T04:02:49.247 INFO:tasks.rados.rados.0.smithi122.stdout:3648: writing smithi12223906-2 from 2103350 to 2575786 tid 2 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3648: writing smithi12223906-2 from 3227358 to 3580188 tid 3 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3649: read oid 49 snap -1 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3649: expect (ObjNum 1173 snap 384 seq_num 1173) 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3650: snap_create 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3647: finishing write tid 1 to smithi12223906-6 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3647: finishing write tid 2 to smithi12223906-6 2024-09-15T04:02:49.254 INFO:tasks.rados.rados.0.smithi122.stdout:3647: finishing write tid 3 to smithi12223906-6 2024-09-15T04:02:49.255 INFO:tasks.rados.rados.0.smithi122.stdout:3647: finishing write tid 4 to smithi12223906-6 2024-09-15T04:02:49.255 INFO:tasks.rados.rados.0.smithi122.stdout:3647: finishing write tid 5 to smithi12223906-6 2024-09-15T04:02:49.255 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 6 v 2496 (ObjNum 1283 snap 424 seq_num 1283) dirty exists 2024-09-15T04:02:49.255 INFO:tasks.rados.rados.0.smithi122.stdout:3647: left oid 6 (ObjNum 1283 snap 424 seq_num 1283) 2024-09-15T04:02:49.255 INFO:tasks.rados.rados.0.smithi122.stdout:3648: finishing write tid 1 to smithi12223906-2 2024-09-15T04:02:49.256 INFO:tasks.rados.rados.0.smithi122.stdout:3648: finishing write tid 2 to smithi12223906-2 2024-09-15T04:02:49.263 INFO:tasks.rados.rados.0.smithi122.stdout:3648: finishing write tid 3 to smithi12223906-2 2024-09-15T04:02:49.264 INFO:tasks.rados.rados.0.smithi122.stdout:3648: finishing write tid 4 to smithi12223906-2 2024-09-15T04:02:49.264 INFO:tasks.rados.rados.0.smithi122.stdout:3648: finishing write tid 5 to smithi12223906-2 2024-09-15T04:02:49.264 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 2898 (ObjNum 1284 snap 424 seq_num 1284) dirty exists 2024-09-15T04:02:49.264 INFO:tasks.rados.rados.0.smithi122.stdout:3648: left oid 2 (ObjNum 1284 snap 424 seq_num 1284) 2024-09-15T04:02:49.265 INFO:tasks.rados.rados.0.smithi122.stdout:3649: expect (ObjNum 1173 snap 384 seq_num 1173) 2024-09-15T04:02:49.536 INFO:tasks.rados.rados.0.smithi122.stdout:3647: done (3 left) 2024-09-15T04:02:49.536 INFO:tasks.rados.rados.0.smithi122.stdout:3648: done (2 left) 2024-09-15T04:02:49.536 INFO:tasks.rados.rados.0.smithi122.stdout:3649: done (1 left) 2024-09-15T04:02:49.536 INFO:tasks.rados.rados.0.smithi122.stdout:3650: done (0 left) 2024-09-15T04:02:49.536 INFO:tasks.rados.rados.0.smithi122.stdout:3651: delete oid 12 current snap is 425 2024-09-15T04:02:49.541 INFO:tasks.rados.rados.0.smithi122.stdout:3651: done (0 left) 2024-09-15T04:02:49.541 INFO:tasks.rados.rados.0.smithi122.stdout:3652: rollback oid 32 current snap is 425 2024-09-15T04:02:49.541 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 32 to 366 2024-09-15T04:02:49.541 INFO:tasks.rados.rados.0.smithi122.stdout:3653: snap_remove snap 349 2024-09-15T04:02:50.543 INFO:tasks.rados.rados.0.smithi122.stdout:3653: done (1 left) 2024-09-15T04:02:50.543 INFO:tasks.rados.rados.0.smithi122.stdout:3654: snap_create 2024-09-15T04:02:50.543 INFO:tasks.rados.rados.0.smithi122.stdout:3652: finishing rollback tid 0 to smithi12223906-32 2024-09-15T04:02:50.543 INFO:tasks.rados.rados.0.smithi122.stdout:3652: finishing rollback tid 1 to smithi12223906-32 2024-09-15T04:02:50.543 INFO:tasks.rados.rados.0.smithi122.stdout:3652: finishing rollback tid 2 to smithi12223906-32 2024-09-15T04:02:50.543 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 3078 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:02:51.552 INFO:tasks.rados.rados.0.smithi122.stdout:3652: done (1 left) 2024-09-15T04:02:51.552 INFO:tasks.rados.rados.0.smithi122.stdout:3654: done (0 left) 2024-09-15T04:02:51.552 INFO:tasks.rados.rados.0.smithi122.stdout:3655: read oid 4 snap -1 2024-09-15T04:02:51.552 INFO:tasks.rados.rados.0.smithi122.stdout:3655: expect deleted 2024-09-15T04:02:51.552 INFO:tasks.rados.rados.0.smithi122.stdout:3656: delete oid 11 current snap is 426 2024-09-15T04:02:51.559 INFO:tasks.rados.rados.0.smithi122.stdout:3655: done (1 left) 2024-09-15T04:02:51.559 INFO:tasks.rados.rados.0.smithi122.stdout:3656: done (0 left) 2024-09-15T04:02:51.559 INFO:tasks.rados.rados.0.smithi122.stdout:3657: read oid 24 snap -1 2024-09-15T04:02:51.559 INFO:tasks.rados.rados.0.smithi122.stdout:3657: expect (ObjNum 1255 snap 410 seq_num 1255) 2024-09-15T04:02:51.559 INFO:tasks.rados.rados.0.smithi122.stdout:3658: snap_remove snap 369 2024-09-15T04:02:52.561 INFO:tasks.rados.rados.0.smithi122.stdout:3657: expect (ObjNum 1255 snap 410 seq_num 1255) 2024-09-15T04:02:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:3657: done (1 left) 2024-09-15T04:02:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:3658: done (0 left) 2024-09-15T04:02:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:3659: write (excl) oid 26 current snap is 426 2024-09-15T04:02:52.827 INFO:tasks.rados.rados.0.smithi122.stdout:3659: seq_num 1287 ranges {430609=509120,1478796=736116,2909627=602257} 2024-09-15T04:02:52.842 INFO:tasks.rados.rados.0.smithi122.stdout:3659: writing smithi12223906-26 from 430609 to 939729 tid 1 2024-09-15T04:02:52.849 INFO:tasks.rados.rados.0.smithi122.stdout:3659: writing smithi12223906-26 from 1478796 to 2214912 tid 2 2024-09-15T04:02:52.857 INFO:tasks.rados.rados.0.smithi122.stdout:3659: writing smithi12223906-26 from 2909627 to 3511884 tid 3 2024-09-15T04:02:52.857 INFO:tasks.rados.rados.0.smithi122.stdout:3660: read oid 3 snap -1 2024-09-15T04:02:52.857 INFO:tasks.rados.rados.0.smithi122.stdout:3660: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T04:02:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:3659: finishing write tid 1 to smithi12223906-26 2024-09-15T04:02:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:3661: read oid 11 snap -1 2024-09-15T04:02:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:3661: expect deleted 2024-09-15T04:02:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:3662: delete oid 41 current snap is 426 2024-09-15T04:02:52.858 INFO:tasks.rados.rados.0.smithi122.stdout:3659: finishing write tid 2 to smithi12223906-26 2024-09-15T04:02:52.863 INFO:tasks.rados.rados.0.smithi122.stdout:3661: done (3 left) 2024-09-15T04:02:52.863 INFO:tasks.rados.rados.0.smithi122.stdout:3662: done (2 left) 2024-09-15T04:02:52.863 INFO:tasks.rados.rados.0.smithi122.stdout:3663: delete oid 36 current snap is 426 2024-09-15T04:02:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:3659: finishing write tid 3 to smithi12223906-26 2024-09-15T04:02:52.867 INFO:tasks.rados.rados.0.smithi122.stdout:3659: finishing write tid 4 to smithi12223906-26 2024-09-15T04:02:52.869 INFO:tasks.rados.rados.0.smithi122.stdout:3659: finishing write tid 5 to smithi12223906-26 2024-09-15T04:02:52.869 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 2166 (ObjNum 1287 snap 426 seq_num 1287) dirty exists 2024-09-15T04:02:52.869 INFO:tasks.rados.rados.0.smithi122.stdout:3659: left oid 26 (ObjNum 1287 snap 426 seq_num 1287) 2024-09-15T04:02:52.870 INFO:tasks.rados.rados.0.smithi122.stdout:3660: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T04:02:53.005 INFO:tasks.rados.rados.0.smithi122.stdout:3659: done (2 left) 2024-09-15T04:02:53.005 INFO:tasks.rados.rados.0.smithi122.stdout:3660: done (1 left) 2024-09-15T04:02:53.005 INFO:tasks.rados.rados.0.smithi122.stdout:3663: done (0 left) 2024-09-15T04:02:53.005 INFO:tasks.rados.rados.0.smithi122.stdout:3664: write oid 34 current snap is 426 2024-09-15T04:02:53.005 INFO:tasks.rados.rados.0.smithi122.stdout:3664: seq_num 1290 ranges {481030=734867,1937889=483443,2875543=552426} 2024-09-15T04:02:53.018 INFO:tasks.rados.rados.0.smithi122.stdout:3664: writing smithi12223906-34 from 481030 to 1215897 tid 1 2024-09-15T04:02:53.024 INFO:tasks.rados.rados.0.smithi122.stdout:3664: writing smithi12223906-34 from 1937889 to 2421332 tid 2 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3664: writing smithi12223906-34 from 2875543 to 3427969 tid 3 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3665: rollback oid 10 current snap is 426 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3664: finishing write tid 1 to smithi12223906-34 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 420 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3664: finishing write tid 2 to smithi12223906-34 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3666: read oid 15 snap -1 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3666: expect deleted 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3667: write (excl) oid 7 current snap is 426 2024-09-15T04:02:53.031 INFO:tasks.rados.rados.0.smithi122.stdout:3667: seq_num 1291 ranges {497837=410116,1596371=743454,2902966=763698} 2024-09-15T04:02:53.036 INFO:tasks.rados.rados.0.smithi122.stdout:3667: writing smithi12223906-7 from 497837 to 907953 tid 1 2024-09-15T04:02:53.044 INFO:tasks.rados.rados.0.smithi122.stdout:3667: writing smithi12223906-7 from 1596371 to 2339825 tid 2 2024-09-15T04:02:53.051 INFO:tasks.rados.rados.0.smithi122.stdout:3667: writing smithi12223906-7 from 2902966 to 3666664 tid 3 2024-09-15T04:02:53.052 INFO:tasks.rados.rados.0.smithi122.stdout:3668: snap_remove snap 386 2024-09-15T04:02:53.581 INFO:tasks.rados.rados.0.smithi122.stdout:3668: done (4 left) 2024-09-15T04:02:53.581 INFO:tasks.rados.rados.0.smithi122.stdout:3669: read oid 38 snap -1 2024-09-15T04:02:53.581 INFO:tasks.rados.rados.0.smithi122.stdout:3669: expect (ObjNum 1279 snap 424 seq_num 1279) 2024-09-15T04:02:53.583 INFO:tasks.rados.rados.0.smithi122.stdout:3670: copy_from oid 40 from oid 21 current snap is 426 2024-09-15T04:02:53.583 INFO:tasks.rados.rados.0.smithi122.stdout:3671: rollback oid 23 current snap is 426 2024-09-15T04:02:53.583 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 416 2024-09-15T04:02:53.583 INFO:tasks.rados.rados.0.smithi122.stdout:3672: copy_from oid 46 from oid 32 current snap is 426 2024-09-15T04:02:53.583 INFO:tasks.rados.rados.0.smithi122.stdout:3665: finishing rollback tid 1 to smithi12223906-10 2024-09-15T04:02:53.583 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 2143 (ObjNum 129 snap 0 seq_num 1842556752) dirty dne 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3664: finishing write tid 3 to smithi12223906-34 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3664: finishing write tid 4 to smithi12223906-34 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3664: finishing write tid 5 to smithi12223906-34 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 2141 (ObjNum 1290 snap 426 seq_num 1290) dirty exists 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3664: left oid 34 (ObjNum 1290 snap 426 seq_num 1290) 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3667: finishing write tid 1 to smithi12223906-7 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3667: finishing write tid 2 to smithi12223906-7 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3664: done (7 left) 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3665: done (6 left) 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3666: done (5 left) 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3673: read oid 16 snap -1 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3673: expect (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3667: finishing write tid 3 to smithi12223906-7 2024-09-15T04:02:53.584 INFO:tasks.rados.rados.0.smithi122.stdout:3667: finishing write tid 4 to smithi12223906-7 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3667: finishing write tid 5 to smithi12223906-7 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 2440 (ObjNum 1291 snap 426 seq_num 1291) dirty exists 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3667: left oid 7 (ObjNum 1291 snap 426 seq_num 1291) 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3667: done (5 left) 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3674: rollback oid 17 current snap is 426 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 17 to 416 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3675: read oid 31 snap -1 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3675: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:02:53.585 INFO:tasks.rados.rados.0.smithi122.stdout:3676: snap_remove snap 389 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3676: done (7 left) 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3677: read oid 36 snap 424 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3677: expect (ObjNum 1259 snap 417 seq_num 1259) 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3678: read oid 25 snap -1 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3678: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3679: delete oid 45 current snap is 426 2024-09-15T04:02:54.533 INFO:tasks.rados.rados.0.smithi122.stdout:3673: expect (ObjNum 1153 snap 380 seq_num 1153) 2024-09-15T04:02:54.732 INFO:tasks.rados.rados.0.smithi122.stdout:3674: finishing rollback tid 0 to smithi12223906-17 2024-09-15T04:02:54.732 INFO:tasks.rados.rados.0.smithi122.stdout:3674: finishing rollback tid 1 to smithi12223906-17 2024-09-15T04:02:54.732 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 2900 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-09-15T04:02:54.732 INFO:tasks.rados.rados.0.smithi122.stdout:3670: finishing copy_from racing read to smithi12223906-40 2024-09-15T04:02:54.733 INFO:tasks.rados.rados.0.smithi122.stdout:3671: finishing rollback tid 1 to smithi12223906-23 2024-09-15T04:02:54.733 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 2941 (ObjNum 129 snap 0 seq_num 1842718888) dirty dne 2024-09-15T04:02:54.733 INFO:tasks.rados.rados.0.smithi122.stdout:3672: finishing copy_from racing read to smithi12223906-46 2024-09-15T04:02:54.733 INFO:tasks.rados.rados.0.smithi122.stdout:3675: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:02:54.907 INFO:tasks.rados.rados.0.smithi122.stdout:3672: finishing copy_from to smithi12223906-46 2024-09-15T04:02:54.907 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 3078 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:02:54.908 INFO:tasks.rados.rados.0.smithi122.stdout:3670: finishing copy_from to smithi12223906-40 2024-09-15T04:02:54.908 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:02:54.908 INFO:tasks.rados.rados.0.smithi122.stdout:3669: expect (ObjNum 1279 snap 424 seq_num 1279) 2024-09-15T04:02:55.091 INFO:tasks.rados.rados.0.smithi122.stdout:3677: expect (ObjNum 1259 snap 417 seq_num 1259) 2024-09-15T04:02:55.221 INFO:tasks.rados.rados.0.smithi122.stdout:3678: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:02:55.311 INFO:tasks.rados.rados.0.smithi122.stdout:3669: done (9 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3670: done (8 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3671: done (7 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3672: done (6 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3673: done (5 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3674: done (4 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3675: done (3 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3677: done (2 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3678: done (1 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3679: done (0 left) 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:3680: rollback oid 30 current snap is 426 2024-09-15T04:02:55.312 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 30 to 396 2024-09-15T04:02:55.313 INFO:tasks.rados.rados.0.smithi122.stdout:3681: snap_create 2024-09-15T04:02:55.318 INFO:tasks.rados.rados.0.smithi122.stdout:3680: finishing rollback tid 0 to smithi12223906-30 2024-09-15T04:02:55.319 INFO:tasks.rados.rados.0.smithi122.stdout:3680: finishing rollback tid 1 to smithi12223906-30 2024-09-15T04:02:55.320 INFO:tasks.rados.rados.0.smithi122.stdout:3680: finishing rollback tid 2 to smithi12223906-30 2024-09-15T04:02:55.321 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 2901 (ObjNum 1066 snap 345 seq_num 1066) dirty exists 2024-09-15T04:02:55.591 INFO:tasks.rados.rados.0.smithi122.stdout:3680: done (1 left) 2024-09-15T04:02:55.591 INFO:tasks.rados.rados.0.smithi122.stdout:3681: done (0 left) 2024-09-15T04:02:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3682: write (excl) oid 13 current snap is 427 2024-09-15T04:02:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3682: seq_num 1293 ranges {531451=676967,1713335=747123,2957812=502595} 2024-09-15T04:02:55.602 INFO:tasks.rados.rados.0.smithi122.stdout:3682: writing smithi12223906-13 from 531451 to 1208418 tid 1 2024-09-15T04:02:55.609 INFO:tasks.rados.rados.0.smithi122.stdout:3682: writing smithi12223906-13 from 1713335 to 2460458 tid 2 2024-09-15T04:02:55.615 INFO:tasks.rados.rados.0.smithi122.stdout:3682: writing smithi12223906-13 from 2957812 to 3460407 tid 3 2024-09-15T04:02:55.615 INFO:tasks.rados.rados.0.smithi122.stdout:3683: write (excl) oid 40 current snap is 427 2024-09-15T04:02:55.615 INFO:tasks.rados.rados.0.smithi122.stdout:3683: seq_num 1294 ranges {548258=752216,1771817=490781,2985235=713867} 2024-09-15T04:02:55.622 INFO:tasks.rados.rados.0.smithi122.stdout:3683: writing smithi12223906-40 from 548258 to 1300474 tid 1 2024-09-15T04:02:55.628 INFO:tasks.rados.rados.0.smithi122.stdout:3683: writing smithi12223906-40 from 1771817 to 2262598 tid 2 2024-09-15T04:02:55.637 INFO:tasks.rados.rados.0.smithi122.stdout:3683: writing smithi12223906-40 from 2985235 to 3699102 tid 3 2024-09-15T04:02:55.637 INFO:tasks.rados.rados.0.smithi122.stdout:3684: snap_remove snap 400 2024-09-15T04:02:56.601 INFO:tasks.rados.rados.0.smithi122.stdout:3682: finishing write tid 1 to smithi12223906-13 2024-09-15T04:02:56.601 INFO:tasks.rados.rados.0.smithi122.stdout:3682: finishing write tid 2 to smithi12223906-13 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3682: finishing write tid 3 to smithi12223906-13 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3682: finishing write tid 4 to smithi12223906-13 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3682: finishing write tid 5 to smithi12223906-13 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 3082 (ObjNum 1293 snap 427 seq_num 1293) dirty exists 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3682: left oid 13 (ObjNum 1293 snap 427 seq_num 1293) 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3683: finishing write tid 1 to smithi12223906-40 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3683: finishing write tid 2 to smithi12223906-40 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3683: finishing write tid 3 to smithi12223906-40 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3683: finishing write tid 4 to smithi12223906-40 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3683: finishing write tid 5 to smithi12223906-40 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2905 (ObjNum 1294 snap 427 seq_num 1294) dirty exists 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3683: left oid 40 (ObjNum 1294 snap 427 seq_num 1294) 2024-09-15T04:02:56.602 INFO:tasks.rados.rados.0.smithi122.stdout:3682: done (2 left) 2024-09-15T04:02:56.603 INFO:tasks.rados.rados.0.smithi122.stdout:3683: done (1 left) 2024-09-15T04:02:56.603 INFO:tasks.rados.rados.0.smithi122.stdout:3684: done (0 left) 2024-09-15T04:02:56.603 INFO:tasks.rados.rados.0.smithi122.stdout:3685: read oid 2 snap -1 2024-09-15T04:02:56.603 INFO:tasks.rados.rados.0.smithi122.stdout:3685: expect (ObjNum 1284 snap 424 seq_num 1284) 2024-09-15T04:02:56.603 INFO:tasks.rados.rados.0.smithi122.stdout:3686: snap_create 2024-09-15T04:02:56.619 INFO:tasks.rados.rados.0.smithi122.stdout:3685: expect (ObjNum 1284 snap 424 seq_num 1284) 2024-09-15T04:02:57.605 INFO:tasks.rados.rados.0.smithi122.stdout:3685: done (1 left) 2024-09-15T04:02:57.605 INFO:tasks.rados.rados.0.smithi122.stdout:3686: done (0 left) 2024-09-15T04:02:57.605 INFO:tasks.rados.rados.0.smithi122.stdout:3687: snap_create 2024-09-15T04:02:58.643 INFO:tasks.rados.rados.0.smithi122.stdout:3687: done (0 left) 2024-09-15T04:02:58.643 INFO:tasks.rados.rados.0.smithi122.stdout:3688: write oid 32 current snap is 429 2024-09-15T04:02:58.643 INFO:tasks.rados.rados.0.smithi122.stdout:3688: seq_num 1295 ranges {565065=427465,1430299=634439,2496305=525139,3472104=292961} 2024-09-15T04:02:58.653 INFO:tasks.rados.rados.0.smithi122.stdout:3688: writing smithi12223906-32 from 565065 to 992530 tid 1 2024-09-15T04:02:58.659 INFO:tasks.rados.rados.0.smithi122.stdout:3688: writing smithi12223906-32 from 1430299 to 2064738 tid 2 2024-09-15T04:02:58.665 INFO:tasks.rados.rados.0.smithi122.stdout:3688: writing smithi12223906-32 from 2496305 to 3021444 tid 3 2024-09-15T04:02:58.669 INFO:tasks.rados.rados.0.smithi122.stdout:3688: writing smithi12223906-32 from 3472104 to 3765065 tid 4 2024-09-15T04:02:58.670 INFO:tasks.rados.rados.0.smithi122.stdout:3689: copy_from oid 8 from oid 46 current snap is 429 2024-09-15T04:02:58.670 INFO:tasks.rados.rados.0.smithi122.stdout:3690: snap_remove snap 405 2024-09-15T04:02:59.534 INFO:tasks.rados.rados.0.smithi122.stdout:3690: done (2 left) 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3691: snap_create 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: finishing write tid 1 to smithi12223906-32 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: finishing write tid 2 to smithi12223906-32 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: finishing write tid 3 to smithi12223906-32 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: finishing write tid 4 to smithi12223906-32 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: finishing write tid 5 to smithi12223906-32 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: finishing write tid 6 to smithi12223906-32 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 3143 (ObjNum 1295 snap 429 seq_num 1295) dirty exists 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3688: left oid 32 (ObjNum 1295 snap 429 seq_num 1295) 2024-09-15T04:02:59.535 INFO:tasks.rados.rados.0.smithi122.stdout:3689: finishing copy_from racing read to smithi12223906-8 2024-09-15T04:02:59.536 INFO:tasks.rados.rados.0.smithi122.stdout:3689: finishing copy_from to smithi12223906-8 2024-09-15T04:02:59.536 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 3078 (ObjNum 331 snap 99 seq_num 331) dirty exists 2024-09-15T04:03:00.534 INFO:tasks.rados.rados.0.smithi122.stdout:3688: done (2 left) 2024-09-15T04:03:00.534 INFO:tasks.rados.rados.0.smithi122.stdout:3689: done (1 left) 2024-09-15T04:03:00.534 INFO:tasks.rados.rados.0.smithi122.stdout:3691: done (0 left) 2024-09-15T04:03:00.534 INFO:tasks.rados.rados.0.smithi122.stdout:3692: read oid 49 snap 412 2024-09-15T04:03:00.534 INFO:tasks.rados.rados.0.smithi122.stdout:3692: expect deleted 2024-09-15T04:03:00.534 INFO:tasks.rados.rados.0.smithi122.stdout:3693: snap_create 2024-09-15T04:03:01.536 INFO:tasks.rados.rados.0.smithi122.stdout:3692: done (1 left) 2024-09-15T04:03:01.536 INFO:tasks.rados.rados.0.smithi122.stdout:3693: done (0 left) 2024-09-15T04:03:01.536 INFO:tasks.rados.rados.0.smithi122.stdout:3694: snap_remove snap 425 2024-09-15T04:03:02.545 INFO:tasks.rados.rados.0.smithi122.stdout:3694: done (0 left) 2024-09-15T04:03:02.545 INFO:tasks.rados.rados.0.smithi122.stdout:3695: delete oid 41 current snap is 431 2024-09-15T04:03:02.554 INFO:tasks.rados.rados.0.smithi122.stdout:3695: done (0 left) 2024-09-15T04:03:02.554 INFO:tasks.rados.rados.0.smithi122.stdout:3696: write oid 44 current snap is 431 2024-09-15T04:03:02.554 INFO:tasks.rados.rados.0.smithi122.stdout:3696: seq_num 1297 ranges {598679=577963,1947263=638108,3351151=447528} 2024-09-15T04:03:02.561 INFO:tasks.rados.rados.0.smithi122.stdout:3696: writing smithi12223906-44 from 598679 to 1176642 tid 1 2024-09-15T04:03:02.569 INFO:tasks.rados.rados.0.smithi122.stdout:3696: writing smithi12223906-44 from 1947263 to 2585371 tid 2 2024-09-15T04:03:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3696: writing smithi12223906-44 from 3351151 to 3798679 tid 3 2024-09-15T04:03:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3697: delete oid 5 current snap is 431 2024-09-15T04:03:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3696: finishing write tid 1 to smithi12223906-44 2024-09-15T04:03:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3696: finishing write tid 2 to smithi12223906-44 2024-09-15T04:03:02.585 INFO:tasks.rados.rados.0.smithi122.stdout:3697: done (1 left) 2024-09-15T04:03:02.585 INFO:tasks.rados.rados.0.smithi122.stdout:3698: read oid 8 snap -1 2024-09-15T04:03:02.585 INFO:tasks.rados.rados.0.smithi122.stdout:3698: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:03:02.586 INFO:tasks.rados.rados.0.smithi122.stdout:3699: write oid 48 current snap is 431 2024-09-15T04:03:02.586 INFO:tasks.rados.rados.0.smithi122.stdout:3696: finishing write tid 3 to smithi12223906-44 2024-09-15T04:03:02.586 INFO:tasks.rados.rados.0.smithi122.stdout:3699: seq_num 1299 ranges {632293=728461,1947874=641777,3005997=686580} 2024-09-15T04:03:02.596 INFO:tasks.rados.rados.0.smithi122.stdout:3699: writing smithi12223906-48 from 632293 to 1360754 tid 1 2024-09-15T04:03:02.604 INFO:tasks.rados.rados.0.smithi122.stdout:3699: writing smithi12223906-48 from 1947874 to 2589651 tid 2 2024-09-15T04:03:02.611 INFO:tasks.rados.rados.0.smithi122.stdout:3699: writing smithi12223906-48 from 3005997 to 3692577 tid 3 2024-09-15T04:03:02.611 INFO:tasks.rados.rados.0.smithi122.stdout:3700: write oid 40 current snap is 431 2024-09-15T04:03:02.611 INFO:tasks.rados.rados.0.smithi122.stdout:3700: seq_num 1300 ranges {649100=403710,1606356=501788,2633420=497852,3636005=213095} 2024-09-15T04:03:02.617 INFO:tasks.rados.rados.0.smithi122.stdout:3700: writing smithi12223906-40 from 649100 to 1052810 tid 1 2024-09-15T04:03:02.623 INFO:tasks.rados.rados.0.smithi122.stdout:3700: writing smithi12223906-40 from 1606356 to 2108144 tid 2 2024-09-15T04:03:02.629 INFO:tasks.rados.rados.0.smithi122.stdout:3700: writing smithi12223906-40 from 2633420 to 3131272 tid 3 2024-09-15T04:03:02.633 INFO:tasks.rados.rados.0.smithi122.stdout:3700: writing smithi12223906-40 from 3636005 to 3849100 tid 4 2024-09-15T04:03:02.633 INFO:tasks.rados.rados.0.smithi122.stdout:3701: snap_remove snap 371 2024-09-15T04:03:03.558 INFO:tasks.rados.rados.0.smithi122.stdout:3696: finishing write tid 4 to smithi12223906-44 2024-09-15T04:03:03.559 INFO:tasks.rados.rados.0.smithi122.stdout:3696: finishing write tid 5 to smithi12223906-44 2024-09-15T04:03:03.559 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 3157 (ObjNum 1297 snap 431 seq_num 1297) dirty exists 2024-09-15T04:03:03.559 INFO:tasks.rados.rados.0.smithi122.stdout:3696: left oid 44 (ObjNum 1297 snap 431 seq_num 1297) 2024-09-15T04:03:03.559 INFO:tasks.rados.rados.0.smithi122.stdout:3698: expect (ObjNum 331 snap 99 seq_num 331) 2024-09-15T04:03:03.746 INFO:tasks.rados.rados.0.smithi122.stdout:3699: finishing write tid 1 to smithi12223906-48 2024-09-15T04:03:03.746 INFO:tasks.rados.rados.0.smithi122.stdout:3699: finishing write tid 2 to smithi12223906-48 2024-09-15T04:03:03.746 INFO:tasks.rados.rados.0.smithi122.stdout:3699: finishing write tid 3 to smithi12223906-48 2024-09-15T04:03:03.746 INFO:tasks.rados.rados.0.smithi122.stdout:3699: finishing write tid 4 to smithi12223906-48 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3699: finishing write tid 5 to smithi12223906-48 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2147 (ObjNum 1299 snap 431 seq_num 1299) dirty exists 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3699: left oid 48 (ObjNum 1299 snap 431 seq_num 1299) 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: finishing write tid 1 to smithi12223906-40 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: finishing write tid 2 to smithi12223906-40 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: finishing write tid 3 to smithi12223906-40 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: finishing write tid 4 to smithi12223906-40 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: finishing write tid 5 to smithi12223906-40 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: finishing write tid 6 to smithi12223906-40 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2910 (ObjNum 1300 snap 431 seq_num 1300) dirty exists 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3700: left oid 40 (ObjNum 1300 snap 431 seq_num 1300) 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3696: done (4 left) 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3698: done (3 left) 2024-09-15T04:03:03.747 INFO:tasks.rados.rados.0.smithi122.stdout:3699: done (2 left) 2024-09-15T04:03:03.748 INFO:tasks.rados.rados.0.smithi122.stdout:3700: done (1 left) 2024-09-15T04:03:03.748 INFO:tasks.rados.rados.0.smithi122.stdout:3701: done (0 left) 2024-09-15T04:03:03.748 INFO:tasks.rados.rados.0.smithi122.stdout:3702: write (excl) oid 37 current snap is 431 2024-09-15T04:03:03.748 INFO:tasks.rados.rados.0.smithi122.stdout:3702: seq_num 1301 ranges {665907=595312,1781191=645446,3177196=688711} 2024-09-15T04:03:03.761 INFO:tasks.rados.rados.0.smithi122.stdout:3702: writing smithi12223906-37 from 665907 to 1261219 tid 1 2024-09-15T04:03:03.768 INFO:tasks.rados.rados.0.smithi122.stdout:3702: writing smithi12223906-37 from 1781191 to 2426637 tid 2 2024-09-15T04:03:03.777 INFO:tasks.rados.rados.0.smithi122.stdout:3702: writing smithi12223906-37 from 3177196 to 3865907 tid 3 2024-09-15T04:03:03.777 INFO:tasks.rados.rados.0.smithi122.stdout:3703: delete oid 28 current snap is 431 2024-09-15T04:03:03.777 INFO:tasks.rados.rados.0.smithi122.stdout:3702: finishing write tid 1 to smithi12223906-37 2024-09-15T04:03:03.778 INFO:tasks.rados.rados.0.smithi122.stdout:3702: finishing write tid 2 to smithi12223906-37 2024-09-15T04:03:03.782 INFO:tasks.rados.rados.0.smithi122.stdout:3703: done (1 left) 2024-09-15T04:03:03.782 INFO:tasks.rados.rados.0.smithi122.stdout:3704: delete oid 19 current snap is 431 2024-09-15T04:03:03.783 INFO:tasks.rados.rados.0.smithi122.stdout:3702: finishing write tid 3 to smithi12223906-37 2024-09-15T04:03:03.783 INFO:tasks.rados.rados.0.smithi122.stdout:3702: finishing write tid 4 to smithi12223906-37 2024-09-15T04:03:03.784 INFO:tasks.rados.rados.0.smithi122.stdout:3702: finishing write tid 5 to smithi12223906-37 2024-09-15T04:03:03.784 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 2151 (ObjNum 1301 snap 431 seq_num 1301) dirty exists 2024-09-15T04:03:03.784 INFO:tasks.rados.rados.0.smithi122.stdout:3702: left oid 37 (ObjNum 1301 snap 431 seq_num 1301) 2024-09-15T04:03:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:3702: done (1 left) 2024-09-15T04:03:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:3704: done (0 left) 2024-09-15T04:03:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:3705: read oid 42 snap 418 2024-09-15T04:03:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:3705: expect (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:03:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:3706: write (excl) oid 18 current snap is 431 2024-09-15T04:03:03.789 INFO:tasks.rados.rados.0.smithi122.stdout:3706: seq_num 1304 ranges {716328=421059,1840284=509126,3143112=659293} 2024-09-15T04:03:03.796 INFO:tasks.rados.rados.0.smithi122.stdout:3706: writing smithi12223906-18 from 716328 to 1137387 tid 1 2024-09-15T04:03:03.805 INFO:tasks.rados.rados.0.smithi122.stdout:3706: writing smithi12223906-18 from 1840284 to 2349410 tid 2 2024-09-15T04:03:03.814 INFO:tasks.rados.rados.0.smithi122.stdout:3706: writing smithi12223906-18 from 3143112 to 3802405 tid 3 2024-09-15T04:03:03.814 INFO:tasks.rados.rados.0.smithi122.stdout:3707: delete oid 37 current snap is 431 2024-09-15T04:03:03.814 INFO:tasks.rados.rados.0.smithi122.stdout:3706: finishing write tid 1 to smithi12223906-18 2024-09-15T04:03:03.814 INFO:tasks.rados.rados.0.smithi122.stdout:3705: expect (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:03:04.000 INFO:tasks.rados.rados.0.smithi122.stdout:3706: finishing write tid 2 to smithi12223906-18 2024-09-15T04:03:04.000 INFO:tasks.rados.rados.0.smithi122.stdout:3706: finishing write tid 3 to smithi12223906-18 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3706: finishing write tid 4 to smithi12223906-18 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3706: finishing write tid 5 to smithi12223906-18 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3176 (ObjNum 1304 snap 431 seq_num 1304) dirty exists 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3706: left oid 18 (ObjNum 1304 snap 431 seq_num 1304) 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3705: done (2 left) 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3706: done (1 left) 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3707: done (0 left) 2024-09-15T04:03:04.001 INFO:tasks.rados.rados.0.smithi122.stdout:3708: snap_remove snap 382 2024-09-15T04:03:04.537 INFO:tasks.rados.rados.0.smithi122.stdout:3708: done (0 left) 2024-09-15T04:03:04.537 INFO:tasks.rados.rados.0.smithi122.stdout:3709: write oid 18 current snap is 431 2024-09-15T04:03:04.537 INFO:tasks.rados.rados.0.smithi122.stdout:3709: seq_num 1306 ranges {749942=571557,1840895=796442,3365252=584690} 2024-09-15T04:03:04.545 INFO:tasks.rados.rados.0.smithi122.stdout:3709: writing smithi12223906-18 from 749942 to 1321499 tid 1 2024-09-15T04:03:04.553 INFO:tasks.rados.rados.0.smithi122.stdout:3709: writing smithi12223906-18 from 1840895 to 2637337 tid 2 2024-09-15T04:03:04.562 INFO:tasks.rados.rados.0.smithi122.stdout:3709: writing smithi12223906-18 from 3365252 to 3949942 tid 3 2024-09-15T04:03:04.562 INFO:tasks.rados.rados.0.smithi122.stdout:3710: write oid 17 current snap is 431 2024-09-15T04:03:04.562 INFO:tasks.rados.rados.0.smithi122.stdout:3710: seq_num 1307 ranges {766749=646806,1899377=656453,3109028=609462} 2024-09-15T04:03:04.569 INFO:tasks.rados.rados.0.smithi122.stdout:3710: writing smithi12223906-17 from 766749 to 1413555 tid 1 2024-09-15T04:03:04.576 INFO:tasks.rados.rados.0.smithi122.stdout:3710: writing smithi12223906-17 from 1899377 to 2555830 tid 2 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3710: writing smithi12223906-17 from 3109028 to 3718490 tid 3 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3711: delete oid 32 current snap is 431 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3709: finishing write tid 1 to smithi12223906-18 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3709: finishing write tid 2 to smithi12223906-18 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3709: finishing write tid 3 to smithi12223906-18 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3709: finishing write tid 4 to smithi12223906-18 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3709: finishing write tid 5 to smithi12223906-18 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3184 (ObjNum 1306 snap 431 seq_num 1306) dirty exists 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3709: left oid 18 (ObjNum 1306 snap 431 seq_num 1306) 2024-09-15T04:03:04.583 INFO:tasks.rados.rados.0.smithi122.stdout:3710: finishing write tid 1 to smithi12223906-17 2024-09-15T04:03:04.584 INFO:tasks.rados.rados.0.smithi122.stdout:3710: finishing write tid 2 to smithi12223906-17 2024-09-15T04:03:04.588 INFO:tasks.rados.rados.0.smithi122.stdout:3709: done (2 left) 2024-09-15T04:03:04.588 INFO:tasks.rados.rados.0.smithi122.stdout:3711: done (1 left) 2024-09-15T04:03:04.588 INFO:tasks.rados.rados.0.smithi122.stdout:3712: write oid 33 current snap is 431 2024-09-15T04:03:04.588 INFO:tasks.rados.rados.0.smithi122.stdout:3712: seq_num 1309 ranges {400363=513657,1332694=660122} 2024-09-15T04:03:04.596 INFO:tasks.rados.rados.0.smithi122.stdout:3712: writing smithi12223906-33 from 400363 to 914020 tid 1 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3712: writing smithi12223906-33 from 1332694 to 1992816 tid 2 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3713: rollback oid 20 current snap is 431 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 20 to 380 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3710: finishing write tid 3 to smithi12223906-17 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3710: finishing write tid 4 to smithi12223906-17 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3710: finishing write tid 5 to smithi12223906-17 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 2904 (ObjNum 1307 snap 431 seq_num 1307) dirty exists 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3710: left oid 17 (ObjNum 1307 snap 431 seq_num 1307) 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3710: done (2 left) 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3714: write (excl) oid 4 current snap is 431 2024-09-15T04:03:04.604 INFO:tasks.rados.rados.0.smithi122.stdout:3714: seq_num 1310 ranges {417170=588906,1674823=342347} 2024-09-15T04:03:04.610 INFO:tasks.rados.rados.0.smithi122.stdout:3714: writing smithi12223906-4 from 417170 to 1006076 tid 1 2024-09-15T04:03:04.616 INFO:tasks.rados.rados.0.smithi122.stdout:3714: writing smithi12223906-4 from 1674823 to 2017170 tid 2 2024-09-15T04:03:04.616 INFO:tasks.rados.rados.0.smithi122.stdout:3715: snap_remove snap 366 2024-09-15T04:03:05.573 INFO:tasks.rados.rados.0.smithi122.stdout:3712: finishing write tid 1 to smithi12223906-33 2024-09-15T04:03:05.573 INFO:tasks.rados.rados.0.smithi122.stdout:3713: finishing rollback tid 0 to smithi12223906-20 2024-09-15T04:03:05.573 INFO:tasks.rados.rados.0.smithi122.stdout:3713: finishing rollback tid 1 to smithi12223906-20 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 1586 (ObjNum 369 snap 0 seq_num 1842672304) dirty dne 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:3712: finishing write tid 2 to smithi12223906-33 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:3712: finishing write tid 3 to smithi12223906-33 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:3712: finishing write tid 4 to smithi12223906-33 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 3196 (ObjNum 1309 snap 431 seq_num 1309) dirty exists 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:3712: left oid 33 (ObjNum 1309 snap 431 seq_num 1309) 2024-09-15T04:03:05.574 INFO:tasks.rados.rados.0.smithi122.stdout:3714: finishing write tid 1 to smithi12223906-4 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3714: finishing write tid 2 to smithi12223906-4 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3714: finishing write tid 3 to smithi12223906-4 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3714: finishing write tid 4 to smithi12223906-4 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2632 (ObjNum 1310 snap 431 seq_num 1310) dirty exists 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3714: left oid 4 (ObjNum 1310 snap 431 seq_num 1310) 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3712: done (3 left) 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3713: done (2 left) 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3714: done (1 left) 2024-09-15T04:03:05.575 INFO:tasks.rados.rados.0.smithi122.stdout:3715: done (0 left) 2024-09-15T04:03:05.576 INFO:tasks.rados.rados.0.smithi122.stdout:3716: snap_create 2024-09-15T04:03:06.569 INFO:tasks.rados.rados.0.smithi122.stdout:3716: done (0 left) 2024-09-15T04:03:06.569 INFO:tasks.rados.rados.0.smithi122.stdout:3717: read oid 30 snap -1 2024-09-15T04:03:06.569 INFO:tasks.rados.rados.0.smithi122.stdout:3717: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:03:06.569 INFO:tasks.rados.rados.0.smithi122.stdout:3718: delete oid 43 current snap is 432 2024-09-15T04:03:06.578 INFO:tasks.rados.rados.0.smithi122.stdout:3717: expect (ObjNum 1066 snap 345 seq_num 1066) 2024-09-15T04:03:06.699 INFO:tasks.rados.rados.0.smithi122.stdout:3717: done (1 left) 2024-09-15T04:03:06.700 INFO:tasks.rados.rados.0.smithi122.stdout:3718: done (0 left) 2024-09-15T04:03:06.700 INFO:tasks.rados.rados.0.smithi122.stdout:3719: write (excl) oid 41 current snap is 432 2024-09-15T04:03:06.700 INFO:tasks.rados.rados.0.smithi122.stdout:3719: seq_num 1312 ranges {450784=739404,1791787=258997} 2024-09-15T04:03:06.716 INFO:tasks.rados.rados.0.smithi122.stdout:3719: writing smithi12223906-41 from 450784 to 1190188 tid 1 2024-09-15T04:03:06.721 INFO:tasks.rados.rados.0.smithi122.stdout:3719: writing smithi12223906-41 from 1791787 to 2050784 tid 2 2024-09-15T04:03:06.721 INFO:tasks.rados.rados.0.smithi122.stdout:3720: write (excl) oid 18 current snap is 432 2024-09-15T04:03:06.721 INFO:tasks.rados.rados.0.smithi122.stdout:3720: seq_num 1313 ranges {467591=414653,1450269=617322} 2024-09-15T04:03:06.725 INFO:tasks.rados.rados.0.smithi122.stdout:3720: writing smithi12223906-18 from 467591 to 882244 tid 1 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3720: writing smithi12223906-18 from 1450269 to 2067591 tid 2 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3721: snap_create 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3719: finishing write tid 1 to smithi12223906-41 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3719: finishing write tid 2 to smithi12223906-41 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3719: finishing write tid 3 to smithi12223906-41 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3719: finishing write tid 4 to smithi12223906-41 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2913 (ObjNum 1312 snap 432 seq_num 1312) dirty exists 2024-09-15T04:03:06.733 INFO:tasks.rados.rados.0.smithi122.stdout:3719: left oid 41 (ObjNum 1312 snap 432 seq_num 1312) 2024-09-15T04:03:06.745 INFO:tasks.rados.rados.0.smithi122.stdout:3720: finishing write tid 1 to smithi12223906-18 2024-09-15T04:03:06.746 INFO:tasks.rados.rados.0.smithi122.stdout:3720: finishing write tid 2 to smithi12223906-18 2024-09-15T04:03:06.746 INFO:tasks.rados.rados.0.smithi122.stdout:3720: finishing write tid 3 to smithi12223906-18 2024-09-15T04:03:06.746 INFO:tasks.rados.rados.0.smithi122.stdout:3720: finishing write tid 4 to smithi12223906-18 2024-09-15T04:03:06.747 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3212 (ObjNum 1313 snap 432 seq_num 1313) dirty exists 2024-09-15T04:03:06.747 INFO:tasks.rados.rados.0.smithi122.stdout:3720: left oid 18 (ObjNum 1313 snap 432 seq_num 1313) 2024-09-15T04:03:07.584 INFO:tasks.rados.rados.0.smithi122.stdout:3719: done (2 left) 2024-09-15T04:03:07.584 INFO:tasks.rados.rados.0.smithi122.stdout:3720: done (1 left) 2024-09-15T04:03:07.584 INFO:tasks.rados.rados.0.smithi122.stdout:3721: done (0 left) 2024-09-15T04:03:07.584 INFO:tasks.rados.rados.0.smithi122.stdout:3722: snap_create 2024-09-15T04:03:08.585 INFO:tasks.rados.rados.0.smithi122.stdout:3722: done (0 left) 2024-09-15T04:03:08.585 INFO:tasks.rados.rados.0.smithi122.stdout:3723: copy_from oid 13 from oid 4 current snap is 434 2024-09-15T04:03:08.585 INFO:tasks.rados.rados.0.smithi122.stdout:3724: copy_from oid 38 from oid 47 current snap is 434 2024-09-15T04:03:08.585 INFO:tasks.rados.rados.0.smithi122.stdout:3725: delete oid 25 current snap is 434 2024-09-15T04:03:08.606 INFO:tasks.rados.rados.0.smithi122.stdout:3723: finishing copy_from racing read to smithi12223906-13 2024-09-15T04:03:08.607 INFO:tasks.rados.rados.0.smithi122.stdout:3725: done (2 left) 2024-09-15T04:03:08.607 INFO:tasks.rados.rados.0.smithi122.stdout:3726: delete oid 22 current snap is 434 2024-09-15T04:03:08.615 INFO:tasks.rados.rados.0.smithi122.stdout:3724: finishing copy_from racing read to smithi12223906-38 2024-09-15T04:03:08.615 INFO:tasks.rados.rados.0.smithi122.stdout:3723: finishing copy_from to smithi12223906-13 2024-09-15T04:03:08.615 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 13 v 2632 (ObjNum 1310 snap 431 seq_num 1310) dirty exists 2024-09-15T04:03:08.617 INFO:tasks.rados.rados.0.smithi122.stdout:3724: finishing copy_from to smithi12223906-38 2024-09-15T04:03:08.617 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 2623 (ObjNum 1273 snap 420 seq_num 1273) dirty exists 2024-09-15T04:03:08.623 INFO:tasks.rados.rados.0.smithi122.stdout:3723: done (2 left) 2024-09-15T04:03:08.623 INFO:tasks.rados.rados.0.smithi122.stdout:3724: done (1 left) 2024-09-15T04:03:08.623 INFO:tasks.rados.rados.0.smithi122.stdout:3726: done (0 left) 2024-09-15T04:03:08.623 INFO:tasks.rados.rados.0.smithi122.stdout:3727: delete oid 32 current snap is 434 2024-09-15T04:03:08.629 INFO:tasks.rados.rados.0.smithi122.stdout:3727: done (0 left) 2024-09-15T04:03:08.629 INFO:tasks.rados.rados.0.smithi122.stdout:3728: delete oid 12 current snap is 434 2024-09-15T04:03:08.632 INFO:tasks.rados.rados.0.smithi122.stdout:3728: done (0 left) 2024-09-15T04:03:08.632 INFO:tasks.rados.rados.0.smithi122.stdout:3729: read oid 43 snap -1 2024-09-15T04:03:08.632 INFO:tasks.rados.rados.0.smithi122.stdout:3729: expect deleted 2024-09-15T04:03:08.632 INFO:tasks.rados.rados.0.smithi122.stdout:3730: snap_remove snap 411 2024-09-15T04:03:09.540 INFO:tasks.rados.rados.0.smithi122.stdout:3730: done (1 left) 2024-09-15T04:03:09.540 INFO:tasks.rados.rados.0.smithi122.stdout:3731: write (excl) oid 38 current snap is 434 2024-09-15T04:03:09.540 INFO:tasks.rados.rados.0.smithi122.stdout:3731: seq_num 1318 ranges {551626=507251,1626326=418456} 2024-09-15T04:03:09.550 INFO:tasks.rados.rados.0.smithi122.stdout:3731: writing smithi12223906-38 from 551626 to 1058877 tid 1 2024-09-15T04:03:09.558 INFO:tasks.rados.rados.0.smithi122.stdout:3731: writing smithi12223906-38 from 1626326 to 2044782 tid 2 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3729: done (1 left) 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3732: read oid 46 snap 424 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3732: expect (ObjNum 1272 snap 420 seq_num 1272) 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3733: copy_from oid 7 from oid 21 current snap is 434 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3734: copy_from oid 50 from oid 32 current snap is 434 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3735: copy_from oid 49 from oid 26 current snap is 434 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3736: write (excl) oid 39 current snap is 434 2024-09-15T04:03:09.559 INFO:tasks.rados.rados.0.smithi122.stdout:3736: seq_num 1319 ranges {568433=582500,1684808=483625} 2024-09-15T04:03:09.566 INFO:tasks.rados.rados.0.smithi122.stdout:3736: writing smithi12223906-39 from 568433 to 1150933 tid 1 2024-09-15T04:03:09.572 INFO:tasks.rados.rados.0.smithi122.stdout:3736: writing smithi12223906-39 from 1684808 to 2168433 tid 2 2024-09-15T04:03:09.572 INFO:tasks.rados.rados.0.smithi122.stdout:3737: delete oid 10 current snap is 434 2024-09-15T04:03:09.573 INFO:tasks.rados.rados.0.smithi122.stdout:3733: finishing copy_from racing read to smithi12223906-7 2024-09-15T04:03:09.578 INFO:tasks.rados.rados.0.smithi122.stdout:3732: expect (ObjNum 1272 snap 420 seq_num 1272) 2024-09-15T04:03:09.760 INFO:tasks.rados.rados.0.smithi122.stdout:3734: finishing copy_from to smithi12223906-50 2024-09-15T04:03:09.760 INFO:tasks.rados.rados.0.smithi122.stdout:3734: got expected ENOENT (src dne) 2024-09-15T04:03:09.760 INFO:tasks.rados.rados.0.smithi122.stdout:3734: finishing copy_from racing read to smithi12223906-50 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3735: finishing copy_from racing read to smithi12223906-49 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3731: finishing write tid 1 to smithi12223906-38 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3731: finishing write tid 2 to smithi12223906-38 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3731: finishing write tid 3 to smithi12223906-38 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3733: finishing copy_from to smithi12223906-7 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3731: finishing write tid 4 to smithi12223906-38 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 3081 (ObjNum 1318 snap 434 seq_num 1318) dirty exists 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3731: left oid 38 (ObjNum 1318 snap 434 seq_num 1318) 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3735: finishing copy_from to smithi12223906-49 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 2166 (ObjNum 1287 snap 426 seq_num 1287) dirty exists 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3736: finishing write tid 1 to smithi12223906-39 2024-09-15T04:03:09.761 INFO:tasks.rados.rados.0.smithi122.stdout:3736: finishing write tid 2 to smithi12223906-39 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3736: finishing write tid 3 to smithi12223906-39 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3731: done (6 left) 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3732: done (5 left) 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3733: done (4 left) 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3734: done (3 left) 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3735: done (2 left) 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3737: done (1 left) 2024-09-15T04:03:09.762 INFO:tasks.rados.rados.0.smithi122.stdout:3738: snap_remove snap 335 2024-09-15T04:03:10.555 INFO:tasks.rados.rados.0.smithi122.stdout:3738: done (1 left) 2024-09-15T04:03:10.556 INFO:tasks.rados.rados.0.smithi122.stdout:3739: write (excl) oid 19 current snap is 434 2024-09-15T04:03:10.556 INFO:tasks.rados.rados.0.smithi122.stdout:3739: seq_num 1321 ranges {602047=732998,1801772=400275} 2024-09-15T04:03:10.567 INFO:tasks.rados.rados.0.smithi122.stdout:3739: writing smithi12223906-19 from 602047 to 1335045 tid 1 2024-09-15T04:03:10.573 INFO:tasks.rados.rados.0.smithi122.stdout:3739: writing smithi12223906-19 from 1801772 to 2202047 tid 2 2024-09-15T04:03:10.573 INFO:tasks.rados.rados.0.smithi122.stdout:3740: write oid 8 current snap is 434 2024-09-15T04:03:10.574 INFO:tasks.rados.rados.0.smithi122.stdout:3736: finishing write tid 4 to smithi12223906-39 2024-09-15T04:03:10.574 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 3225 (ObjNum 1319 snap 434 seq_num 1319) dirty exists 2024-09-15T04:03:10.574 INFO:tasks.rados.rados.0.smithi122.stdout:3736: left oid 39 (ObjNum 1319 snap 434 seq_num 1319) 2024-09-15T04:03:10.574 INFO:tasks.rados.rados.0.smithi122.stdout:3740: seq_num 1322 ranges {618854=408247,1743901=425794} 2024-09-15T04:03:10.579 INFO:tasks.rados.rados.0.smithi122.stdout:3740: writing smithi12223906-8 from 618854 to 1027101 tid 1 2024-09-15T04:03:10.587 INFO:tasks.rados.rados.0.smithi122.stdout:3740: writing smithi12223906-8 from 1743901 to 2169695 tid 2 2024-09-15T04:03:10.587 INFO:tasks.rados.rados.0.smithi122.stdout:3736: done (2 left) 2024-09-15T04:03:10.587 INFO:tasks.rados.rados.0.smithi122.stdout:3741: snap_remove snap 432 2024-09-15T04:03:11.564 INFO:tasks.rados.rados.0.smithi122.stdout:3741: done (2 left) 2024-09-15T04:03:11.564 INFO:tasks.rados.rados.0.smithi122.stdout:3742: read oid 36 snap 423 2024-09-15T04:03:11.564 INFO:tasks.rados.rados.0.smithi122.stdout:3742: expect (ObjNum 1259 snap 417 seq_num 1259) 2024-09-15T04:03:11.564 INFO:tasks.rados.rados.0.smithi122.stdout:3743: snap_remove snap 358 2024-09-15T04:03:12.568 INFO:tasks.rados.rados.0.smithi122.stdout:3743: done (3 left) 2024-09-15T04:03:12.568 INFO:tasks.rados.rados.0.smithi122.stdout:3744: write (excl) oid 48 current snap is 434 2024-09-15T04:03:12.568 INFO:tasks.rados.rados.0.smithi122.stdout:3744: seq_num 1323 ranges {635661=599849,1918736=316925} 2024-09-15T04:03:12.579 INFO:tasks.rados.rados.0.smithi122.stdout:3744: writing smithi12223906-48 from 635661 to 1235510 tid 1 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3744: writing smithi12223906-48 from 1918736 to 2235661 tid 2 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3745: delete oid 44 current snap is 434 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3739: finishing write tid 1 to smithi12223906-19 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3739: finishing write tid 2 to smithi12223906-19 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3739: finishing write tid 3 to smithi12223906-19 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3739: finishing write tid 4 to smithi12223906-19 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 3086 (ObjNum 1321 snap 434 seq_num 1321) dirty exists 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3739: left oid 19 (ObjNum 1321 snap 434 seq_num 1321) 2024-09-15T04:03:12.586 INFO:tasks.rados.rados.0.smithi122.stdout:3740: finishing write tid 1 to smithi12223906-8 2024-09-15T04:03:12.587 INFO:tasks.rados.rados.0.smithi122.stdout:3740: finishing write tid 2 to smithi12223906-8 2024-09-15T04:03:12.587 INFO:tasks.rados.rados.0.smithi122.stdout:3740: finishing write tid 3 to smithi12223906-8 2024-09-15T04:03:12.587 INFO:tasks.rados.rados.0.smithi122.stdout:3740: finishing write tid 4 to smithi12223906-8 2024-09-15T04:03:12.587 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 3084 (ObjNum 1322 snap 434 seq_num 1322) dirty exists 2024-09-15T04:03:12.587 INFO:tasks.rados.rados.0.smithi122.stdout:3740: left oid 8 (ObjNum 1322 snap 434 seq_num 1322) 2024-09-15T04:03:12.587 INFO:tasks.rados.rados.0.smithi122.stdout:3742: expect (ObjNum 1259 snap 417 seq_num 1259) 2024-09-15T04:03:12.726 INFO:tasks.rados.rados.0.smithi122.stdout:3744: finishing write tid 1 to smithi12223906-48 2024-09-15T04:03:12.726 INFO:tasks.rados.rados.0.smithi122.stdout:3744: finishing write tid 2 to smithi12223906-48 2024-09-15T04:03:12.727 INFO:tasks.rados.rados.0.smithi122.stdout:3744: finishing write tid 3 to smithi12223906-48 2024-09-15T04:03:12.727 INFO:tasks.rados.rados.0.smithi122.stdout:3744: finishing write tid 4 to smithi12223906-48 2024-09-15T04:03:12.727 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2155 (ObjNum 1323 snap 434 seq_num 1323) dirty exists 2024-09-15T04:03:12.727 INFO:tasks.rados.rados.0.smithi122.stdout:3744: left oid 48 (ObjNum 1323 snap 434 seq_num 1323) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3739: done (4 left) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3740: done (3 left) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3742: done (2 left) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3744: done (1 left) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3745: done (0 left) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3746: read oid 50 snap -1 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3746: expect (ObjNum 1271 snap 419 seq_num 1271) 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3747: write (excl) oid 9 current snap is 434 2024-09-15T04:03:12.735 INFO:tasks.rados.rados.0.smithi122.stdout:3747: seq_num 1325 ranges {669275=750347,2035700=233575} 2024-09-15T04:03:12.752 INFO:tasks.rados.rados.0.smithi122.stdout:3747: writing smithi12223906-9 from 669275 to 1419622 tid 1 2024-09-15T04:03:12.757 INFO:tasks.rados.rados.0.smithi122.stdout:3747: writing smithi12223906-9 from 2035700 to 2269275 tid 2 2024-09-15T04:03:12.757 INFO:tasks.rados.rados.0.smithi122.stdout:3748: rollback oid 42 current snap is 434 2024-09-15T04:03:12.757 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 42 to 409 2024-09-15T04:03:12.757 INFO:tasks.rados.rados.0.smithi122.stdout:3746: expect (ObjNum 1271 snap 419 seq_num 1271) 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3747: finishing write tid 1 to smithi12223906-9 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3747: finishing write tid 2 to smithi12223906-9 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3747: finishing write tid 3 to smithi12223906-9 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3747: finishing write tid 4 to smithi12223906-9 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2499 (ObjNum 1325 snap 434 seq_num 1325) dirty exists 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3747: left oid 9 (ObjNum 1325 snap 434 seq_num 1325) 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3748: finishing rollback tid 0 to smithi12223906-42 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3748: finishing rollback tid 1 to smithi12223906-42 2024-09-15T04:03:12.967 INFO:tasks.rados.rados.0.smithi122.stdout:3748: finishing rollback tid 2 to smithi12223906-42 2024-09-15T04:03:12.968 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 42 v 3253 (ObjNum 1186 snap 390 seq_num 1186) dirty exists 2024-09-15T04:03:12.968 INFO:tasks.rados.rados.0.smithi122.stdout:3746: done (2 left) 2024-09-15T04:03:12.968 INFO:tasks.rados.rados.0.smithi122.stdout:3747: done (1 left) 2024-09-15T04:03:12.968 INFO:tasks.rados.rados.0.smithi122.stdout:3748: done (0 left) 2024-09-15T04:03:12.968 INFO:tasks.rados.rados.0.smithi122.stdout:3749: write (excl) oid 46 current snap is 434 2024-09-15T04:03:12.968 INFO:tasks.rados.rados.0.smithi122.stdout:3749: seq_num 1326 ranges {686082=425596,1577829=433132} 2024-09-15T04:03:12.982 INFO:tasks.rados.rados.0.smithi122.stdout:3749: writing smithi12223906-46 from 686082 to 1111678 tid 1 2024-09-15T04:03:12.988 INFO:tasks.rados.rados.0.smithi122.stdout:3749: writing smithi12223906-46 from 1577829 to 2010961 tid 2 2024-09-15T04:03:12.988 INFO:tasks.rados.rados.0.smithi122.stdout:3750: rollback oid 40 current snap is 434 2024-09-15T04:03:12.989 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 40 to 393 2024-09-15T04:03:12.989 INFO:tasks.rados.rados.0.smithi122.stdout:3751: write (excl) oid 12 current snap is 434 2024-09-15T04:03:12.989 INFO:tasks.rados.rados.0.smithi122.stdout:3751: seq_num 1327 ranges {702889=500845,1636311=576790} 2024-09-15T04:03:12.994 INFO:tasks.rados.rados.0.smithi122.stdout:3751: writing smithi12223906-12 from 702889 to 1203734 tid 1 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3751: writing smithi12223906-12 from 1636311 to 2213101 tid 2 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3752: snap_create 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3751: finishing write tid 1 to smithi12223906-12 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3749: finishing write tid 1 to smithi12223906-46 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3749: finishing write tid 2 to smithi12223906-46 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3750: finishing rollback tid 0 to smithi12223906-40 2024-09-15T04:03:13.001 INFO:tasks.rados.rados.0.smithi122.stdout:3750: finishing rollback tid 1 to smithi12223906-40 2024-09-15T04:03:13.002 INFO:tasks.rados.rados.0.smithi122.stdout:3750: finishing rollback tid 2 to smithi12223906-40 2024-09-15T04:03:13.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2916 (ObjNum 1160 snap 381 seq_num 1160) dirty exists 2024-09-15T04:03:13.002 INFO:tasks.rados.rados.0.smithi122.stdout:3749: finishing write tid 3 to smithi12223906-46 2024-09-15T04:03:13.002 INFO:tasks.rados.rados.0.smithi122.stdout:3749: finishing write tid 4 to smithi12223906-46 2024-09-15T04:03:13.002 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 3089 (ObjNum 1326 snap 434 seq_num 1326) dirty exists 2024-09-15T04:03:13.002 INFO:tasks.rados.rados.0.smithi122.stdout:3749: left oid 46 (ObjNum 1326 snap 434 seq_num 1326) 2024-09-15T04:03:13.004 INFO:tasks.rados.rados.0.smithi122.stdout:3751: finishing write tid 2 to smithi12223906-12 2024-09-15T04:03:13.004 INFO:tasks.rados.rados.0.smithi122.stdout:3751: finishing write tid 3 to smithi12223906-12 2024-09-15T04:03:13.005 INFO:tasks.rados.rados.0.smithi122.stdout:3751: finishing write tid 4 to smithi12223906-12 2024-09-15T04:03:13.005 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 2144 (ObjNum 1327 snap 434 seq_num 1327) dirty exists 2024-09-15T04:03:13.005 INFO:tasks.rados.rados.0.smithi122.stdout:3751: left oid 12 (ObjNum 1327 snap 434 seq_num 1327) 2024-09-15T04:03:13.583 INFO:tasks.rados.rados.0.smithi122.stdout:3749: done (3 left) 2024-09-15T04:03:13.583 INFO:tasks.rados.rados.0.smithi122.stdout:3750: done (2 left) 2024-09-15T04:03:13.583 INFO:tasks.rados.rados.0.smithi122.stdout:3751: done (1 left) 2024-09-15T04:03:13.583 INFO:tasks.rados.rados.0.smithi122.stdout:3752: done (0 left) 2024-09-15T04:03:13.583 INFO:tasks.rados.rados.0.smithi122.stdout:3753: snap_remove snap 418 2024-09-15T04:03:14.542 INFO:tasks.rados.rados.0.smithi122.stdout:3753: done (0 left) 2024-09-15T04:03:14.542 INFO:tasks.rados.rados.0.smithi122.stdout:3754: write (excl) oid 31 current snap is 435 2024-09-15T04:03:14.542 INFO:tasks.rados.rados.0.smithi122.stdout:3754: seq_num 1328 ranges {719696=576094,2094793=224903} 2024-09-15T04:03:14.552 INFO:tasks.rados.rados.0.smithi122.stdout:3754: writing smithi12223906-31 from 719696 to 1295790 tid 1 2024-09-15T04:03:14.559 INFO:tasks.rados.rados.0.smithi122.stdout:3754: writing smithi12223906-31 from 2094793 to 2319696 tid 2 2024-09-15T04:03:14.559 INFO:tasks.rados.rados.0.smithi122.stdout:3755: write oid 15 current snap is 435 2024-09-15T04:03:14.559 INFO:tasks.rados.rados.0.smithi122.stdout:3755: seq_num 1329 ranges {736503=651343,2153275=183228} 2024-09-15T04:03:14.567 INFO:tasks.rados.rados.0.smithi122.stdout:3755: writing smithi12223906-15 from 736503 to 1387846 tid 1 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:3755: writing smithi12223906-15 from 2153275 to 2336503 tid 2 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:3756: copy_from oid 17 from oid 7 current snap is 435 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:3754: finishing write tid 1 to smithi12223906-31 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:3754: finishing write tid 2 to smithi12223906-31 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:3754: finishing write tid 3 to smithi12223906-31 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:3754: finishing write tid 4 to smithi12223906-31 2024-09-15T04:03:14.574 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 2635 (ObjNum 1328 snap 435 seq_num 1328) dirty exists 2024-09-15T04:03:14.575 INFO:tasks.rados.rados.0.smithi122.stdout:3754: left oid 31 (ObjNum 1328 snap 435 seq_num 1328) 2024-09-15T04:03:14.575 INFO:tasks.rados.rados.0.smithi122.stdout:3755: finishing write tid 1 to smithi12223906-15 2024-09-15T04:03:14.575 INFO:tasks.rados.rados.0.smithi122.stdout:3754: done (2 left) 2024-09-15T04:03:14.575 INFO:tasks.rados.rados.0.smithi122.stdout:3757: write oid 30 current snap is 435 2024-09-15T04:03:14.575 INFO:tasks.rados.rados.0.smithi122.stdout:3757: seq_num 1330 ranges {753310=726592,2095404=257906} 2024-09-15T04:03:14.581 INFO:tasks.rados.rados.0.smithi122.stdout:3757: writing smithi12223906-30 from 753310 to 1479902 tid 1 2024-09-15T04:03:14.587 INFO:tasks.rados.rados.0.smithi122.stdout:3757: writing smithi12223906-30 from 2095404 to 2353310 tid 2 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3758: read oid 28 snap 359 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3758: expect (ObjNum 1039 snap 330 seq_num 1039) 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3755: finishing write tid 2 to smithi12223906-15 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3755: finishing write tid 3 to smithi12223906-15 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3755: finishing write tid 4 to smithi12223906-15 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1942 (ObjNum 1329 snap 435 seq_num 1329) dirty exists 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3755: left oid 15 (ObjNum 1329 snap 435 seq_num 1329) 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3756: finishing copy_from racing read to smithi12223906-17 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3755: done (3 left) 2024-09-15T04:03:14.588 INFO:tasks.rados.rados.0.smithi122.stdout:3759: snap_create 2024-09-15T04:03:14.589 INFO:tasks.rados.rados.0.smithi122.stdout:3756: finishing copy_from to smithi12223906-17 2024-09-15T04:03:14.589 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:03:14.590 INFO:tasks.rados.rados.0.smithi122.stdout:3757: finishing write tid 1 to smithi12223906-30 2024-09-15T04:03:14.594 INFO:tasks.rados.rados.0.smithi122.stdout:3757: finishing write tid 2 to smithi12223906-30 2024-09-15T04:03:14.594 INFO:tasks.rados.rados.0.smithi122.stdout:3757: finishing write tid 3 to smithi12223906-30 2024-09-15T04:03:14.595 INFO:tasks.rados.rados.0.smithi122.stdout:3757: finishing write tid 4 to smithi12223906-30 2024-09-15T04:03:14.595 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 30 v 2919 (ObjNum 1330 snap 435 seq_num 1330) dirty exists 2024-09-15T04:03:14.595 INFO:tasks.rados.rados.0.smithi122.stdout:3757: left oid 30 (ObjNum 1330 snap 435 seq_num 1330) 2024-09-15T04:03:14.598 INFO:tasks.rados.rados.0.smithi122.stdout:3758: expect (ObjNum 1039 snap 330 seq_num 1039) 2024-09-15T04:03:15.595 INFO:tasks.rados.rados.0.smithi122.stdout:3756: done (3 left) 2024-09-15T04:03:15.595 INFO:tasks.rados.rados.0.smithi122.stdout:3757: done (2 left) 2024-09-15T04:03:15.595 INFO:tasks.rados.rados.0.smithi122.stdout:3758: done (1 left) 2024-09-15T04:03:15.596 INFO:tasks.rados.rados.0.smithi122.stdout:3759: done (0 left) 2024-09-15T04:03:15.596 INFO:tasks.rados.rados.0.smithi122.stdout:3760: rollback oid 19 current snap is 436 2024-09-15T04:03:15.596 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 19 to 336 2024-09-15T04:03:15.596 INFO:tasks.rados.rados.0.smithi122.stdout:3761: write (excl) oid 44 current snap is 436 2024-09-15T04:03:15.596 INFO:tasks.rados.rados.0.smithi122.stdout:3761: seq_num 1331 ranges {770117=518194,1870239=499878} 2024-09-15T04:03:15.606 INFO:tasks.rados.rados.0.smithi122.stdout:3761: writing smithi12223906-44 from 770117 to 1288311 tid 1 2024-09-15T04:03:15.613 INFO:tasks.rados.rados.0.smithi122.stdout:3761: writing smithi12223906-44 from 1870239 to 2370117 tid 2 2024-09-15T04:03:15.613 INFO:tasks.rados.rados.0.smithi122.stdout:3762: rollback oid 41 current snap is 436 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 359 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3760: finishing rollback tid 0 to smithi12223906-19 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3760: finishing rollback tid 1 to smithi12223906-19 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3760: finishing rollback tid 2 to smithi12223906-19 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 3092 (ObjNum 848 snap 276 seq_num 848) dirty exists 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3761: finishing write tid 1 to smithi12223906-44 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3760: done (2 left) 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3763: read oid 23 snap -1 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3763: expect deleted 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3764: read oid 27 snap -1 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3764: expect deleted 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3765: copy_from oid 46 from oid 21 current snap is 436 2024-09-15T04:03:15.614 INFO:tasks.rados.rados.0.smithi122.stdout:3766: read oid 31 snap 364 2024-09-15T04:03:15.615 INFO:tasks.rados.rados.0.smithi122.stdout:3766: expect (ObjNum 1088 snap 355 seq_num 1088) 2024-09-15T04:03:15.615 INFO:tasks.rados.rados.0.smithi122.stdout:3767: delete oid 49 current snap is 436 2024-09-15T04:03:15.620 INFO:tasks.rados.rados.0.smithi122.stdout:3762: finishing rollback tid 0 to smithi12223906-41 2024-09-15T04:03:15.620 INFO:tasks.rados.rados.0.smithi122.stdout:3767: done (6 left) 2024-09-15T04:03:15.620 INFO:tasks.rados.rados.0.smithi122.stdout:3768: read oid 12 snap 370 2024-09-15T04:03:15.620 INFO:tasks.rados.rados.0.smithi122.stdout:3768: expect deleted 2024-09-15T04:03:15.621 INFO:tasks.rados.rados.0.smithi122.stdout:3769: snap_create 2024-09-15T04:03:15.622 INFO:tasks.rados.rados.0.smithi122.stdout:3762: finishing rollback tid 1 to smithi12223906-41 2024-09-15T04:03:15.624 INFO:tasks.rados.rados.0.smithi122.stdout:3761: finishing write tid 2 to smithi12223906-44 2024-09-15T04:03:15.624 INFO:tasks.rados.rados.0.smithi122.stdout:3761: finishing write tid 3 to smithi12223906-44 2024-09-15T04:03:15.624 INFO:tasks.rados.rados.0.smithi122.stdout:3761: finishing write tid 4 to smithi12223906-44 2024-09-15T04:03:15.624 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 3260 (ObjNum 1331 snap 436 seq_num 1331) dirty exists 2024-09-15T04:03:15.624 INFO:tasks.rados.rados.0.smithi122.stdout:3761: left oid 44 (ObjNum 1331 snap 436 seq_num 1331) 2024-09-15T04:03:15.624 INFO:tasks.rados.rados.0.smithi122.stdout:3762: finishing rollback tid 2 to smithi12223906-41 2024-09-15T04:03:15.625 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2922 (ObjNum 1050 snap 335 seq_num 1050) dirty exists 2024-09-15T04:03:15.629 INFO:tasks.rados.rados.0.smithi122.stdout:3766: expect (ObjNum 1088 snap 355 seq_num 1088) 2024-09-15T04:03:15.707 INFO:tasks.rados.rados.0.smithi122.stdout:3765: finishing copy_from racing read to smithi12223906-46 2024-09-15T04:03:15.707 INFO:tasks.rados.rados.0.smithi122.stdout:3765: finishing copy_from to smithi12223906-46 2024-09-15T04:03:15.707 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:03:16.603 INFO:tasks.rados.rados.0.smithi122.stdout:3761: done (7 left) 2024-09-15T04:03:16.603 INFO:tasks.rados.rados.0.smithi122.stdout:3762: done (6 left) 2024-09-15T04:03:16.603 INFO:tasks.rados.rados.0.smithi122.stdout:3763: done (5 left) 2024-09-15T04:03:16.603 INFO:tasks.rados.rados.0.smithi122.stdout:3764: done (4 left) 2024-09-15T04:03:16.604 INFO:tasks.rados.rados.0.smithi122.stdout:3765: done (3 left) 2024-09-15T04:03:16.604 INFO:tasks.rados.rados.0.smithi122.stdout:3766: done (2 left) 2024-09-15T04:03:16.604 INFO:tasks.rados.rados.0.smithi122.stdout:3768: done (1 left) 2024-09-15T04:03:16.604 INFO:tasks.rados.rados.0.smithi122.stdout:3769: done (0 left) 2024-09-15T04:03:16.604 INFO:tasks.rados.rados.0.smithi122.stdout:3770: write oid 50 current snap is 437 2024-09-15T04:03:16.604 INFO:tasks.rados.rados.0.smithi122.stdout:3770: seq_num 1333 ranges {403731=668692,1587203=587797} 2024-09-15T04:03:16.613 INFO:tasks.rados.rados.0.smithi122.stdout:3770: writing smithi12223906-50 from 403731 to 1072423 tid 1 2024-09-15T04:03:16.619 INFO:tasks.rados.rados.0.smithi122.stdout:3770: writing smithi12223906-50 from 1587203 to 2175000 tid 2 2024-09-15T04:03:16.619 INFO:tasks.rados.rados.0.smithi122.stdout:3771: delete oid 46 current snap is 437 2024-09-15T04:03:16.626 INFO:tasks.rados.rados.0.smithi122.stdout:3771: done (1 left) 2024-09-15T04:03:16.626 INFO:tasks.rados.rados.0.smithi122.stdout:3772: read oid 29 snap 381 2024-09-15T04:03:16.626 INFO:tasks.rados.rados.0.smithi122.stdout:3772: expect (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T04:03:16.626 INFO:tasks.rados.rados.0.smithi122.stdout:3773: rollback oid 16 current snap is 437 2024-09-15T04:03:16.626 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 383 2024-09-15T04:03:16.626 INFO:tasks.rados.rados.0.smithi122.stdout:3774: snap_create 2024-09-15T04:03:16.629 INFO:tasks.rados.rados.0.smithi122.stdout:3770: finishing write tid 1 to smithi12223906-50 2024-09-15T04:03:16.629 INFO:tasks.rados.rados.0.smithi122.stdout:3770: finishing write tid 2 to smithi12223906-50 2024-09-15T04:03:16.630 INFO:tasks.rados.rados.0.smithi122.stdout:3770: finishing write tid 3 to smithi12223906-50 2024-09-15T04:03:16.630 INFO:tasks.rados.rados.0.smithi122.stdout:3770: finishing write tid 4 to smithi12223906-50 2024-09-15T04:03:16.630 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 3264 (ObjNum 1333 snap 437 seq_num 1333) dirty exists 2024-09-15T04:03:16.631 INFO:tasks.rados.rados.0.smithi122.stdout:3770: left oid 50 (ObjNum 1333 snap 437 seq_num 1333) 2024-09-15T04:03:16.636 INFO:tasks.rados.rados.0.smithi122.stdout:3772: expect (ObjNum 1095 snap 356 seq_num 1095) 2024-09-15T04:03:16.754 INFO:tasks.rados.rados.0.smithi122.stdout:3773: finishing rollback tid 0 to smithi12223906-16 2024-09-15T04:03:16.754 INFO:tasks.rados.rados.0.smithi122.stdout:3773: finishing rollback tid 1 to smithi12223906-16 2024-09-15T04:03:16.755 INFO:tasks.rados.rados.0.smithi122.stdout:3773: finishing rollback tid 2 to smithi12223906-16 2024-09-15T04:03:16.755 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1945 (ObjNum 1153 snap 380 seq_num 1153) dirty exists 2024-09-15T04:03:17.612 INFO:tasks.rados.rados.0.smithi122.stdout:3770: done (3 left) 2024-09-15T04:03:17.612 INFO:tasks.rados.rados.0.smithi122.stdout:3772: done (2 left) 2024-09-15T04:03:17.612 INFO:tasks.rados.rados.0.smithi122.stdout:3773: done (1 left) 2024-09-15T04:03:17.612 INFO:tasks.rados.rados.0.smithi122.stdout:3774: done (0 left) 2024-09-15T04:03:17.612 INFO:tasks.rados.rados.0.smithi122.stdout:3775: delete oid 22 current snap is 438 2024-09-15T04:03:17.616 INFO:tasks.rados.rados.0.smithi122.stdout:3775: done (0 left) 2024-09-15T04:03:17.617 INFO:tasks.rados.rados.0.smithi122.stdout:3776: snap_create 2024-09-15T04:03:18.621 INFO:tasks.rados.rados.0.smithi122.stdout:3776: done (0 left) 2024-09-15T04:03:18.621 INFO:tasks.rados.rados.0.smithi122.stdout:3777: snap_create 2024-09-15T04:03:19.548 INFO:tasks.rados.rados.0.smithi122.stdout:3777: done (0 left) 2024-09-15T04:03:19.549 INFO:tasks.rados.rados.0.smithi122.stdout:3778: read oid 1 snap 431 2024-09-15T04:03:19.549 INFO:tasks.rados.rados.0.smithi122.stdout:3778: expect deleted 2024-09-15T04:03:19.549 INFO:tasks.rados.rados.0.smithi122.stdout:3779: delete oid 33 current snap is 440 2024-09-15T04:03:19.554 INFO:tasks.rados.rados.0.smithi122.stdout:3778: done (1 left) 2024-09-15T04:03:19.554 INFO:tasks.rados.rados.0.smithi122.stdout:3779: done (0 left) 2024-09-15T04:03:19.555 INFO:tasks.rados.rados.0.smithi122.stdout:3780: copy_from oid 46 from oid 40 current snap is 440 2024-09-15T04:03:19.555 INFO:tasks.rados.rados.0.smithi122.stdout:3781: delete oid 8 current snap is 440 2024-09-15T04:03:19.561 INFO:tasks.rados.rados.0.smithi122.stdout:3781: done (1 left) 2024-09-15T04:03:19.562 INFO:tasks.rados.rados.0.smithi122.stdout:3782: read oid 5 snap -1 2024-09-15T04:03:19.562 INFO:tasks.rados.rados.0.smithi122.stdout:3782: expect deleted 2024-09-15T04:03:19.562 INFO:tasks.rados.rados.0.smithi122.stdout:3783: read oid 4 snap -1 2024-09-15T04:03:19.562 INFO:tasks.rados.rados.0.smithi122.stdout:3783: expect (ObjNum 1310 snap 431 seq_num 1310) 2024-09-15T04:03:19.562 INFO:tasks.rados.rados.0.smithi122.stdout:3784: write (excl) oid 16 current snap is 440 2024-09-15T04:03:19.563 INFO:tasks.rados.rados.0.smithi122.stdout:3784: seq_num 1338 ranges {487766=644937,1646907=738793} 2024-09-15T04:03:19.568 INFO:tasks.rados.rados.0.smithi122.stdout:3784: writing smithi12223906-16 from 487766 to 1132703 tid 1 2024-09-15T04:03:19.577 INFO:tasks.rados.rados.0.smithi122.stdout:3784: writing smithi12223906-16 from 1646907 to 2385700 tid 2 2024-09-15T04:03:19.577 INFO:tasks.rados.rados.0.smithi122.stdout:3785: write oid 39 current snap is 440 2024-09-15T04:03:19.578 INFO:tasks.rados.rados.0.smithi122.stdout:3785: seq_num 1339 ranges {504573=436539,1421742=598804,2437505=67068} 2024-09-15T04:03:19.582 INFO:tasks.rados.rados.0.smithi122.stdout:3785: writing smithi12223906-39 from 504573 to 941112 tid 1 2024-09-15T04:03:19.588 INFO:tasks.rados.rados.0.smithi122.stdout:3785: writing smithi12223906-39 from 1421742 to 2020546 tid 2 2024-09-15T04:03:19.593 INFO:tasks.rados.rados.0.smithi122.stdout:3785: writing smithi12223906-39 from 2437505 to 2504573 tid 3 2024-09-15T04:03:19.594 INFO:tasks.rados.rados.0.smithi122.stdout:3786: read oid 26 snap -1 2024-09-15T04:03:19.594 INFO:tasks.rados.rados.0.smithi122.stdout:3786: expect (ObjNum 1287 snap 426 seq_num 1287) 2024-09-15T04:03:19.594 INFO:tasks.rados.rados.0.smithi122.stdout:3787: snap_remove snap 439 2024-09-15T04:03:20.557 INFO:tasks.rados.rados.0.smithi122.stdout:3787: done (6 left) 2024-09-15T04:03:20.557 INFO:tasks.rados.rados.0.smithi122.stdout:3788: snap_remove snap 419 2024-09-15T04:03:20.557 INFO:tasks.rados.rados.0.smithi122.stdout:3780: finishing copy_from racing read to smithi12223906-46 2024-09-15T04:03:20.557 INFO:tasks.rados.rados.0.smithi122.stdout:3783: expect (ObjNum 1310 snap 431 seq_num 1310) 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:3780: finishing copy_from to smithi12223906-46 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 2916 (ObjNum 1160 snap 381 seq_num 1160) dirty exists 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:3784: finishing write tid 1 to smithi12223906-16 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:3784: finishing write tid 2 to smithi12223906-16 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:3784: finishing write tid 3 to smithi12223906-16 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:3784: finishing write tid 4 to smithi12223906-16 2024-09-15T04:03:20.626 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1948 (ObjNum 1338 snap 440 seq_num 1338) dirty exists 2024-09-15T04:03:20.627 INFO:tasks.rados.rados.0.smithi122.stdout:3784: left oid 16 (ObjNum 1338 snap 440 seq_num 1338) 2024-09-15T04:03:20.627 INFO:tasks.rados.rados.0.smithi122.stdout:3785: finishing write tid 1 to smithi12223906-39 2024-09-15T04:03:20.627 INFO:tasks.rados.rados.0.smithi122.stdout:3786: expect (ObjNum 1287 snap 426 seq_num 1287) 2024-09-15T04:03:20.833 INFO:tasks.rados.rados.0.smithi122.stdout:3785: finishing write tid 2 to smithi12223906-39 2024-09-15T04:03:20.833 INFO:tasks.rados.rados.0.smithi122.stdout:3785: finishing write tid 3 to smithi12223906-39 2024-09-15T04:03:20.833 INFO:tasks.rados.rados.0.smithi122.stdout:3785: finishing write tid 4 to smithi12223906-39 2024-09-15T04:03:20.833 INFO:tasks.rados.rados.0.smithi122.stdout:3785: finishing write tid 5 to smithi12223906-39 2024-09-15T04:03:20.833 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 3268 (ObjNum 1339 snap 440 seq_num 1339) dirty exists 2024-09-15T04:03:20.833 INFO:tasks.rados.rados.0.smithi122.stdout:3785: left oid 39 (ObjNum 1339 snap 440 seq_num 1339) 2024-09-15T04:03:21.642 INFO:tasks.rados.rados.0.smithi122.stdout:3780: done (6 left) 2024-09-15T04:03:21.642 INFO:tasks.rados.rados.0.smithi122.stdout:3782: done (5 left) 2024-09-15T04:03:21.642 INFO:tasks.rados.rados.0.smithi122.stdout:3783: done (4 left) 2024-09-15T04:03:21.642 INFO:tasks.rados.rados.0.smithi122.stdout:3784: done (3 left) 2024-09-15T04:03:21.642 INFO:tasks.rados.rados.0.smithi122.stdout:3785: done (2 left) 2024-09-15T04:03:21.643 INFO:tasks.rados.rados.0.smithi122.stdout:3786: done (1 left) 2024-09-15T04:03:21.643 INFO:tasks.rados.rados.0.smithi122.stdout:3788: done (0 left) 2024-09-15T04:03:21.643 INFO:tasks.rados.rados.0.smithi122.stdout:3789: delete oid 47 current snap is 440 2024-09-15T04:03:21.656 INFO:tasks.rados.rados.0.smithi122.stdout:3789: done (0 left) 2024-09-15T04:03:21.656 INFO:tasks.rados.rados.0.smithi122.stdout:3790: write oid 4 current snap is 440 2024-09-15T04:03:21.656 INFO:tasks.rados.rados.0.smithi122.stdout:3790: seq_num 1341 ranges {538187=587037,1538706=602473} 2024-09-15T04:03:21.663 INFO:tasks.rados.rados.0.smithi122.stdout:3790: writing smithi12223906-4 from 538187 to 1125224 tid 1 2024-09-15T04:03:21.670 INFO:tasks.rados.rados.0.smithi122.stdout:3790: writing smithi12223906-4 from 1538706 to 2141179 tid 2 2024-09-15T04:03:21.670 INFO:tasks.rados.rados.0.smithi122.stdout:3791: delete oid 14 current snap is 440 2024-09-15T04:03:21.670 INFO:tasks.rados.rados.0.smithi122.stdout:3790: finishing write tid 1 to smithi12223906-4 2024-09-15T04:03:21.675 INFO:tasks.rados.rados.0.smithi122.stdout:3791: done (1 left) 2024-09-15T04:03:21.675 INFO:tasks.rados.rados.0.smithi122.stdout:3792: snap_remove snap 394 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3790: finishing write tid 2 to smithi12223906-4 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3790: finishing write tid 3 to smithi12223906-4 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3790: finishing write tid 4 to smithi12223906-4 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2638 (ObjNum 1341 snap 440 seq_num 1341) dirty exists 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3790: left oid 4 (ObjNum 1341 snap 440 seq_num 1341) 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3790: done (1 left) 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3792: done (0 left) 2024-09-15T04:03:22.646 INFO:tasks.rados.rados.0.smithi122.stdout:3793: snap_remove snap 412 2024-09-15T04:03:23.659 INFO:tasks.rados.rados.0.smithi122.stdout:3793: done (0 left) 2024-09-15T04:03:23.659 INFO:tasks.rados.rados.0.smithi122.stdout:3794: read oid 38 snap -1 2024-09-15T04:03:23.659 INFO:tasks.rados.rados.0.smithi122.stdout:3794: expect (ObjNum 1318 snap 434 seq_num 1318) 2024-09-15T04:03:23.659 INFO:tasks.rados.rados.0.smithi122.stdout:3795: read oid 29 snap -1 2024-09-15T04:03:23.660 INFO:tasks.rados.rados.0.smithi122.stdout:3795: expect deleted 2024-09-15T04:03:23.660 INFO:tasks.rados.rados.0.smithi122.stdout:3796: read oid 10 snap -1 2024-09-15T04:03:23.660 INFO:tasks.rados.rados.0.smithi122.stdout:3796: expect deleted 2024-09-15T04:03:23.660 INFO:tasks.rados.rados.0.smithi122.stdout:3797: delete oid 7 current snap is 440 2024-09-15T04:03:23.675 INFO:tasks.rados.rados.0.smithi122.stdout:3795: done (3 left) 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:3796: done (2 left) 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:3797: done (1 left) 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:3798: rollback oid 46 current snap is 440 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 46 to 437 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:3799: read oid 3 snap -1 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:3799: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T04:03:23.676 INFO:tasks.rados.rados.0.smithi122.stdout:3800: snap_create 2024-09-15T04:03:23.680 INFO:tasks.rados.rados.0.smithi122.stdout:3794: expect (ObjNum 1318 snap 434 seq_num 1318) 2024-09-15T04:03:23.811 INFO:tasks.rados.rados.0.smithi122.stdout:3799: expect (ObjNum 973 snap 307 seq_num 973) 2024-09-15T04:03:23.938 INFO:tasks.rados.rados.0.smithi122.stdout:3798: finishing rollback tid 0 to smithi12223906-46 2024-09-15T04:03:23.938 INFO:tasks.rados.rados.0.smithi122.stdout:3798: finishing rollback tid 1 to smithi12223906-46 2024-09-15T04:03:23.938 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 46 v 3095 (ObjNum 1 snap 0 seq_num 304) dirty dne 2024-09-15T04:03:24.547 INFO:tasks.rados.rados.0.smithi122.stdout:3794: done (3 left) 2024-09-15T04:03:24.547 INFO:tasks.rados.rados.0.smithi122.stdout:3798: done (2 left) 2024-09-15T04:03:24.547 INFO:tasks.rados.rados.0.smithi122.stdout:3799: done (1 left) 2024-09-15T04:03:24.547 INFO:tasks.rados.rados.0.smithi122.stdout:3800: done (0 left) 2024-09-15T04:03:24.547 INFO:tasks.rados.rados.0.smithi122.stdout:3801: write oid 49 current snap is 441 2024-09-15T04:03:24.547 INFO:tasks.rados.rados.0.smithi122.stdout:3801: seq_num 1344 ranges {588608=412784,1597799=749800} 2024-09-15T04:03:24.556 INFO:tasks.rados.rados.0.smithi122.stdout:3801: writing smithi12223906-49 from 588608 to 1001392 tid 1 2024-09-15T04:03:24.564 INFO:tasks.rados.rados.0.smithi122.stdout:3801: writing smithi12223906-49 from 1597799 to 2347599 tid 2 2024-09-15T04:03:24.564 INFO:tasks.rados.rados.0.smithi122.stdout:3802: read oid 39 snap -1 2024-09-15T04:03:24.564 INFO:tasks.rados.rados.0.smithi122.stdout:3802: expect (ObjNum 1339 snap 440 seq_num 1339) 2024-09-15T04:03:24.564 INFO:tasks.rados.rados.0.smithi122.stdout:3803: write (excl) oid 5 current snap is 441 2024-09-15T04:03:24.564 INFO:tasks.rados.rados.0.smithi122.stdout:3801: finishing write tid 1 to smithi12223906-49 2024-09-15T04:03:24.565 INFO:tasks.rados.rados.0.smithi122.stdout:3803: seq_num 1345 ranges {605415=488033,1656281=493458} 2024-09-15T04:03:24.571 INFO:tasks.rados.rados.0.smithi122.stdout:3803: writing smithi12223906-5 from 605415 to 1093448 tid 1 2024-09-15T04:03:24.577 INFO:tasks.rados.rados.0.smithi122.stdout:3803: writing smithi12223906-5 from 1656281 to 2149739 tid 2 2024-09-15T04:03:24.578 INFO:tasks.rados.rados.0.smithi122.stdout:3804: read oid 8 snap -1 2024-09-15T04:03:24.578 INFO:tasks.rados.rados.0.smithi122.stdout:3804: expect deleted 2024-09-15T04:03:24.578 INFO:tasks.rados.rados.0.smithi122.stdout:3805: write oid 22 current snap is 441 2024-09-15T04:03:24.578 INFO:tasks.rados.rados.0.smithi122.stdout:3801: finishing write tid 2 to smithi12223906-49 2024-09-15T04:03:24.578 INFO:tasks.rados.rados.0.smithi122.stdout:3805: seq_num 1346 ranges {622222=679635,1831116=753469} 2024-09-15T04:03:24.584 INFO:tasks.rados.rados.0.smithi122.stdout:3805: writing smithi12223906-22 from 622222 to 1301857 tid 1 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:3805: writing smithi12223906-22 from 1831116 to 2584585 tid 2 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:3806: snap_create 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:3801: finishing write tid 3 to smithi12223906-49 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:3801: finishing write tid 4 to smithi12223906-49 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 2908 (ObjNum 1344 snap 441 seq_num 1344) dirty exists 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:3801: left oid 49 (ObjNum 1344 snap 441 seq_num 1344) 2024-09-15T04:03:24.592 INFO:tasks.rados.rados.0.smithi122.stdout:3802: expect (ObjNum 1339 snap 440 seq_num 1339) 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3803: finishing write tid 1 to smithi12223906-5 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3803: finishing write tid 2 to smithi12223906-5 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3803: finishing write tid 3 to smithi12223906-5 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3803: finishing write tid 4 to smithi12223906-5 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 2627 (ObjNum 1345 snap 441 seq_num 1345) dirty exists 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3803: left oid 5 (ObjNum 1345 snap 441 seq_num 1345) 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3805: finishing write tid 1 to smithi12223906-22 2024-09-15T04:03:24.754 INFO:tasks.rados.rados.0.smithi122.stdout:3805: finishing write tid 2 to smithi12223906-22 2024-09-15T04:03:24.755 INFO:tasks.rados.rados.0.smithi122.stdout:3805: finishing write tid 3 to smithi12223906-22 2024-09-15T04:03:24.755 INFO:tasks.rados.rados.0.smithi122.stdout:3805: finishing write tid 4 to smithi12223906-22 2024-09-15T04:03:24.755 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 3304 (ObjNum 1346 snap 441 seq_num 1346) dirty exists 2024-09-15T04:03:24.755 INFO:tasks.rados.rados.0.smithi122.stdout:3805: left oid 22 (ObjNum 1346 snap 441 seq_num 1346) 2024-09-15T04:03:25.549 INFO:tasks.rados.rados.0.smithi122.stdout:3801: done (5 left) 2024-09-15T04:03:25.549 INFO:tasks.rados.rados.0.smithi122.stdout:3802: done (4 left) 2024-09-15T04:03:25.549 INFO:tasks.rados.rados.0.smithi122.stdout:3803: done (3 left) 2024-09-15T04:03:25.550 INFO:tasks.rados.rados.0.smithi122.stdout:3804: done (2 left) 2024-09-15T04:03:25.550 INFO:tasks.rados.rados.0.smithi122.stdout:3805: done (1 left) 2024-09-15T04:03:25.550 INFO:tasks.rados.rados.0.smithi122.stdout:3806: done (0 left) 2024-09-15T04:03:25.550 INFO:tasks.rados.rados.0.smithi122.stdout:3807: read oid 45 snap -1 2024-09-15T04:03:25.550 INFO:tasks.rados.rados.0.smithi122.stdout:3807: expect deleted 2024-09-15T04:03:25.550 INFO:tasks.rados.rados.0.smithi122.stdout:3808: write oid 2 current snap is 442 2024-09-15T04:03:25.551 INFO:tasks.rados.rados.0.smithi122.stdout:3808: seq_num 1347 ranges {639029=754884,2173245=465784} 2024-09-15T04:03:25.560 INFO:tasks.rados.rados.0.smithi122.stdout:3808: writing smithi12223906-2 from 639029 to 1393913 tid 1 2024-09-15T04:03:25.568 INFO:tasks.rados.rados.0.smithi122.stdout:3808: writing smithi12223906-2 from 2173245 to 2639029 tid 2 2024-09-15T04:03:25.568 INFO:tasks.rados.rados.0.smithi122.stdout:3809: write (excl) oid 37 current snap is 442 2024-09-15T04:03:25.568 INFO:tasks.rados.rados.0.smithi122.stdout:3809: seq_num 1348 ranges {655836=430133,1831727=757138} 2024-09-15T04:03:25.574 INFO:tasks.rados.rados.0.smithi122.stdout:3809: writing smithi12223906-37 from 655836 to 1085969 tid 1 2024-09-15T04:03:25.583 INFO:tasks.rados.rados.0.smithi122.stdout:3809: writing smithi12223906-37 from 1831727 to 2588865 tid 2 2024-09-15T04:03:25.583 INFO:tasks.rados.rados.0.smithi122.stdout:3810: write oid 32 current snap is 442 2024-09-15T04:03:25.583 INFO:tasks.rados.rados.0.smithi122.stdout:3810: seq_num 1349 ranges {672643=505382,1890209=500796} 2024-09-15T04:03:25.590 INFO:tasks.rados.rados.0.smithi122.stdout:3810: writing smithi12223906-32 from 672643 to 1178025 tid 1 2024-09-15T04:03:25.597 INFO:tasks.rados.rados.0.smithi122.stdout:3810: writing smithi12223906-32 from 1890209 to 2391005 tid 2 2024-09-15T04:03:25.597 INFO:tasks.rados.rados.0.smithi122.stdout:3811: read oid 31 snap 391 2024-09-15T04:03:25.597 INFO:tasks.rados.rados.0.smithi122.stdout:3811: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:03:25.597 INFO:tasks.rados.rados.0.smithi122.stdout:3812: write oid 5 current snap is 442 2024-09-15T04:03:25.597 INFO:tasks.rados.rados.0.smithi122.stdout:3812: seq_num 1350 ranges {689450=580631,1948691=740759} 2024-09-15T04:03:25.604 INFO:tasks.rados.rados.0.smithi122.stdout:3812: writing smithi12223906-5 from 689450 to 1270081 tid 1 2024-09-15T04:03:25.613 INFO:tasks.rados.rados.0.smithi122.stdout:3812: writing smithi12223906-5 from 1948691 to 2689450 tid 2 2024-09-15T04:03:25.613 INFO:tasks.rados.rados.0.smithi122.stdout:3813: read oid 21 snap -1 2024-09-15T04:03:25.613 INFO:tasks.rados.rados.0.smithi122.stdout:3813: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:03:25.613 INFO:tasks.rados.rados.0.smithi122.stdout:3814: read oid 28 snap 391 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3814: expect (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3815: copy_from oid 34 from oid 48 current snap is 442 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3808: finishing write tid 1 to smithi12223906-2 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3808: finishing write tid 2 to smithi12223906-2 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3808: finishing write tid 3 to smithi12223906-2 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3808: finishing write tid 4 to smithi12223906-2 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 2911 (ObjNum 1347 snap 442 seq_num 1347) dirty exists 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3808: left oid 2 (ObjNum 1347 snap 442 seq_num 1347) 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3809: finishing write tid 1 to smithi12223906-37 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3807: done (8 left) 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3808: done (7 left) 2024-09-15T04:03:25.614 INFO:tasks.rados.rados.0.smithi122.stdout:3816: copy_from oid 40 from oid 45 current snap is 442 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3817: read oid 33 snap 415 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3817: expect (ObjNum 1173 snap 384 seq_num 1173) 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3818: rollback oid 18 current snap is 442 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 441 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3819: read oid 1 snap 339 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3819: expect (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3820: read oid 27 snap -1 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3820: expect deleted 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3821: copy_from oid 8 from oid 35 current snap is 442 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3822: write (excl) oid 43 current snap is 442 2024-09-15T04:03:25.615 INFO:tasks.rados.rados.0.smithi122.stdout:3822: seq_num 1351 ranges {706257=655880,1890820=504465} 2024-09-15T04:03:25.621 INFO:tasks.rados.rados.0.smithi122.stdout:3822: writing smithi12223906-43 from 706257 to 1362137 tid 1 2024-09-15T04:03:25.628 INFO:tasks.rados.rados.0.smithi122.stdout:3822: writing smithi12223906-43 from 1890820 to 2395285 tid 2 2024-09-15T04:03:25.628 INFO:tasks.rados.rados.0.smithi122.stdout:3823: write oid 19 current snap is 442 2024-09-15T04:03:25.628 INFO:tasks.rados.rados.0.smithi122.stdout:3823: seq_num 1352 ranges {723064=731129,1949302=764476} 2024-09-15T04:03:25.638 INFO:tasks.rados.rados.0.smithi122.stdout:3823: writing smithi12223906-19 from 723064 to 1454193 tid 1 2024-09-15T04:03:25.645 INFO:tasks.rados.rados.0.smithi122.stdout:3823: writing smithi12223906-19 from 1949302 to 2713778 tid 2 2024-09-15T04:03:25.645 INFO:tasks.rados.rados.0.smithi122.stdout:3824: copy_from oid 4 from oid 6 current snap is 442 2024-09-15T04:03:25.645 INFO:tasks.rados.rados.0.smithi122.stdout:3809: finishing write tid 2 to smithi12223906-37 2024-09-15T04:03:25.646 INFO:tasks.rados.rados.0.smithi122.stdout:3809: finishing write tid 3 to smithi12223906-37 2024-09-15T04:03:25.646 INFO:tasks.rados.rados.0.smithi122.stdout:3809: finishing write tid 4 to smithi12223906-37 2024-09-15T04:03:25.646 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 37 v 2158 (ObjNum 1348 snap 442 seq_num 1348) dirty exists 2024-09-15T04:03:25.646 INFO:tasks.rados.rados.0.smithi122.stdout:3809: left oid 37 (ObjNum 1348 snap 442 seq_num 1348) 2024-09-15T04:03:25.646 INFO:tasks.rados.rados.0.smithi122.stdout:3810: finishing write tid 1 to smithi12223906-32 2024-09-15T04:03:25.646 INFO:tasks.rados.rados.0.smithi122.stdout:3810: finishing write tid 2 to smithi12223906-32 2024-09-15T04:03:25.647 INFO:tasks.rados.rados.0.smithi122.stdout:3810: finishing write tid 3 to smithi12223906-32 2024-09-15T04:03:25.647 INFO:tasks.rados.rados.0.smithi122.stdout:3810: finishing write tid 4 to smithi12223906-32 2024-09-15T04:03:25.647 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 32 v 3307 (ObjNum 1349 snap 442 seq_num 1349) dirty exists 2024-09-15T04:03:25.647 INFO:tasks.rados.rados.0.smithi122.stdout:3810: left oid 32 (ObjNum 1349 snap 442 seq_num 1349) 2024-09-15T04:03:25.647 INFO:tasks.rados.rados.0.smithi122.stdout:3811: expect (ObjNum 1146 snap 377 seq_num 1146) 2024-09-15T04:03:25.835 INFO:tasks.rados.rados.0.smithi122.stdout:3812: finishing write tid 1 to smithi12223906-5 2024-09-15T04:03:25.835 INFO:tasks.rados.rados.0.smithi122.stdout:3816: finishing copy_from to smithi12223906-40 2024-09-15T04:03:25.835 INFO:tasks.rados.rados.0.smithi122.stdout:3816: got expected ENOENT (src dne) 2024-09-15T04:03:25.835 INFO:tasks.rados.rados.0.smithi122.stdout:3816: finishing copy_from racing read to smithi12223906-40 2024-09-15T04:03:25.835 INFO:tasks.rados.rados.0.smithi122.stdout:3819: expect (ObjNum 1011 snap 316 seq_num 1011) 2024-09-15T04:03:25.932 INFO:tasks.rados.rados.0.smithi122.stdout:3813: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:03:26.028 INFO:tasks.rados.rados.0.smithi122.stdout:3814: expect (ObjNum 1170 snap 383 seq_num 1170) 2024-09-15T04:03:26.225 INFO:tasks.rados.rados.0.smithi122.stdout:3821: finishing copy_from racing read to smithi12223906-8 2024-09-15T04:03:26.225 INFO:tasks.rados.rados.0.smithi122.stdout:3812: finishing write tid 2 to smithi12223906-5 2024-09-15T04:03:26.225 INFO:tasks.rados.rados.0.smithi122.stdout:3812: finishing write tid 3 to smithi12223906-5 2024-09-15T04:03:26.226 INFO:tasks.rados.rados.0.smithi122.stdout:3812: finishing write tid 4 to smithi12223906-5 2024-09-15T04:03:26.226 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 2630 (ObjNum 1350 snap 442 seq_num 1350) dirty exists 2024-09-15T04:03:26.226 INFO:tasks.rados.rados.0.smithi122.stdout:3812: left oid 5 (ObjNum 1350 snap 442 seq_num 1350) 2024-09-15T04:03:26.226 INFO:tasks.rados.rados.0.smithi122.stdout:3817: expect (ObjNum 1173 snap 384 seq_num 1173) 2024-09-15T04:03:26.469 INFO:tasks.rados.rados.0.smithi122.stdout:3818: finishing rollback tid 0 to smithi12223906-18 2024-09-15T04:03:26.469 INFO:tasks.rados.rados.0.smithi122.stdout:3818: finishing rollback tid 1 to smithi12223906-18 2024-09-15T04:03:26.469 INFO:tasks.rados.rados.0.smithi122.stdout:3818: finishing rollback tid 2 to smithi12223906-18 2024-09-15T04:03:26.469 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3311 (ObjNum 1313 snap 432 seq_num 1313) dirty exists 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3815: finishing copy_from racing read to smithi12223906-34 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3822: finishing write tid 1 to smithi12223906-43 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3821: finishing copy_from to smithi12223906-8 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 8 v 1849 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3822: finishing write tid 2 to smithi12223906-43 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3815: finishing copy_from to smithi12223906-34 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 2155 (ObjNum 1323 snap 434 seq_num 1323) dirty exists 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3822: finishing write tid 3 to smithi12223906-43 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3822: finishing write tid 4 to smithi12223906-43 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 43 v 3311 (ObjNum 1351 snap 442 seq_num 1351) dirty exists 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3822: left oid 43 (ObjNum 1351 snap 442 seq_num 1351) 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3823: finishing write tid 1 to smithi12223906-19 2024-09-15T04:03:26.470 INFO:tasks.rados.rados.0.smithi122.stdout:3823: finishing write tid 2 to smithi12223906-19 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3823: finishing write tid 3 to smithi12223906-19 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3823: finishing write tid 4 to smithi12223906-19 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 3098 (ObjNum 1352 snap 442 seq_num 1352) dirty exists 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3823: left oid 19 (ObjNum 1352 snap 442 seq_num 1352) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3824: finishing copy_from racing read to smithi12223906-4 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3824: finishing copy_from to smithi12223906-4 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 2496 (ObjNum 1283 snap 424 seq_num 1283) dirty exists 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3809: done (15 left) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3810: done (14 left) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3811: done (13 left) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3812: done (12 left) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3813: done (11 left) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3814: done (10 left) 2024-09-15T04:03:26.471 INFO:tasks.rados.rados.0.smithi122.stdout:3815: done (9 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3816: done (8 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3817: done (7 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3818: done (6 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3819: done (5 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3820: done (4 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3821: done (3 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3822: done (2 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3823: done (1 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3824: done (0 left) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3825: copy_from oid 19 from oid 22 current snap is 442 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3826: read oid 6 snap -1 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3826: expect (ObjNum 1283 snap 424 seq_num 1283) 2024-09-15T04:03:26.472 INFO:tasks.rados.rados.0.smithi122.stdout:3827: snap_create 2024-09-15T04:03:26.478 INFO:tasks.rados.rados.0.smithi122.stdout:3826: expect (ObjNum 1283 snap 424 seq_num 1283) 2024-09-15T04:03:26.646 INFO:tasks.rados.rados.0.smithi122.stdout:3825: finishing copy_from racing read to smithi12223906-19 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:3825: finishing copy_from to smithi12223906-19 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 3304 (ObjNum 1346 snap 441 seq_num 1346) dirty exists 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:3825: done (2 left) 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:3826: done (1 left) 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:3827: done (0 left) 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:3828: write oid 31 current snap is 443 2024-09-15T04:03:26.647 INFO:tasks.rados.rados.0.smithi122.stdout:3828: seq_num 1353 ranges {739871=406378,1607784=508134} 2024-09-15T04:03:26.663 INFO:tasks.rados.rados.0.smithi122.stdout:3828: writing smithi12223906-31 from 739871 to 1146249 tid 1 2024-09-15T04:03:26.669 INFO:tasks.rados.rados.0.smithi122.stdout:3828: writing smithi12223906-31 from 1607784 to 2115918 tid 2 2024-09-15T04:03:26.669 INFO:tasks.rados.rados.0.smithi122.stdout:3829: snap_remove snap 407 2024-09-15T04:03:27.569 INFO:tasks.rados.rados.0.smithi122.stdout:3829: done (1 left) 2024-09-15T04:03:27.569 INFO:tasks.rados.rados.0.smithi122.stdout:3830: write oid 28 current snap is 443 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:3828: finishing write tid 1 to smithi12223906-31 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:3828: finishing write tid 2 to smithi12223906-31 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:3828: finishing write tid 3 to smithi12223906-31 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:3828: finishing write tid 4 to smithi12223906-31 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 2641 (ObjNum 1353 snap 443 seq_num 1353) dirty exists 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:3828: left oid 31 (ObjNum 1353 snap 443 seq_num 1353) 2024-09-15T04:03:27.570 INFO:tasks.rados.rados.0.smithi122.stdout:3830: seq_num 1354 ranges {756678=597980,1782619=768145} 2024-09-15T04:03:27.580 INFO:tasks.rados.rados.0.smithi122.stdout:3830: writing smithi12223906-28 from 756678 to 1354658 tid 1 2024-09-15T04:03:27.588 INFO:tasks.rados.rados.0.smithi122.stdout:3830: writing smithi12223906-28 from 1782619 to 2550764 tid 2 2024-09-15T04:03:27.588 INFO:tasks.rados.rados.0.smithi122.stdout:3828: done (1 left) 2024-09-15T04:03:27.588 INFO:tasks.rados.rados.0.smithi122.stdout:3831: copy_from oid 41 from oid 8 current snap is 443 2024-09-15T04:03:27.589 INFO:tasks.rados.rados.0.smithi122.stdout:3830: finishing write tid 1 to smithi12223906-28 2024-09-15T04:03:27.589 INFO:tasks.rados.rados.0.smithi122.stdout:3832: copy_from oid 33 from oid 4 current snap is 443 2024-09-15T04:03:27.589 INFO:tasks.rados.rados.0.smithi122.stdout:3833: delete oid 23 current snap is 443 2024-09-15T04:03:27.598 INFO:tasks.rados.rados.0.smithi122.stdout:3830: finishing write tid 2 to smithi12223906-28 2024-09-15T04:03:27.598 INFO:tasks.rados.rados.0.smithi122.stdout:3830: finishing write tid 3 to smithi12223906-28 2024-09-15T04:03:27.598 INFO:tasks.rados.rados.0.smithi122.stdout:3830: finishing write tid 4 to smithi12223906-28 2024-09-15T04:03:27.598 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 2445 (ObjNum 1354 snap 443 seq_num 1354) dirty exists 2024-09-15T04:03:27.598 INFO:tasks.rados.rados.0.smithi122.stdout:3830: left oid 28 (ObjNum 1354 snap 443 seq_num 1354) 2024-09-15T04:03:27.603 INFO:tasks.rados.rados.0.smithi122.stdout:3832: finishing copy_from racing read to smithi12223906-33 2024-09-15T04:03:27.603 INFO:tasks.rados.rados.0.smithi122.stdout:3830: done (3 left) 2024-09-15T04:03:27.603 INFO:tasks.rados.rados.0.smithi122.stdout:3833: done (2 left) 2024-09-15T04:03:27.603 INFO:tasks.rados.rados.0.smithi122.stdout:3834: snap_remove snap 329 2024-09-15T04:03:28.578 INFO:tasks.rados.rados.0.smithi122.stdout:3834: done (2 left) 2024-09-15T04:03:28.578 INFO:tasks.rados.rados.0.smithi122.stdout:3835: write (excl) oid 27 current snap is 443 2024-09-15T04:03:28.578 INFO:tasks.rados.rados.0.smithi122.stdout:3832: finishing copy_from to smithi12223906-33 2024-09-15T04:03:28.578 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2496 (ObjNum 1283 snap 424 seq_num 1283) dirty exists 2024-09-15T04:03:28.578 INFO:tasks.rados.rados.0.smithi122.stdout:3831: finishing copy_from racing read to smithi12223906-41 2024-09-15T04:03:28.578 INFO:tasks.rados.rados.0.smithi122.stdout:3831: finishing copy_from to smithi12223906-41 2024-09-15T04:03:28.579 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 1849 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:03:28.579 INFO:tasks.rados.rados.0.smithi122.stdout:3835: seq_num 1356 ranges {790292=748478,2183230=607062} 2024-09-15T04:03:28.590 INFO:tasks.rados.rados.0.smithi122.stdout:3835: writing smithi12223906-27 from 790292 to 1538770 tid 1 2024-09-15T04:03:28.597 INFO:tasks.rados.rados.0.smithi122.stdout:3835: writing smithi12223906-27 from 2183230 to 2790292 tid 2 2024-09-15T04:03:28.597 INFO:tasks.rados.rados.0.smithi122.stdout:3831: done (2 left) 2024-09-15T04:03:28.598 INFO:tasks.rados.rados.0.smithi122.stdout:3832: done (1 left) 2024-09-15T04:03:28.598 INFO:tasks.rados.rados.0.smithi122.stdout:3836: delete oid 5 current snap is 443 2024-09-15T04:03:28.620 INFO:tasks.rados.rados.0.smithi122.stdout:3835: finishing write tid 1 to smithi12223906-27 2024-09-15T04:03:28.624 INFO:tasks.rados.rados.0.smithi122.stdout:3835: finishing write tid 2 to smithi12223906-27 2024-09-15T04:03:28.625 INFO:tasks.rados.rados.0.smithi122.stdout:3835: finishing write tid 3 to smithi12223906-27 2024-09-15T04:03:28.625 INFO:tasks.rados.rados.0.smithi122.stdout:3835: finishing write tid 4 to smithi12223906-27 2024-09-15T04:03:28.625 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 3328 (ObjNum 1356 snap 443 seq_num 1356) dirty exists 2024-09-15T04:03:28.625 INFO:tasks.rados.rados.0.smithi122.stdout:3835: left oid 27 (ObjNum 1356 snap 443 seq_num 1356) 2024-09-15T04:03:28.626 INFO:tasks.rados.rados.0.smithi122.stdout:3835: done (1 left) 2024-09-15T04:03:28.626 INFO:tasks.rados.rados.0.smithi122.stdout:3836: done (0 left) 2024-09-15T04:03:28.626 INFO:tasks.rados.rados.0.smithi122.stdout:3837: write (excl) oid 11 current snap is 443 2024-09-15T04:03:28.626 INFO:tasks.rados.rados.0.smithi122.stdout:3837: seq_num 1358 ranges {423906=498976,1500194=659130,2609483=214423} 2024-09-15T04:03:28.634 INFO:tasks.rados.rados.0.smithi122.stdout:3837: writing smithi12223906-11 from 423906 to 922882 tid 1 2024-09-15T04:03:28.641 INFO:tasks.rados.rados.0.smithi122.stdout:3837: writing smithi12223906-11 from 1500194 to 2159324 tid 2 2024-09-15T04:03:28.645 INFO:tasks.rados.rados.0.smithi122.stdout:3837: writing smithi12223906-11 from 2609483 to 2823906 tid 3 2024-09-15T04:03:28.645 INFO:tasks.rados.rados.0.smithi122.stdout:3838: delete oid 50 current snap is 443 2024-09-15T04:03:28.645 INFO:tasks.rados.rados.0.smithi122.stdout:3837: finishing write tid 1 to smithi12223906-11 2024-09-15T04:03:28.649 INFO:tasks.rados.rados.0.smithi122.stdout:3837: finishing write tid 2 to smithi12223906-11 2024-09-15T04:03:28.651 INFO:tasks.rados.rados.0.smithi122.stdout:3837: finishing write tid 3 to smithi12223906-11 2024-09-15T04:03:28.651 INFO:tasks.rados.rados.0.smithi122.stdout:3837: finishing write tid 4 to smithi12223906-11 2024-09-15T04:03:28.652 INFO:tasks.rados.rados.0.smithi122.stdout:3837: finishing write tid 5 to smithi12223906-11 2024-09-15T04:03:28.652 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 11 v 1952 (ObjNum 1358 snap 443 seq_num 1358) dirty exists 2024-09-15T04:03:28.652 INFO:tasks.rados.rados.0.smithi122.stdout:3837: left oid 11 (ObjNum 1358 snap 443 seq_num 1358) 2024-09-15T04:03:28.652 INFO:tasks.rados.rados.0.smithi122.stdout:3837: done (1 left) 2024-09-15T04:03:28.652 INFO:tasks.rados.rados.0.smithi122.stdout:3838: done (0 left) 2024-09-15T04:03:28.652 INFO:tasks.rados.rados.0.smithi122.stdout:3839: snap_create 2024-09-15T04:03:29.550 INFO:tasks.rados.rados.0.smithi122.stdout:3839: done (0 left) 2024-09-15T04:03:29.550 INFO:tasks.rados.rados.0.smithi122.stdout:3840: snap_remove snap 336 2024-09-15T04:03:30.556 INFO:tasks.rados.rados.0.smithi122.stdout:3840: done (0 left) 2024-09-15T04:03:30.556 INFO:tasks.rados.rados.0.smithi122.stdout:3841: snap_create 2024-09-15T04:03:31.554 INFO:tasks.rados.rados.0.smithi122.stdout:3841: done (0 left) 2024-09-15T04:03:31.554 INFO:tasks.rados.rados.0.smithi122.stdout:3842: read oid 37 snap -1 2024-09-15T04:03:31.554 INFO:tasks.rados.rados.0.smithi122.stdout:3842: expect (ObjNum 1348 snap 442 seq_num 1348) 2024-09-15T04:03:31.554 INFO:tasks.rados.rados.0.smithi122.stdout:3843: delete oid 13 current snap is 445 2024-09-15T04:03:31.557 INFO:tasks.rados.rados.0.smithi122.stdout:3843: done (1 left) 2024-09-15T04:03:31.558 INFO:tasks.rados.rados.0.smithi122.stdout:3844: read oid 10 snap -1 2024-09-15T04:03:31.558 INFO:tasks.rados.rados.0.smithi122.stdout:3844: expect deleted 2024-09-15T04:03:31.558 INFO:tasks.rados.rados.0.smithi122.stdout:3845: read oid 45 snap -1 2024-09-15T04:03:31.558 INFO:tasks.rados.rados.0.smithi122.stdout:3845: expect deleted 2024-09-15T04:03:31.558 INFO:tasks.rados.rados.0.smithi122.stdout:3846: write oid 35 current snap is 445 2024-09-15T04:03:31.558 INFO:tasks.rados.rados.0.smithi122.stdout:3846: seq_num 1361 ranges {474327=441076,1675640=522810,2808105=66222} 2024-09-15T04:03:31.566 INFO:tasks.rados.rados.0.smithi122.stdout:3846: writing smithi12223906-35 from 474327 to 915403 tid 1 2024-09-15T04:03:31.575 INFO:tasks.rados.rados.0.smithi122.stdout:3846: writing smithi12223906-35 from 1675640 to 2198450 tid 2 2024-09-15T04:03:31.579 INFO:tasks.rados.rados.0.smithi122.stdout:3846: writing smithi12223906-35 from 2808105 to 2874327 tid 3 2024-09-15T04:03:31.579 INFO:tasks.rados.rados.0.smithi122.stdout:3847: copy_from oid 48 from oid 1 current snap is 445 2024-09-15T04:03:31.579 INFO:tasks.rados.rados.0.smithi122.stdout:3842: expect (ObjNum 1348 snap 442 seq_num 1348) 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3846: finishing write tid 1 to smithi12223906-35 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3846: finishing write tid 2 to smithi12223906-35 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3846: finishing write tid 3 to smithi12223906-35 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3847: finishing copy_from to smithi12223906-48 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3847: got expected ENOENT (src dne) 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3846: finishing write tid 4 to smithi12223906-35 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3847: finishing copy_from racing read to smithi12223906-48 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3846: finishing write tid 5 to smithi12223906-35 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1853 (ObjNum 1361 snap 445 seq_num 1361) dirty exists 2024-09-15T04:03:31.676 INFO:tasks.rados.rados.0.smithi122.stdout:3846: left oid 35 (ObjNum 1361 snap 445 seq_num 1361) 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3842: done (4 left) 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3844: done (3 left) 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3845: done (2 left) 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3846: done (1 left) 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3847: done (0 left) 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3848: write (excl) oid 22 current snap is 445 2024-09-15T04:03:31.677 INFO:tasks.rados.rados.0.smithi122.stdout:3848: seq_num 1362 ranges {491134=516325,1734122=666468} 2024-09-15T04:03:31.690 INFO:tasks.rados.rados.0.smithi122.stdout:3848: writing smithi12223906-22 from 491134 to 1007459 tid 1 2024-09-15T04:03:31.698 INFO:tasks.rados.rados.0.smithi122.stdout:3848: writing smithi12223906-22 from 1734122 to 2400590 tid 2 2024-09-15T04:03:31.698 INFO:tasks.rados.rados.0.smithi122.stdout:3849: read oid 13 snap -1 2024-09-15T04:03:31.698 INFO:tasks.rados.rados.0.smithi122.stdout:3849: expect deleted 2024-09-15T04:03:31.698 INFO:tasks.rados.rados.0.smithi122.stdout:3850: snap_create 2024-09-15T04:03:31.708 INFO:tasks.rados.rados.0.smithi122.stdout:3848: finishing write tid 1 to smithi12223906-22 2024-09-15T04:03:31.709 INFO:tasks.rados.rados.0.smithi122.stdout:3848: finishing write tid 2 to smithi12223906-22 2024-09-15T04:03:31.709 INFO:tasks.rados.rados.0.smithi122.stdout:3848: finishing write tid 3 to smithi12223906-22 2024-09-15T04:03:31.709 INFO:tasks.rados.rados.0.smithi122.stdout:3848: finishing write tid 4 to smithi12223906-22 2024-09-15T04:03:31.709 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 3354 (ObjNum 1362 snap 445 seq_num 1362) dirty exists 2024-09-15T04:03:31.710 INFO:tasks.rados.rados.0.smithi122.stdout:3848: left oid 22 (ObjNum 1362 snap 445 seq_num 1362) 2024-09-15T04:03:32.563 INFO:tasks.rados.rados.0.smithi122.stdout:3848: done (2 left) 2024-09-15T04:03:32.563 INFO:tasks.rados.rados.0.smithi122.stdout:3849: done (1 left) 2024-09-15T04:03:32.563 INFO:tasks.rados.rados.0.smithi122.stdout:3850: done (0 left) 2024-09-15T04:03:32.564 INFO:tasks.rados.rados.0.smithi122.stdout:3851: snap_remove snap 396 2024-09-15T04:03:33.572 INFO:tasks.rados.rados.0.smithi122.stdout:3851: done (0 left) 2024-09-15T04:03:33.573 INFO:tasks.rados.rados.0.smithi122.stdout:3852: read oid 30 snap -1 2024-09-15T04:03:33.573 INFO:tasks.rados.rados.0.smithi122.stdout:3852: expect (ObjNum 1330 snap 435 seq_num 1330) 2024-09-15T04:03:33.573 INFO:tasks.rados.rados.0.smithi122.stdout:3853: copy_from oid 20 from oid 21 current snap is 446 2024-09-15T04:03:33.573 INFO:tasks.rados.rados.0.smithi122.stdout:3854: snap_create 2024-09-15T04:03:33.594 INFO:tasks.rados.rados.0.smithi122.stdout:3852: expect (ObjNum 1330 snap 435 seq_num 1330) 2024-09-15T04:03:33.704 INFO:tasks.rados.rados.0.smithi122.stdout:3853: finishing copy_from racing read to smithi12223906-20 2024-09-15T04:03:33.704 INFO:tasks.rados.rados.0.smithi122.stdout:3853: finishing copy_from to smithi12223906-20 2024-09-15T04:03:33.704 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 20 v 2138 (ObjNum 1247 snap 407 seq_num 1247) dirty exists 2024-09-15T04:03:34.554 INFO:tasks.rados.rados.0.smithi122.stdout:3852: done (2 left) 2024-09-15T04:03:34.554 INFO:tasks.rados.rados.0.smithi122.stdout:3853: done (1 left) 2024-09-15T04:03:34.554 INFO:tasks.rados.rados.0.smithi122.stdout:3854: done (0 left) 2024-09-15T04:03:34.554 INFO:tasks.rados.rados.0.smithi122.stdout:3855: snap_remove snap 445 2024-09-15T04:03:35.557 INFO:tasks.rados.rados.0.smithi122.stdout:3855: done (0 left) 2024-09-15T04:03:35.557 INFO:tasks.rados.rados.0.smithi122.stdout:3856: read oid 26 snap -1 2024-09-15T04:03:35.557 INFO:tasks.rados.rados.0.smithi122.stdout:3856: expect (ObjNum 1287 snap 426 seq_num 1287) 2024-09-15T04:03:35.557 INFO:tasks.rados.rados.0.smithi122.stdout:3857: snap_remove snap 428 2024-09-15T04:03:36.566 INFO:tasks.rados.rados.0.smithi122.stdout:3857: done (1 left) 2024-09-15T04:03:36.567 INFO:tasks.rados.rados.0.smithi122.stdout:3858: snap_remove snap 353 2024-09-15T04:03:37.577 INFO:tasks.rados.rados.0.smithi122.stdout:3858: done (1 left) 2024-09-15T04:03:37.577 INFO:tasks.rados.rados.0.smithi122.stdout:3859: rollback oid 27 current snap is 447 2024-09-15T04:03:37.577 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 27 to 345 2024-09-15T04:03:37.577 INFO:tasks.rados.rados.0.smithi122.stdout:3860: snap_remove snap 380 2024-09-15T04:03:38.587 INFO:tasks.rados.rados.0.smithi122.stdout:3860: done (2 left) 2024-09-15T04:03:38.587 INFO:tasks.rados.rados.0.smithi122.stdout:3861: snap_remove snap 435 2024-09-15T04:03:39.559 INFO:tasks.rados.rados.0.smithi122.stdout:3861: done (2 left) 2024-09-15T04:03:39.559 INFO:tasks.rados.rados.0.smithi122.stdout:3862: delete oid 49 current snap is 447 2024-09-15T04:03:39.559 INFO:tasks.rados.rados.0.smithi122.stdout:3856: expect (ObjNum 1287 snap 426 seq_num 1287) 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:3859: finishing rollback tid 0 to smithi12223906-27 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:3859: finishing rollback tid 1 to smithi12223906-27 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 3387 (ObjNum 129 snap 0 seq_num 1842533784) dirty dne 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:3856: done (2 left) 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:3859: done (1 left) 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:3862: done (0 left) 2024-09-15T04:03:39.769 INFO:tasks.rados.rados.0.smithi122.stdout:3863: snap_remove snap 414 2024-09-15T04:03:40.615 INFO:tasks.rados.rados.0.smithi122.stdout:3863: done (0 left) 2024-09-15T04:03:40.615 INFO:tasks.rados.rados.0.smithi122.stdout:3864: copy_from oid 40 from oid 30 current snap is 447 2024-09-15T04:03:40.618 INFO:tasks.rados.rados.0.smithi122.stdout:3865: write (excl) oid 31 current snap is 447 2024-09-15T04:03:40.618 INFO:tasks.rados.rados.0.smithi122.stdout:3865: seq_num 1364 ranges {524748=666823,1734733=670137} 2024-09-15T04:03:40.623 INFO:tasks.rados.rados.0.smithi122.stdout:3865: writing smithi12223906-31 from 524748 to 1191571 tid 1 2024-09-15T04:03:40.632 INFO:tasks.rados.rados.0.smithi122.stdout:3865: writing smithi12223906-31 from 1734733 to 2404870 tid 2 2024-09-15T04:03:40.632 INFO:tasks.rados.rados.0.smithi122.stdout:3866: snap_remove snap 438 2024-09-15T04:03:41.614 INFO:tasks.rados.rados.0.smithi122.stdout:3864: finishing copy_from racing read to smithi12223906-40 2024-09-15T04:03:41.614 INFO:tasks.rados.rados.0.smithi122.stdout:3865: finishing write tid 1 to smithi12223906-31 2024-09-15T04:03:41.614 INFO:tasks.rados.rados.0.smithi122.stdout:3864: finishing copy_from to smithi12223906-40 2024-09-15T04:03:41.614 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2919 (ObjNum 1330 snap 435 seq_num 1330) dirty exists 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3865: finishing write tid 2 to smithi12223906-31 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3865: finishing write tid 3 to smithi12223906-31 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3865: finishing write tid 4 to smithi12223906-31 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 2644 (ObjNum 1364 snap 447 seq_num 1364) dirty exists 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3865: left oid 31 (ObjNum 1364 snap 447 seq_num 1364) 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3864: done (2 left) 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3865: done (1 left) 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3866: done (0 left) 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3867: copy_from oid 12 from oid 39 current snap is 447 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3868: write (excl) oid 48 current snap is 447 2024-09-15T04:03:41.615 INFO:tasks.rados.rados.0.smithi122.stdout:3868: seq_num 1365 ranges {541555=742072,1793215=530148,2801444=140111} 2024-09-15T04:03:41.624 INFO:tasks.rados.rados.0.smithi122.stdout:3868: writing smithi12223906-48 from 541555 to 1283627 tid 1 2024-09-15T04:03:41.630 INFO:tasks.rados.rados.0.smithi122.stdout:3868: writing smithi12223906-48 from 1793215 to 2323363 tid 2 2024-09-15T04:03:41.633 INFO:tasks.rados.rados.0.smithi122.stdout:3868: writing smithi12223906-48 from 2801444 to 2941555 tid 3 2024-09-15T04:03:41.634 INFO:tasks.rados.rados.0.smithi122.stdout:3869: read oid 19 snap -1 2024-09-15T04:03:41.634 INFO:tasks.rados.rados.0.smithi122.stdout:3869: expect (ObjNum 1346 snap 441 seq_num 1346) 2024-09-15T04:03:41.634 INFO:tasks.rados.rados.0.smithi122.stdout:3867: finishing copy_from racing read to smithi12223906-12 2024-09-15T04:03:41.634 INFO:tasks.rados.rados.0.smithi122.stdout:3870: snap_create 2024-09-15T04:03:41.635 INFO:tasks.rados.rados.0.smithi122.stdout:3867: finishing copy_from to smithi12223906-12 2024-09-15T04:03:41.635 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 3268 (ObjNum 1339 snap 440 seq_num 1339) dirty exists 2024-09-15T04:03:41.636 INFO:tasks.rados.rados.0.smithi122.stdout:3868: finishing write tid 1 to smithi12223906-48 2024-09-15T04:03:41.637 INFO:tasks.rados.rados.0.smithi122.stdout:3868: finishing write tid 2 to smithi12223906-48 2024-09-15T04:03:41.638 INFO:tasks.rados.rados.0.smithi122.stdout:3868: finishing write tid 3 to smithi12223906-48 2024-09-15T04:03:41.639 INFO:tasks.rados.rados.0.smithi122.stdout:3868: finishing write tid 4 to smithi12223906-48 2024-09-15T04:03:41.639 INFO:tasks.rados.rados.0.smithi122.stdout:3868: finishing write tid 5 to smithi12223906-48 2024-09-15T04:03:41.639 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2162 (ObjNum 1365 snap 447 seq_num 1365) dirty exists 2024-09-15T04:03:41.639 INFO:tasks.rados.rados.0.smithi122.stdout:3868: left oid 48 (ObjNum 1365 snap 447 seq_num 1365) 2024-09-15T04:03:41.645 INFO:tasks.rados.rados.0.smithi122.stdout:3869: expect (ObjNum 1346 snap 441 seq_num 1346) 2024-09-15T04:03:42.634 INFO:tasks.rados.rados.0.smithi122.stdout:3867: done (3 left) 2024-09-15T04:03:42.634 INFO:tasks.rados.rados.0.smithi122.stdout:3868: done (2 left) 2024-09-15T04:03:42.635 INFO:tasks.rados.rados.0.smithi122.stdout:3869: done (1 left) 2024-09-15T04:03:42.635 INFO:tasks.rados.rados.0.smithi122.stdout:3870: done (0 left) 2024-09-15T04:03:42.635 INFO:tasks.rados.rados.0.smithi122.stdout:3871: snap_remove snap 442 2024-09-15T04:03:43.668 INFO:tasks.rados.rados.0.smithi122.stdout:3871: done (0 left) 2024-09-15T04:03:43.668 INFO:tasks.rados.rados.0.smithi122.stdout:3872: write oid 40 current snap is 448 2024-09-15T04:03:43.668 INFO:tasks.rados.rados.0.smithi122.stdout:3872: seq_num 1366 ranges {558362=417321,1451697=673806,2712514=245848} 2024-09-15T04:03:43.676 INFO:tasks.rados.rados.0.smithi122.stdout:3872: writing smithi12223906-40 from 558362 to 975683 tid 1 2024-09-15T04:03:43.683 INFO:tasks.rados.rados.0.smithi122.stdout:3872: writing smithi12223906-40 from 1451697 to 2125503 tid 2 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3872: writing smithi12223906-40 from 2712514 to 2958362 tid 3 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3873: rollback oid 33 current snap is 448 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 33 to 410 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3872: finishing write tid 1 to smithi12223906-40 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3872: finishing write tid 2 to smithi12223906-40 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3874: read oid 14 snap -1 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3874: expect deleted 2024-09-15T04:03:43.689 INFO:tasks.rados.rados.0.smithi122.stdout:3875: read oid 45 snap -1 2024-09-15T04:03:43.690 INFO:tasks.rados.rados.0.smithi122.stdout:3875: expect deleted 2024-09-15T04:03:43.690 INFO:tasks.rados.rados.0.smithi122.stdout:3876: rollback oid 49 current snap is 448 2024-09-15T04:03:43.690 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 399 2024-09-15T04:03:43.690 INFO:tasks.rados.rados.0.smithi122.stdout:3877: read oid 12 snap 422 2024-09-15T04:03:43.690 INFO:tasks.rados.rados.0.smithi122.stdout:3877: expect (ObjNum 1172 snap 383 seq_num 1172) 2024-09-15T04:03:43.690 INFO:tasks.rados.rados.0.smithi122.stdout:3878: delete oid 3 current snap is 448 2024-09-15T04:03:43.695 INFO:tasks.rados.rados.0.smithi122.stdout:3877: expect (ObjNum 1172 snap 383 seq_num 1172) 2024-09-15T04:03:43.864 INFO:tasks.rados.rados.0.smithi122.stdout:3876: finishing rollback tid 1 to smithi12223906-49 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3876: finishing rollback tid 2 to smithi12223906-49 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 2914 (ObjNum 1066 snap 345 seq_num 1066) dirty exists 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3872: finishing write tid 3 to smithi12223906-40 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3872: finishing write tid 4 to smithi12223906-40 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3872: finishing write tid 5 to smithi12223906-40 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 40 v 2926 (ObjNum 1366 snap 448 seq_num 1366) dirty exists 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3872: left oid 40 (ObjNum 1366 snap 448 seq_num 1366) 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3873: finishing rollback tid 0 to smithi12223906-33 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3873: finishing rollback tid 1 to smithi12223906-33 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:3873: finishing rollback tid 2 to smithi12223906-33 2024-09-15T04:03:43.865 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 3433 (ObjNum 1173 snap 384 seq_num 1173) dirty exists 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3872: done (6 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3873: done (5 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3874: done (4 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3875: done (3 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3876: done (2 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3877: done (1 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3878: done (0 left) 2024-09-15T04:03:43.866 INFO:tasks.rados.rados.0.smithi122.stdout:3879: delete oid 20 current snap is 448 2024-09-15T04:03:43.876 INFO:tasks.rados.rados.0.smithi122.stdout:3879: done (0 left) 2024-09-15T04:03:43.876 INFO:tasks.rados.rados.0.smithi122.stdout:3880: read oid 11 snap -1 2024-09-15T04:03:43.876 INFO:tasks.rados.rados.0.smithi122.stdout:3880: expect (ObjNum 1358 snap 443 seq_num 1358) 2024-09-15T04:03:43.876 INFO:tasks.rados.rados.0.smithi122.stdout:3881: write oid 38 current snap is 448 2024-09-15T04:03:43.876 INFO:tasks.rados.rados.0.smithi122.stdout:3881: seq_num 1369 ranges {608783=759421,2027143=421133} 2024-09-15T04:03:43.890 INFO:tasks.rados.rados.0.smithi122.stdout:3881: writing smithi12223906-38 from 608783 to 1368204 tid 1 2024-09-15T04:03:43.896 INFO:tasks.rados.rados.0.smithi122.stdout:3881: writing smithi12223906-38 from 2027143 to 2448276 tid 2 2024-09-15T04:03:43.897 INFO:tasks.rados.rados.0.smithi122.stdout:3882: write (excl) oid 39 current snap is 448 2024-09-15T04:03:43.897 INFO:tasks.rados.rados.0.smithi122.stdout:3882: seq_num 1370 ranges {625590=434670,1685625=681144,2822206=203384} 2024-09-15T04:03:43.902 INFO:tasks.rados.rados.0.smithi122.stdout:3882: writing smithi12223906-39 from 625590 to 1060260 tid 1 2024-09-15T04:03:43.910 INFO:tasks.rados.rados.0.smithi122.stdout:3882: writing smithi12223906-39 from 1685625 to 2366769 tid 2 2024-09-15T04:03:43.914 INFO:tasks.rados.rados.0.smithi122.stdout:3882: writing smithi12223906-39 from 2822206 to 3025590 tid 3 2024-09-15T04:03:43.914 INFO:tasks.rados.rados.0.smithi122.stdout:3883: write oid 34 current snap is 448 2024-09-15T04:03:43.914 INFO:tasks.rados.rados.0.smithi122.stdout:3883: seq_num 1371 ranges {642397=509919,1627754=424802,2733276=309121} 2024-09-15T04:03:43.920 INFO:tasks.rados.rados.0.smithi122.stdout:3883: writing smithi12223906-34 from 642397 to 1152316 tid 1 2024-09-15T04:03:43.925 INFO:tasks.rados.rados.0.smithi122.stdout:3883: writing smithi12223906-34 from 1627754 to 2052556 tid 2 2024-09-15T04:03:43.930 INFO:tasks.rados.rados.0.smithi122.stdout:3883: writing smithi12223906-34 from 2733276 to 3042397 tid 3 2024-09-15T04:03:43.930 INFO:tasks.rados.rados.0.smithi122.stdout:3884: snap_remove snap 410 2024-09-15T04:03:44.560 INFO:tasks.rados.rados.0.smithi122.stdout:3884: done (4 left) 2024-09-15T04:03:44.560 INFO:tasks.rados.rados.0.smithi122.stdout:3885: write oid 18 current snap is 448 2024-09-15T04:03:44.560 INFO:tasks.rados.rados.0.smithi122.stdout:3885: seq_num 1372 ranges {659204=585168,1686236=684813} 2024-09-15T04:03:44.568 INFO:tasks.rados.rados.0.smithi122.stdout:3885: writing smithi12223906-18 from 659204 to 1244372 tid 1 2024-09-15T04:03:44.576 INFO:tasks.rados.rados.0.smithi122.stdout:3885: writing smithi12223906-18 from 1686236 to 2371049 tid 2 2024-09-15T04:03:44.577 INFO:tasks.rados.rados.0.smithi122.stdout:3886: write (excl) oid 50 current snap is 448 2024-09-15T04:03:44.577 INFO:tasks.rados.rados.0.smithi122.stdout:3880: expect (ObjNum 1358 snap 443 seq_num 1358) 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3881: finishing write tid 1 to smithi12223906-38 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3881: finishing write tid 2 to smithi12223906-38 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3881: finishing write tid 3 to smithi12223906-38 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3881: finishing write tid 4 to smithi12223906-38 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 3088 (ObjNum 1369 snap 448 seq_num 1369) dirty exists 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3881: left oid 38 (ObjNum 1369 snap 448 seq_num 1369) 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3882: finishing write tid 1 to smithi12223906-39 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3882: finishing write tid 2 to smithi12223906-39 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3882: finishing write tid 3 to smithi12223906-39 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3882: finishing write tid 4 to smithi12223906-39 2024-09-15T04:03:44.670 INFO:tasks.rados.rados.0.smithi122.stdout:3882: finishing write tid 5 to smithi12223906-39 2024-09-15T04:03:44.671 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 39 v 3416 (ObjNum 1370 snap 448 seq_num 1370) dirty exists 2024-09-15T04:03:44.671 INFO:tasks.rados.rados.0.smithi122.stdout:3882: left oid 39 (ObjNum 1370 snap 448 seq_num 1370) 2024-09-15T04:03:44.671 INFO:tasks.rados.rados.0.smithi122.stdout:3886: seq_num 1373 ranges {676011=660417,1744718=428471,2788122=287889} 2024-09-15T04:03:44.683 INFO:tasks.rados.rados.0.smithi122.stdout:3886: writing smithi12223906-50 from 676011 to 1336428 tid 1 2024-09-15T04:03:44.688 INFO:tasks.rados.rados.0.smithi122.stdout:3886: writing smithi12223906-50 from 1744718 to 2173189 tid 2 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:3886: writing smithi12223906-50 from 2788122 to 3076011 tid 3 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:3880: done (5 left) 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:3881: done (4 left) 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:3882: done (3 left) 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:3887: rollback oid 12 current snap is 448 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 12 to 417 2024-09-15T04:03:44.694 INFO:tasks.rados.rados.0.smithi122.stdout:3883: finishing write tid 1 to smithi12223906-34 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3883: finishing write tid 2 to smithi12223906-34 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3883: finishing write tid 3 to smithi12223906-34 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3883: finishing write tid 4 to smithi12223906-34 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3883: finishing write tid 5 to smithi12223906-34 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 3272 (ObjNum 1371 snap 448 seq_num 1371) dirty exists 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3883: left oid 34 (ObjNum 1371 snap 448 seq_num 1371) 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3885: finishing write tid 1 to smithi12223906-18 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3885: finishing write tid 2 to smithi12223906-18 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3885: finishing write tid 3 to smithi12223906-18 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3885: finishing write tid 4 to smithi12223906-18 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3441 (ObjNum 1372 snap 448 seq_num 1372) dirty exists 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3885: left oid 18 (ObjNum 1372 snap 448 seq_num 1372) 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3883: done (3 left) 2024-09-15T04:03:44.695 INFO:tasks.rados.rados.0.smithi122.stdout:3885: done (2 left) 2024-09-15T04:03:44.696 INFO:tasks.rados.rados.0.smithi122.stdout:3888: delete oid 20 current snap is 448 2024-09-15T04:03:44.696 INFO:tasks.rados.rados.0.smithi122.stdout:3888: done (2 left) 2024-09-15T04:03:44.696 INFO:tasks.rados.rados.0.smithi122.stdout:3889: rollback oid 41 current snap is 448 2024-09-15T04:03:44.696 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 383 2024-09-15T04:03:44.696 INFO:tasks.rados.rados.0.smithi122.stdout:3890: snap_create 2024-09-15T04:03:44.701 INFO:tasks.rados.rados.0.smithi122.stdout:3889: finishing rollback tid 0 to smithi12223906-41 2024-09-15T04:03:44.701 INFO:tasks.rados.rados.0.smithi122.stdout:3887: finishing rollback tid 0 to smithi12223906-12 2024-09-15T04:03:44.701 INFO:tasks.rados.rados.0.smithi122.stdout:3887: finishing rollback tid 1 to smithi12223906-12 2024-09-15T04:03:44.702 INFO:tasks.rados.rados.0.smithi122.stdout:3889: finishing rollback tid 1 to smithi12223906-41 2024-09-15T04:03:44.702 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2928 (ObjNum 129 snap 0 seq_num 1843039344) dirty dne 2024-09-15T04:03:44.703 INFO:tasks.rados.rados.0.smithi122.stdout:3887: finishing rollback tid 2 to smithi12223906-12 2024-09-15T04:03:44.703 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 12 v 3275 (ObjNum 1242 snap 406 seq_num 1242) dirty exists 2024-09-15T04:03:44.703 INFO:tasks.rados.rados.0.smithi122.stdout:3886: finishing write tid 1 to smithi12223906-50 2024-09-15T04:03:44.703 INFO:tasks.rados.rados.0.smithi122.stdout:3886: finishing write tid 2 to smithi12223906-50 2024-09-15T04:03:44.704 INFO:tasks.rados.rados.0.smithi122.stdout:3886: finishing write tid 3 to smithi12223906-50 2024-09-15T04:03:44.704 INFO:tasks.rados.rados.0.smithi122.stdout:3886: finishing write tid 4 to smithi12223906-50 2024-09-15T04:03:44.705 INFO:tasks.rados.rados.0.smithi122.stdout:3886: finishing write tid 5 to smithi12223906-50 2024-09-15T04:03:44.705 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 50 v 3449 (ObjNum 1373 snap 448 seq_num 1373) dirty exists 2024-09-15T04:03:44.705 INFO:tasks.rados.rados.0.smithi122.stdout:3886: left oid 50 (ObjNum 1373 snap 448 seq_num 1373) 2024-09-15T04:03:45.667 INFO:tasks.rados.rados.0.smithi122.stdout:3886: done (3 left) 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:3887: done (2 left) 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:3889: done (1 left) 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:3890: done (0 left) 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:3891: rollback oid 14 current snap is 449 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 14 to 423 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:3892: write oid 49 current snap is 449 2024-09-15T04:03:45.668 INFO:tasks.rados.rados.0.smithi122.stdout:3892: seq_num 1375 ranges {709625=410915,1745329=432140,2726615=383010} 2024-09-15T04:03:45.681 INFO:tasks.rados.rados.0.smithi122.stdout:3892: writing smithi12223906-49 from 709625 to 1120540 tid 1 2024-09-15T04:03:45.688 INFO:tasks.rados.rados.0.smithi122.stdout:3892: writing smithi12223906-49 from 1745329 to 2177469 tid 2 2024-09-15T04:03:45.693 INFO:tasks.rados.rados.0.smithi122.stdout:3892: writing smithi12223906-49 from 2726615 to 3109625 tid 3 2024-09-15T04:03:45.694 INFO:tasks.rados.rados.0.smithi122.stdout:3893: read oid 42 snap -1 2024-09-15T04:03:45.694 INFO:tasks.rados.rados.0.smithi122.stdout:3893: expect (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:03:45.694 INFO:tasks.rados.rados.0.smithi122.stdout:3894: snap_remove snap 409 2024-09-15T04:03:46.668 INFO:tasks.rados.rados.0.smithi122.stdout:3894: done (3 left) 2024-09-15T04:03:46.668 INFO:tasks.rados.rados.0.smithi122.stdout:3895: read oid 17 snap -1 2024-09-15T04:03:46.669 INFO:tasks.rados.rados.0.smithi122.stdout:3895: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:3891: finishing rollback tid 1 to smithi12223906-14 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:3891: finishing rollback tid 2 to smithi12223906-14 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 3432 (ObjNum 1018 snap 319 seq_num 1018) dirty exists 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:3892: finishing write tid 1 to smithi12223906-49 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:3892: finishing write tid 2 to smithi12223906-49 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:3892: finishing write tid 3 to smithi12223906-49 2024-09-15T04:03:46.670 INFO:tasks.rados.rados.0.smithi122.stdout:3892: finishing write tid 4 to smithi12223906-49 2024-09-15T04:03:46.671 INFO:tasks.rados.rados.0.smithi122.stdout:3892: finishing write tid 5 to smithi12223906-49 2024-09-15T04:03:46.671 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 2918 (ObjNum 1375 snap 449 seq_num 1375) dirty exists 2024-09-15T04:03:46.671 INFO:tasks.rados.rados.0.smithi122.stdout:3892: left oid 49 (ObjNum 1375 snap 449 seq_num 1375) 2024-09-15T04:03:46.671 INFO:tasks.rados.rados.0.smithi122.stdout:3893: expect (ObjNum 1186 snap 390 seq_num 1186) 2024-09-15T04:03:46.859 INFO:tasks.rados.rados.0.smithi122.stdout:3895: expect (ObjNum 1247 snap 407 seq_num 1247) 2024-09-15T04:03:46.956 INFO:tasks.rados.rados.0.smithi122.stdout:3891: done (3 left) 2024-09-15T04:03:46.956 INFO:tasks.rados.rados.0.smithi122.stdout:3892: done (2 left) 2024-09-15T04:03:46.956 INFO:tasks.rados.rados.0.smithi122.stdout:3893: done (1 left) 2024-09-15T04:03:46.956 INFO:tasks.rados.rados.0.smithi122.stdout:3895: done (0 left) 2024-09-15T04:03:46.956 INFO:tasks.rados.rados.0.smithi122.stdout:3896: copy_from oid 37 from oid 5 current snap is 449 2024-09-15T04:03:46.957 INFO:tasks.rados.rados.0.smithi122.stdout:3897: write (excl) oid 17 current snap is 449 2024-09-15T04:03:46.957 INFO:tasks.rados.rados.0.smithi122.stdout:3897: seq_num 1376 ranges {726432=486164,1803811=692151} 2024-09-15T04:03:46.972 INFO:tasks.rados.rados.0.smithi122.stdout:3897: writing smithi12223906-17 from 726432 to 1212596 tid 1 2024-09-15T04:03:46.980 INFO:tasks.rados.rados.0.smithi122.stdout:3897: writing smithi12223906-17 from 1803811 to 2495962 tid 2 2024-09-15T04:03:46.980 INFO:tasks.rados.rados.0.smithi122.stdout:3898: delete oid 21 current snap is 449 2024-09-15T04:03:46.980 INFO:tasks.rados.rados.0.smithi122.stdout:3896: finishing copy_from to smithi12223906-37 2024-09-15T04:03:46.980 INFO:tasks.rados.rados.0.smithi122.stdout:3896: got expected ENOENT (src dne) 2024-09-15T04:03:46.980 INFO:tasks.rados.rados.0.smithi122.stdout:3896: finishing copy_from racing read to smithi12223906-37 2024-09-15T04:03:46.984 INFO:tasks.rados.rados.0.smithi122.stdout:3897: finishing write tid 1 to smithi12223906-17 2024-09-15T04:03:46.986 INFO:tasks.rados.rados.0.smithi122.stdout:3897: finishing write tid 2 to smithi12223906-17 2024-09-15T04:03:46.989 INFO:tasks.rados.rados.0.smithi122.stdout:3896: done (2 left) 2024-09-15T04:03:46.989 INFO:tasks.rados.rados.0.smithi122.stdout:3898: done (1 left) 2024-09-15T04:03:46.989 INFO:tasks.rados.rados.0.smithi122.stdout:3899: read oid 38 snap -1 2024-09-15T04:03:46.989 INFO:tasks.rados.rados.0.smithi122.stdout:3899: expect (ObjNum 1369 snap 448 seq_num 1369) 2024-09-15T04:03:46.989 INFO:tasks.rados.rados.0.smithi122.stdout:3900: write (excl) oid 23 current snap is 449 2024-09-15T04:03:46.989 INFO:tasks.rados.rados.0.smithi122.stdout:3900: seq_num 1378 ranges {760046=753015,2037128=695820} 2024-09-15T04:03:46.998 INFO:tasks.rados.rados.0.smithi122.stdout:3900: writing smithi12223906-23 from 760046 to 1513061 tid 1 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3900: writing smithi12223906-23 from 2037128 to 2732948 tid 2 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3901: read oid 48 snap -1 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3901: expect (ObjNum 1365 snap 447 seq_num 1365) 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3902: read oid 31 snap -1 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3902: expect (ObjNum 1364 snap 447 seq_num 1364) 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3903: snap_create 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3897: finishing write tid 3 to smithi12223906-17 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3897: finishing write tid 4 to smithi12223906-17 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 17 v 2921 (ObjNum 1376 snap 449 seq_num 1376) dirty exists 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3897: left oid 17 (ObjNum 1376 snap 449 seq_num 1376) 2024-09-15T04:03:47.007 INFO:tasks.rados.rados.0.smithi122.stdout:3899: expect (ObjNum 1369 snap 448 seq_num 1369) 2024-09-15T04:03:47.184 INFO:tasks.rados.rados.0.smithi122.stdout:3900: finishing write tid 1 to smithi12223906-23 2024-09-15T04:03:47.184 INFO:tasks.rados.rados.0.smithi122.stdout:3901: expect (ObjNum 1365 snap 447 seq_num 1365) 2024-09-15T04:03:47.379 INFO:tasks.rados.rados.0.smithi122.stdout:3900: finishing write tid 2 to smithi12223906-23 2024-09-15T04:03:47.379 INFO:tasks.rados.rados.0.smithi122.stdout:3900: finishing write tid 3 to smithi12223906-23 2024-09-15T04:03:47.379 INFO:tasks.rados.rados.0.smithi122.stdout:3900: finishing write tid 4 to smithi12223906-23 2024-09-15T04:03:47.379 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 3460 (ObjNum 1378 snap 449 seq_num 1378) dirty exists 2024-09-15T04:03:47.379 INFO:tasks.rados.rados.0.smithi122.stdout:3900: left oid 23 (ObjNum 1378 snap 449 seq_num 1378) 2024-09-15T04:03:47.379 INFO:tasks.rados.rados.0.smithi122.stdout:3902: expect (ObjNum 1364 snap 447 seq_num 1364) 2024-09-15T04:03:47.685 INFO:tasks.rados.rados.0.smithi122.stdout:3897: done (5 left) 2024-09-15T04:03:47.685 INFO:tasks.rados.rados.0.smithi122.stdout:3899: done (4 left) 2024-09-15T04:03:47.686 INFO:tasks.rados.rados.0.smithi122.stdout:3900: done (3 left) 2024-09-15T04:03:47.686 INFO:tasks.rados.rados.0.smithi122.stdout:3901: done (2 left) 2024-09-15T04:03:47.686 INFO:tasks.rados.rados.0.smithi122.stdout:3902: done (1 left) 2024-09-15T04:03:47.686 INFO:tasks.rados.rados.0.smithi122.stdout:3903: done (0 left) 2024-09-15T04:03:47.686 INFO:tasks.rados.rados.0.smithi122.stdout:3904: write (excl) oid 38 current snap is 450 2024-09-15T04:03:47.686 INFO:tasks.rados.rados.0.smithi122.stdout:3904: seq_num 1379 ranges {776853=428264,1979257=439478,2836307=340546} 2024-09-15T04:03:47.697 INFO:tasks.rados.rados.0.smithi122.stdout:3904: writing smithi12223906-38 from 776853 to 1205117 tid 1 2024-09-15T04:03:47.704 INFO:tasks.rados.rados.0.smithi122.stdout:3904: writing smithi12223906-38 from 1979257 to 2418735 tid 2 2024-09-15T04:03:47.708 INFO:tasks.rados.rados.0.smithi122.stdout:3904: writing smithi12223906-38 from 2836307 to 3176853 tid 3 2024-09-15T04:03:47.708 INFO:tasks.rados.rados.0.smithi122.stdout:3905: rollback oid 18 current snap is 450 2024-09-15T04:03:47.709 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 18 to 437 2024-09-15T04:03:47.709 INFO:tasks.rados.rados.0.smithi122.stdout:3906: snap_remove snap 333 2024-09-15T04:03:48.701 INFO:tasks.rados.rados.0.smithi122.stdout:3906: done (2 left) 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3907: delete oid 35 current snap is 450 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3904: finishing write tid 1 to smithi12223906-38 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3904: finishing write tid 2 to smithi12223906-38 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3904: finishing write tid 3 to smithi12223906-38 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3904: finishing write tid 4 to smithi12223906-38 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3904: finishing write tid 5 to smithi12223906-38 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 3092 (ObjNum 1379 snap 450 seq_num 1379) dirty exists 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3904: left oid 38 (ObjNum 1379 snap 450 seq_num 1379) 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3905: finishing rollback tid 0 to smithi12223906-18 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3905: finishing rollback tid 1 to smithi12223906-18 2024-09-15T04:03:48.702 INFO:tasks.rados.rados.0.smithi122.stdout:3905: finishing rollback tid 2 to smithi12223906-18 2024-09-15T04:03:48.703 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 18 v 3464 (ObjNum 1313 snap 432 seq_num 1313) dirty exists 2024-09-15T04:03:48.731 INFO:tasks.rados.rados.0.smithi122.stdout:3904: done (2 left) 2024-09-15T04:03:48.731 INFO:tasks.rados.rados.0.smithi122.stdout:3905: done (1 left) 2024-09-15T04:03:48.731 INFO:tasks.rados.rados.0.smithi122.stdout:3907: done (0 left) 2024-09-15T04:03:48.732 INFO:tasks.rados.rados.0.smithi122.stdout:3908: delete oid 31 current snap is 450 2024-09-15T04:03:48.736 INFO:tasks.rados.rados.0.smithi122.stdout:3908: done (0 left) 2024-09-15T04:03:48.736 INFO:tasks.rados.rados.0.smithi122.stdout:3909: read oid 40 snap 441 2024-09-15T04:03:48.736 INFO:tasks.rados.rados.0.smithi122.stdout:3909: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:03:48.736 INFO:tasks.rados.rados.0.smithi122.stdout:3910: snap_create 2024-09-15T04:03:48.744 INFO:tasks.rados.rados.0.smithi122.stdout:3909: expect (ObjNum 1160 snap 381 seq_num 1160) 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3909: done (1 left) 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3910: done (0 left) 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3911: copy_from oid 24 from oid 29 current snap is 451 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3912: read oid 43 snap -1 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3912: expect (ObjNum 1351 snap 442 seq_num 1351) 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3913: write (excl) oid 47 current snap is 451 2024-09-15T04:03:49.564 INFO:tasks.rados.rados.0.smithi122.stdout:3913: seq_num 1382 ranges {427274=654011,1754703=586805,2918576=308698} 2024-09-15T04:03:49.574 INFO:tasks.rados.rados.0.smithi122.stdout:3913: writing smithi12223906-47 from 427274 to 1081285 tid 1 2024-09-15T04:03:49.582 INFO:tasks.rados.rados.0.smithi122.stdout:3913: writing smithi12223906-47 from 1754703 to 2341508 tid 2 2024-09-15T04:03:49.588 INFO:tasks.rados.rados.0.smithi122.stdout:3913: writing smithi12223906-47 from 2918576 to 3227274 tid 3 2024-09-15T04:03:49.588 INFO:tasks.rados.rados.0.smithi122.stdout:3914: rollback oid 38 current snap is 451 2024-09-15T04:03:49.589 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 38 to 415 2024-09-15T04:03:49.589 INFO:tasks.rados.rados.0.smithi122.stdout:3911: finishing copy_from to smithi12223906-24 2024-09-15T04:03:49.589 INFO:tasks.rados.rados.0.smithi122.stdout:3911: got expected ENOENT (src dne) 2024-09-15T04:03:49.589 INFO:tasks.rados.rados.0.smithi122.stdout:3911: finishing copy_from racing read to smithi12223906-24 2024-09-15T04:03:49.589 INFO:tasks.rados.rados.0.smithi122.stdout:3912: expect (ObjNum 1351 snap 442 seq_num 1351) 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3913: finishing write tid 1 to smithi12223906-47 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3913: finishing write tid 2 to smithi12223906-47 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3913: finishing write tid 3 to smithi12223906-47 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3914: finishing rollback tid 0 to smithi12223906-38 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3913: finishing write tid 4 to smithi12223906-47 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3913: finishing write tid 5 to smithi12223906-47 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 47 v 2635 (ObjNum 1382 snap 451 seq_num 1382) dirty exists 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3913: left oid 47 (ObjNum 1382 snap 451 seq_num 1382) 2024-09-15T04:03:49.679 INFO:tasks.rados.rados.0.smithi122.stdout:3914: finishing rollback tid 1 to smithi12223906-38 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3914: finishing rollback tid 2 to smithi12223906-38 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 3095 (ObjNum 868 snap 281 seq_num 868) dirty exists 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3911: done (3 left) 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3912: done (2 left) 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3913: done (1 left) 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3914: done (0 left) 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3915: write (excl) oid 23 current snap is 451 2024-09-15T04:03:49.680 INFO:tasks.rados.rados.0.smithi122.stdout:3915: seq_num 1383 ranges {444081=729260,1813185=446816,2945999=298082} 2024-09-15T04:03:49.695 INFO:tasks.rados.rados.0.smithi122.stdout:3915: writing smithi12223906-23 from 444081 to 1173341 tid 1 2024-09-15T04:03:49.701 INFO:tasks.rados.rados.0.smithi122.stdout:3915: writing smithi12223906-23 from 1813185 to 2260001 tid 2 2024-09-15T04:03:49.706 INFO:tasks.rados.rados.0.smithi122.stdout:3915: writing smithi12223906-23 from 2945999 to 3244081 tid 3 2024-09-15T04:03:49.707 INFO:tasks.rados.rados.0.smithi122.stdout:3916: delete oid 42 current snap is 451 2024-09-15T04:03:49.713 INFO:tasks.rados.rados.0.smithi122.stdout:3916: done (1 left) 2024-09-15T04:03:49.713 INFO:tasks.rados.rados.0.smithi122.stdout:3917: snap_remove snap 423 2024-09-15T04:03:50.711 INFO:tasks.rados.rados.0.smithi122.stdout:3917: done (1 left) 2024-09-15T04:03:50.712 INFO:tasks.rados.rados.0.smithi122.stdout:3918: write oid 14 current snap is 451 2024-09-15T04:03:50.712 INFO:tasks.rados.rados.0.smithi122.stdout:3915: finishing write tid 1 to smithi12223906-23 2024-09-15T04:03:50.712 INFO:tasks.rados.rados.0.smithi122.stdout:3918: seq_num 1385 ranges {477695=596111,1530149=450485,2600845=666679} 2024-09-15T04:03:50.723 INFO:tasks.rados.rados.0.smithi122.stdout:3918: writing smithi12223906-14 from 477695 to 1073806 tid 1 2024-09-15T04:03:50.728 INFO:tasks.rados.rados.0.smithi122.stdout:3918: writing smithi12223906-14 from 1530149 to 1980634 tid 2 2024-09-15T04:03:50.736 INFO:tasks.rados.rados.0.smithi122.stdout:3918: writing smithi12223906-14 from 2600845 to 3267524 tid 3 2024-09-15T04:03:50.737 INFO:tasks.rados.rados.0.smithi122.stdout:3919: write oid 48 current snap is 451 2024-09-15T04:03:50.737 INFO:tasks.rados.rados.0.smithi122.stdout:3915: finishing write tid 2 to smithi12223906-23 2024-09-15T04:03:50.737 INFO:tasks.rados.rados.0.smithi122.stdout:3919: seq_num 1386 ranges {494502=671360,1588631=594143,2628268=477951} 2024-09-15T04:03:50.743 INFO:tasks.rados.rados.0.smithi122.stdout:3919: writing smithi12223906-48 from 494502 to 1165862 tid 1 2024-09-15T04:03:50.749 INFO:tasks.rados.rados.0.smithi122.stdout:3919: writing smithi12223906-48 from 1588631 to 2182774 tid 2 2024-09-15T04:03:50.755 INFO:tasks.rados.rados.0.smithi122.stdout:3919: writing smithi12223906-48 from 2628268 to 3106219 tid 3 2024-09-15T04:03:50.755 INFO:tasks.rados.rados.0.smithi122.stdout:3920: read oid 50 snap -1 2024-09-15T04:03:50.755 INFO:tasks.rados.rados.0.smithi122.stdout:3920: expect (ObjNum 1373 snap 448 seq_num 1373) 2024-09-15T04:03:50.755 INFO:tasks.rados.rados.0.smithi122.stdout:3921: read oid 36 snap -1 2024-09-15T04:03:50.755 INFO:tasks.rados.rados.0.smithi122.stdout:3921: expect deleted 2024-09-15T04:03:50.755 INFO:tasks.rados.rados.0.smithi122.stdout:3922: snap_remove snap 437 2024-09-15T04:03:51.723 INFO:tasks.rados.rados.0.smithi122.stdout:3915: finishing write tid 3 to smithi12223906-23 2024-09-15T04:03:51.723 INFO:tasks.rados.rados.0.smithi122.stdout:3915: finishing write tid 4 to smithi12223906-23 2024-09-15T04:03:51.723 INFO:tasks.rados.rados.0.smithi122.stdout:3915: finishing write tid 5 to smithi12223906-23 2024-09-15T04:03:51.723 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 3481 (ObjNum 1383 snap 451 seq_num 1383) dirty exists 2024-09-15T04:03:51.723 INFO:tasks.rados.rados.0.smithi122.stdout:3915: left oid 23 (ObjNum 1383 snap 451 seq_num 1383) 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3918: finishing write tid 1 to smithi12223906-14 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3918: finishing write tid 2 to smithi12223906-14 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3918: finishing write tid 3 to smithi12223906-14 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3918: finishing write tid 4 to smithi12223906-14 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3918: finishing write tid 5 to smithi12223906-14 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 3473 (ObjNum 1385 snap 451 seq_num 1385) dirty exists 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3918: left oid 14 (ObjNum 1385 snap 451 seq_num 1385) 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3919: finishing write tid 1 to smithi12223906-48 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3919: finishing write tid 2 to smithi12223906-48 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3919: finishing write tid 3 to smithi12223906-48 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3919: finishing write tid 4 to smithi12223906-48 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:3919: finishing write tid 5 to smithi12223906-48 2024-09-15T04:03:51.724 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2167 (ObjNum 1386 snap 451 seq_num 1386) dirty exists 2024-09-15T04:03:51.725 INFO:tasks.rados.rados.0.smithi122.stdout:3919: left oid 48 (ObjNum 1386 snap 451 seq_num 1386) 2024-09-15T04:03:51.725 INFO:tasks.rados.rados.0.smithi122.stdout:3920: expect (ObjNum 1373 snap 448 seq_num 1373) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3915: done (5 left) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3918: done (4 left) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3919: done (3 left) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3920: done (2 left) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3921: done (1 left) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3922: done (0 left) 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:3923: rollback oid 23 current snap is 451 2024-09-15T04:03:51.826 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 23 to 449 2024-09-15T04:03:51.827 INFO:tasks.rados.rados.0.smithi122.stdout:3924: delete oid 36 current snap is 451 2024-09-15T04:03:51.835 INFO:tasks.rados.rados.0.smithi122.stdout:3924: done (1 left) 2024-09-15T04:03:51.835 INFO:tasks.rados.rados.0.smithi122.stdout:3925: copy_from oid 4 from oid 44 current snap is 451 2024-09-15T04:03:51.835 INFO:tasks.rados.rados.0.smithi122.stdout:3926: snap_remove snap 364 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:3923: finishing rollback tid 0 to smithi12223906-23 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:3923: finishing rollback tid 1 to smithi12223906-23 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:3923: finishing rollback tid 2 to smithi12223906-23 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 23 v 3500 (ObjNum 1378 snap 449 seq_num 1378) dirty exists 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:3925: finishing copy_from racing read to smithi12223906-4 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:3925: finishing copy_from to smithi12223906-4 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 4 v 3260 (ObjNum 1331 snap 436 seq_num 1331) dirty exists 2024-09-15T04:03:52.733 INFO:tasks.rados.rados.0.smithi122.stdout:3923: done (2 left) 2024-09-15T04:03:52.734 INFO:tasks.rados.rados.0.smithi122.stdout:3925: done (1 left) 2024-09-15T04:03:52.734 INFO:tasks.rados.rados.0.smithi122.stdout:3926: done (0 left) 2024-09-15T04:03:52.734 INFO:tasks.rados.rados.0.smithi122.stdout:3927: copy_from oid 47 from oid 25 current snap is 451 2024-09-15T04:03:52.734 INFO:tasks.rados.rados.0.smithi122.stdout:3928: write oid 27 current snap is 451 2024-09-15T04:03:52.734 INFO:tasks.rados.rados.0.smithi122.stdout:3928: seq_num 1388 ranges {528116=421858,1589242=597812,2966761=361355} 2024-09-15T04:03:52.742 INFO:tasks.rados.rados.0.smithi122.stdout:3928: writing smithi12223906-27 from 528116 to 949974 tid 1 2024-09-15T04:03:52.751 INFO:tasks.rados.rados.0.smithi122.stdout:3928: writing smithi12223906-27 from 1589242 to 2187054 tid 2 2024-09-15T04:03:52.757 INFO:tasks.rados.rados.0.smithi122.stdout:3928: writing smithi12223906-27 from 2966761 to 3328116 tid 3 2024-09-15T04:03:52.757 INFO:tasks.rados.rados.0.smithi122.stdout:3929: write (excl) oid 41 current snap is 451 2024-09-15T04:03:52.757 INFO:tasks.rados.rados.0.smithi122.stdout:3927: finishing copy_from to smithi12223906-47 2024-09-15T04:03:52.757 INFO:tasks.rados.rados.0.smithi122.stdout:3927: got expected ENOENT (src dne) 2024-09-15T04:03:52.757 INFO:tasks.rados.rados.0.smithi122.stdout:3929: seq_num 1389 ranges {544923=497107,1647724=457823,2710537=428120} 2024-09-15T04:03:52.762 INFO:tasks.rados.rados.0.smithi122.stdout:3929: writing smithi12223906-41 from 544923 to 1042030 tid 1 2024-09-15T04:03:52.768 INFO:tasks.rados.rados.0.smithi122.stdout:3929: writing smithi12223906-41 from 1647724 to 2105547 tid 2 2024-09-15T04:03:52.775 INFO:tasks.rados.rados.0.smithi122.stdout:3929: writing smithi12223906-41 from 2710537 to 3138657 tid 3 2024-09-15T04:03:52.775 INFO:tasks.rados.rados.0.smithi122.stdout:3930: rollback oid 49 current snap is 451 2024-09-15T04:03:52.775 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 49 to 440 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3927: finishing copy_from racing read to smithi12223906-47 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: finishing write tid 1 to smithi12223906-27 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: finishing write tid 2 to smithi12223906-27 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: finishing write tid 3 to smithi12223906-27 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: finishing write tid 4 to smithi12223906-27 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: finishing write tid 5 to smithi12223906-27 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 27 v 3516 (ObjNum 1388 snap 451 seq_num 1388) dirty exists 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: left oid 27 (ObjNum 1388 snap 451 seq_num 1388) 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3929: finishing write tid 1 to smithi12223906-41 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3929: finishing write tid 2 to smithi12223906-41 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3927: done (3 left) 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3928: done (2 left) 2024-09-15T04:03:52.776 INFO:tasks.rados.rados.0.smithi122.stdout:3931: rollback oid 45 current snap is 451 2024-09-15T04:03:52.777 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 45 to 433 2024-09-15T04:03:52.777 INFO:tasks.rados.rados.0.smithi122.stdout:3932: snap_remove snap 446 2024-09-15T04:03:53.739 INFO:tasks.rados.rados.0.smithi122.stdout:3931: finishing rollback tid 1 to smithi12223906-45 2024-09-15T04:03:53.739 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 45 v 2629 (ObjNum 177 snap 0 seq_num 1842551104) dirty dne 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3929: finishing write tid 3 to smithi12223906-41 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3929: finishing write tid 4 to smithi12223906-41 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3929: finishing write tid 5 to smithi12223906-41 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2932 (ObjNum 1389 snap 451 seq_num 1389) dirty exists 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3929: left oid 41 (ObjNum 1389 snap 451 seq_num 1389) 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3930: finishing rollback tid 0 to smithi12223906-49 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3930: finishing rollback tid 1 to smithi12223906-49 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 2923 (ObjNum 129 snap 0 seq_num 1842920920) dirty dne 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3929: done (3 left) 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3930: done (2 left) 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3931: done (1 left) 2024-09-15T04:03:53.740 INFO:tasks.rados.rados.0.smithi122.stdout:3932: done (0 left) 2024-09-15T04:03:53.741 INFO:tasks.rados.rados.0.smithi122.stdout:3933: delete oid 32 current snap is 451 2024-09-15T04:03:53.763 INFO:tasks.rados.rados.0.smithi122.stdout:3933: done (0 left) 2024-09-15T04:03:53.764 INFO:tasks.rados.rados.0.smithi122.stdout:3934: read oid 41 snap -1 2024-09-15T04:03:53.764 INFO:tasks.rados.rados.0.smithi122.stdout:3934: expect (ObjNum 1389 snap 451 seq_num 1389) 2024-09-15T04:03:53.764 INFO:tasks.rados.rados.0.smithi122.stdout:3935: copy_from oid 29 from oid 10 current snap is 451 2024-09-15T04:03:53.764 INFO:tasks.rados.rados.0.smithi122.stdout:3936: snap_remove snap 391 2024-09-15T04:03:54.572 INFO:tasks.rados.rados.0.smithi122.stdout:3935: finishing copy_from to smithi12223906-29 2024-09-15T04:03:54.572 INFO:tasks.rados.rados.0.smithi122.stdout:3935: got expected ENOENT (src dne) 2024-09-15T04:03:54.572 INFO:tasks.rados.rados.0.smithi122.stdout:3935: finishing copy_from racing read to smithi12223906-29 2024-09-15T04:03:54.572 INFO:tasks.rados.rados.0.smithi122.stdout:3934: expect (ObjNum 1389 snap 451 seq_num 1389) 2024-09-15T04:03:54.686 INFO:tasks.rados.rados.0.smithi122.stdout:3934: done (2 left) 2024-09-15T04:03:54.686 INFO:tasks.rados.rados.0.smithi122.stdout:3935: done (1 left) 2024-09-15T04:03:54.686 INFO:tasks.rados.rados.0.smithi122.stdout:3936: done (0 left) 2024-09-15T04:03:54.686 INFO:tasks.rados.rados.0.smithi122.stdout:3937: snap_remove snap 436 2024-09-15T04:03:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3937: done (0 left) 2024-09-15T04:03:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3938: rollback oid 22 current snap is 451 2024-09-15T04:03:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 22 to 433 2024-09-15T04:03:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3939: copy_from oid 3 from oid 42 current snap is 451 2024-09-15T04:03:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3940: write (excl) oid 19 current snap is 451 2024-09-15T04:03:55.592 INFO:tasks.rados.rados.0.smithi122.stdout:3940: seq_num 1391 ranges {578537=647605,1764688=745139,3049030=329507} 2024-09-15T04:03:55.598 INFO:tasks.rados.rados.0.smithi122.stdout:3940: writing smithi12223906-19 from 578537 to 1226142 tid 1 2024-09-15T04:03:55.605 INFO:tasks.rados.rados.0.smithi122.stdout:3940: writing smithi12223906-19 from 1764688 to 2509827 tid 2 2024-09-15T04:03:55.610 INFO:tasks.rados.rados.0.smithi122.stdout:3940: writing smithi12223906-19 from 3049030 to 3378537 tid 3 2024-09-15T04:03:55.610 INFO:tasks.rados.rados.0.smithi122.stdout:3941: write oid 33 current snap is 451 2024-09-15T04:03:55.610 INFO:tasks.rados.rados.0.smithi122.stdout:3941: seq_num 1392 ranges {595344=439207,1823170=605150,3076453=318891} 2024-09-15T04:03:55.616 INFO:tasks.rados.rados.0.smithi122.stdout:3941: writing smithi12223906-33 from 595344 to 1034551 tid 1 2024-09-15T04:03:55.624 INFO:tasks.rados.rados.0.smithi122.stdout:3941: writing smithi12223906-33 from 1823170 to 2428320 tid 2 2024-09-15T04:03:55.629 INFO:tasks.rados.rados.0.smithi122.stdout:3941: writing smithi12223906-33 from 3076453 to 3395344 tid 3 2024-09-15T04:03:55.630 INFO:tasks.rados.rados.0.smithi122.stdout:3942: write oid 35 current snap is 451 2024-09-15T04:03:55.630 INFO:tasks.rados.rados.0.smithi122.stdout:3939: finishing copy_from to smithi12223906-3 2024-09-15T04:03:55.630 INFO:tasks.rados.rados.0.smithi122.stdout:3939: got expected ENOENT (src dne) 2024-09-15T04:03:55.630 INFO:tasks.rados.rados.0.smithi122.stdout:3942: seq_num 1393 ranges {612151=514456,1881652=748808,3103876=308275} 2024-09-15T04:03:55.636 INFO:tasks.rados.rados.0.smithi122.stdout:3942: writing smithi12223906-35 from 612151 to 1126607 tid 1 2024-09-15T04:03:55.644 INFO:tasks.rados.rados.0.smithi122.stdout:3942: writing smithi12223906-35 from 1881652 to 2630460 tid 2 2024-09-15T04:03:55.649 INFO:tasks.rados.rados.0.smithi122.stdout:3942: writing smithi12223906-35 from 3103876 to 3412151 tid 3 2024-09-15T04:03:55.649 INFO:tasks.rados.rados.0.smithi122.stdout:3943: delete oid 36 current snap is 451 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3939: finishing copy_from racing read to smithi12223906-3 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3940: finishing write tid 1 to smithi12223906-19 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3938: finishing rollback tid 0 to smithi12223906-22 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3938: finishing rollback tid 1 to smithi12223906-22 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3938: finishing rollback tid 2 to smithi12223906-22 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 3544 (ObjNum 1216 snap 401 seq_num 1216) dirty exists 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3940: finishing write tid 2 to smithi12223906-19 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3940: finishing write tid 3 to smithi12223906-19 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3940: finishing write tid 4 to smithi12223906-19 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3940: finishing write tid 5 to smithi12223906-19 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 19 v 3309 (ObjNum 1391 snap 451 seq_num 1391) dirty exists 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3940: left oid 19 (ObjNum 1391 snap 451 seq_num 1391) 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3941: finishing write tid 1 to smithi12223906-33 2024-09-15T04:03:55.650 INFO:tasks.rados.rados.0.smithi122.stdout:3941: finishing write tid 2 to smithi12223906-33 2024-09-15T04:03:55.651 INFO:tasks.rados.rados.0.smithi122.stdout:3941: finishing write tid 3 to smithi12223906-33 2024-09-15T04:03:55.651 INFO:tasks.rados.rados.0.smithi122.stdout:3941: finishing write tid 4 to smithi12223906-33 2024-09-15T04:03:55.651 INFO:tasks.rados.rados.0.smithi122.stdout:3941: finishing write tid 5 to smithi12223906-33 2024-09-15T04:03:55.651 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 3551 (ObjNum 1392 snap 451 seq_num 1392) dirty exists 2024-09-15T04:03:55.651 INFO:tasks.rados.rados.0.smithi122.stdout:3941: left oid 33 (ObjNum 1392 snap 451 seq_num 1392) 2024-09-15T04:03:55.651 INFO:tasks.rados.rados.0.smithi122.stdout:3942: finishing write tid 1 to smithi12223906-35 2024-09-15T04:03:55.653 INFO:tasks.rados.rados.0.smithi122.stdout:3942: finishing write tid 2 to smithi12223906-35 2024-09-15T04:03:55.654 INFO:tasks.rados.rados.0.smithi122.stdout:3942: finishing write tid 3 to smithi12223906-35 2024-09-15T04:03:55.654 INFO:tasks.rados.rados.0.smithi122.stdout:3942: finishing write tid 4 to smithi12223906-35 2024-09-15T04:03:55.654 INFO:tasks.rados.rados.0.smithi122.stdout:3942: finishing write tid 5 to smithi12223906-35 2024-09-15T04:03:55.654 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1858 (ObjNum 1393 snap 451 seq_num 1393) dirty exists 2024-09-15T04:03:55.654 INFO:tasks.rados.rados.0.smithi122.stdout:3942: left oid 35 (ObjNum 1393 snap 451 seq_num 1393) 2024-09-15T04:03:55.655 INFO:tasks.rados.rados.0.smithi122.stdout:3938: done (5 left) 2024-09-15T04:03:55.655 INFO:tasks.rados.rados.0.smithi122.stdout:3939: done (4 left) 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3940: done (3 left) 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3941: done (2 left) 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3942: done (1 left) 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3943: done (0 left) 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3944: read oid 19 snap -1 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3944: expect (ObjNum 1391 snap 451 seq_num 1391) 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3945: write (excl) oid 34 current snap is 451 2024-09-15T04:03:55.656 INFO:tasks.rados.rados.0.smithi122.stdout:3945: seq_num 1395 ranges {645765=664954,1998616=752477,3158722=287043} 2024-09-15T04:03:55.665 INFO:tasks.rados.rados.0.smithi122.stdout:3945: writing smithi12223906-34 from 645765 to 1310719 tid 1 2024-09-15T04:03:55.673 INFO:tasks.rados.rados.0.smithi122.stdout:3945: writing smithi12223906-34 from 1998616 to 2751093 tid 2 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3945: writing smithi12223906-34 from 3158722 to 3445765 tid 3 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3946: read oid 5 snap -1 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3946: expect deleted 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3947: read oid 2 snap 448 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3947: expect (ObjNum 1347 snap 442 seq_num 1347) 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3948: delete oid 45 current snap is 451 2024-09-15T04:03:55.678 INFO:tasks.rados.rados.0.smithi122.stdout:3944: expect (ObjNum 1391 snap 451 seq_num 1391) 2024-09-15T04:03:55.827 INFO:tasks.rados.rados.0.smithi122.stdout:3945: finishing write tid 1 to smithi12223906-34 2024-09-15T04:03:55.827 INFO:tasks.rados.rados.0.smithi122.stdout:3944: done (4 left) 2024-09-15T04:03:55.827 INFO:tasks.rados.rados.0.smithi122.stdout:3948: done (3 left) 2024-09-15T04:03:55.827 INFO:tasks.rados.rados.0.smithi122.stdout:3949: delete oid 32 current snap is 451 2024-09-15T04:03:55.827 INFO:tasks.rados.rados.0.smithi122.stdout:3945: finishing write tid 2 to smithi12223906-34 2024-09-15T04:03:55.827 INFO:tasks.rados.rados.0.smithi122.stdout:3947: expect (ObjNum 1347 snap 442 seq_num 1347) 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:3945: finishing write tid 3 to smithi12223906-34 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:3945: finishing write tid 4 to smithi12223906-34 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:3945: finishing write tid 5 to smithi12223906-34 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 34 v 3279 (ObjNum 1395 snap 451 seq_num 1395) dirty exists 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:3945: left oid 34 (ObjNum 1395 snap 451 seq_num 1395) 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:3945: done (3 left) 2024-09-15T04:03:56.030 INFO:tasks.rados.rados.0.smithi122.stdout:3946: done (2 left) 2024-09-15T04:03:56.031 INFO:tasks.rados.rados.0.smithi122.stdout:3947: done (1 left) 2024-09-15T04:03:56.031 INFO:tasks.rados.rados.0.smithi122.stdout:3949: done (0 left) 2024-09-15T04:03:56.031 INFO:tasks.rados.rados.0.smithi122.stdout:3950: write (excl) oid 3 current snap is 451 2024-09-15T04:03:56.031 INFO:tasks.rados.rados.0.smithi122.stdout:3950: seq_num 1398 ranges {696186=490701,1657709=616157,2724638=562274} 2024-09-15T04:03:56.036 INFO:tasks.rados.rados.0.smithi122.stdout:3950: writing smithi12223906-3 from 696186 to 1186887 tid 1 2024-09-15T04:03:56.043 INFO:tasks.rados.rados.0.smithi122.stdout:3950: writing smithi12223906-3 from 1657709 to 2273866 tid 2 2024-09-15T04:03:56.049 INFO:tasks.rados.rados.0.smithi122.stdout:3950: writing smithi12223906-3 from 2724638 to 3286912 tid 3 2024-09-15T04:03:56.049 INFO:tasks.rados.rados.0.smithi122.stdout:3951: write oid 38 current snap is 451 2024-09-15T04:03:56.049 INFO:tasks.rados.rados.0.smithi122.stdout:3951: seq_num 1399 ranges {712993=682303,1832544=759815,3268414=244579} 2024-09-15T04:03:56.056 INFO:tasks.rados.rados.0.smithi122.stdout:3951: writing smithi12223906-38 from 712993 to 1395296 tid 1 2024-09-15T04:03:56.063 INFO:tasks.rados.rados.0.smithi122.stdout:3951: writing smithi12223906-38 from 1832544 to 2592359 tid 2 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3951: writing smithi12223906-38 from 3268414 to 3512993 tid 3 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3952: copy_from oid 35 from oid 16 current snap is 451 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3950: finishing write tid 1 to smithi12223906-3 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3950: finishing write tid 2 to smithi12223906-3 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3950: finishing write tid 3 to smithi12223906-3 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3950: finishing write tid 4 to smithi12223906-3 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:3950: finishing write tid 5 to smithi12223906-3 2024-09-15T04:03:56.069 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 3 v 3527 (ObjNum 1398 snap 451 seq_num 1398) dirty exists 2024-09-15T04:03:56.070 INFO:tasks.rados.rados.0.smithi122.stdout:3950: left oid 3 (ObjNum 1398 snap 451 seq_num 1398) 2024-09-15T04:03:56.070 INFO:tasks.rados.rados.0.smithi122.stdout:3951: finishing write tid 1 to smithi12223906-38 2024-09-15T04:03:56.070 INFO:tasks.rados.rados.0.smithi122.stdout:3950: done (2 left) 2024-09-15T04:03:56.070 INFO:tasks.rados.rados.0.smithi122.stdout:3953: delete oid 32 current snap is 451 2024-09-15T04:03:56.073 INFO:tasks.rados.rados.0.smithi122.stdout:3951: finishing write tid 2 to smithi12223906-38 2024-09-15T04:03:56.074 INFO:tasks.rados.rados.0.smithi122.stdout:3951: finishing write tid 3 to smithi12223906-38 2024-09-15T04:03:56.075 INFO:tasks.rados.rados.0.smithi122.stdout:3951: finishing write tid 4 to smithi12223906-38 2024-09-15T04:03:56.075 INFO:tasks.rados.rados.0.smithi122.stdout:3951: finishing write tid 5 to smithi12223906-38 2024-09-15T04:03:56.075 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 38 v 3099 (ObjNum 1399 snap 451 seq_num 1399) dirty exists 2024-09-15T04:03:56.076 INFO:tasks.rados.rados.0.smithi122.stdout:3951: left oid 38 (ObjNum 1399 snap 451 seq_num 1399) 2024-09-15T04:03:56.076 INFO:tasks.rados.rados.0.smithi122.stdout:3951: done (2 left) 2024-09-15T04:03:56.076 INFO:tasks.rados.rados.0.smithi122.stdout:3953: done (1 left) 2024-09-15T04:03:56.076 INFO:tasks.rados.rados.0.smithi122.stdout:3954: snap_remove snap 381 2024-09-15T04:03:56.596 INFO:tasks.rados.rados.0.smithi122.stdout:3954: done (1 left) 2024-09-15T04:03:56.597 INFO:tasks.rados.rados.0.smithi122.stdout:3955: read oid 46 snap -1 2024-09-15T04:03:56.597 INFO:tasks.rados.rados.0.smithi122.stdout:3955: expect deleted 2024-09-15T04:03:56.597 INFO:tasks.rados.rados.0.smithi122.stdout:3956: copy_from oid 49 from oid 27 current snap is 451 2024-09-15T04:03:56.597 INFO:tasks.rados.rados.0.smithi122.stdout:3957: write oid 14 current snap is 451 2024-09-15T04:03:56.597 INFO:tasks.rados.rados.0.smithi122.stdout:3957: seq_num 1401 ranges {746607=432801,1833155=763484,3206907=339700} 2024-09-15T04:03:56.605 INFO:tasks.rados.rados.0.smithi122.stdout:3957: writing smithi12223906-14 from 746607 to 1179408 tid 1 2024-09-15T04:03:56.613 INFO:tasks.rados.rados.0.smithi122.stdout:3957: writing smithi12223906-14 from 1833155 to 2596639 tid 2 2024-09-15T04:03:56.620 INFO:tasks.rados.rados.0.smithi122.stdout:3957: writing smithi12223906-14 from 3206907 to 3546607 tid 3 2024-09-15T04:03:56.620 INFO:tasks.rados.rados.0.smithi122.stdout:3958: delete oid 5 current snap is 451 2024-09-15T04:03:56.621 INFO:tasks.rados.rados.0.smithi122.stdout:3952: finishing copy_from racing read to smithi12223906-35 2024-09-15T04:03:56.621 INFO:tasks.rados.rados.0.smithi122.stdout:3952: finishing copy_from to smithi12223906-35 2024-09-15T04:03:56.621 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 35 v 1948 (ObjNum 1338 snap 440 seq_num 1338) dirty exists 2024-09-15T04:03:56.623 INFO:tasks.rados.rados.0.smithi122.stdout:3956: finishing copy_from racing read to smithi12223906-49 2024-09-15T04:03:56.624 INFO:tasks.rados.rados.0.smithi122.stdout:3952: done (4 left) 2024-09-15T04:03:56.624 INFO:tasks.rados.rados.0.smithi122.stdout:3955: done (3 left) 2024-09-15T04:03:56.625 INFO:tasks.rados.rados.0.smithi122.stdout:3958: done (2 left) 2024-09-15T04:03:56.625 INFO:tasks.rados.rados.0.smithi122.stdout:3959: snap_create 2024-09-15T04:03:56.627 INFO:tasks.rados.rados.0.smithi122.stdout:3957: finishing write tid 1 to smithi12223906-14 2024-09-15T04:03:56.628 INFO:tasks.rados.rados.0.smithi122.stdout:3957: finishing write tid 2 to smithi12223906-14 2024-09-15T04:03:56.629 INFO:tasks.rados.rados.0.smithi122.stdout:3956: finishing copy_from to smithi12223906-49 2024-09-15T04:03:56.629 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 49 v 3516 (ObjNum 1388 snap 451 seq_num 1388) dirty exists 2024-09-15T04:03:56.629 INFO:tasks.rados.rados.0.smithi122.stdout:3957: finishing write tid 3 to smithi12223906-14 2024-09-15T04:03:56.630 INFO:tasks.rados.rados.0.smithi122.stdout:3957: finishing write tid 4 to smithi12223906-14 2024-09-15T04:03:56.630 INFO:tasks.rados.rados.0.smithi122.stdout:3957: finishing write tid 5 to smithi12223906-14 2024-09-15T04:03:56.630 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 14 v 3541 (ObjNum 1401 snap 451 seq_num 1401) dirty exists 2024-09-15T04:03:56.630 INFO:tasks.rados.rados.0.smithi122.stdout:3957: left oid 14 (ObjNum 1401 snap 451 seq_num 1401) 2024-09-15T04:03:57.607 INFO:tasks.rados.rados.0.smithi122.stdout:3956: done (2 left) 2024-09-15T04:03:57.607 INFO:tasks.rados.rados.0.smithi122.stdout:3957: done (1 left) 2024-09-15T04:03:57.607 INFO:tasks.rados.rados.0.smithi122.stdout:3959: done (0 left) 2024-09-15T04:03:57.607 INFO:tasks.rados.rados.0.smithi122.stdout:3960: read oid 39 snap -1 2024-09-15T04:03:57.607 INFO:tasks.rados.rados.0.smithi122.stdout:3960: expect (ObjNum 1370 snap 448 seq_num 1370) 2024-09-15T04:03:57.607 INFO:tasks.rados.rados.0.smithi122.stdout:3961: snap_remove snap 443 2024-09-15T04:03:58.613 INFO:tasks.rados.rados.0.smithi122.stdout:3961: done (1 left) 2024-09-15T04:03:58.614 INFO:tasks.rados.rados.0.smithi122.stdout:3962: read oid 31 snap -1 2024-09-15T04:03:58.614 INFO:tasks.rados.rados.0.smithi122.stdout:3962: expect deleted 2024-09-15T04:03:58.614 INFO:tasks.rados.rados.0.smithi122.stdout:3960: expect (ObjNum 1370 snap 448 seq_num 1370) 2024-09-15T04:03:58.831 INFO:tasks.rados.rados.0.smithi122.stdout:3960: done (1 left) 2024-09-15T04:03:58.831 INFO:tasks.rados.rados.0.smithi122.stdout:3963: snap_remove snap 422 2024-09-15T04:03:59.573 INFO:tasks.rados.rados.0.smithi122.stdout:3963: done (1 left) 2024-09-15T04:03:59.573 INFO:tasks.rados.rados.0.smithi122.stdout:3964: snap_remove snap 444 2024-09-15T04:04:00.578 INFO:tasks.rados.rados.0.smithi122.stdout:3964: done (1 left) 2024-09-15T04:04:00.578 INFO:tasks.rados.rados.0.smithi122.stdout:3965: read oid 29 snap -1 2024-09-15T04:04:00.578 INFO:tasks.rados.rados.0.smithi122.stdout:3965: expect deleted 2024-09-15T04:04:00.579 INFO:tasks.rados.rados.0.smithi122.stdout:3966: copy_from oid 33 from oid 24 current snap is 452 2024-09-15T04:04:00.579 INFO:tasks.rados.rados.0.smithi122.stdout:3967: write oid 9 current snap is 452 2024-09-15T04:04:00.579 INFO:tasks.rados.rados.0.smithi122.stdout:3967: seq_num 1403 ranges {780221=583299,1950119=767153,3261753=318468} 2024-09-15T04:04:00.586 INFO:tasks.rados.rados.0.smithi122.stdout:3967: writing smithi12223906-9 from 780221 to 1363520 tid 1 2024-09-15T04:04:00.594 INFO:tasks.rados.rados.0.smithi122.stdout:3967: writing smithi12223906-9 from 1950119 to 2717272 tid 2 2024-09-15T04:04:00.599 INFO:tasks.rados.rados.0.smithi122.stdout:3967: writing smithi12223906-9 from 3261753 to 3580221 tid 3 2024-09-15T04:04:00.599 INFO:tasks.rados.rados.0.smithi122.stdout:3968: snap_remove snap 440 2024-09-15T04:04:01.575 INFO:tasks.rados.rados.0.smithi122.stdout:3968: done (4 left) 2024-09-15T04:04:01.575 INFO:tasks.rados.rados.0.smithi122.stdout:3969: rollback oid 15 current snap is 452 2024-09-15T04:04:01.575 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 15 to 337 2024-09-15T04:04:01.575 INFO:tasks.rados.rados.0.smithi122.stdout:3967: finishing write tid 1 to smithi12223906-9 2024-09-15T04:04:01.575 INFO:tasks.rados.rados.0.smithi122.stdout:3967: finishing write tid 2 to smithi12223906-9 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:3966: finishing copy_from racing read to smithi12223906-33 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:3967: finishing write tid 3 to smithi12223906-9 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:3967: finishing write tid 4 to smithi12223906-9 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:3966: finishing copy_from to smithi12223906-33 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 33 v 2134 (ObjNum 1255 snap 410 seq_num 1255) dirty exists 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:3967: finishing write tid 5 to smithi12223906-9 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2503 (ObjNum 1403 snap 452 seq_num 1403) dirty exists 2024-09-15T04:04:01.576 INFO:tasks.rados.rados.0.smithi122.stdout:3967: left oid 9 (ObjNum 1403 snap 452 seq_num 1403) 2024-09-15T04:04:01.577 INFO:tasks.rados.rados.0.smithi122.stdout:3962: done (4 left) 2024-09-15T04:04:01.577 INFO:tasks.rados.rados.0.smithi122.stdout:3965: done (3 left) 2024-09-15T04:04:01.577 INFO:tasks.rados.rados.0.smithi122.stdout:3966: done (2 left) 2024-09-15T04:04:01.577 INFO:tasks.rados.rados.0.smithi122.stdout:3967: done (1 left) 2024-09-15T04:04:01.577 INFO:tasks.rados.rados.0.smithi122.stdout:3970: copy_from oid 7 from oid 14 current snap is 452 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:3971: rollback oid 10 current snap is 452 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 10 to 431 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:3972: read oid 16 snap 449 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:3972: expect (ObjNum 1338 snap 440 seq_num 1338) 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:3973: rollback oid 9 current snap is 452 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 9 to 447 2024-09-15T04:04:01.578 INFO:tasks.rados.rados.0.smithi122.stdout:3974: delete oid 41 current snap is 452 2024-09-15T04:04:01.587 INFO:tasks.rados.rados.0.smithi122.stdout:3973: finishing rollback tid 0 to smithi12223906-9 2024-09-15T04:04:01.590 INFO:tasks.rados.rados.0.smithi122.stdout:3974: done (5 left) 2024-09-15T04:04:01.590 INFO:tasks.rados.rados.0.smithi122.stdout:3975: snap_create 2024-09-15T04:04:01.591 INFO:tasks.rados.rados.0.smithi122.stdout:3972: expect (ObjNum 1338 snap 440 seq_num 1338) 2024-09-15T04:04:01.749 INFO:tasks.rados.rados.0.smithi122.stdout:3969: finishing rollback tid 0 to smithi12223906-15 2024-09-15T04:04:01.749 INFO:tasks.rados.rados.0.smithi122.stdout:3969: finishing rollback tid 1 to smithi12223906-15 2024-09-15T04:04:01.749 INFO:tasks.rados.rados.0.smithi122.stdout:3969: finishing rollback tid 2 to smithi12223906-15 2024-09-15T04:04:01.749 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 15 v 1955 (ObjNum 1029 snap 321 seq_num 1029) dirty exists 2024-09-15T04:04:01.749 INFO:tasks.rados.rados.0.smithi122.stdout:3973: finishing rollback tid 1 to smithi12223906-9 2024-09-15T04:04:01.749 INFO:tasks.rados.rados.0.smithi122.stdout:3973: finishing rollback tid 2 to smithi12223906-9 2024-09-15T04:04:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 9 v 2506 (ObjNum 1325 snap 434 seq_num 1325) dirty exists 2024-09-15T04:04:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:3971: finishing rollback tid 1 to smithi12223906-10 2024-09-15T04:04:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 10 v 2143 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-15T04:04:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:3970: finishing copy_from racing read to smithi12223906-7 2024-09-15T04:04:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:3970: finishing copy_from to smithi12223906-7 2024-09-15T04:04:01.750 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 7 v 3541 (ObjNum 1401 snap 451 seq_num 1401) dirty exists 2024-09-15T04:04:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3969: done (5 left) 2024-09-15T04:04:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3970: done (4 left) 2024-09-15T04:04:02.576 INFO:tasks.rados.rados.0.smithi122.stdout:3971: done (3 left) 2024-09-15T04:04:02.577 INFO:tasks.rados.rados.0.smithi122.stdout:3972: done (2 left) 2024-09-15T04:04:02.577 INFO:tasks.rados.rados.0.smithi122.stdout:3973: done (1 left) 2024-09-15T04:04:02.577 INFO:tasks.rados.rados.0.smithi122.stdout:3975: done (0 left) 2024-09-15T04:04:02.577 INFO:tasks.rados.rados.0.smithi122.stdout:3976: snap_create 2024-09-15T04:04:03.579 INFO:tasks.rados.rados.0.smithi122.stdout:3976: done (0 left) 2024-09-15T04:04:03.579 INFO:tasks.rados.rados.0.smithi122.stdout:3977: copy_from oid 36 from oid 42 current snap is 454 2024-09-15T04:04:03.579 INFO:tasks.rados.rados.0.smithi122.stdout:3978: read oid 9 snap 339 2024-09-15T04:04:03.579 INFO:tasks.rados.rados.0.smithi122.stdout:3978: expect (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T04:04:03.579 INFO:tasks.rados.rados.0.smithi122.stdout:3979: read oid 8 snap -1 2024-09-15T04:04:03.579 INFO:tasks.rados.rados.0.smithi122.stdout:3979: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T04:04:03.580 INFO:tasks.rados.rados.0.smithi122.stdout:3980: read oid 31 snap -1 2024-09-15T04:04:03.580 INFO:tasks.rados.rados.0.smithi122.stdout:3980: expect deleted 2024-09-15T04:04:03.580 INFO:tasks.rados.rados.0.smithi122.stdout:3981: snap_remove snap 420 2024-09-15T04:04:04.575 INFO:tasks.rados.rados.0.smithi122.stdout:3981: done (4 left) 2024-09-15T04:04:04.575 INFO:tasks.rados.rados.0.smithi122.stdout:3982: delete oid 23 current snap is 454 2024-09-15T04:04:04.575 INFO:tasks.rados.rados.0.smithi122.stdout:3977: finishing copy_from to smithi12223906-36 2024-09-15T04:04:04.575 INFO:tasks.rados.rados.0.smithi122.stdout:3977: got expected ENOENT (src dne) 2024-09-15T04:04:04.575 INFO:tasks.rados.rados.0.smithi122.stdout:3977: finishing copy_from racing read to smithi12223906-36 2024-09-15T04:04:04.575 INFO:tasks.rados.rados.0.smithi122.stdout:3979: expect (ObjNum 1018 snap 319 seq_num 1018) 2024-09-15T04:04:04.685 INFO:tasks.rados.rados.0.smithi122.stdout:3978: expect (ObjNum 1035 snap 326 seq_num 1035) 2024-09-15T04:04:04.789 INFO:tasks.rados.rados.0.smithi122.stdout:3977: done (4 left) 2024-09-15T04:04:04.789 INFO:tasks.rados.rados.0.smithi122.stdout:3978: done (3 left) 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3979: done (2 left) 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3980: done (1 left) 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3982: done (0 left) 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3983: read oid 15 snap -1 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3983: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3984: read oid 33 snap -1 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3984: expect (ObjNum 1255 snap 410 seq_num 1255) 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3985: write (excl) oid 48 current snap is 454 2024-09-15T04:04:04.790 INFO:tasks.rados.rados.0.smithi122.stdout:3985: seq_num 1406 ranges {430642=409046,1609212=514480,2827669=768803} 2024-09-15T04:04:04.801 INFO:tasks.rados.rados.0.smithi122.stdout:3985: writing smithi12223906-48 from 430642 to 839688 tid 1 2024-09-15T04:04:04.811 INFO:tasks.rados.rados.0.smithi122.stdout:3985: writing smithi12223906-48 from 1609212 to 2123692 tid 2 2024-09-15T04:04:04.819 INFO:tasks.rados.rados.0.smithi122.stdout:3985: writing smithi12223906-48 from 2827669 to 3596472 tid 3 2024-09-15T04:04:04.820 INFO:tasks.rados.rados.0.smithi122.stdout:3986: write oid 28 current snap is 454 2024-09-15T04:04:04.820 INFO:tasks.rados.rados.0.smithi122.stdout:3986: seq_num 1407 ranges {447449=600648,1784047=774491,2971445=676004} 2024-09-15T04:04:04.825 INFO:tasks.rados.rados.0.smithi122.stdout:3986: writing smithi12223906-28 from 447449 to 1048097 tid 1 2024-09-15T04:04:04.834 INFO:tasks.rados.rados.0.smithi122.stdout:3986: writing smithi12223906-28 from 1784047 to 2558538 tid 2 2024-09-15T04:04:04.840 INFO:tasks.rados.rados.0.smithi122.stdout:3986: writing smithi12223906-28 from 2971445 to 3647449 tid 3 2024-09-15T04:04:04.840 INFO:tasks.rados.rados.0.smithi122.stdout:3987: snap_create 2024-09-15T04:04:04.841 INFO:tasks.rados.rados.0.smithi122.stdout:3983: expect (ObjNum 1029 snap 321 seq_num 1029) 2024-09-15T04:04:05.120 INFO:tasks.rados.rados.0.smithi122.stdout:3984: expect (ObjNum 1255 snap 410 seq_num 1255) 2024-09-15T04:04:05.367 INFO:tasks.rados.rados.0.smithi122.stdout:3985: finishing write tid 1 to smithi12223906-48 2024-09-15T04:04:05.367 INFO:tasks.rados.rados.0.smithi122.stdout:3985: finishing write tid 2 to smithi12223906-48 2024-09-15T04:04:05.367 INFO:tasks.rados.rados.0.smithi122.stdout:3985: finishing write tid 3 to smithi12223906-48 2024-09-15T04:04:05.367 INFO:tasks.rados.rados.0.smithi122.stdout:3985: finishing write tid 4 to smithi12223906-48 2024-09-15T04:04:05.367 INFO:tasks.rados.rados.0.smithi122.stdout:3985: finishing write tid 5 to smithi12223906-48 2024-09-15T04:04:05.367 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 48 v 2171 (ObjNum 1406 snap 454 seq_num 1406) dirty exists 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3985: left oid 48 (ObjNum 1406 snap 454 seq_num 1406) 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3986: finishing write tid 1 to smithi12223906-28 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3986: finishing write tid 2 to smithi12223906-28 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3986: finishing write tid 3 to smithi12223906-28 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3986: finishing write tid 4 to smithi12223906-28 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3986: finishing write tid 5 to smithi12223906-28 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 28 v 3545 (ObjNum 1407 snap 454 seq_num 1407) dirty exists 2024-09-15T04:04:05.368 INFO:tasks.rados.rados.0.smithi122.stdout:3986: left oid 28 (ObjNum 1407 snap 454 seq_num 1407) 2024-09-15T04:04:05.626 INFO:tasks.rados.rados.0.smithi122.stdout:3983: done (4 left) 2024-09-15T04:04:05.626 INFO:tasks.rados.rados.0.smithi122.stdout:3984: done (3 left) 2024-09-15T04:04:05.626 INFO:tasks.rados.rados.0.smithi122.stdout:3985: done (2 left) 2024-09-15T04:04:05.626 INFO:tasks.rados.rados.0.smithi122.stdout:3986: done (1 left) 2024-09-15T04:04:05.626 INFO:tasks.rados.rados.0.smithi122.stdout:3987: done (0 left) 2024-09-15T04:04:05.626 INFO:tasks.rados.rados.0.smithi122.stdout:3988: copy_from oid 22 from oid 15 current snap is 455 2024-09-15T04:04:05.627 INFO:tasks.rados.rados.0.smithi122.stdout:3989: rollback oid 41 current snap is 455 2024-09-15T04:04:05.627 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 41 to 360 2024-09-15T04:04:05.627 INFO:tasks.rados.rados.0.smithi122.stdout:3990: copy_from oid 26 from oid 24 current snap is 455 2024-09-15T04:04:05.627 INFO:tasks.rados.rados.0.smithi122.stdout:3991: write (excl) oid 31 current snap is 455 2024-09-15T04:04:05.627 INFO:tasks.rados.rados.0.smithi122.stdout:3991: seq_num 1408 ranges {464256=675897,1726176=518149,2882515=507700} 2024-09-15T04:04:05.637 INFO:tasks.rados.rados.0.smithi122.stdout:3991: writing smithi12223906-31 from 464256 to 1140153 tid 1 2024-09-15T04:04:05.644 INFO:tasks.rados.rados.0.smithi122.stdout:3991: writing smithi12223906-31 from 1726176 to 2244325 tid 2 2024-09-15T04:04:05.650 INFO:tasks.rados.rados.0.smithi122.stdout:3991: writing smithi12223906-31 from 2882515 to 3390215 tid 3 2024-09-15T04:04:05.651 INFO:tasks.rados.rados.0.smithi122.stdout:3992: write oid 2 current snap is 455 2024-09-15T04:04:05.651 INFO:tasks.rados.rados.0.smithi122.stdout:3989: finishing rollback tid 1 to smithi12223906-41 2024-09-15T04:04:05.651 INFO:tasks.rados.rados.0.smithi122.stdout:3992: seq_num 1409 ranges {481063=751146,1784658=778160,3309938=371125} 2024-09-15T04:04:05.657 INFO:tasks.rados.rados.0.smithi122.stdout:3992: writing smithi12223906-2 from 481063 to 1232209 tid 1 2024-09-15T04:04:05.665 INFO:tasks.rados.rados.0.smithi122.stdout:3992: writing smithi12223906-2 from 1784658 to 2562818 tid 2 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:3992: writing smithi12223906-2 from 3309938 to 3681063 tid 3 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:3993: snap_create 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:3989: finishing rollback tid 2 to smithi12223906-41 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 41 v 2935 (ObjNum 1050 snap 335 seq_num 1050) dirty exists 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:3990: finishing copy_from racing read to smithi12223906-26 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:3991: finishing write tid 1 to smithi12223906-31 2024-09-15T04:04:05.672 INFO:tasks.rados.rados.0.smithi122.stdout:3990: finishing copy_from to smithi12223906-26 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 26 v 2134 (ObjNum 1255 snap 410 seq_num 1255) dirty exists 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3991: finishing write tid 2 to smithi12223906-31 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3988: finishing copy_from racing read to smithi12223906-22 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3988: finishing copy_from to smithi12223906-22 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 22 v 1955 (ObjNum 1029 snap 321 seq_num 1029) dirty exists 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3991: finishing write tid 3 to smithi12223906-31 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3991: finishing write tid 4 to smithi12223906-31 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3991: finishing write tid 5 to smithi12223906-31 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 31 v 3264 (ObjNum 1408 snap 455 seq_num 1408) dirty exists 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3991: left oid 31 (ObjNum 1408 snap 455 seq_num 1408) 2024-09-15T04:04:05.673 INFO:tasks.rados.rados.0.smithi122.stdout:3992: finishing write tid 1 to smithi12223906-2 2024-09-15T04:04:05.676 INFO:tasks.rados.rados.0.smithi122.stdout:3992: finishing write tid 2 to smithi12223906-2 2024-09-15T04:04:05.676 INFO:tasks.rados.rados.0.smithi122.stdout:3992: finishing write tid 3 to smithi12223906-2 2024-09-15T04:04:05.677 INFO:tasks.rados.rados.0.smithi122.stdout:3992: finishing write tid 4 to smithi12223906-2 2024-09-15T04:04:05.677 INFO:tasks.rados.rados.0.smithi122.stdout:3992: finishing write tid 5 to smithi12223906-2 2024-09-15T04:04:05.677 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 2 v 3520 (ObjNum 1409 snap 455 seq_num 1409) dirty exists 2024-09-15T04:04:05.677 INFO:tasks.rados.rados.0.smithi122.stdout:3992: left oid 2 (ObjNum 1409 snap 455 seq_num 1409) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3988: done (5 left) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3989: done (4 left) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3990: done (3 left) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3991: done (2 left) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3992: done (1 left) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3993: done (0 left) 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:3994: rollback oid 16 current snap is 456 2024-09-15T04:04:06.635 INFO:tasks.rados.rados.0.smithi122.stdout:rollback oid 16 to 417 2024-09-15T04:04:06.636 INFO:tasks.rados.rados.0.smithi122.stdout:3995: read oid 14 snap -1 2024-09-15T04:04:06.636 INFO:tasks.rados.rados.0.smithi122.stdout:3995: expect (ObjNum 1401 snap 451 seq_num 1401) 2024-09-15T04:04:06.636 INFO:tasks.rados.rados.0.smithi122.stdout:3996: write oid 5 current snap is 456 2024-09-15T04:04:06.636 INFO:tasks.rados.rados.0.smithi122.stdout:3996: seq_num 1410 ranges {497870=426395,1443140=521818,2537361=530244,3583943=113927} 2024-09-15T04:04:06.642 INFO:tasks.rados.rados.0.smithi122.stdout:3996: writing smithi12223906-5 from 497870 to 924265 tid 1 2024-09-15T04:04:06.652 INFO:tasks.rados.rados.0.smithi122.stdout:3996: writing smithi12223906-5 from 1443140 to 1964958 tid 2 2024-09-15T04:04:06.656 INFO:tasks.rados.rados.0.smithi122.stdout:3996: writing smithi12223906-5 from 2537361 to 3067605 tid 3 2024-09-15T04:04:06.660 INFO:tasks.rados.rados.0.smithi122.stdout:3996: writing smithi12223906-5 from 3583943 to 3697870 tid 4 2024-09-15T04:04:06.660 INFO:tasks.rados.rados.0.smithi122.stdout:3997: write (excl) oid 44 current snap is 456 2024-09-15T04:04:06.660 INFO:tasks.rados.rados.0.smithi122.stdout:3997: seq_num 1411 ranges {514677=501644,1501622=781829,2964784=741516} 2024-09-15T04:04:06.665 INFO:tasks.rados.rados.0.smithi122.stdout:3997: writing smithi12223906-44 from 514677 to 1016321 tid 1 2024-09-15T04:04:06.672 INFO:tasks.rados.rados.0.smithi122.stdout:3997: writing smithi12223906-44 from 1501622 to 2283451 tid 2 2024-09-15T04:04:06.680 INFO:tasks.rados.rados.0.smithi122.stdout:3997: writing smithi12223906-44 from 2964784 to 3706300 tid 3 2024-09-15T04:04:06.680 INFO:tasks.rados.rados.0.smithi122.stdout:3998: copy_from oid 20 from oid 32 current snap is 456 2024-09-15T04:04:06.680 INFO:tasks.rados.rados.0.smithi122.stdout:3999: write oid 24 current snap is 456 2024-09-15T04:04:06.680 INFO:tasks.rados.rados.0.smithi122.stdout:3994: finishing rollback tid 0 to smithi12223906-16 2024-09-15T04:04:06.680 INFO:tasks.rados.rados.0.smithi122.stdout:3994: finishing rollback tid 1 to smithi12223906-16 2024-09-15T04:04:06.680 INFO:tasks.rados.rados.0.smithi122.stdout:3995: expect (ObjNum 1401 snap 451 seq_num 1401) 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3994: finishing rollback tid 2 to smithi12223906-16 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 16 v 1958 (ObjNum 1153 snap 380 seq_num 1153) dirty exists 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3996: finishing write tid 1 to smithi12223906-5 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3996: finishing write tid 2 to smithi12223906-5 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3996: finishing write tid 3 to smithi12223906-5 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3996: finishing write tid 4 to smithi12223906-5 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3996: finishing write tid 5 to smithi12223906-5 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:3996: finishing write tid 6 to smithi12223906-5 2024-09-15T04:04:06.862 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 5 v 2640 (ObjNum 1410 snap 456 seq_num 1410) dirty exists 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3996: left oid 5 (ObjNum 1410 snap 456 seq_num 1410) 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3997: finishing write tid 1 to smithi12223906-44 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3997: finishing write tid 2 to smithi12223906-44 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3997: finishing write tid 3 to smithi12223906-44 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3998: finishing copy_from to smithi12223906-20 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3998: got expected ENOENT (src dne) 2024-09-15T04:04:06.863 INFO:tasks.rados.rados.0.smithi122.stdout:3999: seq_num 1412 ranges {531484=576893,1843751=525487,2875854=669141} 2024-09-15T04:04:06.876 INFO:tasks.rados.rados.0.smithi122.stdout:3999: writing smithi12223906-24 from 531484 to 1108377 tid 1 2024-09-15T04:04:06.883 INFO:tasks.rados.rados.0.smithi122.stdout:3999: writing smithi12223906-24 from 1843751 to 2369238 tid 2 2024-09-15T04:04:06.889 INFO:tasks.rados.rados.0.smithi122.stdout:3999: writing smithi12223906-24 from 2875854 to 3544995 tid 3 2024-09-15T04:04:06.889 INFO:tasks.rados.rados.0.smithi122.stdout:3994: done (5 left) 2024-09-15T04:04:06.889 INFO:tasks.rados.rados.0.smithi122.stdout:3995: done (4 left) 2024-09-15T04:04:06.889 INFO:tasks.rados.rados.0.smithi122.stdout:3996: done (3 left) 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 3 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:3997: finishing write tid 4 to smithi12223906-44 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:3998: finishing copy_from racing read to smithi12223906-20 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:3997: finishing write tid 5 to smithi12223906-44 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 44 v 3616 (ObjNum 1411 snap 456 seq_num 1411) dirty exists 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:3997: left oid 44 (ObjNum 1411 snap 456 seq_num 1411) 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:3997: done (2 left) 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout:3998: done (1 left) 2024-09-15T04:04:06.890 INFO:tasks.rados.rados.0.smithi122.stdout: waiting on 1 2024-09-15T04:04:06.900 INFO:tasks.rados.rados.0.smithi122.stdout:3999: finishing write tid 1 to smithi12223906-24 2024-09-15T04:04:06.900 INFO:tasks.rados.rados.0.smithi122.stdout:3999: finishing write tid 2 to smithi12223906-24 2024-09-15T04:04:06.901 INFO:tasks.rados.rados.0.smithi122.stdout:3999: finishing write tid 3 to smithi12223906-24 2024-09-15T04:04:06.902 INFO:tasks.rados.rados.0.smithi122.stdout:3999: finishing write tid 4 to smithi12223906-24 2024-09-15T04:04:06.902 INFO:tasks.rados.rados.0.smithi122.stdout:3999: finishing write tid 5 to smithi12223906-24 2024-09-15T04:04:06.903 INFO:tasks.rados.rados.0.smithi122.stdout:update_object_version oid 24 v 3588 (ObjNum 1412 snap 456 seq_num 1412) dirty exists 2024-09-15T04:04:06.903 INFO:tasks.rados.rados.0.smithi122.stdout:3999: left oid 24 (ObjNum 1412 snap 456 seq_num 1412) 2024-09-15T04:04:06.904 INFO:tasks.rados.rados.0.smithi122.stdout:3999: done (0 left) 2024-09-15T04:04:06.908 INFO:tasks.rados.rados.0.smithi122.stderr:0 errors. 2024-09-15T04:04:06.909 INFO:tasks.rados.rados.0.smithi122.stderr: 2024-09-15T04:04:06.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T04:04:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.019+0000 7f8541b27700 10 monclient: get_monmap_and_config 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.019+0000 7f8541b27700 10 monclient: build_initial_monmap 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.019+0000 7f8541b27700 10 monclient: monmap: 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:07.022251+0000 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:07.022251+0000 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:07.027 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.019+0000 7f8541b27700 1 Processor -- start 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.019+0000 7f8541b27700 1 -- start start 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.019+0000 7f8541b27700 10 monclient: init 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient: _add_conns ranks=[3,2,6,1,4,0,7,8,5] 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient(hunting): picked mon.noname-g con 0x7f853c0982f0 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient(hunting): picked mon.noname-b con 0x7f853c09a8a0 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient(hunting): picked mon.noname-d con 0x7f853c135630 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c04ca10 con 0x7f853c09a8a0 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c0986c0 con 0x7f853c0982f0 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c09d630 con 0x7f853c135630 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient(hunting): authenticate will time out at 1991.452206s 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541326700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f853c135630 0x7f853c137a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58392/0 (socket says 172.21.15.26:58392) 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541326700 1 -- 172.21.15.26:0/1432964982 learned_addr learned my addr 172.21.15.26:0/1432964982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857094886 0 0) 0x7f853c09d630 con 0x7f853c135630 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): my global_id is 195252 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8524004680 con 0x7f853c135630 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443004073 0 0) 0x7f853c04ca10 con 0x7f853c09a8a0 2024-09-15T04:04:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): my global_id is 254243 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f853c09d630 con 0x7f853c09a8a0 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086849551 0 0) 0x7f853c0986c0 con 0x7f853c0982f0 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient(hunting): my global_id is 254316 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f853c04ca10 con 0x7f853c0982f0 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3587290786 0 0) 0x7f853c09d630 con 0x7f853c09a8a0 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 >> v1:172.21.15.26:6792/0 conn(0x7f853c135630 legacy=0x7f853c137a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 >> v1:172.21.15.122:6790/0 conn(0x7f853c0982f0 legacy=0x7f853c09c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 monclient: found mon.noname-b 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c098e70 con 0x7f853c09a8a0 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8530001790 con 0x7f853c09a8a0 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 5 monclient: authenticate success, global_id 254243 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: dump: 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f85300019c0 con 0x7f853c09a8a0 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 1 -- 172.21.15.26:0/1432964982 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8530001ef0 con 0x7f853c09a8a0 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 4 set_mon_vals no callback set 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f853b7fe700 10 monclient: dump: 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient: shutdown 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:07.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- 172.21.15.26:0/1432964982 >> v1:172.21.15.26:6790/0 conn(0x7f853c09a8a0 legacy=0x7f853c09cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- 172.21.15.26:0/1432964982 shutdown_connections 2024-09-15T04:04:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- 172.21.15.26:0/1432964982 >> 172.21.15.26:0/1432964982 conn(0x7f853c063d90 msgr2=0x7f853c0a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- 172.21.15.26:0/1432964982 shutdown_connections 2024-09-15T04:04:07.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 1 -- 172.21.15.26:0/1432964982 wait complete. 2024-09-15T04:04:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient: build_initial_monmap 2024-09-15T04:04:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.023+0000 7f8541b27700 10 monclient: monmap: 2024-09-15T04:04:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:07.041 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:07.044 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 Processor -- start 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 -- start start 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: init 2024-09-15T04:04:07.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: _add_conns ranks=[6,0,5,1,2,3,7,4,8] 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient(hunting): picked mon.noname-d con 0x7f853c0982f0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient(hunting): picked mon.noname-a con 0x7f853c09a8a0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient(hunting): picked mon.noname-h con 0x7f853c135630 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c06e250 con 0x7f853c09a8a0 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c06de90 con 0x7f853c135630 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c1228d0 con 0x7f853c0982f0 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient(hunting): authenticate will time out at 1991.457462s 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f853bfff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f853c09a8a0 0x7f853c1347a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54072/0 (socket says 172.21.15.26:54072) 2024-09-15T04:04:07.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f853bfff700 1 -- 172.21.15.26:0/2877274794 learned_addr learned my addr 172.21.15.26:0/2877274794 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541326700 1 -- 172.21.15.26:0/2877274794 >> v1:172.21.15.122:6791/0 conn(0x7f853c135630 legacy=0x7f853c149e80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121956120 0 0) 0x7f853c1228d0 con 0x7f853c0982f0 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): my global_id is 195261 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8528003570 con 0x7f853c0982f0 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515998212 0 0) 0x7f853c06e250 con 0x7f853c09a8a0 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): my global_id is 195174 2024-09-15T04:04:07.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f853c1228d0 con 0x7f853c09a8a0 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2660195748 0 0) 0x7f8528003570 con 0x7f853c0982f0 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f853c06e250 con 0x7f853c0982f0 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f852c001de0 con 0x7f853c0982f0 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.048 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 282547895 0 0) 0x7f853c1228d0 con 0x7f853c09a8a0 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8528003570 con 0x7f853c09a8a0 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 893142033 0 0) 0x7f853c06e250 con 0x7f853c0982f0 2024-09-15T04:04:07.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 >> v1:172.21.15.122:6791/0 conn(0x7f853c135630 legacy=0x7f853c149e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 >> v1:172.21.15.26:6789/0 conn(0x7f853c09a8a0 legacy=0x7f853c1347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 monclient: found mon.d 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c120e10 con 0x7f853c0982f0 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 5 monclient: authenticate success, global_id 195261 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: _renew_subs 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c1428d0 con 0x7f853c0982f0 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: _renew_subs 2024-09-15T04:04:07.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c135050 con 0x7f853c0982f0 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f852c0023a0 con 0x7f853c0982f0 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f852c0028d0 con 0x7f853c0982f0 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f853affd700 4 set_mon_vals no callback set 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.027+0000 7f8538ff9700 10 monclient: dump: 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.052 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f8541b27700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f8541b27700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f8541b27700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c0050b0 con 0x7f853c0982f0 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f852c002ab0 con 0x7f853c0982f0 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(903..903 src has 435..903) v4 ==== 3911+0+0 (unknown 3839595639 0 0) 0x7f852c0419a0 con 0x7f853c0982f0 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.031+0000 7f853bfff700 10 monclient: get_auth_request con 0x7f85280377f0 auth_method 0 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.035+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f852c0127f0 con 0x7f853c0982f0 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.035+0000 7f8538ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.035+0000 7f8538ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:07.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8541b27700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:07.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8541b27700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:07.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8541b27700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:07.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f851c005360 con 0x7f853c0982f0 2024-09-15T04:04:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8538ff9700 1 -- 172.21.15.26:0/2877274794 <== mon.6 v1:172.21.15.26:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v903) v1 ==== 74+0+11630 (unknown 343214300 0 1448002004) 0x7f852c0121f0 con 0x7f853c0982f0 2024-09-15T04:04:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8538ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.199+0000 7f8538ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:07.205 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:07.205 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":903,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T04:04:06.616918+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T03:44:56.976968+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-15T03:44:57.527097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-15T03:45:06.856805+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"903","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":889,"snap_epoch":903,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":803,"length":1},{"begin":847,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T04:04:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 >> v1:172.21.15.122:6812/20895 conn(0x7f85280377f0 legacy=0x7f8528039ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 10 monclient: shutdown 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 >> v1:172.21.15.26:6792/0 conn(0x7f853c0982f0 legacy=0x7f853c141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 shutdown_connections 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 >> 172.21.15.26:0/2877274794 conn(0x7f853c063d90 msgr2=0x7f853c09edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 shutdown_connections 2024-09-15T04:04:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.203+0000 7f8541b27700 1 -- 172.21.15.26:0/2877274794 wait complete. 2024-09-15T04:04:07.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.319+0000 7fd6ed8ee700 10 monclient: get_monmap_and_config 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.319+0000 7fd6ed8ee700 10 monclient: build_initial_monmap 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.319+0000 7fd6ed8ee700 10 monclient: monmap: 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:07.323141+0000 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:07.323141+0000 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:07.328 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 Processor -- start 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 -- start start 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient: init 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient: _add_conns ranks=[8,7,5,3,0,1,4,2,6] 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient(hunting): picked mon.noname-e con 0x7fd6e80a6a80 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient(hunting): picked mon.noname-i con 0x7fd6e809fb80 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient(hunting): picked mon.noname-h con 0x7fd6e8138d30 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e804ca10 con 0x7fd6e8138d30 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80a0f50 con 0x7fd6e809fb80 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e8133680 con 0x7fd6e80a6a80 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ec8ec700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7fd6e80a6a80 legacy=0x7fd6e8132f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient(hunting): authenticate will time out at 1991.753167s 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed0ed700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fd6e8138d30 legacy=0x7fd6e813b110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e7fff700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fd6e809fb80 0x7fd6e809ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:44094/0 (socket says 172.21.15.26:44094) 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e7fff700 1 -- 172.21.15.26:0/800674444 learned_addr learned my addr 172.21.15.26:0/800674444 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801528736 0 0) 0x7fd6e80a0f50 con 0x7fd6e809fb80 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient(hunting): my global_id is 254302 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8003570 con 0x7fd6e809fb80 2024-09-15T04:04:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272920648 0 0) 0x7fd6d8003570 con 0x7fd6e809fb80 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 >> v1:172.21.15.26:6793/0 conn(0x7fd6e80a6a80 legacy=0x7fd6e8132f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 >> v1:172.21.15.122:6791/0 conn(0x7fd6e8138d30 legacy=0x7fd6e813b110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 monclient: found mon.noname-i 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80a6e50 con 0x7fd6e809fb80 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 5 monclient: authenticate success, global_id 254302 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd6d40017b0 con 0x7fd6e809fb80 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: dump: 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6d4001ad0 con 0x7fd6e809fb80 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 4 set_mon_vals no callback set 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 1 -- 172.21.15.26:0/800674444 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd6d4004c30 con 0x7fd6e809fb80 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6e77fe700 10 monclient: dump: 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 10 monclient: shutdown 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/800674444 >> v1:172.21.15.122:6792/0 conn(0x7fd6e809fb80 legacy=0x7fd6e809ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.323+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/800674444 shutdown_connections 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/800674444 >> 172.21.15.26:0/800674444 conn(0x7fd6e806e380 msgr2=0x7fd6e80a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/800674444 shutdown_connections 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/800674444 wait complete. 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient: build_initial_monmap 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient: monmap: 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:07.339 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 Processor -- start 2024-09-15T04:04:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- start start 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient: init 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient: _add_conns ranks=[1,4,8,3,7,5,0,6,2] 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient(hunting): picked mon.noname-f con 0x7fd6e809fb80 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient(hunting): picked mon.noname-c con 0x7fd6e80a6a80 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient(hunting): picked mon.noname-e con 0x7fd6e8138d30 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e809cfc0 con 0x7fd6e809fb80 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e811cdf0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e811d0d0 con 0x7fd6e8138d30 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed0ed700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7fd6e8138d30 legacy=0x7fd6e814a3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ed8ee700 10 monclient(hunting): authenticate will time out at 1991.758324s 2024-09-15T04:04:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6ec8ec700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fd6e809fb80 legacy=0x7fd6e8142190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e7fff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7fd6e80a6a80 0x7fd6e81428a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:52826/0 (socket says 172.21.15.26:52826) 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e7fff700 1 -- 172.21.15.26:0/3647108323 learned_addr learned my addr 172.21.15.26:0/3647108323 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162136725 0 0) 0x7fd6e811cdf0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): my global_id is 205096 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6cc003570 con 0x7fd6e80a6a80 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3958337947 0 0) 0x7fd6cc003570 con 0x7fd6e80a6a80 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6e811cdf0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd6d4004b70 con 0x7fd6e80a6a80 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1282302036 0 0) 0x7fd6e811cdf0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 >> v1:172.21.15.26:6793/0 conn(0x7fd6e8138d30 legacy=0x7fd6e814a3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 >> v1:172.21.15.122:6789/0 conn(0x7fd6e809fb80 legacy=0x7fd6e8142190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 monclient: found mon.c 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.327+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8142fb0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 5 monclient: authenticate success, global_id 205096 2024-09-15T04:04:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 monclient: _renew_subs 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e8137310 con 0x7fd6e80a6a80 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 monclient: _renew_subs 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e8138120 con 0x7fd6e80a6a80 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd6d4004ef0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd6d40053a0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e6ffd700 4 set_mon_vals no callback set 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 10 monclient: dump: 2024-09-15T04:04:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd6d400fd60 con 0x7fd6e80a6a80 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c80050b0 con 0x7fd6e80a6a80 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(903..903 src has 435..903) v4 ==== 3911+0+0 (unknown 3839595639 0 0) 0x7fd6d4044180 con 0x7fd6e80a6a80 2024-09-15T04:04:07.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.331+0000 7fd6ec8ec700 10 monclient: get_auth_request con 0x7fd6cc0374f0 auth_method 0 2024-09-15T04:04:07.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.339+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fd6d4015070 con 0x7fd6e80a6a80 2024-09-15T04:04:07.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.339+0000 7fd6e4ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.339+0000 7fd6e4ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:07.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.503+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6c8000b90 con 0x7fd6cc0374f0 2024-09-15T04:04:07.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.511+0000 7fd6e4ff9700 1 -- 172.21.15.26:0/3647108323 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+97274 (unknown 2965378022 0 1991784072) 0x7fd6c8000b90 con 0x7fd6cc0374f0 2024-09-15T04:04:07.518 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 >> v1:172.21.15.122:6812/20895 conn(0x7fd6cc0374f0 legacy=0x7fd6cc0399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 10 monclient: shutdown 2024-09-15T04:04:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 >> v1:172.21.15.26:6791/0 conn(0x7fd6e80a6a80 legacy=0x7fd6e81428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 shutdown_connections 2024-09-15T04:04:07.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 >> 172.21.15.26:0/3647108323 conn(0x7fd6e806e380 msgr2=0x7fd6e809db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:07.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 shutdown_connections 2024-09-15T04:04:07.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.515+0000 7fd6ed8ee700 1 -- 172.21.15.26:0/3647108323 wait complete. 2024-09-15T04:04:07.521 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T04:04:07.532 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":1473,"stamp":"2024-09-15T04:04:06.631046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":843776130,"num_objects":372,"num_object_clones":318,"num_object_copies":744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":372,"num_whiteouts":15,"num_read":7925,"num_read_kb":8000721,"num_write":5655,"num_write_kb":2137183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22799,"ondisk_log_size":22799,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":2799296,"kb_used_data":965212,"kb_used_omap":9,"kb_used_meta":1833974,"kb_avail":563431744,"statfs":{"total":579820584960,"available":576954105856,"internally_reserved":0,"allocated":988377088,"data_stored":1334745458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1877989392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16895527,"num_objects":4,"num_object_clones":4,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":-1,"num_read":41,"num_read_kb":46521,"num_write":22,"num_write_kb":12208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.060697"},"pg_stats":[{"pgid":"3.f","version":"901'376","reported_seq":3100,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.583374+0000","last_change":"2024-09-15T04:04:04.583374+0000","last_active":"2024-09-15T04:04:04.583374+0000","last_peered":"2024-09-15T04:04:04.583374+0000","last_clean":"2024-09-15T04:04:04.583374+0000","last_became_active":"2024-09-15T03:45:32.891832+0000","last_became_peered":"2024-09-15T03:45:32.891832+0000","last_unstale":"2024-09-15T04:04:04.583374+0000","last_undegraded":"2024-09-15T04:04:04.583374+0000","last_fullsized":"2024-09-15T04:04:04.583374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:33.116588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033540390000000001,"stat_sum":{"num_bytes":13110450,"num_objects":5,"num_object_clones":4,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":120,"num_read_kb":126995,"num_write":109,"num_write_kb":38810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"3.e","version":"901'1454","reported_seq":4105,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.847523+0000","last_change":"2024-09-15T04:04:04.603579+0000","last_active":"2024-09-15T04:04:04.847523+0000","last_peered":"2024-09-15T04:04:04.847523+0000","last_clean":"2024-09-15T04:04:04.847523+0000","last_became_active":"2024-09-15T03:45:32.897132+0000","last_became_peered":"2024-09-15T03:45:32.897132+0000","last_unstale":"2024-09-15T04:04:04.847523+0000","last_undegraded":"2024-09-15T04:04:04.847523+0000","last_fullsized":"2024-09-15T04:04:04.847523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1454,"log_dups_size":0,"ondisk_log_size":1454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:41:23.973135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.023585881,"stat_sum":{"num_bytes":49809355,"num_objects":22,"num_object_clones":19,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":1,"num_read":503,"num_read_kb":532308,"num_write":313,"num_write_kb":121855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"3.d","version":"901'922","reported_seq":3599,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.578128+0000","last_change":"2024-09-15T04:04:04.578128+0000","last_active":"2024-09-15T04:04:04.578128+0000","last_peered":"2024-09-15T04:04:04.578128+0000","last_clean":"2024-09-15T04:04:04.578128+0000","last_became_active":"2024-09-15T03:45:32.891061+0000","last_became_peered":"2024-09-15T03:45:32.891061+0000","last_unstale":"2024-09-15T04:04:04.578128+0000","last_undegraded":"2024-09-15T04:04:04.578128+0000","last_fullsized":"2024-09-15T04:04:04.578128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:22:54.809044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00712592,"stat_sum":{"num_bytes":49205723,"num_objects":18,"num_object_clones":16,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":240,"num_read_kb":203928,"num_write":214,"num_write_kb":88724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"3.c","version":"901'921","reported_seq":3594,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.592394+0000","last_change":"2024-09-15T04:04:04.592394+0000","last_active":"2024-09-15T04:04:04.592394+0000","last_peered":"2024-09-15T04:04:04.592394+0000","last_clean":"2024-09-15T04:04:04.592394+0000","last_became_active":"2024-09-15T03:45:32.885882+0000","last_became_peered":"2024-09-15T03:45:32.885882+0000","last_unstale":"2024-09-15T04:04:04.592394+0000","last_undegraded":"2024-09-15T04:04:04.592394+0000","last_fullsized":"2024-09-15T04:04:04.592394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:33:26.113214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012274007,"stat_sum":{"num_bytes":35147718,"num_objects":15,"num_object_clones":13,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":282,"num_read_kb":266957,"num_write":234,"num_write_kb":94411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":2680,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-15T04:04:01.572453+0000","last_change":"2024-09-15T04:04:01.572725+0000","last_active":"2024-09-15T04:04:01.572453+0000","last_peered":"2024-09-15T04:04:01.572453+0000","last_clean":"2024-09-15T04:04:01.572453+0000","last_became_active":"2024-09-15T03:45:32.881275+0000","last_became_peered":"2024-09-15T03:45:32.881275+0000","last_unstale":"2024-09-15T04:04:01.572453+0000","last_undegraded":"2024-09-15T04:04:01.572453+0000","last_fullsized":"2024-09-15T04:04:01.572453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:04.087267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011711289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d9","length":"1"},{"start":"329","length":"1"},{"start":"34f","length":"1"},{"start":"354","length":"1"},{"start":"356","length":"1"}]},{"pgid":"3.a","version":"902'1475","reported_seq":4123,"reported_epoch":902,"state":"active+clean","last_fresh":"2024-09-15T04:04:05.677316+0000","last_change":"2024-09-15T04:04:04.600422+0000","last_active":"2024-09-15T04:04:05.677316+0000","last_peered":"2024-09-15T04:04:05.677316+0000","last_clean":"2024-09-15T04:04:05.677316+0000","last_became_active":"2024-09-15T03:45:32.897200+0000","last_became_peered":"2024-09-15T03:45:32.897200+0000","last_unstale":"2024-09-15T04:04:05.677316+0000","last_undegraded":"2024-09-15T04:04:05.677316+0000","last_fullsized":"2024-09-15T04:04:05.677316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1475,"log_dups_size":0,"ondisk_log_size":1475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:14.249480+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.029567945000000002,"stat_sum":{"num_bytes":66797681,"num_objects":25,"num_object_clones":22,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":455,"num_read_kb":452092,"num_write":405,"num_write_kb":154109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"3.9","version":"898'1053","reported_seq":3786,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-15T04:04:01.591869+0000","last_change":"2024-09-15T04:04:01.592207+0000","last_active":"2024-09-15T04:04:01.591869+0000","last_peered":"2024-09-15T04:04:01.591869+0000","last_clean":"2024-09-15T04:04:01.591869+0000","last_became_active":"2024-09-15T03:45:32.896679+0000","last_became_peered":"2024-09-15T03:45:32.896679+0000","last_unstale":"2024-09-15T04:04:01.591869+0000","last_undegraded":"2024-09-15T04:04:01.591869+0000","last_fullsized":"2024-09-15T04:04:01.591869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:55:25.625391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010637038,"stat_sum":{"num_bytes":61483012,"num_objects":25,"num_object_clones":22,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":1,"num_read":528,"num_read_kb":621461,"num_write":261,"num_write_kb":96809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d9","length":"1"},{"start":"329","length":"1"},{"start":"34f","length":"1"},{"start":"354","length":"1"},{"start":"356","length":"1"}]},{"pgid":"3.8","version":"902'3611","reported_seq":6101,"reported_epoch":902,"state":"active+clean","last_fresh":"2024-09-15T04:04:05.646465+0000","last_change":"2024-09-15T04:04:04.606092+0000","last_active":"2024-09-15T04:04:05.646465+0000","last_peered":"2024-09-15T04:04:05.646465+0000","last_clean":"2024-09-15T04:04:05.646465+0000","last_became_active":"2024-09-15T03:45:32.890878+0000","last_became_peered":"2024-09-15T03:45:32.890878+0000","last_unstale":"2024-09-15T04:04:05.646465+0000","last_undegraded":"2024-09-15T04:04:05.646465+0000","last_fullsized":"2024-09-15T04:04:05.646465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":3611,"log_dups_size":0,"ondisk_log_size":3611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:07:53.936950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039679408999999999,"stat_sum":{"num_bytes":157718665,"num_objects":68,"num_object_clones":60,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":2,"num_read":1186,"num_read_kb":1261802,"num_write":923,"num_write_kb":384385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":888,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.570516+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T04:04:04.570516+0000","last_peered":"2024-09-15T04:04:04.570516+0000","last_clean":"2024-09-15T04:04:04.570516+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T04:04:04.570516+0000","last_undegraded":"2024-09-15T04:04:04.570516+0000","last_fullsized":"2024-09-15T04:04:04.570516+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"898'1428","reported_seq":4093,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-15T04:04:01.584177+0000","last_change":"2024-09-15T04:04:01.584902+0000","last_active":"2024-09-15T04:04:01.584177+0000","last_peered":"2024-09-15T04:04:01.584177+0000","last_clean":"2024-09-15T04:04:01.584177+0000","last_became_active":"2024-09-15T03:45:32.880272+0000","last_became_peered":"2024-09-15T03:45:32.880272+0000","last_unstale":"2024-09-15T04:04:01.584177+0000","last_undegraded":"2024-09-15T04:04:01.584177+0000","last_fullsized":"2024-09-15T04:04:01.584177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1428,"log_dups_size":0,"ondisk_log_size":1428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T11:41:46.098759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013596993999999999,"stat_sum":{"num_bytes":59659718,"num_objects":29,"num_object_clones":26,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":501,"num_read_kb":495301,"num_write":357,"num_write_kb":140489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d9","length":"1"},{"start":"329","length":"1"},{"start":"34f","length":"1"},{"start":"354","length":"1"},{"start":"356","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":892,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.579886+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T04:04:04.579886+0000","last_peered":"2024-09-15T04:04:04.579886+0000","last_clean":"2024-09-15T04:04:04.579886+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T04:04:04.579886+0000","last_undegraded":"2024-09-15T04:04:04.579886+0000","last_fullsized":"2024-09-15T04:04:04.579886+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"898'1775","reported_seq":4408,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-15T04:04:01.592635+0000","last_change":"2024-09-15T04:04:01.592825+0000","last_active":"2024-09-15T04:04:01.592635+0000","last_peered":"2024-09-15T04:04:01.592635+0000","last_clean":"2024-09-15T04:04:01.592635+0000","last_became_active":"2024-09-15T03:45:32.896844+0000","last_became_peered":"2024-09-15T03:45:32.896844+0000","last_unstale":"2024-09-15T04:04:01.592635+0000","last_undegraded":"2024-09-15T04:04:01.592635+0000","last_fullsized":"2024-09-15T04:04:01.592635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1775,"log_dups_size":0,"ondisk_log_size":1775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:12:54.387570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011081440999999999,"stat_sum":{"num_bytes":49687152,"num_objects":23,"num_object_clones":19,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":2,"num_read":628,"num_read_kb":608229,"num_write":441,"num_write_kb":144877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d9","length":"1"},{"start":"329","length":"1"},{"start":"34f","length":"1"},{"start":"354","length":"1"},{"start":"356","length":"1"}]},{"pgid":"1.2","version":"16'2","reported_seq":878,"reported_epoch":897,"state":"active+clean","last_fresh":"2024-09-15T04:04:00.573360+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T04:04:00.573360+0000","last_peered":"2024-09-15T04:04:00.573360+0000","last_clean":"2024-09-15T04:04:00.573360+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T04:04:00.573360+0000","last_undegraded":"2024-09-15T04:04:00.573360+0000","last_fullsized":"2024-09-15T04:04:00.573360+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"896'884","reported_seq":3582,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-15T04:04:01.571789+0000","last_change":"2024-09-15T04:04:01.572390+0000","last_active":"2024-09-15T04:04:01.571789+0000","last_peered":"2024-09-15T04:04:01.571789+0000","last_clean":"2024-09-15T04:04:01.571789+0000","last_became_active":"2024-09-15T03:45:32.889212+0000","last_became_peered":"2024-09-15T03:45:32.889212+0000","last_unstale":"2024-09-15T04:04:01.571789+0000","last_undegraded":"2024-09-15T04:04:01.571789+0000","last_fullsized":"2024-09-15T04:04:01.571789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:09:49.468131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010439449999999999,"stat_sum":{"num_bytes":23333506,"num_objects":10,"num_object_clones":8,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":319,"num_read_kb":319100,"num_write":223,"num_write_kb":80532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d9","length":"1"},{"start":"329","length":"1"},{"start":"34f","length":"1"},{"start":"354","length":"1"},{"start":"356","length":"1"}]},{"pgid":"1.3","version":"14'1","reported_seq":887,"reported_epoch":897,"state":"active+clean","last_fresh":"2024-09-15T04:04:00.569451+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T04:04:00.569451+0000","last_peered":"2024-09-15T04:04:00.569451+0000","last_clean":"2024-09-15T04:04:00.569451+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T04:04:00.569451+0000","last_undegraded":"2024-09-15T04:04:00.569451+0000","last_fullsized":"2024-09-15T04:04:00.569451+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":973,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.570586+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T04:04:04.570586+0000","last_peered":"2024-09-15T04:04:04.570586+0000","last_clean":"2024-09-15T04:04:04.570586+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T04:04:04.570586+0000","last_undegraded":"2024-09-15T04:04:04.570586+0000","last_fullsized":"2024-09-15T04:04:04.570586+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"902'1100","reported_seq":3767,"reported_epoch":902,"state":"active+clean","last_fresh":"2024-09-15T04:04:05.652025+0000","last_change":"2024-09-15T04:04:04.596079+0000","last_active":"2024-09-15T04:04:05.652025+0000","last_peered":"2024-09-15T04:04:05.652025+0000","last_clean":"2024-09-15T04:04:05.652025+0000","last_became_active":"2024-09-15T03:45:32.897069+0000","last_became_peered":"2024-09-15T03:45:32.897069+0000","last_unstale":"2024-09-15T04:04:05.652025+0000","last_undegraded":"2024-09-15T04:04:05.652025+0000","last_fullsized":"2024-09-15T04:04:05.652025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:36.978197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.023277587999999998,"stat_sum":{"num_bytes":39490589,"num_objects":17,"num_object_clones":15,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":351,"num_read_kb":345770,"num_write":298,"num_write_kb":107757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"3.6","version":"901'559","reported_seq":3283,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.580042+0000","last_change":"2024-09-15T04:04:04.580042+0000","last_active":"2024-09-15T04:04:04.580042+0000","last_peered":"2024-09-15T04:04:04.580042+0000","last_clean":"2024-09-15T04:04:04.580042+0000","last_became_active":"2024-09-15T03:45:32.882448+0000","last_became_peered":"2024-09-15T03:45:32.882448+0000","last_unstale":"2024-09-15T04:04:04.580042+0000","last_undegraded":"2024-09-15T04:04:04.580042+0000","last_fullsized":"2024-09-15T04:04:04.580042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:15:36.051122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072279040000000003,"stat_sum":{"num_bytes":9361690,"num_objects":6,"num_object_clones":5,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":126419,"num_write":137,"num_write_kb":51562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":886,"reported_epoch":897,"state":"active+clean","last_fresh":"2024-09-15T04:04:00.571709+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T04:04:00.571709+0000","last_peered":"2024-09-15T04:04:00.571709+0000","last_clean":"2024-09-15T04:04:00.571709+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T04:04:00.571709+0000","last_undegraded":"2024-09-15T04:04:00.571709+0000","last_fullsized":"2024-09-15T04:04:00.571709+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"901'1759","reported_seq":4420,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.593664+0000","last_change":"2024-09-15T04:04:04.593664+0000","last_active":"2024-09-15T04:04:04.593664+0000","last_peered":"2024-09-15T04:04:04.593664+0000","last_clean":"2024-09-15T04:04:04.593664+0000","last_became_active":"2024-09-15T03:45:32.889536+0000","last_became_peered":"2024-09-15T03:45:32.889536+0000","last_unstale":"2024-09-15T04:04:04.593664+0000","last_undegraded":"2024-09-15T04:04:04.593664+0000","last_fullsized":"2024-09-15T04:04:04.593664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1759,"log_dups_size":0,"ondisk_log_size":1759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:23:16.194280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027091006000000001,"stat_sum":{"num_bytes":61537656,"num_objects":29,"num_object_clones":25,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":3,"num_read":629,"num_read_kb":632554,"num_write":417,"num_write_kb":170254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":876,"reported_epoch":897,"state":"active+clean","last_fresh":"2024-09-15T04:04:00.573323+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T04:04:00.573323+0000","last_peered":"2024-09-15T04:04:00.573323+0000","last_clean":"2024-09-15T04:04:00.573323+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T04:04:00.573323+0000","last_undegraded":"2024-09-15T04:04:00.573323+0000","last_fullsized":"2024-09-15T04:04:00.573323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"898'3571","reported_seq":6047,"reported_epoch":898,"state":"active+clean","last_fresh":"2024-09-15T04:04:01.592459+0000","last_change":"2024-09-15T04:04:01.592793+0000","last_active":"2024-09-15T04:04:01.592459+0000","last_peered":"2024-09-15T04:04:01.592459+0000","last_clean":"2024-09-15T04:04:01.592459+0000","last_became_active":"2024-09-15T03:45:32.889192+0000","last_became_peered":"2024-09-15T03:45:32.889192+0000","last_unstale":"2024-09-15T04:04:01.592459+0000","last_undegraded":"2024-09-15T04:04:01.592459+0000","last_fullsized":"2024-09-15T04:04:01.592459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":3571,"log_dups_size":0,"ondisk_log_size":3571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:00:33.417434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.020649151000000001,"stat_sum":{"num_bytes":121046668,"num_objects":55,"num_object_clones":47,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":3,"num_read":1213,"num_read_kb":1246593,"num_write":804,"num_write_kb":316415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d9","length":"1"},{"start":"329","length":"1"},{"start":"34f","length":"1"},{"start":"354","length":"1"},{"start":"356","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":888,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.571514+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T04:04:04.571514+0000","last_peered":"2024-09-15T04:04:04.571514+0000","last_clean":"2024-09-15T04:04:04.571514+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T04:04:04.571514+0000","last_undegraded":"2024-09-15T04:04:04.571514+0000","last_fullsized":"2024-09-15T04:04:04.571514+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"901'1836","reported_seq":4507,"reported_epoch":902,"state":"active+clean","last_fresh":"2024-09-15T04:04:05.631242+0000","last_change":"2024-09-15T04:04:04.589711+0000","last_active":"2024-09-15T04:04:05.631242+0000","last_peered":"2024-09-15T04:04:05.631242+0000","last_clean":"2024-09-15T04:04:05.631242+0000","last_became_active":"2024-09-15T03:45:32.890936+0000","last_became_peered":"2024-09-15T03:45:32.890936+0000","last_unstale":"2024-09-15T04:04:05.631242+0000","last_undegraded":"2024-09-15T04:04:05.631242+0000","last_fullsized":"2024-09-15T04:04:05.631242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:45:07.778025+0000","last_clean_scrub_stamp":"2024-09-15T03:45:07.778025+0000","objects_scrubbed":0,"log_size":1836,"log_dups_size":0,"ondisk_log_size":1836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:59.679215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096993070000000008,"stat_sum":{"num_bytes":45796160,"num_objects":21,"num_object_clones":17,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":1,"num_read":756,"num_read_kb":761157,"num_write":416,"num_write_kb":144564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"323","length":"1"},{"start":"34f","length":"1"}]},{"pgid":"1.7","version":"0'0","reported_seq":892,"reported_epoch":901,"state":"active+clean","last_fresh":"2024-09-15T04:04:04.580073+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T04:04:04.580073+0000","last_peered":"2024-09-15T04:04:04.580073+0000","last_clean":"2024-09-15T04:04:04.580073+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T04:04:04.580073+0000","last_undegraded":"2024-09-15T04:04:04.580073+0000","last_fullsized":"2024-09-15T04:04:04.580073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":843185743,"num_objects":368,"num_object_clones":318,"num_object_copies":736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":368,"num_whiteouts":15,"num_read":7859,"num_read_kb":8000666,"num_write":5552,"num_write_kb":2135553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":967159808,"data_stored":1323890961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22724,"ondisk_log_size":22724,"up":32,"acting":32,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640489,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427104,"kb_used_data":121420,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93944736,"statfs":{"total":96636764160,"available":96199409664,"internally_reserved":0,"allocated":124334080,"data_stored":163587232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.63400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.224,"5min":1.224,"15min":5.1609999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.81399999999999995,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.374,"15min":0.372},"max":{"1min":1.1799999999999999,"5min":1.2609999999999999,"15min":3.5099999999999998},"last":0.90700000000000003}]},{"osd":1,"last update":"Sun Sep 15 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.54700000000000004,"5min":0.40200000000000002,"15min":0.36499999999999999},"max":{"1min":1.0329999999999999,"5min":1.6619999999999999,"15min":2.2890000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.61899999999999999,"5min":0.44500000000000001,"15min":0.41199999999999998},"max":{"1min":1.2230000000000001,"5min":1.264,"15min":1.341},"last":1.022}]},{"osd":2,"last update":"Sun Sep 15 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.90600000000000003,"15min":0.86199999999999999},"min":{"1min":0.60299999999999998,"5min":0.51700000000000002,"15min":0.45500000000000002},"max":{"1min":1.3380000000000001,"5min":9.8149999999999995,"15min":9.8149999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.93000000000000005,"15min":0.88},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.36699999999999999},"max":{"1min":1.258,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.81599999999999995}]},{"osd":3,"last update":"Sun Sep 15 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.56499999999999995,"5min":0.39900000000000002,"15min":0.27200000000000002},"max":{"1min":0.90200000000000002,"5min":1.3640000000000001,"15min":5.2539999999999996},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":1.018,"5min":1.391,"15min":2.3700000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Sun Sep 15 04:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":0.99099999999999999,"5min":1.4279999999999999,"15min":5.2830000000000004},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.93000000000000005,"15min":0.93799999999999994},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.1699999999999999,"5min":1.3899999999999999,"15min":5.3650000000000002},"last":1.052}]}]},{"osd":4,"up_from":11,"seq":47244640488,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454180,"kb_used_data":148504,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93917660,"statfs":{"total":96636764160,"available":96171683840,"internally_reserved":0,"allocated":152068096,"data_stored":197804258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80200000000000005,"15min":0.79700000000000004},"min":{"1min":0.58899999999999997,"5min":0.40000000000000002,"15min":0.29499999999999998},"max":{"1min":0.99399999999999999,"5min":1.1699999999999999,"15min":1.3420000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.89200000000000002,"15min":0.84199999999999997},"min":{"1min":0.71499999999999997,"5min":0.40300000000000002,"15min":0.35099999999999998},"max":{"1min":1.27,"5min":1.401,"15min":1.401},"last":0.79800000000000004}]},{"osd":1,"last update":"Sun Sep 15 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.879,"15min":0.84999999999999998},"min":{"1min":0.56399999999999995,"5min":0.45800000000000002,"15min":0.41999999999999998},"max":{"1min":1.145,"5min":1.327,"15min":1.8100000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.86399999999999999,"15min":0.875},"min":{"1min":0.76900000000000002,"5min":0.54900000000000004,"15min":0.42599999999999999},"max":{"1min":1.228,"5min":1.228,"15min":2.2050000000000001},"last":0.91800000000000004}]},{"osd":2,"last update":"Sun Sep 15 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.55000000000000004,"5min":0.44700000000000001,"15min":0.38600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":3.2440000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.873,"15min":0.89000000000000001},"min":{"1min":0.66600000000000004,"5min":0.505,"15min":0.45500000000000002},"max":{"1min":1.139,"5min":1.226,"15min":2.5059999999999998},"last":0.74199999999999999}]},{"osd":3,"last update":"Sun Sep 15 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72799999999999998,"15min":0.76600000000000001},"min":{"1min":0.48899999999999999,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.335},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.85999999999999999,"15min":0.86499999999999999},"min":{"1min":0.64100000000000001,"5min":0.47699999999999998,"15min":0.39300000000000002},"max":{"1min":1.1519999999999999,"5min":1.1759999999999999,"15min":1.5409999999999999},"last":0.83099999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:03:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.78800000000000003,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.40699999999999997,"15min":0.308},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.474},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.629,"5min":0.46400000000000002,"15min":0.315},"max":{"1min":1.1619999999999999,"5min":1.242,"15min":1.26},"last":0.94599999999999995}]}]},{"osd":3,"up_from":11,"seq":47244640490,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":385524,"kb_used_data":71528,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":93986316,"statfs":{"total":96636764160,"available":96241987584,"internally_reserved":0,"allocated":73244672,"data_stored":96756359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.499,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.2749999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.66200000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2230000000000001,"5min":1.3129999999999999,"15min":1.3680000000000001},"last":0.80600000000000005}]},{"osd":1,"last update":"Sun Sep 15 04:03:21 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83999999999999997,"15min":0.83599999999999997},"min":{"1min":0.54700000000000004,"5min":0.42499999999999999,"15min":0.40699999999999997},"max":{"1min":1.135,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91500000000000004,"15min":0.873},"min":{"1min":0.65200000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1890000000000001,"5min":1.954,"15min":1.954},"last":0.85099999999999998}]},{"osd":2,"last update":"Sun Sep 15 04:03:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84299999999999997,"15min":0.85599999999999998},"min":{"1min":0.60399999999999998,"5min":0.432,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":1.2709999999999999,"15min":1.3799999999999999},"last":0.754},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82199999999999995,"15min":0.84999999999999998},"min":{"1min":0.63700000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.98299999999999998,"5min":1.2310000000000001,"15min":1.4330000000000001},"last":0.76800000000000002}]},{"osd":4,"last update":"Sun Sep 15 04:03:21 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.53100000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.177,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.63},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.59099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.69899999999999995}]},{"osd":5,"last update":"Sun Sep 15 04:03:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71199999999999997,"15min":0.73499999999999999},"min":{"1min":0.61399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.5740000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.83799999999999997,"15min":0.85399999999999998},"min":{"1min":0.75700000000000001,"5min":0.45600000000000002,"15min":0.435},"max":{"1min":1.1339999999999999,"5min":1.1379999999999999,"15min":1.621},"last":0.79300000000000004}]}]},{"osd":2,"up_from":11,"seq":47244640489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594668,"kb_used_data":288984,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":93777172,"statfs":{"total":96636764160,"available":96027824128,"internally_reserved":0,"allocated":295919616,"data_stored":411272829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57299999999999995,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.27700000000000002,"15min":0.253},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":6.3019999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73799999999999999,"15min":0.76000000000000001},"min":{"1min":0.38200000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.002,"5min":1.113,"15min":6.3550000000000004},"last":0.83399999999999996}]},{"osd":1,"last update":"Sun Sep 15 04:03:28 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70699999999999996,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.036,"5min":1.121,"15min":1.952},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76900000000000002,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.25,"15min":0.25},"max":{"1min":1.022,"5min":1.119,"15min":2.0019999999999998},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Sep 15 04:03:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.94899999999999995,"15min":0.95799999999999996},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.976},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94399999999999995,"15min":0.93500000000000005},"min":{"1min":0.82599999999999996,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.86099999999999999}]},{"osd":4,"last update":"Sun Sep 15 04:03:33 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.88400000000000001,"15min":0.89100000000000001},"min":{"1min":0.626,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":0.95899999999999996,"15min":0.91300000000000003},"min":{"1min":0.75700000000000001,"5min":0.503,"15min":0.503},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":0.96799999999999997}]},{"osd":5,"last update":"Sun Sep 15 04:03:28 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.93999999999999995,"15min":0.92500000000000004},"min":{"1min":0.68999999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2170000000000001,"5min":1.4810000000000001,"15min":2.2330000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.92600000000000005,"15min":0.91800000000000004},"min":{"1min":0.63900000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.131,"5min":1.5449999999999999,"15min":1.637},"last":1.01}]}]},{"osd":1,"up_from":11,"seq":47244640490,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552328,"kb_used_data":250808,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93819512,"statfs":{"total":96636764160,"available":96071180288,"internally_reserved":0,"allocated":256827392,"data_stored":353463207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.59699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.883},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":1.976},"last":0.78200000000000003}]},{"osd":2,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64900000000000002,"15min":0.66400000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85699999999999998,"5min":1.5249999999999999,"15min":2.536},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.012,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.57099999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0820000000000001,"5min":1.2509999999999999,"15min":1.288},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0149999999999999,"15min":0.95499999999999996},"min":{"1min":0.65200000000000002,"5min":0.51900000000000002,"15min":0.50800000000000001},"max":{"1min":1.1970000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.1639999999999999}]},{"osd":4,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.024,"5min":1.677,"15min":1.677},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.68600000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2070000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.91500000000000004},"min":{"1min":0.628,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.115,"5min":1.617,"15min":1.617},"last":1.2789999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.278,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.056}]}]},{"osd":0,"up_from":11,"seq":47244640490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":385492,"kb_used_data":83968,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":93986348,"statfs":{"total":96636764160,"available":96242020352,"internally_reserved":0,"allocated":85983232,"data_stored":111861573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57499999999999996,"15min":0.64900000000000002},"min":{"1min":0.35599999999999998,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.76800000000000002,"5min":1.1659999999999999,"15min":18.481000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.70199999999999996,"15min":0.751},"min":{"1min":0.47599999999999998,"5min":0.247,"15min":0.247},"max":{"1min":2.222,"5min":2.222,"15min":19.239000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Sun Sep 15 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.67700000000000005,"15min":0.69299999999999995},"min":{"1min":0.52400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.038,"5min":0.91500000000000004,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":1.079}]},{"osd":3,"last update":"Sun Sep 15 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.872,"15min":0.86599999999999999},"min":{"1min":0.51300000000000001,"5min":0.44400000000000001,"15min":0.40200000000000002},"max":{"1min":2.395,"5min":2.395,"15min":2.5449999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.68799999999999994,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.94899999999999995}]},{"osd":4,"last update":"Sun Sep 15 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92100000000000004,"15min":0.89200000000000002},"min":{"1min":0.60299999999999998,"5min":0.55600000000000005,"15min":0.46800000000000003},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.746},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.01,"5min":0.99299999999999999,"15min":0.95299999999999996},"min":{"1min":0.57499999999999996,"5min":0.438,"15min":0.438},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.91300000000000003}]},{"osd":5,"last update":"Sun Sep 15 04:03:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.86699999999999999,"15min":0.89700000000000002},"min":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.44500000000000001},"max":{"1min":1.21,"5min":1.238,"15min":1.2989999999999999},"last":1.054},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96099999999999997,"15min":0.98899999999999999},"min":{"1min":0.63300000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.282,"5min":1.4379999999999999,"15min":2.6970000000000001},"last":1.123}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":82137088,"data_stored":109750294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252493824,"data_stored":350845642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292077568,"data_stored":409161569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":68870144,"data_stored":94116171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":151089152,"data_stored":198541313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":120492032,"data_stored":161475972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T04:04:07.536 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-15T04:04:07.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-15T04:04:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient: get_monmap_and_config 2024-09-15T04:04:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient: build_initial_monmap 2024-09-15T04:04:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient: monmap: 2024-09-15T04:04:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:07.642077+0000 2024-09-15T04:04:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:07.642077+0000 2024-09-15T04:04:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:07.649 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 1 Processor -- start 2024-09-15T04:04:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 1 -- start start 2024-09-15T04:04:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient: init 2024-09-15T04:04:07.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:07.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient: _add_conns ranks=[6,5,7,4,3,0,1,8,2] 2024-09-15T04:04:07.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient(hunting): picked mon.noname-d con 0x7fd9f4132ce0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:07.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient(hunting): picked mon.noname-h con 0x7fd9f4133370 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:07.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient(hunting): picked mon.noname-i con 0x7fd9f4133740 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f404c150 con 0x7fd9f4133370 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f40a0750 con 0x7fd9f4132ce0 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f41330b0 con 0x7fd9f4133740 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f9e4a700 10 monclient(hunting): authenticate will time out at 1992.071792s 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f37fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fd9f4132ce0 0x7fd9f40a0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58418/0 (socket says 172.21.15.26:58418) 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f2ffd700 1 -- >> v1:172.21.15.122:6791/0 conn(0x7fd9f4133370 legacy=0x7fd9f4138030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.122:6791/0 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.639+0000 7fd9f37fe700 1 -- 172.21.15.26:0/1642486274 learned_addr learned my addr 172.21.15.26:0/1642486274 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735520947 0 0) 0x7fd9f40a0750 con 0x7fd9f4132ce0 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient(hunting): my global_id is 195270 2024-09-15T04:04:07.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e0003570 con 0x7fd9f4132ce0 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624922658 0 0) 0x7fd9f41330b0 con 0x7fd9f4133740 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient(hunting): my global_id is 254311 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f40a0750 con 0x7fd9f4133740 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1853829353 0 0) 0x7fd9e0003570 con 0x7fd9f4132ce0 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 >> v1:172.21.15.122:6792/0 conn(0x7fd9f4133740 legacy=0x7fd9f413b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 >> v1:172.21.15.122:6791/0 conn(0x7fd9f4133370 legacy=0x7fd9f4138030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 monclient: found mon.noname-d 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:07.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40a2ae0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd9ec001b30 con 0x7fd9f4132ce0 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 5 monclient: authenticate success, global_id 195270 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: dump: 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.659 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd9ec001e10 con 0x7fd9f4132ce0 2024-09-15T04:04:07.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 1 -- 172.21.15.26:0/1642486274 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd9ec004fb0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 4 set_mon_vals no callback set 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f27fc700 10 monclient: dump: 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.661 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: shutdown 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:07.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/1642486274 >> v1:172.21.15.26:6792/0 conn(0x7fd9f4132ce0 legacy=0x7fd9f40a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/1642486274 shutdown_connections 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/1642486274 >> 172.21.15.26:0/1642486274 conn(0x7fd9f406e580 msgr2=0x7fd9f40a6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/1642486274 shutdown_connections 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/1642486274 wait complete. 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: build_initial_monmap 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: monmap: 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:07.663 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 Processor -- start 2024-09-15T04:04:07.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- start start 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: init 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient: _add_conns ranks=[7,8,1,3,5,2,6,4,0] 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient(hunting): picked mon.noname-i con 0x7fd9f4132ce0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient(hunting): picked mon.noname-e con 0x7fd9f4133370 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient(hunting): picked mon.noname-f con 0x7fd9f4133740 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f40a1f10 con 0x7fd9f4133740 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f409d1b0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f4122140 con 0x7fd9f4133370 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f9e4a700 10 monclient(hunting): authenticate will time out at 1992.075129s 2024-09-15T04:04:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f2ffd700 1 -- >> v1:172.21.15.26:6793/0 conn(0x7fd9f4133370 legacy=0x7fd9f4147040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.26:6793/0 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f3fff700 1 -- >> v1:172.21.15.122:6789/0 conn(0x7fd9f4133740 legacy=0x7fd9f414a770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.122:6789/0 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f37fe700 1 --1- >> v1:172.21.15.122:6792/0 conn(0x7fd9f4132ce0 0x7fd9f4131a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:44112/0 (socket says 172.21.15.26:44112) 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9f37fe700 1 -- 172.21.15.26:0/2002764292 learned_addr learned my addr 172.21.15.26:0/2002764292 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828478961 0 0) 0x7fd9f409d1b0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9eb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9eb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9eb7fe700 10 monclient(hunting): my global_id is 254320 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.643+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d4003570 con 0x7fd9f4132ce0 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2104608411 0 0) 0x7fd9d4003570 con 0x7fd9f4132ce0 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd9f409d1b0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd9ec004ef0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient(hunting): dump: 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.667 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.668 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.668 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.668 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.668 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.668 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3692358815 0 0) 0x7fd9f409d1b0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 >> v1:172.21.15.26:6793/0 conn(0x7fd9f4133370 legacy=0x7fd9f4147040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 >> v1:172.21.15.122:6789/0 conn(0x7fd9f4133740 legacy=0x7fd9f414a770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 monclient: found mon.i 2024-09-15T04:04:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4121b80 con 0x7fd9f4132ce0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 5 monclient: authenticate success, global_id 254320 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 monclient: _renew_subs 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f41200c0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 monclient: _renew_subs 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f41207c0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd9ec005280 con 0x7fd9f4132ce0 2024-09-15T04:04:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd9ec005730 con 0x7fd9f4132ce0 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f1ffb700 4 set_mon_vals no callback set 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 10 monclient: dump: 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d80050b0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd9ec0100f0 con 0x7fd9f4132ce0 2024-09-15T04:04:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 8 ==== osd_map(903..903 src has 435..903) v4 ==== 3911+0+0 (unknown 3839595639 0 0) 0x7fd9ec044550 con 0x7fd9f4132ce0 2024-09-15T04:04:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.647+0000 7fd9f2ffd700 10 monclient: get_auth_request con 0x7fd9d40373a0 auth_method 0 2024-09-15T04:04:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.655+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fd9ec015440 con 0x7fd9f4132ce0 2024-09-15T04:04:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.655+0000 7fd9eb7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:07.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.655+0000 7fd9eb7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:07.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.819+0000 7fd9f9e4a700 10 start_mon_command cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-15T04:04:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.819+0000 7fd9f9e4a700 10 monclient: _send_command 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-15T04:04:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.819+0000 7fd9f9e4a700 10 monclient: _send_mon_message to mon.i at v1:172.21.15.122:6792/0 2024-09-15T04:04:07.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:07.819+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 --> v1:172.21.15.122:6792/0 -- mon_command({"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9d8005360 con 0x7fd9f4132ce0 2024-09-15T04:04:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.663+0000 7fd9eb7fe700 1 -- 172.21.15.26:0/2002764292 <== mon.7 v1:172.21.15.122:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]=0 pool 'unique_pool_0' removed v904) v1 ==== 175+0+0 (unknown 3150083700 0 0) 0x7fd9ec014e40 con 0x7fd9f4132ce0 2024-09-15T04:04:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.663+0000 7fd9eb7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}] 2024-09-15T04:04:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.663+0000 7fd9eb7fe700 10 monclient: _finish_command 2 = system:0 pool 'unique_pool_0' removed 2024-09-15T04:04:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 >> v1:172.21.15.122:6812/20895 conn(0x7fd9d40373a0 legacy=0x7fd9d4039850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 10 monclient: shutdown 2024-09-15T04:04:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 >> v1:172.21.15.122:6792/0 conn(0x7fd9f4132ce0 legacy=0x7fd9f4131a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:08.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 shutdown_connections 2024-09-15T04:04:08.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 >> 172.21.15.26:0/2002764292 conn(0x7fd9f406e580 msgr2=0x7fd9f409e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:08.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 shutdown_connections 2024-09-15T04:04:08.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:08.671+0000 7fd9f9e4a700 1 -- 172.21.15.26:0/2002764292 wait complete. 2024-09-15T04:04:08.680 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'unique_pool_0' removed 2024-09-15T04:04:08.697 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-15T04:04:08.710 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-15T04:04:18.436 INFO:tasks.mon_thrash.mon_thrasher:killing mon.h 2024-09-15T04:04:18.436 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.h 2024-09-15T04:04:18.436 INFO:tasks.ceph.mon.h:Restarting daemon 2024-09-15T04:04:18.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-09-15T04:04:18.439 INFO:tasks.ceph.mon.h:Started 2024-09-15T04:04:18.440 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-09-15T04:04:18.440 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-09-15T04:04:18.440 INFO:tasks.ceph.mon.e:Restarting daemon 2024-09-15T04:04:18.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-09-15T04:04:18.445 INFO:tasks.ceph.mon.e:Started 2024-09-15T04:04:18.445 INFO:tasks.mon_thrash.mon_thrasher:killing mon.f 2024-09-15T04:04:18.446 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.f 2024-09-15T04:04:18.446 INFO:tasks.ceph.mon.f:Restarting daemon 2024-09-15T04:04:18.446 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-09-15T04:04:18.448 INFO:tasks.ceph.mon.f:Started 2024-09-15T04:04:18.448 INFO:tasks.mon_thrash:Sending STOP to mon g 2024-09-15T04:04:18.448 INFO:tasks.ceph.mon.g:Sent signal 19 2024-09-15T04:04:18.448 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-15T04:04:33.449 INFO:tasks.mon_thrash:Sending CONT to mon g 2024-09-15T04:04:33.450 INFO:tasks.ceph.mon.g:Sent signal 18 2024-09-15T04:04:33.450 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T04:04:33.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:04:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient: get_monmap_and_config 2024-09-15T04:04:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient: build_initial_monmap 2024-09-15T04:04:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient: monmap: 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:33.561890+0000 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:33.561890+0000 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:33.567 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 1 Processor -- start 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 1 -- start start 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient: init 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient: _add_conns ranks=[1,7,5,3,2,0,6,8,4] 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient(hunting): picked mon.noname-f con 0x7f81f4099310 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient(hunting): picked mon.noname-i con 0x7f81f4132fa0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient(hunting): picked mon.noname-h con 0x7f81f41333f0 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f404ca10 con 0x7f81f4099310 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f4097e50 con 0x7f81f41333f0 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f40a1410 con 0x7f81f4132fa0 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fbaf3700 10 monclient(hunting): authenticate will time out at 2017.991317s 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81f988f700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f81f4099310 0x7f81f40996e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:51082/0 (socket says 172.21.15.26:51082) 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81fa090700 1 --1- >> v1:172.21.15.122:6791/0 conn(0x7f81f41333f0 0x7f81f413b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:33738/0 (socket says 172.21.15.26:33738) 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81f988f700 1 -- 172.21.15.26:0/4152324552 learned_addr learned my addr 172.21.15.26:0/4152324552 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.558+0000 7f81f908e700 1 --1- 172.21.15.26:0/4152324552 >> v1:172.21.15.122:6792/0 conn(0x7f81f4132fa0 0x7f81f41380a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:41610/0 (socket says 172.21.15.26:41610) 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054432 0 0) 0x7f81f4097e50 con 0x7f81f41333f0 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): my global_id is 274109 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e0003570 con 0x7f81f41333f0 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166313807 0 0) 0x7f81f40a1410 con 0x7f81f4132fa0 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): my global_id is 254329 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f4097e50 con 0x7f81f4132fa0 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231865819 0 0) 0x7f81f404ca10 con 0x7f81f4099310 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient(hunting): my global_id is 274105 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f40a1410 con 0x7f81f4099310 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3195105702 0 0) 0x7f81f4097e50 con 0x7f81f4132fa0 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 >> v1:172.21.15.122:6791/0 conn(0x7f81f41333f0 legacy=0x7f81f413b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 >> v1:172.21.15.122:6789/0 conn(0x7f81f4099310 legacy=0x7f81f40996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: _finish_hunting 0 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 monclient: found mon.noname-i 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.122:6792/0 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 --> v1:172.21.15.122:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f40a2710 con 0x7f81f4132fa0 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: _finish_auth 0 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81e4001700 con 0x7f81f4132fa0 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 5 monclient: authenticate success, global_id 254329 2024-09-15T04:04:33.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: dump: 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.7 v1:172.21.15.122:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81e4001920 con 0x7f81f4132fa0 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 1 -- 172.21.15.26:0/4152324552 <== mon.7 v1:172.21.15.122:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81e4004830 con 0x7f81f4132fa0 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 4 set_mon_vals no callback set 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:33.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f888d700 10 monclient: dump: 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: shutdown 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- 172.21.15.26:0/4152324552 >> v1:172.21.15.122:6792/0 conn(0x7f81f4132fa0 legacy=0x7f81f41380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- 172.21.15.26:0/4152324552 shutdown_connections 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- 172.21.15.26:0/4152324552 >> 172.21.15.26:0/4152324552 conn(0x7f81f40a6c30 msgr2=0x7f81f406dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- 172.21.15.26:0/4152324552 shutdown_connections 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- 172.21.15.26:0/4152324552 wait complete. 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: build_initial_monmap 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: monmap: 2024-09-15T04:04:33.582 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:33.583 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 Processor -- start 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- start start 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: init 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient: _add_conns ranks=[6,8,7,2,3,4,0,5,1] 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient(hunting): picked mon.noname-d con 0x7f81f4099310 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient(hunting): picked mon.noname-e con 0x7f81f4132fa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient(hunting): picked mon.noname-i con 0x7f81f41333f0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f411f980 con 0x7f81f4099310 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f40a1d70 con 0x7f81f41333f0 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f40a1fa0 con 0x7f81f4132fa0 2024-09-15T04:04:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81fbaf3700 10 monclient(hunting): authenticate will time out at 2017.994959s 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f988f700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f81f4099310 0x7f81f411c890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:57468/0 (socket says 172.21.15.26:57468) 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81f988f700 1 -- 172.21.15.26:0/1738305306 learned_addr learned my addr 172.21.15.26:0/1738305306 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782235512 0 0) 0x7f81f411f980 con 0x7f81f4099310 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): my global_id is 195279 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d8003570 con 0x7f81f4099310 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77195806 0 0) 0x7f81f40a1fa0 con 0x7f81f4132fa0 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): my global_id is 274112 2024-09-15T04:04:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f411f980 con 0x7f81f4132fa0 2024-09-15T04:04:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203365570 0 0) 0x7f81f40a1d70 con 0x7f81f41333f0 2024-09-15T04:04:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): my global_id is 254338 2024-09-15T04:04:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f40a1fa0 con 0x7f81f41333f0 2024-09-15T04:04:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1059908327 0 0) 0x7f81f411f980 con 0x7f81f4132fa0 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81f40a1d70 con 0x7f81f4132fa0 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81e40016c0 con 0x7f81f4132fa0 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.562+0000 7f81e9ffb700 10 monclient(hunting): dump: 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:33.588 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 716286924 0 0) 0x7f81d8003570 con 0x7f81f4099310 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81f411f980 con 0x7f81f4099310 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81f0001510 con 0x7f81f4099310 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient(hunting): dump: 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.589 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2582591177 0 0) 0x7f81f40a1d70 con 0x7f81f4132fa0 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 >> v1:172.21.15.122:6792/0 conn(0x7f81f41333f0 legacy=0x7f81f411d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 >> v1:172.21.15.26:6792/0 conn(0x7f81f4099310 legacy=0x7f81f411c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 monclient: found mon.e 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f4120070 con 0x7f81f4132fa0 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: _finish_auth 0 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 5 monclient: authenticate success, global_id 274112 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 monclient: _renew_subs 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f411e2a0 con 0x7f81f4132fa0 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 monclient: _renew_subs 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f4120350 con 0x7f81f4132fa0 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f81e4001180 con 0x7f81f4132fa0 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f81e4005170 con 0x7f81f4132fa0 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 10 monclient: dump: 2024-09-15T04:04:33.592 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:33.593 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81ebfff700 4 set_mon_vals no callback set 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f81e400fb30 con 0x7f81f4132fa0 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d40050b0 con 0x7f81f4132fa0 2024-09-15T04:04:33.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f81e4043f00 con 0x7f81f4132fa0 2024-09-15T04:04:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.566+0000 7f81f988f700 10 monclient: get_auth_request con 0x7f81d8037c10 auth_method 0 2024-09-15T04:04:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.574+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f81e4010fb0 con 0x7f81f4132fa0 2024-09-15T04:04:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.574+0000 7f81e9ffb700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.574+0000 7f81e9ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:33.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81fbaf3700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:04:33.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81fbaf3700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:33.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81fbaf3700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:33.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f81d40058e0 con 0x7f81f4132fa0 2024-09-15T04:04:33.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81e9ffb700 1 -- 172.21.15.26:0/1738305306 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2695 (unknown 254746657 0 3822096725) 0x7f81e40148f0 con 0x7f81f4132fa0 2024-09-15T04:04:33.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81e9ffb700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:33.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.802+0000 7f81e9ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:33.810 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":120,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":4,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 >> v1:172.21.15.122:6812/20895 conn(0x7f81d8037c10 legacy=0x7f81d803a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 10 monclient: shutdown 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 >> v1:172.21.15.26:6793/0 conn(0x7f81f4132fa0 legacy=0x7f81f411cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 shutdown_connections 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 >> 172.21.15.26:0/1738305306 conn(0x7f81f40a6c30 msgr2=0x7f81f4067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 shutdown_connections 2024-09-15T04:04:33.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:33.806+0000 7f81fbaf3700 1 -- 172.21.15.26:0/1738305306 wait complete. 2024-09-15T04:04:36.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:04:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient: get_monmap_and_config 2024-09-15T04:04:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient: build_initial_monmap 2024-09-15T04:04:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient: monmap: 2024-09-15T04:04:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:36.933719+0000 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:36.933719+0000 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:36.938 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 1 Processor -- start 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 1 -- start start 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient: init 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient: _add_conns ranks=[5,8,3,6,1,2,4,7,0] 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient(hunting): picked mon.noname-h con 0x7f8e3c063c20 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient(hunting): picked mon.noname-e con 0x7f8e3c09aca0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient(hunting): picked mon.noname-g con 0x7f8e3c09e100 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c04ca10 con 0x7f8e3c09e100 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0696a0 con 0x7f8e3c063c20 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0687d0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e40892700 10 monclient(hunting): authenticate will time out at 2021.363267s 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e3affd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8e3c09aca0 0x7f8e3c137ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39264/0 (socket says 172.21.15.26:39264) 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e3affd700 1 -- 172.21.15.26:0/774497367 learned_addr learned my addr 172.21.15.26:0/774497367 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393694455 0 0) 0x7f8e3c0687d0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.930+0000 7f8e3a7fc700 10 monclient(hunting): my global_id is 274121 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e24003570 con 0x7f8e3c09aca0 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551314062 0 0) 0x7f8e3c0696a0 con 0x7f8e3c063c20 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient(hunting): my global_id is 274118 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e3c0687d0 con 0x7f8e3c063c20 2024-09-15T04:04:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 658755523 0 0) 0x7f8e24003570 con 0x7f8e3c09aca0 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 >> v1:172.21.15.122:6791/0 conn(0x7f8e3c063c20 legacy=0x7f8e3c09d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 >> v1:172.21.15.122:6790/0 conn(0x7f8e3c09e100 legacy=0x7f8e3c13b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 monclient: found mon.noname-e 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0a0250 con 0x7f8e3c09aca0 2024-09-15T04:04:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e2c001b30 con 0x7f8e3c09aca0 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: dump: 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 5 monclient: authenticate success, global_id 274121 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8e2c001da0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 1 -- 172.21.15.26:0/774497367 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e2c001f90 con 0x7f8e3c09aca0 2024-09-15T04:04:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 4 set_mon_vals no callback set 2024-09-15T04:04:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e3a7fc700 10 monclient: dump: 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 10 monclient: shutdown 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 1 -- 172.21.15.26:0/774497367 >> v1:172.21.15.26:6793/0 conn(0x7f8e3c09aca0 legacy=0x7f8e3c137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 1 -- 172.21.15.26:0/774497367 shutdown_connections 2024-09-15T04:04:36.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 1 -- 172.21.15.26:0/774497367 >> 172.21.15.26:0/774497367 conn(0x7f8e3c0a4950 msgr2=0x7f8e3c063340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:36.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 1 -- 172.21.15.26:0/774497367 shutdown_connections 2024-09-15T04:04:36.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 1 -- 172.21.15.26:0/774497367 wait complete. 2024-09-15T04:04:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 10 monclient: build_initial_monmap 2024-09-15T04:04:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.934+0000 7f8e40892700 10 monclient: monmap: 2024-09-15T04:04:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 Processor -- start 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 -- start start 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: init 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: _add_conns ranks=[8,2,7,0,6,3,5,4,1] 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient(hunting): picked mon.noname-e con 0x7f8e3c09aca0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient(hunting): picked mon.noname-b con 0x7f8e3c09e100 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient(hunting): picked mon.noname-i con 0x7f8e3c120680 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c1368b0 con 0x7f8e3c09e100 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0678b0 con 0x7f8e3c120680 2024-09-15T04:04:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e3b7fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f8e3c09aca0 0x7f8e3c135b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39268/0 (socket says 172.21.15.26:39268) 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e3b7fe700 1 -- 172.21.15.26:0/2094197435 learned_addr learned my addr 172.21.15.26:0/2094197435 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c06e120 con 0x7f8e3c09aca0 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient(hunting): authenticate will time out at 2021.370321s 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517507586 0 0) 0x7f8e3c06e120 con 0x7f8e3c09aca0 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): my global_id is 274130 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e28003570 con 0x7f8e3c09aca0 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462061715 0 0) 0x7f8e3c1368b0 con 0x7f8e3c09e100 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): my global_id is 254252 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e3c06e120 con 0x7f8e3c09e100 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103867758 0 0) 0x7f8e3c0678b0 con 0x7f8e3c120680 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): my global_id is 254347 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e3c1368b0 con 0x7f8e3c120680 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2572020888 0 0) 0x7f8e28003570 con 0x7f8e3c09aca0 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e3c0678b0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e30002a70 con 0x7f8e3c09aca0 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): dump: 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:36.960 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4211927976 0 0) 0x7f8e3c06e120 con 0x7f8e3c09e100 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e28003570 con 0x7f8e3c09e100 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e2c001f10 con 0x7f8e3c09e100 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient(hunting): dump: 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 438353531 0 0) 0x7f8e3c0678b0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 >> v1:172.21.15.122:6792/0 conn(0x7f8e3c120680 legacy=0x7f8e3c14a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 >> v1:172.21.15.26:6790/0 conn(0x7f8e3c09e100 legacy=0x7f8e3c147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 monclient: found mon.e 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c142bb0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e237fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 5 monclient: authenticate success, global_id 274130 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: _renew_subs 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c121d40 con 0x7f8e3c09aca0 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: _renew_subs 2024-09-15T04:04:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.938+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c121170 con 0x7f8e3c09aca0 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f8e30002d30 con 0x7f8e3c09aca0 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f8e300019a0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e39ffb700 4 set_mon_vals no callback set 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 10 monclient: dump: 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f8e30001b00 con 0x7f8e3c09aca0 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e40892700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e40892700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e40892700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c0050b0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f8e30043c00 con 0x7f8e3c09aca0 2024-09-15T04:04:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.942+0000 7f8e3affd700 10 monclient: get_auth_request con 0x7f8e28037d50 auth_method 0 2024-09-15T04:04:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.950+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f8e30014bf0 con 0x7f8e3c09aca0 2024-09-15T04:04:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.950+0000 7f8e237fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:36.950+0000 7f8e237fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e40892700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:04:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e40892700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e40892700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:37.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8e1c0058e0 con 0x7f8e3c09aca0 2024-09-15T04:04:37.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e237fe700 1 -- 172.21.15.26:0/2094197435 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2695 (unknown 254746657 0 1860335562) 0x7f8e300145f0 con 0x7f8e3c09aca0 2024-09-15T04:04:37.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e237fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:37.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.174+0000 7f8e237fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:37.183 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":120,"quorum":[0,1,2,4,5,6,7,8],"quorum_names":["a","f","b","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":7,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 >> v1:172.21.15.122:6812/20895 conn(0x7f8e28037d50 legacy=0x7f8e2803a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 10 monclient: shutdown 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 >> v1:172.21.15.26:6793/0 conn(0x7f8e3c09aca0 legacy=0x7f8e3c135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 shutdown_connections 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 >> 172.21.15.26:0/2094197435 conn(0x7f8e3c0a4950 msgr2=0x7f8e3c09ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:37.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 shutdown_connections 2024-09-15T04:04:37.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:37.178+0000 7f8e40892700 1 -- 172.21.15.26:0/2094197435 wait complete. 2024-09-15T04:04:40.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:04:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.306+0000 7feec97e3700 10 monclient: get_monmap_and_config 2024-09-15T04:04:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.306+0000 7feec97e3700 10 monclient: build_initial_monmap 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.306+0000 7feec97e3700 10 monclient: monmap: 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:40.311077+0000 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:40.311077+0000 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:40.316 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.306+0000 7feec97e3700 1 Processor -- start 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.306+0000 7feec97e3700 1 -- start start 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.306+0000 7feec97e3700 10 monclient: init 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: _add_conns ranks=[7,1,8,2,4,6,0,3,5] 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient(hunting): picked mon.noname-i con 0x7feec40a4a30 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient(hunting): picked mon.noname-f con 0x7feec4067f60 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient(hunting): picked mon.noname-e con 0x7feec41393a0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec404ca10 con 0x7feec4067f60 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4068aa0 con 0x7feec40a4a30 2024-09-15T04:04:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec40a4e20 con 0x7feec41393a0 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient(hunting): authenticate will time out at 2024.740495s 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec8fe2700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7feec41393a0 0x7feec413b780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39272/0 (socket says 172.21.15.26:39272) 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec8fe2700 1 -- 172.21.15.26:0/497942525 learned_addr learned my addr 172.21.15.26:0/497942525 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835811076 0 0) 0x7feec40a4e20 con 0x7feec41393a0 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): my global_id is 274139 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb0003570 con 0x7feec41393a0 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344864047 0 0) 0x7feec404ca10 con 0x7feec4067f60 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): my global_id is 274114 2024-09-15T04:04:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec40a4e20 con 0x7feec4067f60 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111527246 0 0) 0x7feeac001110 con 0x7feec40a4a30 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient(hunting): my global_id is 254356 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec404ca10 con 0x7feec40a4a30 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006935652 0 0) 0x7feeb0003570 con 0x7feec41393a0 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 >> v1:172.21.15.122:6792/0 conn(0x7feec40a4a30 legacy=0x7feec409aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 >> v1:172.21.15.122:6789/0 conn(0x7feec4067f60 legacy=0x7feec4137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 monclient: found mon.noname-e 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40a0610 con 0x7feec41393a0 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 5 monclient: authenticate success, global_id 274139 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7feebc001ab0 con 0x7feec41393a0 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: dump: 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feebc001cd0 con 0x7feec41393a0 2024-09-15T04:04:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 4 set_mon_vals no callback set 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 1 -- 172.21.15.26:0/497942525 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7feebc004c70 con 0x7feec41393a0 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec37fe700 10 monclient: dump: 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: shutdown 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- 172.21.15.26:0/497942525 >> v1:172.21.15.26:6793/0 conn(0x7feec41393a0 legacy=0x7feec413b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- 172.21.15.26:0/497942525 shutdown_connections 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- 172.21.15.26:0/497942525 >> 172.21.15.26:0/497942525 conn(0x7feec4063b90 msgr2=0x7feec40a40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- 172.21.15.26:0/497942525 shutdown_connections 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- 172.21.15.26:0/497942525 wait complete. 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: build_initial_monmap 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: monmap: 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:40.332 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 Processor -- start 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 1 -- start start 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.310+0000 7feec97e3700 10 monclient: init 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient: _add_conns ranks=[7,0,4,3,5,1,8,6,2] 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient(hunting): picked mon.noname-i con 0x7feec4067f60 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient(hunting): picked mon.noname-a con 0x7feec40a4a30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient(hunting): picked mon.noname-c con 0x7feec41393a0 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec41231a0 con 0x7feec40a4a30 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4139170 con 0x7feec41393a0 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec4121090 con 0x7feec4067f60 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient(hunting): authenticate will time out at 2024.744556s 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec8fe2700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7feec41393a0 0x7feec414a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40504/0 (socket says 172.21.15.26:40504) 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec8fe2700 1 -- 172.21.15.26:0/1129625240 learned_addr learned my addr 172.21.15.26:0/1129625240 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718104149 0 0) 0x7feec41231a0 con 0x7feec40a4a30 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): my global_id is 195183 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea4003570 con 0x7feec40a4a30 2024-09-15T04:04:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834541384 0 0) 0x7feec4139170 con 0x7feec41393a0 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): my global_id is 205105 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec41231a0 con 0x7feec41393a0 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408668904 0 0) 0x7feec4121090 con 0x7feec4067f60 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): my global_id is 254365 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec4139170 con 0x7feec4067f60 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 512948878 0 0) 0x7feea4003570 con 0x7feec40a4a30 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feec4121090 con 0x7feec40a4a30 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7feeb4001a60 con 0x7feec40a4a30 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3218737594 0 0) 0x7feec41231a0 con 0x7feec41393a0 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feea4003570 con 0x7feec41393a0 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7feebc001a30 con 0x7feec41393a0 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:40.337 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.338 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 617099505 0 0) 0x7feec4139170 con 0x7feec4067f60 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feec41231a0 con 0x7feec4067f60 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7feeac001bf0 con 0x7feec4067f60 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4088380634 0 0) 0x7feec4121090 con 0x7feec40a4a30 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 >> v1:172.21.15.122:6792/0 conn(0x7feec4067f60 legacy=0x7feec4138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 >> v1:172.21.15.26:6791/0 conn(0x7feec41393a0 legacy=0x7feec414a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 monclient: found mon.a 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec413f370 con 0x7feec40a4a30 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 5 monclient: authenticate success, global_id 195183 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient: _renew_subs 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec4122b50 con 0x7feec40a4a30 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient: _renew_subs 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feec4121de0 con 0x7feec40a4a30 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7feeb4001ff0 con 0x7feec40a4a30 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7feeb4002520 con 0x7feec40a4a30 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 10 monclient: dump: 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.342 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec2ffd700 4 set_mon_vals no callback set 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.314+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7feeb4002680 con 0x7feec40a4a30 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.318+0000 7feec97e3700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.318+0000 7feec97e3700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.318+0000 7feec97e3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.318+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea80050b0 con 0x7feec40a4a30 2024-09-15T04:04:40.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.318+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7feeb40413a0 con 0x7feec40a4a30 2024-09-15T04:04:40.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.318+0000 7feec3fff700 10 monclient: get_auth_request con 0x7feea40380e0 auth_method 0 2024-09-15T04:04:40.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.322+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7feeb4011f10 con 0x7feec40a4a30 2024-09-15T04:04:40.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.322+0000 7feec0ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.322+0000 7feec0ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec97e3700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:04:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec97e3700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec97e3700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feea80058e0 con 0x7feec40a4a30 2024-09-15T04:04:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec0ff9700 1 -- 172.21.15.26:0/1129625240 <== mon.0 v1:172.21.15.26:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 1687777907) 0x7feeb4011d30 con 0x7feec40a4a30 2024-09-15T04:04:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec0ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:40.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.550+0000 7feec0ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:40.559 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":128,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":0,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 >> v1:172.21.15.122:6812/20895 conn(0x7feea40380e0 legacy=0x7feea403a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 10 monclient: shutdown 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 >> v1:172.21.15.26:6789/0 conn(0x7feec40a4a30 legacy=0x7feec41470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 shutdown_connections 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 >> 172.21.15.26:0/1129625240 conn(0x7feec4063b90 msgr2=0x7feec4099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 shutdown_connections 2024-09-15T04:04:40.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.554+0000 7feec97e3700 1 -- 172.21.15.26:0/1129625240 wait complete. 2024-09-15T04:04:40.573 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T04:04:40.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-09-15T04:04:40.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T04:04:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient: get_monmap_and_config 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient: build_initial_monmap 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient: monmap: 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:40.674276+0000 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:40.674276+0000 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:40.679 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 1 Processor -- start 2024-09-15T04:04:40.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 1 -- start start 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient: init 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient: _add_conns ranks=[2,7,0,6,5,8,1,4,3] 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient(hunting): picked mon.noname-b con 0x7f07940a6880 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient(hunting): picked mon.noname-i con 0x7f079409faa0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient(hunting): picked mon.noname-a con 0x7f0794138d30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079404ca10 con 0x7f0794138d30 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940a0580 con 0x7f07940a6880 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0794133780 con 0x7f079409faa0 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079ab46700 10 monclient(hunting): authenticate will time out at 2025.103677s 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f0799b44700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f07940a6880 0x7f0794133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57140/0 (socket says 172.21.15.26:57140) 2024-09-15T04:04:40.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f0799b44700 1 -- 172.21.15.26:0/540257340 learned_addr learned my addr 172.21.15.26:0/540257340 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.670+0000 7f079a345700 1 --1- 172.21.15.26:0/540257340 >> v1:172.21.15.26:6789/0 conn(0x7f0794138d30 0x7f079413b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49210/0 (socket says 172.21.15.26:49210) 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214614235 0 0) 0x7f0790001110 con 0x7f07940a6880 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): my global_id is 254261 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0780003570 con 0x7f07940a6880 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647693762 0 0) 0x7f079404ca10 con 0x7f0794138d30 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): my global_id is 195192 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0790001110 con 0x7f0794138d30 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961959923 0 0) 0x7f0794133780 con 0x7f079409faa0 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient(hunting): my global_id is 254374 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f079404ca10 con 0x7f079409faa0 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630226119 0 0) 0x7f0780003570 con 0x7f07940a6880 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 >> v1:172.21.15.122:6792/0 conn(0x7f079409faa0 legacy=0x7f079409fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 >> v1:172.21.15.26:6789/0 conn(0x7f0794138d30 legacy=0x7f079413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: _finish_hunting 0 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 monclient: found mon.noname-b 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07940a6c50 con 0x7f07940a6880 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: _finish_auth 0 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 5 monclient: authenticate success, global_id 254261 2024-09-15T04:04:40.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f07900018c0 con 0x7f07940a6880 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: dump: 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.690 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f0790001ae0 con 0x7f07940a6880 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:40.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 4 set_mon_vals no callback set 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 1 -- 172.21.15.26:0/540257340 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0790004c30 con 0x7f07940a6880 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0798b42700 10 monclient: dump: 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.692 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: shutdown 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- 172.21.15.26:0/540257340 >> v1:172.21.15.26:6790/0 conn(0x7f07940a6880 legacy=0x7f0794133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- 172.21.15.26:0/540257340 shutdown_connections 2024-09-15T04:04:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- 172.21.15.26:0/540257340 >> 172.21.15.26:0/540257340 conn(0x7f079406e2a0 msgr2=0x7f07940a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- 172.21.15.26:0/540257340 shutdown_connections 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- 172.21.15.26:0/540257340 wait complete. 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: build_initial_monmap 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: monmap: 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:40.694 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 Processor -- start 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- start start 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: init 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient: _add_conns ranks=[7,4,6,5,8,1,2,3,0] 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient(hunting): picked mon.noname-i con 0x7f079409faa0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient(hunting): picked mon.noname-c con 0x7f07940a6880 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient(hunting): picked mon.noname-d con 0x7f0794138d30 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079409b900 con 0x7f07940a6880 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f079409b650 con 0x7f0794138d30 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07940a18c0 con 0x7f079409faa0 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f079ab46700 10 monclient(hunting): authenticate will time out at 2025.106794s 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0799343700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f07940a6880 0x7f0794099d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40516/0 (socket says 172.21.15.26:40516) 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0799343700 1 -- 172.21.15.26:0/2511899510 learned_addr learned my addr 172.21.15.26:0/2511899510 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009925198 0 0) 0x7f079409b900 con 0x7f07940a6880 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): my global_id is 205114 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0778003570 con 0x7f07940a6880 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002086414 0 0) 0x7f079409b650 con 0x7f0794138d30 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): my global_id is 195288 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f079409b900 con 0x7f0794138d30 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277541501 0 0) 0x7f07940a18c0 con 0x7f079409faa0 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): my global_id is 254383 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f079409b650 con 0x7f079409faa0 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3385785567 0 0) 0x7f0778003570 con 0x7f07940a6880 2024-09-15T04:04:40.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f07940a18c0 con 0x7f07940a6880 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0784002b70 con 0x7f07940a6880 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): dump: 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2794275755 0 0) 0x7f079409b900 con 0x7f0794138d30 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0778003570 con 0x7f0794138d30 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f078c001bf0 con 0x7f0794138d30 2024-09-15T04:04:40.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.674+0000 7f0789ffb700 10 monclient(hunting): dump: 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1532032854 0 0) 0x7f07940a18c0 con 0x7f07940a6880 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.122:6792/0 conn(0x7f079409faa0 legacy=0x7f0794099640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.26:6792/0 conn(0x7f0794138d30 legacy=0x7f079409a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 monclient: found mon.c 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079409b140 con 0x7f07940a6880 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: _finish_auth 0 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 5 monclient: authenticate success, global_id 205114 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 10 monclient: _renew_subs 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079409bd10 con 0x7f07940a6880 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 10 monclient: _renew_subs 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:40.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f079414b390 con 0x7f07940a6880 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f07840016e0 con 0x7f07940a6880 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0784001c10 con 0x7f07940a6880 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f078b7fe700 4 set_mon_vals no callback set 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: dump: 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0774006590 con 0x7f0774002840 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f0784001df0 con 0x7f07940a6880 2024-09-15T04:04:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f0784043e50 con 0x7f07940a6880 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333625611 0 0) 0x7f0774006590 con 0x7f0774002840 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient(hunting): my global_id is 195201 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07940a18c0 con 0x7f0774002840 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f079a345700 10 monclient: get_auth_request con 0x7f0778037bf0 auth_method 0 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1137693746 0 0) 0x7f07940a18c0 con 0x7f0774002840 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0774006590 con 0x7f0774002840 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0790004cd0 con 0x7f0774002840 2024-09-15T04:04:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: dump: 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.707 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1618918167 0 0) 0x7f0774006590 con 0x7f0774002840 2024-09-15T04:04:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f077400a130 con 0x7f0774002840 2024-09-15T04:04:40.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f0790005380 con 0x7f0774002840 2024-09-15T04:04:40.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.26:6789/0 conn(0x7f0774002840 legacy=0x7f0774004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.678+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.26:6789/0 conn(0x7f0774002840 legacy=0x7f0774004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f0774009620 con 0x7f0774009090 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924739877 0 0) 0x7f0774009620 con 0x7f0774009090 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient(hunting): my global_id is 195210 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0774006590 con 0x7f0774009090 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1605011289 0 0) 0x7f0774006590 con 0x7f0774009090 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0774009620 con 0x7f0774009090 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f0790005b90 con 0x7f0774009090 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient: dump: 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.710 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3433002960 0 0) 0x7f0774009620 con 0x7f0774009090 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 --> v1:172.21.15.26:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f07740098d0 con 0x7f0774009090 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:40.711 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:40.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:40.713 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.714 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:40.715 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:40.716 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:40.717 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.718 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.719 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:40.720 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.721 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:40.722 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.723 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.724 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.725 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.726 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.727 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:40.729 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.730 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 3 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.731 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": [ 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:40.732 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 <== mon.0 v1:172.21.15.26:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7739 (unknown 0 0 4087475107) 0x7f0790008370 con 0x7f0774009090 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.26:6789/0 conn(0x7f0774009090 legacy=0x7f0774008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f0789ffb700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.26:6789/0 conn(0x7f0774009090 legacy=0x7f0774008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.122:6812/20895 conn(0x7f0778037bf0 legacy=0x7f077803a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 10 monclient: shutdown 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 >> v1:172.21.15.26:6791/0 conn(0x7f07940a6880 legacy=0x7f0794099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079a345700 1 -- 172.21.15.26:0/2511899510 reap_dead start 2024-09-15T04:04:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079a345700 1 -- 172.21.15.26:0/2511899510 reap_dead start 2024-09-15T04:04:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 shutdown_connections 2024-09-15T04:04:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 >> 172.21.15.26:0/2511899510 conn(0x7f079406e2a0 msgr2=0x7f079409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 shutdown_connections 2024-09-15T04:04:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.690+0000 7f079ab46700 1 -- 172.21.15.26:0/2511899510 wait complete. 2024-09-15T04:04:40.735 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:40.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-09-15T04:04:40.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T04:04:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: get_monmap_and_config 2024-09-15T04:04:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: build_initial_monmap 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: monmap: 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:40.847959+0000 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:40.847959+0000 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:40.853 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 Processor -- start 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- start start 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: init 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: _add_conns ranks=[7,6,5,0,4,8,3,2,1] 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient(hunting): picked mon.noname-i con 0x7fa7500a6880 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient(hunting): picked mon.noname-d con 0x7fa75009faa0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient(hunting): picked mon.noname-h con 0x7fa750138d30 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:40.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75004ca10 con 0x7fa750138d30 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7500a0580 con 0x7fa75009faa0 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa750133780 con 0x7fa7500a6880 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient(hunting): authenticate will time out at 2025.277540s 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa754c78700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fa75009faa0 0x7fa75009fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58318/0 (socket says 172.21.15.26:58318) 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa754c78700 1 -- 172.21.15.26:0/997598672 learned_addr learned my addr 172.21.15.26:0/997598672 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193515634 0 0) 0x7fa740001110 con 0x7fa75009faa0 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): my global_id is 195297 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa73c003570 con 0x7fa75009faa0 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907203573 0 0) 0x7fa750133780 con 0x7fa7500a6880 2024-09-15T04:04:40.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): my global_id is 254392 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa740001110 con 0x7fa7500a6880 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530314298 0 0) 0x7fa75004ca10 con 0x7fa750138d30 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient(hunting): my global_id is 274127 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa750133780 con 0x7fa750138d30 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529209501 0 0) 0x7fa73c003570 con 0x7fa75009faa0 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 >> v1:172.21.15.122:6792/0 conn(0x7fa7500a6880 legacy=0x7fa750133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 >> v1:172.21.15.122:6791/0 conn(0x7fa750138d30 legacy=0x7fa75013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 monclient: found mon.noname-d 2024-09-15T04:04:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7500a6c50 con 0x7fa75009faa0 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: _finish_auth 0 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 5 monclient: authenticate success, global_id 195297 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa740001940 con 0x7fa75009faa0 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:40.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: dump: 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.860 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa740001c20 con 0x7fa75009faa0 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 4 set_mon_vals no callback set 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 1 -- 172.21.15.26:0/997598672 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa740004c30 con 0x7fa75009faa0 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa747fff700 10 monclient: dump: 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.861 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 10 monclient: shutdown 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- 172.21.15.26:0/997598672 >> v1:172.21.15.26:6792/0 conn(0x7fa75009faa0 legacy=0x7fa75009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- 172.21.15.26:0/997598672 shutdown_connections 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- 172.21.15.26:0/997598672 >> 172.21.15.26:0/997598672 conn(0x7fa75006e2a0 msgr2=0x7fa7500a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- 172.21.15.26:0/997598672 shutdown_connections 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.846+0000 7fa75647b700 1 -- 172.21.15.26:0/997598672 wait complete. 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: build_initial_monmap 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: monmap: 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:40.863 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 Processor -- start 2024-09-15T04:04:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 -- start start 2024-09-15T04:04:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: init 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: _add_conns ranks=[7,1,0,5,8,2,4,6,3] 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient(hunting): picked mon.noname-i con 0x7fa75009faa0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient(hunting): picked mon.noname-f con 0x7fa7500a6880 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient(hunting): picked mon.noname-a con 0x7fa750138d30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7500a18c0 con 0x7fa750138d30 2024-09-15T04:04:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75009cf10 con 0x7fa7500a6880 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75011d330 con 0x7fa75009faa0 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient(hunting): authenticate will time out at 2025.281637s 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa755c7a700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa750138d30 0x7fa750136350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49236/0 (socket says 172.21.15.26:49236) 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa755c7a700 1 -- 172.21.15.26:0/857753850 learned_addr learned my addr 172.21.15.26:0/857753850 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa754c78700 1 --1- 172.21.15.26:0/857753850 >> v1:172.21.15.122:6789/0 conn(0x7fa7500a6880 0x7fa750135c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:38628/0 (socket says 172.21.15.26:38628) 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa755479700 1 --1- 172.21.15.26:0/857753850 >> v1:172.21.15.122:6792/0 conn(0x7fa75009faa0 0x7fa750135530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6792/0 says I am v1:172.21.15.26:48296/0 (socket says 172.21.15.26:48296) 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129409044 0 0) 0x7fa7500a18c0 con 0x7fa750138d30 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): my global_id is 195219 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa734003570 con 0x7fa750138d30 2024-09-15T04:04:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918012752 0 0) 0x7fa75011d330 con 0x7fa75009faa0 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): my global_id is 254401 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7500a18c0 con 0x7fa75009faa0 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424491137 0 0) 0x7fa75009cf10 con 0x7fa7500a6880 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): my global_id is 274123 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75011d330 con 0x7fa7500a6880 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3967499087 0 0) 0x7fa734003570 con 0x7fa750138d30 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa75009cf10 con 0x7fa750138d30 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa748002d90 con 0x7fa750138d30 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): dump: 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1611813848 0 0) 0x7fa7500a18c0 con 0x7fa75009faa0 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa734003570 con 0x7fa75009faa0 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa74c001bf0 con 0x7fa75009faa0 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient(hunting): dump: 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.873 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3768826354 0 0) 0x7fa75009cf10 con 0x7fa750138d30 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.122:6792/0 conn(0x7fa75009faa0 legacy=0x7fa750135530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.122:6789/0 conn(0x7fa7500a6880 legacy=0x7fa750135c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient: _finish_hunting 0 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 monclient: found mon.a 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750137590 con 0x7fa750138d30 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient: _finish_auth 0 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 5 monclient: authenticate success, global_id 195219 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: _renew_subs 2024-09-15T04:04:40.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa750137080 con 0x7fa750138d30 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: _renew_subs 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa750137c50 con 0x7fa750138d30 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa748003000 con 0x7fa750138d30 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa748001b80 con 0x7fa750138d30 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7457fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.850+0000 7fa7477fe700 4 set_mon_vals no callback set 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: dump: 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.877 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa75647b700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa730006590 con 0x7fa730002840 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa748001ce0 con 0x7fa750138d30 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fa748043fa0 con 0x7fa750138d30 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768170603 0 0) 0x7fa730006590 con 0x7fa730002840 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient(hunting): my global_id is 254270 2024-09-15T04:04:40.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75009cf10 con 0x7fa730002840 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa754c78700 10 monclient: get_auth_request con 0x7fa734037d00 auth_method 0 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2723387257 0 0) 0x7fa75009cf10 con 0x7fa730002840 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa730006590 con 0x7fa730002840 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa74c002340 con 0x7fa730002840 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: dump: 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.879 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 6270114 0 0) 0x7fa730006590 con 0x7fa730002840 2024-09-15T04:04:40.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa73000a130 con 0x7fa730002840 2024-09-15T04:04:40.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa74c0027f0 con 0x7fa730002840 2024-09-15T04:04:40.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:40.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:40.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.26:6790/0 conn(0x7fa730002840 legacy=0x7fa730004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.854+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.26:6790/0 conn(0x7fa730002840 legacy=0x7fa730004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.862+0000 7fa75647b700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa730007520 con 0x7fa7300068d0 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899840963 0 0) 0x7fa730007520 con 0x7fa7300068d0 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient(hunting): my global_id is 254279 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa730006590 con 0x7fa7300068d0 2024-09-15T04:04:40.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1097280734 0 0) 0x7fa730006590 con 0x7fa7300068d0 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa730007520 con 0x7fa7300068d0 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa74c002d10 con 0x7fa7300068d0 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient: dump: 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:40.884 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1570019252 0 0) 0x7fa730007520 con 0x7fa7300068d0 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 --> v1:172.21.15.26:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa730008760 con 0x7fa7300068d0 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 <== mon.2 v1:172.21.15.26:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 681490219) 0x7fa74c007cf0 con 0x7fa7300068d0 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:40.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.26:6790/0 conn(0x7fa7300068d0 legacy=0x7fa730006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa7457fa700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.26:6790/0 conn(0x7fa7300068d0 legacy=0x7fa730006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:40.886 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:40.887 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:40.888 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:40.889 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:40.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.893 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:40.894 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.895 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.897 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:40.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.900 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:40.901 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:40.902 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.903 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:40.904 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:40.905 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:40.905 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:40.905 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:40.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:40.905 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:40.905 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:40.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.122:6812/20895 conn(0x7fa734037d00 legacy=0x7fa73403a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 10 monclient: shutdown 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 >> v1:172.21.15.26:6789/0 conn(0x7fa750138d30 legacy=0x7fa750136350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa755c7a700 1 -- 172.21.15.26:0/857753850 reap_dead start 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa755c7a700 1 -- 172.21.15.26:0/857753850 reap_dead start 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 shutdown_connections 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 >> 172.21.15.26:0/857753850 conn(0x7fa75006e2a0 msgr2=0x7fa750068110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 shutdown_connections 2024-09-15T04:04:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:40.866+0000 7fa75647b700 1 -- 172.21.15.26:0/857753850 wait complete. 2024-09-15T04:04:40.907 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:40.908 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-09-15T04:04:40.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T04:04:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.014+0000 7f1e12595700 10 monclient: get_monmap_and_config 2024-09-15T04:04:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.014+0000 7f1e12595700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient: monmap: 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:41.020011+0000 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:41.020011+0000 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.025 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 1 Processor -- start 2024-09-15T04:04:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 1 -- start start 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient: init 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient: _add_conns ranks=[5,8,3,7,6,0,2,1,4] 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient(hunting): picked mon.noname-h con 0x7f1e0c09f600 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient(hunting): picked mon.noname-e con 0x7f1e0c09c640 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient(hunting): picked mon.noname-g con 0x7f1e0c09b2c0 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c04ca10 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c09af10 con 0x7f1e0c09f600 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c09f9d0 con 0x7f1e0c09c640 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e12595700 10 monclient(hunting): authenticate will time out at 2025.451050s 2024-09-15T04:04:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e10d92700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f1e0c09c640 0x7f1e0c09e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39284/0 (socket says 172.21.15.26:39284) 2024-09-15T04:04:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e10d92700 1 -- 172.21.15.26:0/3668925100 learned_addr learned my addr 172.21.15.26:0/3668925100 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201736769 0 0) 0x7f1e0c09f9d0 con 0x7f1e0c09c640 2024-09-15T04:04:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e03fff700 10 monclient(hunting): my global_id is 274148 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.018+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df8004680 con 0x7f1e0c09c640 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951001310 0 0) 0x7f1e0c04ca10 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient(hunting): my global_id is 254325 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c09f9d0 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383516437 0 0) 0x7f1e0c09af10 con 0x7f1e0c09f600 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient(hunting): my global_id is 274136 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c04ca10 con 0x7f1e0c09f600 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1050591508 0 0) 0x7f1df8004680 con 0x7f1e0c09c640 2024-09-15T04:04:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 >> v1:172.21.15.122:6791/0 conn(0x7f1e0c09f600 legacy=0x7f1e0c09de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 >> v1:172.21.15.122:6790/0 conn(0x7f1e0c09b2c0 legacy=0x7f1e0c1375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 monclient: found mon.noname-e 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c09ec20 con 0x7f1e0c09c640 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1dfc0017b0 con 0x7f1e0c09c640 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: dump: 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 5 monclient: authenticate success, global_id 274148 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1dfc001a20 con 0x7f1e0c09c640 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 1 -- 172.21.15.26:0/3668925100 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1dfc004c30 con 0x7f1e0c09c640 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 4 set_mon_vals no callback set 2024-09-15T04:04:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e03fff700 10 monclient: dump: 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.038 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: shutdown 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- 172.21.15.26:0/3668925100 >> v1:172.21.15.26:6793/0 conn(0x7f1e0c09c640 legacy=0x7f1e0c09e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- 172.21.15.26:0/3668925100 shutdown_connections 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- 172.21.15.26:0/3668925100 >> 172.21.15.26:0/3668925100 conn(0x7f1e0c0a2ea0 msgr2=0x7f1e0c09eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- 172.21.15.26:0/3668925100 shutdown_connections 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- 172.21.15.26:0/3668925100 wait complete. 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: monmap: 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 Processor -- start 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- start start 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: init 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient: _add_conns ranks=[7,3,2,8,6,4,1,0,5] 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient(hunting): picked mon.noname-i con 0x7f1e0c09b2c0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient(hunting): picked mon.noname-g con 0x7f1e0c09c640 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient(hunting): picked mon.noname-b con 0x7f1e0c09f600 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c09a1f0 con 0x7f1e0c09f600 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0918b0 con 0x7f1e0c09c640 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0915d0 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e12595700 10 monclient(hunting): authenticate will time out at 2025.454694s 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e11d94700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f1e0c09f600 0x7f1e0c149e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57164/0 (socket says 172.21.15.26:57164) 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e11d94700 1 -- 172.21.15.26:0/2159739881 learned_addr learned my addr 172.21.15.26:0/2159739881 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642172410 0 0) 0x7f1e0c09a1f0 con 0x7f1e0c09f600 2024-09-15T04:04:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): my global_id is 254288 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df0003570 con 0x7f1e0c09f600 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345890832 0 0) 0x7f1e0c0915d0 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): my global_id is 254410 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c09a1f0 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759351774 0 0) 0x7f1e0c09ec20 con 0x7f1e0c09c640 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): my global_id is 254334 2024-09-15T04:04:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c0915d0 con 0x7f1e0c09c640 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1617996218 0 0) 0x7f1df0003570 con 0x7f1e0c09f600 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1e0c09ec20 con 0x7f1e0c09f600 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e04002e10 con 0x7f1e0c09f600 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): dump: 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.045 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.022+0000 7f1e017fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2472048286 0 0) 0x7f1e0c09a1f0 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1df0003570 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e080019c0 con 0x7f1e0c09b2c0 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient(hunting): dump: 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4160733816 0 0) 0x7f1e0c09ec20 con 0x7f1e0c09f600 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.122:6792/0 conn(0x7f1e0c09b2c0 legacy=0x7f1e0c141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.122:6790/0 conn(0x7f1e0c09c640 legacy=0x7f1e0c0953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 monclient: found mon.b 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c091b70 con 0x7f1e0c09f600 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 5 monclient: authenticate success, global_id 254288 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 10 monclient: _renew_subs 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c08fb00 con 0x7f1e0c09f600 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 10 monclient: _renew_subs 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c08fed0 con 0x7f1e0c09f600 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1e040018e0 con 0x7f1e0c09f600 2024-09-15T04:04:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e04001e10 con 0x7f1e0c09f600 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e037fe700 4 set_mon_vals no callback set 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: dump: 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1e04001ff0 con 0x7f1e0c09f600 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1dec006590 con 0x7f1dec002840 2024-09-15T04:04:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f1e040441c0 con 0x7f1e0c09f600 2024-09-15T04:04:41.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306187563 0 0) 0x7f1dec006590 con 0x7f1dec002840 2024-09-15T04:04:41.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient(hunting): my global_id is 205123 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c09ec20 con 0x7f1dec002840 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e10d92700 10 monclient: get_auth_request con 0x7f1df0037c30 auth_method 0 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3580623521 0 0) 0x7f1e0c09ec20 con 0x7f1dec002840 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1dec006590 con 0x7f1dec002840 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e080021d0 con 0x7f1dec002840 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:41.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 10 monclient: dump: 2024-09-15T04:04:41.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 870661395 0 0) 0x7f1dec006590 con 0x7f1dec002840 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.026+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1dec00a130 con 0x7f1dec002840 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.030+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1e080026f0 con 0x7f1dec002840 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.030+0000 7f1e017fa700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.030+0000 7f1e017fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.030+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.26:6791/0 conn(0x7f1dec002840 legacy=0x7f1dec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.030+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.26:6791/0 conn(0x7f1dec002840 legacy=0x7f1dec004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e12595700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1dec007520 con 0x7f1dec0068d0 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082755897 0 0) 0x7f1dec007520 con 0x7f1dec0068d0 2024-09-15T04:04:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 10 monclient(hunting): my global_id is 205132 2024-09-15T04:04:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dec006590 con 0x7f1dec0068d0 2024-09-15T04:04:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2736521071 0 0) 0x7f1dec006590 con 0x7f1dec0068d0 2024-09-15T04:04:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.038+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1dec007520 con 0x7f1dec0068d0 2024-09-15T04:04:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1e08003000 con 0x7f1dec0068d0 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 10 monclient: dump: 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 135012132 0 0) 0x7f1dec007520 con 0x7f1dec0068d0 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 --> v1:172.21.15.26:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1dec008760 con 0x7f1dec0068d0 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 <== mon.4 v1:172.21.15.26:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 2346717200) 0x7f1e08005220 con 0x7f1dec0068d0 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.26:6791/0 conn(0x7f1dec0068d0 legacy=0x7f1dec006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e017fa700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.26:6791/0 conn(0x7f1dec0068d0 legacy=0x7f1dec006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:41.061 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.062 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.063 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.065 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.066 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.067 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.068 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.068 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.068 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.068 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.068 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.069 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.072 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.073 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.076 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.077 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.079 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.079 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:41.079 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:41.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.081 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:41.082 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.122:6812/20895 conn(0x7f1df0037c30 legacy=0x7f1df003a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 10 monclient: shutdown 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 >> v1:172.21.15.26:6790/0 conn(0x7f1e0c09f600 legacy=0x7f1e0c149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e11d94700 1 -- 172.21.15.26:0/2159739881 reap_dead start 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e11d94700 1 -- 172.21.15.26:0/2159739881 reap_dead start 2024-09-15T04:04:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 shutdown_connections 2024-09-15T04:04:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 >> 172.21.15.26:0/2159739881 conn(0x7f1e0c0a2ea0 msgr2=0x7f1e0c0982e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 shutdown_connections 2024-09-15T04:04:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.042+0000 7f1e12595700 1 -- 172.21.15.26:0/2159739881 wait complete. 2024-09-15T04:04:41.085 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:41.085 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-09-15T04:04:41.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: get_monmap_and_config 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: monmap: 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:41.200609+0000 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:41.200609+0000 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 1 Processor -- start 2024-09-15T04:04:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 1 -- start start 2024-09-15T04:04:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: init 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: _add_conns ranks=[2,3,0,4,1,7,6,8,5] 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient(hunting): picked mon.noname-b con 0x7fe0f00a6880 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient(hunting): picked mon.noname-g con 0x7fe0f009faa0 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient(hunting): picked mon.noname-a con 0x7fe0f0138d30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f004ca10 con 0x7fe0f0138d30 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f00a0580 con 0x7fe0f00a6880 2024-09-15T04:04:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f0133780 con 0x7fe0f009faa0 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient(hunting): authenticate will time out at 2025.630185s 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f6aa9700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe0f0138d30 0x7fe0f013b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49250/0 (socket says 172.21.15.26:49250) 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f6aa9700 1 -- 172.21.15.26:0/4149608254 learned_addr learned my addr 172.21.15.26:0/4149608254 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028338924 0 0) 0x7fe0ec001040 con 0x7fe0f00a6880 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): my global_id is 254297 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0dc003570 con 0x7fe0f00a6880 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329286538 0 0) 0x7fe0f004ca10 con 0x7fe0f0138d30 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): my global_id is 195228 2024-09-15T04:04:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ec001040 con 0x7fe0f0138d30 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669399122 0 0) 0x7fe0f0133780 con 0x7fe0f009faa0 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient(hunting): my global_id is 254343 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f004ca10 con 0x7fe0f009faa0 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428539511 0 0) 0x7fe0dc003570 con 0x7fe0f00a6880 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 >> v1:172.21.15.122:6790/0 conn(0x7fe0f009faa0 legacy=0x7fe0f009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 >> v1:172.21.15.26:6789/0 conn(0x7fe0f0138d30 legacy=0x7fe0f013b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 monclient: found mon.noname-b 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f00a6c50 con 0x7fe0f00a6880 2024-09-15T04:04:41.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0ec001870 con 0x7fe0f00a6880 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 5 monclient: authenticate success, global_id 254297 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: dump: 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.217 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0ec001ad0 con 0x7fe0f00a6880 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 1 -- 172.21.15.26:0/4149608254 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0ec002240 con 0x7fe0f00a6880 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f52a6700 10 monclient: dump: 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.219 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.198+0000 7fe0f72aa700 4 set_mon_vals no callback set 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient: shutdown 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- 172.21.15.26:0/4149608254 >> v1:172.21.15.26:6790/0 conn(0x7fe0f00a6880 legacy=0x7fe0f0133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- 172.21.15.26:0/4149608254 shutdown_connections 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- 172.21.15.26:0/4149608254 >> 172.21.15.26:0/4149608254 conn(0x7fe0f006e2a0 msgr2=0x7fe0f00a60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- 172.21.15.26:0/4149608254 shutdown_connections 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- 172.21.15.26:0/4149608254 wait complete. 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient: monmap: 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.221 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 Processor -- start 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- start start 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient: init 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient: _add_conns ranks=[8,1,6,3,7,2,4,0,5] 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient(hunting): picked mon.noname-e con 0x7fe0f009faa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient(hunting): picked mon.noname-f con 0x7fe0f00a6880 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient(hunting): picked mon.noname-d con 0x7fe0f0138d30 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f0137af0 con 0x7fe0f00a6880 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f009cf10 con 0x7fe0f0138d30 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f0131c20 con 0x7fe0f009faa0 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f72aa700 10 monclient(hunting): authenticate will time out at 2025.635241s 2024-09-15T04:04:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f6aa9700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fe0f0138d30 0x7fe0f01363c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58320/0 (socket says 172.21.15.26:58320) 2024-09-15T04:04:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f6aa9700 1 -- 172.21.15.26:0/3077576317 learned_addr learned my addr 172.21.15.26:0/3077576317 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0f62a8700 1 --1- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6793/0 conn(0x7fe0f009faa0 0x7fe0f01355a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39290/0 (socket says 172.21.15.26:39290) 2024-09-15T04:04:41.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982980695 0 0) 0x7fe0f0131c20 con 0x7fe0f009faa0 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.202+0000 7fe0e67fc700 10 monclient(hunting): my global_id is 274157 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d4003570 con 0x7fe0f009faa0 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708953824 0 0) 0x7fe0f009cf10 con 0x7fe0f0138d30 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): my global_id is 195306 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0131c20 con 0x7fe0f0138d30 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622986326 0 0) 0x7fe0f0137af0 con 0x7fe0f00a6880 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): my global_id is 274132 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f009cf10 con 0x7fe0f00a6880 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3718112856 0 0) 0x7fe0f0131c20 con 0x7fe0f0138d30 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0f0137af0 con 0x7fe0f0138d30 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2377582020 0 0) 0x7fe0d4003570 con 0x7fe0f009faa0 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0f0131c20 con 0x7fe0f009faa0 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0e8001d60 con 0x7fe0f0138d30 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): dump: 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.226 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0ec001f80 con 0x7fe0f009faa0 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): dump: 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.228 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.229 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3339869180 0 0) 0x7fe0f009cf10 con 0x7fe0f00a6880 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0d4003570 con 0x7fe0f00a6880 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0e00019e0 con 0x7fe0f00a6880 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): dump: 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.230 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 37868635 0 0) 0x7fe0f0137af0 con 0x7fe0f0138d30 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6793/0 conn(0x7fe0f009faa0 legacy=0x7fe0f01355a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.122:6789/0 conn(0x7fe0f00a6880 legacy=0x7fe0f0135cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 monclient: found mon.d 2024-09-15T04:04:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f00a18c0 con 0x7fe0f0138d30 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 5 monclient: authenticate success, global_id 195306 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 10 monclient: _renew_subs 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f0137600 con 0x7fe0f0138d30 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 10 monclient: _renew_subs 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f0137cc0 con 0x7fe0f0138d30 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe0e8001f90 con 0x7fe0f0138d30 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0e80051a0 con 0x7fe0f0138d30 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f4aa5700 4 set_mon_vals no callback set 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: dump: 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0d8006590 con 0x7fe0d8002840 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490134380 0 0) 0x7fe0d8006590 con 0x7fe0d8002840 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient(hunting): my global_id is 195315 2024-09-15T04:04:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0137af0 con 0x7fe0d8002840 2024-09-15T04:04:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe0e800fbe0 con 0x7fe0f0138d30 2024-09-15T04:04:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fe0e8044130 con 0x7fe0f0138d30 2024-09-15T04:04:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3035417188 0 0) 0x7fe0f0137af0 con 0x7fe0d8002840 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0d8006590 con 0x7fe0d8002840 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0ec0025a0 con 0x7fe0d8002840 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0e67fc700 10 monclient: dump: 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.206+0000 7fe0f5aa7700 10 monclient: get_auth_request con 0x7fe0d40380b0 auth_method 0 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3971228156 0 0) 0x7fe0d8006590 con 0x7fe0d8002840 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0d800a130 con 0x7fe0d8002840 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fe0ec0029c0 con 0x7fe0d8002840 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6792/0 conn(0x7fe0d8002840 legacy=0x7fe0d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.210+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6792/0 conn(0x7fe0d8002840 legacy=0x7fe0d8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.218+0000 7fe0f72aa700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.218+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fe0d8009640 con 0x7fe0d80090b0 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939917019 0 0) 0x7fe0d8009640 con 0x7fe0d80090b0 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient(hunting): my global_id is 195324 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d8006590 con 0x7fe0d80090b0 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4114179239 0 0) 0x7fe0d8006590 con 0x7fe0d80090b0 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0d8009640 con 0x7fe0d80090b0 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe0ec003180 con 0x7fe0d80090b0 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:41.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient: dump: 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.241 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2545975437 0 0) 0x7fe0d8009640 con 0x7fe0d80090b0 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 --> v1:172.21.15.26:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe0d80098f0 con 0x7fe0d80090b0 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 <== mon.6 v1:172.21.15.26:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7571 (unknown 0 0 2071028278) 0x7fe0ec008da0 con 0x7fe0d80090b0 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6792/0 conn(0x7fe0d80090b0 legacy=0x7fe0d8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0e67fc700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6792/0 conn(0x7fe0d80090b0 legacy=0x7fe0d8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.242 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "leader", 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:41.243 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.244 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:41.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.247 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.249 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.251 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:41.252 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.253 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.255 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:41.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:41.258 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.258 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.258 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.259 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.260 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4 2024-09-15T04:04:41.262 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.263 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:41.263 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.122:6812/20895 conn(0x7fe0d40380b0 legacy=0x7fe0d403a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 10 monclient: shutdown 2024-09-15T04:04:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 >> v1:172.21.15.26:6792/0 conn(0x7fe0f0138d30 legacy=0x7fe0f01363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f6aa9700 1 -- 172.21.15.26:0/3077576317 reap_dead start 2024-09-15T04:04:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f6aa9700 1 -- 172.21.15.26:0/3077576317 reap_dead start 2024-09-15T04:04:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 shutdown_connections 2024-09-15T04:04:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 >> 172.21.15.26:0/3077576317 conn(0x7fe0f006e2a0 msgr2=0x7fe0f00a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 shutdown_connections 2024-09-15T04:04:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.222+0000 7fe0f72aa700 1 -- 172.21.15.26:0/3077576317 wait complete. 2024-09-15T04:04:41.265 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:41.266 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-09-15T04:04:41.266 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-09-15T04:04:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: get_monmap_and_config 2024-09-15T04:04:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: monmap: 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:41.379997+0000 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:41.379997+0000 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.385 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 1 Processor -- start 2024-09-15T04:04:41.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 1 -- start start 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: init 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: _add_conns ranks=[2,0,8,6,7,4,3,5,1] 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient(hunting): picked mon.noname-b con 0x7efcf80990c0 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient(hunting): picked mon.noname-a con 0x7efcf80a1310 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient(hunting): picked mon.noname-e con 0x7efcf81335a0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf804ca10 con 0x7efcf80a1310 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80a1b20 con 0x7efcf80990c0 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80a10b0 con 0x7efcf81335a0 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient(hunting): authenticate will time out at 2025.810258s 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf659c700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7efcf80990c0 0x7efcf8132e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57188/0 (socket says 172.21.15.26:57188) 2024-09-15T04:04:41.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf659c700 1 -- 172.21.15.26:0/1651062350 learned_addr learned my addr 172.21.15.26:0/1651062350 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638464461 0 0) 0x7efcf80a1b20 con 0x7efcf80990c0 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): my global_id is 254306 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce4003570 con 0x7efcf80990c0 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506080587 0 0) 0x7efcf804ca10 con 0x7efcf80a1310 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): my global_id is 195237 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf80a1b20 con 0x7efcf80a1310 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895988942 0 0) 0x7efcf80a10b0 con 0x7efcf81335a0 2024-09-15T04:04:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient(hunting): my global_id is 274166 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf804ca10 con 0x7efcf81335a0 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3575848863 0 0) 0x7efcf80a1b20 con 0x7efcf80a1310 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 >> v1:172.21.15.26:6793/0 conn(0x7efcf81335a0 legacy=0x7efcf813afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 >> v1:172.21.15.26:6790/0 conn(0x7efcf80990c0 legacy=0x7efcf8132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 monclient: found mon.noname-a 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8099ba0 con 0x7efcf80a1310 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 5 monclient: authenticate success, global_id 195237 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efce0001790 con 0x7efcf80a1310 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: dump: 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efce0001ab0 con 0x7efcf80a1310 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcfcd5c700 4 set_mon_vals no callback set 2024-09-15T04:04:41.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.378+0000 7efcf559a700 1 -- 172.21.15.26:0/1651062350 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efce0001fe0 con 0x7efcf80a1310 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcf559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcf559a700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcf559a700 10 monclient: dump: 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 10 monclient: shutdown 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 -- 172.21.15.26:0/1651062350 >> v1:172.21.15.26:6789/0 conn(0x7efcf80a1310 legacy=0x7efcf8099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 -- 172.21.15.26:0/1651062350 shutdown_connections 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 -- 172.21.15.26:0/1651062350 >> 172.21.15.26:0/1651062350 conn(0x7efcf806e370 msgr2=0x7efcf80a6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 -- 172.21.15.26:0/1651062350 shutdown_connections 2024-09-15T04:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 -- 172.21.15.26:0/1651062350 wait complete. 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 10 monclient: monmap: 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 Processor -- start 2024-09-15T04:04:41.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 1 -- start start 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 10 monclient: init 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.382+0000 7efcfcd5c700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient: _add_conns ranks=[7,5,8,6,1,4,2,0,3] 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient(hunting): picked mon.noname-i con 0x7efcf80990c0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient(hunting): picked mon.noname-h con 0x7efcf80a1310 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient(hunting): picked mon.noname-e con 0x7efcf81335a0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf811fe90 con 0x7efcf80a1310 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf809a010 con 0x7efcf80990c0 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80a1ce0 con 0x7efcf81335a0 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient(hunting): authenticate will time out at 2025.816160s 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcf6d9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7efcf81335a0 0x7efcf814a050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39300/0 (socket says 172.21.15.26:39300) 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcf6d9d700 1 -- 172.21.15.26:0/815583152 learned_addr learned my addr 172.21.15.26:0/815583152 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786644814 0 0) 0x7efcf80a1ce0 con 0x7efcf81335a0 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): my global_id is 274175 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd8003570 con 0x7efcf81335a0 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 533819620 0 0) 0x7efcd8003570 con 0x7efcf81335a0 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efcf80a1ce0 con 0x7efcf81335a0 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efcf0002d10 con 0x7efcf81335a0 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): dump: 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849877726 0 0) 0x7efcf809a010 con 0x7efcf80990c0 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): my global_id is 254419 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd8003570 con 0x7efcf80990c0 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537276886 0 0) 0x7efcf811fe90 con 0x7efcf80a1310 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): my global_id is 274145 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf809a010 con 0x7efcf80a1310 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2062278778 0 0) 0x7efcf80a1ce0 con 0x7efcf81335a0 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.122:6792/0 conn(0x7efcf80990c0 legacy=0x7efcf8141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.122:6791/0 conn(0x7efcf80a1310 legacy=0x7efcf8142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 monclient: found mon.e 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8142c10 con 0x7efcf81335a0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 5 monclient: authenticate success, global_id 274175 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient: _renew_subs 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf8122d10 con 0x7efcf81335a0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient: _renew_subs 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf8120a60 con 0x7efcf81335a0 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7efcf0003010 con 0x7efcf81335a0 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efcf0001b80 con 0x7efcf81335a0 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efceffff700 4 set_mon_vals no callback set 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient: dump: 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7efcf0001ce0 con 0x7efcf81335a0 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efcdc006590 con 0x7efcdc002840 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7efcf0043f50 con 0x7efcf81335a0 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcf659c700 10 monclient: get_auth_request con 0x7efcd8037860 auth_method 0 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65563322 0 0) 0x7efcdc006590 con 0x7efcdc002840 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 10 monclient(hunting): my global_id is 274184 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.386+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf80a1ce0 con 0x7efcdc002840 2024-09-15T04:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2879996969 0 0) 0x7efcf80a1ce0 con 0x7efcdc002840 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efcdc006590 con 0x7efcdc002840 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efce0001750 con 0x7efcdc002840 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 10 monclient: dump: 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2044510339 0 0) 0x7efcdc006590 con 0x7efcdc002840 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efcdc00a130 con 0x7efcdc002840 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7efce00022a0 con 0x7efcdc002840 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.26:6793/0 conn(0x7efcdc002840 legacy=0x7efcdc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.390+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.26:6793/0 conn(0x7efcdc002840 legacy=0x7efcdc004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcfcd5c700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7efcdc009640 con 0x7efcdc0090b0 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828380172 0 0) 0x7efcdc009640 con 0x7efcdc0090b0 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcee7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcee7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcee7fc700 10 monclient(hunting): my global_id is 274193 2024-09-15T04:04:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.394+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcdc006590 con 0x7efcdc0090b0 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 235347824 0 0) 0x7efcdc006590 con 0x7efcdc0090b0 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efcdc009640 con 0x7efcdc0090b0 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7efce0002b70 con 0x7efcdc0090b0 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 10 monclient: dump: 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1628751400 0 0) 0x7efcdc009640 con 0x7efcdc0090b0 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 --> v1:172.21.15.26:6793/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efcdc0098f0 con 0x7efcdc0090b0 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 <== mon.8 v1:172.21.15.26:6793/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 275238169) 0x7efce0005290 con 0x7efcdc0090b0 2024-09-15T04:04:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.26:6793/0 conn(0x7efcdc0090b0 legacy=0x7efcdc008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcee7fc700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.26:6793/0 conn(0x7efcdc0090b0 legacy=0x7efcdc008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:41.416 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.417 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:41.418 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:41.419 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.420 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.421 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.422 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:41.423 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.424 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.426 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.428 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.429 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.430 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:41.431 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.432 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.433 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.434 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:04:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.435 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:41.435 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.122:6812/20895 conn(0x7efcd8037860 legacy=0x7efcd8039d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 10 monclient: shutdown 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 >> v1:172.21.15.26:6793/0 conn(0x7efcf81335a0 legacy=0x7efcf814a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcf6d9d700 1 -- 172.21.15.26:0/815583152 reap_dead start 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcf6d9d700 1 -- 172.21.15.26:0/815583152 reap_dead start 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 shutdown_connections 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 >> 172.21.15.26:0/815583152 conn(0x7efcf806e370 msgr2=0x7efcf80a4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 shutdown_connections 2024-09-15T04:04:41.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.398+0000 7efcfcd5c700 1 -- 172.21.15.26:0/815583152 wait complete. 2024-09-15T04:04:41.437 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:41.438 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-09-15T04:04:41.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-09-15T04:04:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.546+0000 7fa08eb09700 10 monclient: get_monmap_and_config 2024-09-15T04:04:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.546+0000 7fa08eb09700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.546+0000 7fa08eb09700 10 monclient: monmap: 2024-09-15T04:04:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:41.549601+0000 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:41.549601+0000 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.554 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.555 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.546+0000 7fa08eb09700 1 Processor -- start 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.546+0000 7fa08eb09700 1 -- start start 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.546+0000 7fa08eb09700 10 monclient: init 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient: _add_conns ranks=[6,3,2,5,8,7,1,4,0] 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient(hunting): picked mon.noname-d con 0x7fa0880a0090 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient(hunting): picked mon.noname-g con 0x7fa088099540 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient(hunting): picked mon.noname-b con 0x7fa088135c40 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08804ca10 con 0x7fa088135c40 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08806e600 con 0x7fa088099540 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0880a0460 con 0x7fa0880a0090 2024-09-15T04:04:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient(hunting): authenticate will time out at 2025.980360s 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08e308700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7fa088135c40 0x7fa088138020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57202/0 (socket says 172.21.15.26:57202) 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08e308700 1 -- 172.21.15.26:0/215698954 learned_addr learned my addr 172.21.15.26:0/215698954 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08db07700 1 --1- 172.21.15.26:0/215698954 >> v1:172.21.15.26:6792/0 conn(0x7fa0880a0090 0x7fa08809d8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58348/0 (socket says 172.21.15.26:58348) 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288157613 0 0) 0x7fa0880a0460 con 0x7fa0880a0090 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): my global_id is 195333 2024-09-15T04:04:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa074003570 con 0x7fa0880a0090 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805716639 0 0) 0x7fa08804ca10 con 0x7fa088135c40 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): my global_id is 254315 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0880a0460 con 0x7fa088135c40 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934723656 0 0) 0x7fa08806e600 con 0x7fa088099540 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient(hunting): my global_id is 254352 2024-09-15T04:04:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa08804ca10 con 0x7fa088099540 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861852902 0 0) 0x7fa074003570 con 0x7fa0880a0090 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 >> v1:172.21.15.122:6790/0 conn(0x7fa088099540 legacy=0x7fa08809dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 >> v1:172.21.15.26:6790/0 conn(0x7fa088135c40 legacy=0x7fa088138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 monclient: found mon.noname-d 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08809e6d0 con 0x7fa0880a0090 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa0840017b0 con 0x7fa0880a0090 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: dump: 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 5 monclient: authenticate success, global_id 195333 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa084001a90 con 0x7fa0880a0090 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 1 -- 172.21.15.26:0/215698954 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa084004b70 con 0x7fa0880a0090 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08cb05700 10 monclient: dump: 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.565 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 4 set_mon_vals no callback set 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient: shutdown 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- 172.21.15.26:0/215698954 >> v1:172.21.15.26:6792/0 conn(0x7fa0880a0090 legacy=0x7fa08809d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- 172.21.15.26:0/215698954 shutdown_connections 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- 172.21.15.26:0/215698954 >> 172.21.15.26:0/215698954 conn(0x7fa0880a3430 msgr2=0x7fa0880a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- 172.21.15.26:0/215698954 shutdown_connections 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 1 -- 172.21.15.26:0/215698954 wait complete. 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.550+0000 7fa08eb09700 10 monclient: monmap: 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.568 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 Processor -- start 2024-09-15T04:04:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 -- start start 2024-09-15T04:04:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: init 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: _add_conns ranks=[1,7,6,4,2,0,3,8,5] 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient(hunting): picked mon.noname-f con 0x7fa088099540 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient(hunting): picked mon.noname-i con 0x7fa0880a0090 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient(hunting): picked mon.noname-d con 0x7fa088135c40 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08812d340 con 0x7fa088099540 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa088119520 con 0x7fa088135c40 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08806ddc0 con 0x7fa0880a0090 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient(hunting): authenticate will time out at 2025.985519s 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08e308700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fa088135c40 0x7fa08806c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58362/0 (socket says 172.21.15.26:58362) 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08db07700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fa088099540 0x7fa08806b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:38648/0 (socket says 172.21.15.26:38648) 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08e308700 1 -- 172.21.15.26:0/3800487913 learned_addr learned my addr 172.21.15.26:0/3800487913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094585413 0 0) 0x7fa088119520 con 0x7fa088135c40 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): my global_id is 195342 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa06c003570 con 0x7fa088135c40 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712596451 0 0) 0x7fa08812d340 con 0x7fa088099540 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): my global_id is 274141 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa088119520 con 0x7fa088099540 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134943680 0 0) 0x7fa08804ca10 con 0x7fa0880a0090 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): my global_id is 254428 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa08812d340 con 0x7fa0880a0090 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1392140192 0 0) 0x7fa06c003570 con 0x7fa088135c40 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa08804ca10 con 0x7fa088135c40 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa080001d40 con 0x7fa088135c40 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): dump: 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1294359776 0 0) 0x7fa08804ca10 con 0x7fa088135c40 2024-09-15T04:04:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6792/0 conn(0x7fa0880a0090 legacy=0x7fa08806bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6789/0 conn(0x7fa088099540 legacy=0x7fa08806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 monclient: found mon.d 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08811a560 con 0x7fa088135c40 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 5 monclient: authenticate success, global_id 195342 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: _renew_subs 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08811a280 con 0x7fa088135c40 2024-09-15T04:04:41.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: _renew_subs 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08814b130 con 0x7fa088135c40 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fa080002010 con 0x7fa088135c40 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa080005260 con 0x7fa088135c40 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07ffff700 4 set_mon_vals no callback set 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.554+0000 7fa07dffb700 10 monclient: dump: 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa08eb09700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa068006590 con 0x7fa068002840 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521568958 0 0) 0x7fa068006590 con 0x7fa068002840 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 10 monclient(hunting): my global_id is 274150 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa08804ca10 con 0x7fa068002840 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3975423776 0 0) 0x7fa08804ca10 con 0x7fa068002840 2024-09-15T04:04:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa068006590 con 0x7fa068002840 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa084001c00 con 0x7fa068002840 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 10 monclient: dump: 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fa0800053c0 con 0x7fa088135c40 2024-09-15T04:04:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fa0800440d0 con 0x7fa088135c40 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2869542920 0 0) 0x7fa068006590 con 0x7fa068002840 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa08d306700 10 monclient: get_auth_request con 0x7fa06c037be0 auth_method 0 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.558+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa06800a130 con 0x7fa068002840 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.562+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fa0840052c0 con 0x7fa068002840 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.562+0000 7fa07dffb700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.562+0000 7fa07dffb700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.562+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6789/0 conn(0x7fa068002840 legacy=0x7fa068004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.562+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6789/0 conn(0x7fa068002840 legacy=0x7fa068004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa08eb09700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fa068007510 con 0x7fa0680068c0 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153060748 0 0) 0x7fa068007510 con 0x7fa0680068c0 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 10 monclient(hunting): my global_id is 274159 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa068006590 con 0x7fa0680068c0 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2486399344 0 0) 0x7fa068006590 con 0x7fa0680068c0 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa068007510 con 0x7fa0680068c0 2024-09-15T04:04:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fa084005b10 con 0x7fa0680068c0 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 10 monclient: dump: 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1495057760 0 0) 0x7fa068007510 con 0x7fa0680068c0 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.570+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 --> v1:172.21.15.122:6789/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa068008760 con 0x7fa0680068c0 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 <== mon.1 v1:172.21.15.122:6789/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 2834513402) 0x7fa0840082b0 con 0x7fa0680068c0 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa07dffb700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa07dffb700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6789/0 conn(0x7fa0680068c0 legacy=0x7fa068006d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa07dffb700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6789/0 conn(0x7fa0680068c0 legacy=0x7fa068006d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.588 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:41.589 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1, 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.590 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:41.592 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.593 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.594 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.595 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:41.596 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.597 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.599 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.602 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.603 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.605 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.606 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.607 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.608 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:41.608 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.122:6812/20895 conn(0x7fa06c037be0 legacy=0x7fa06c03a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 10 monclient: shutdown 2024-09-15T04:04:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 >> v1:172.21.15.26:6792/0 conn(0x7fa088135c40 legacy=0x7fa08806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08e308700 1 -- 172.21.15.26:0/3800487913 reap_dead start 2024-09-15T04:04:41.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08e308700 1 -- 172.21.15.26:0/3800487913 reap_dead start 2024-09-15T04:04:41.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 shutdown_connections 2024-09-15T04:04:41.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 >> 172.21.15.26:0/3800487913 conn(0x7fa0880a3430 msgr2=0x7fa088135490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 shutdown_connections 2024-09-15T04:04:41.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.574+0000 7fa08eb09700 1 -- 172.21.15.26:0/3800487913 wait complete. 2024-09-15T04:04:41.610 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:41.611 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-09-15T04:04:41.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-09-15T04:04:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.722+0000 7fddd1ae1700 10 monclient: get_monmap_and_config 2024-09-15T04:04:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.722+0000 7fddd1ae1700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.722+0000 7fddd1ae1700 10 monclient: monmap: 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:41.727028+0000 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:41.727028+0000 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.732 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.722+0000 7fddd1ae1700 1 Processor -- start 2024-09-15T04:04:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.722+0000 7fddd1ae1700 1 -- start start 2024-09-15T04:04:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: init 2024-09-15T04:04:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: _add_conns ranks=[6,7,3,8,4,1,5,0,2] 2024-09-15T04:04:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient(hunting): picked mon.noname-d con 0x7fddcc1335e0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient(hunting): picked mon.noname-i con 0x7fddcc09fc80 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient(hunting): picked mon.noname-g con 0x7fddcc0a0050 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc04ca10 con 0x7fddcc0a0050 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc06eab0 con 0x7fddcc1335e0 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc0a1330 con 0x7fddcc09fc80 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient(hunting): authenticate will time out at 2026.157050s 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd0adf700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fddcc1335e0 0x7fddcc132da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58378/0 (socket says 172.21.15.26:58378) 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd0adf700 1 -- 172.21.15.26:0/2229459103 learned_addr learned my addr 172.21.15.26:0/2229459103 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933608130 0 0) 0x7fddcc06eab0 con 0x7fddcc1335e0 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): my global_id is 195351 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb8003570 con 0x7fddcc1335e0 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284615199 0 0) 0x7fddcc0a1330 con 0x7fddcc09fc80 2024-09-15T04:04:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): my global_id is 254437 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc06eab0 con 0x7fddcc09fc80 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063964520 0 0) 0x7fddcc04ca10 con 0x7fddcc0a0050 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient(hunting): my global_id is 254361 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc0a1330 con 0x7fddcc0a0050 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806935978 0 0) 0x7fddb8003570 con 0x7fddcc1335e0 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 >> v1:172.21.15.122:6792/0 conn(0x7fddcc09fc80 legacy=0x7fddcc137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 >> v1:172.21.15.122:6790/0 conn(0x7fddcc0a0050 legacy=0x7fddcc13b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 monclient: found mon.noname-d 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc0a10d0 con 0x7fddcc1335e0 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddb40017b0 con 0x7fddcc1335e0 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 5 monclient: authenticate success, global_id 195351 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: dump: 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fddb4001a20 con 0x7fddcc1335e0 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 1 -- 172.21.15.26:0/2229459103 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddb4004c30 con 0x7fddcc1335e0 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 4 set_mon_vals no callback set 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddcb7fe700 10 monclient: dump: 2024-09-15T04:04:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: shutdown 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- 172.21.15.26:0/2229459103 >> v1:172.21.15.26:6792/0 conn(0x7fddcc1335e0 legacy=0x7fddcc132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- 172.21.15.26:0/2229459103 shutdown_connections 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- 172.21.15.26:0/2229459103 >> 172.21.15.26:0/2229459103 conn(0x7fddcc06e6e0 msgr2=0x7fddcc0a6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- 172.21.15.26:0/2229459103 shutdown_connections 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 1 -- 172.21.15.26:0/2229459103 wait complete. 2024-09-15T04:04:41.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.726+0000 7fddd1ae1700 10 monclient: monmap: 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 Processor -- start 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 -- start start 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: init 2024-09-15T04:04:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: _add_conns ranks=[8,3,2,1,7,5,0,6,4] 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient(hunting): picked mon.noname-e con 0x7fddcc09fc80 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient(hunting): picked mon.noname-g con 0x7fddcc0a0050 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient(hunting): picked mon.noname-b con 0x7fddcc1335e0 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc11ef00 con 0x7fddcc1335e0 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc09cf10 con 0x7fddcc0a0050 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc0a1b70 con 0x7fddcc09fc80 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient(hunting): authenticate will time out at 2026.161295s 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd0adf700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fddcc09fc80 0x7fddcc141c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39326/0 (socket says 172.21.15.26:39326) 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd0adf700 1 -- 172.21.15.26:0/3384753787 learned_addr learned my addr 172.21.15.26:0/3384753787 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340765515 0 0) 0x7fddcc0a1b70 con 0x7fddcc09fc80 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): my global_id is 274202 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddac003570 con 0x7fddcc09fc80 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933435486 0 0) 0x7fddcc11ef00 con 0x7fddcc1335e0 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): my global_id is 254324 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc0a1b70 con 0x7fddcc1335e0 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582732997 0 0) 0x7fddcc09cf10 con 0x7fddcc0a0050 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): my global_id is 254370 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc11ef00 con 0x7fddcc0a0050 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 903603654 0 0) 0x7fddac003570 con 0x7fddcc09fc80 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddcc09cf10 con 0x7fddcc09fc80 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 365500772 0 0) 0x7fddcc0a1b70 con 0x7fddcc1335e0 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddac003570 con 0x7fddcc1335e0 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddb4001830 con 0x7fddcc09fc80 2024-09-15T04:04:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2370656620 0 0) 0x7fddcc11ef00 con 0x7fddcc0a0050 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddcc0a1b70 con 0x7fddcc0a0050 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddc4001d60 con 0x7fddcc1335e0 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.757 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddbc0019e0 con 0x7fddcc0a0050 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.758 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1697700278 0 0) 0x7fddcc09cf10 con 0x7fddcc09fc80 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.122:6790/0 conn(0x7fddcc0a0050 legacy=0x7fddcc12fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.26:6790/0 conn(0x7fddcc1335e0 legacy=0x7fddcc149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 monclient: found mon.e 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc11f0d0 con 0x7fddcc09fc80 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 5 monclient: authenticate success, global_id 274202 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: _renew_subs 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc11ffc0 con 0x7fddcc09fc80 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: _renew_subs 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc11e500 con 0x7fddcc09fc80 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.730+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fddb4004880 con 0x7fddcc09fc80 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddb40051a0 con 0x7fddcc09fc80 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddd1ae1700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: dump: 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fddcc142380 con 0x7fddcc02ae80 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddcaffd700 4 set_mon_vals no callback set 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fddb400fb60 con 0x7fddcc09fc80 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fddb4043f10 con 0x7fddcc09fc80 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536662521 0 0) 0x7fddcc142380 con 0x7fddcc02ae80 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient(hunting): my global_id is 254379 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc09cf10 con 0x7fddcc02ae80 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddd12e0700 10 monclient: get_auth_request con 0x7fddac0380a0 auth_method 0 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1770642312 0 0) 0x7fddcc09cf10 con 0x7fddcc02ae80 2024-09-15T04:04:41.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddcc142380 con 0x7fddcc02ae80 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddbc002070 con 0x7fddcc02ae80 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: dump: 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.764 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 4246332053 0 0) 0x7fddcc142380 con 0x7fddcc02ae80 2024-09-15T04:04:41.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddcc04bbf0 con 0x7fddcc02ae80 2024-09-15T04:04:41.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7fddbc004460 con 0x7fddcc02ae80 2024-09-15T04:04:41.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.122:6790/0 conn(0x7fddcc02ae80 legacy=0x7fddcc05e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.734+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.122:6790/0 conn(0x7fddcc02ae80 legacy=0x7fddcc05e890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddd1ae1700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7fddcc066de0 con 0x7fddcc098a70 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242354416 0 0) 0x7fddcc066de0 con 0x7fddcc098a70 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 10 monclient(hunting): my global_id is 254388 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc142380 con 0x7fddcc098a70 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3269707561 0 0) 0x7fddcc142380 con 0x7fddcc098a70 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddcc066de0 con 0x7fddcc098a70 2024-09-15T04:04:41.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fddbc002b80 con 0x7fddcc098a70 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 10 monclient: dump: 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.768 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 418646362 0 0) 0x7fddcc066de0 con 0x7fddcc098a70 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.746+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 --> v1:172.21.15.122:6790/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fddcc02b250 con 0x7fddcc098a70 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 <== mon.3 v1:172.21.15.122:6790/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 4100624077) 0x7fddbc0040b0 con 0x7fddcc098a70 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddc8ff9700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddc8ff9700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.122:6790/0 conn(0x7fddcc098a70 legacy=0x7fddcc14f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddc8ff9700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.122:6790/0 conn(0x7fddcc098a70 legacy=0x7fddcc14f700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:41.770 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 2, 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.771 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.772 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:41.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.774 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.775 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.776 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.777 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.779 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.780 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.781 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.782 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.783 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.784 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.785 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.786 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.787 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 2 2024-09-15T04:04:41.788 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.789 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.789 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:41.789 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.122:6812/20895 conn(0x7fddac0380a0 legacy=0x7fddac03a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 10 monclient: shutdown 2024-09-15T04:04:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 >> v1:172.21.15.26:6793/0 conn(0x7fddcc09fc80 legacy=0x7fddcc141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd12e0700 1 -- 172.21.15.26:0/3384753787 reap_dead start 2024-09-15T04:04:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 shutdown_connections 2024-09-15T04:04:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 >> 172.21.15.26:0/3384753787 conn(0x7fddcc06e6e0 msgr2=0x7fddcc131100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 shutdown_connections 2024-09-15T04:04:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.750+0000 7fddd1ae1700 1 -- 172.21.15.26:0/3384753787 wait complete. 2024-09-15T04:04:41.791 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:41.791 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-09-15T04:04:41.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-09-15T04:04:41.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.894+0000 7f6659c6f700 10 monclient: get_monmap_and_config 2024-09-15T04:04:41.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.894+0000 7f6659c6f700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.894+0000 7f6659c6f700 10 monclient: monmap: 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:41.895993+0000 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:41.895993+0000 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.901 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.901 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.901 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.901 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.894+0000 7f6659c6f700 1 Processor -- start 2024-09-15T04:04:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.894+0000 7f6659c6f700 1 -- start start 2024-09-15T04:04:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.894+0000 7f6659c6f700 10 monclient: init 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: _add_conns ranks=[0,7,8,3,4,5,6,1,2] 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient(hunting): picked mon.noname-a con 0x7f6654132c90 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient(hunting): picked mon.noname-i con 0x7f6654133060 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient(hunting): picked mon.noname-e con 0x7f6654133470 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665404ca10 con 0x7f6654132c90 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6654068f90 con 0x7f6654133060 2024-09-15T04:04:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540a1430 con 0x7f6654133470 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient(hunting): authenticate will time out at 2026.328566s 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6653fff700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f6654133470 0x7f665413b6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39328/0 (socket says 172.21.15.26:39328) 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6653fff700 1 -- 172.21.15.26:0/3080597641 learned_addr learned my addr 172.21.15.26:0/3080597641 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996655680 0 0) 0x7f665404ca10 con 0x7f6654132c90 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient(hunting): my global_id is 195246 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f663c003570 con 0x7f6654132c90 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762489094 0 0) 0x7f663c003570 con 0x7f6654132c90 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 >> v1:172.21.15.26:6793/0 conn(0x7f6654133470 legacy=0x7f665413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 >> v1:172.21.15.122:6792/0 conn(0x7f6654133060 legacy=0x7f6654137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 monclient: found mon.noname-a 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665409da90 con 0x7f6654132c90 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 5 monclient: authenticate success, global_id 195246 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f66480016e0 con 0x7f6654132c90 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: dump: 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6648001990 con 0x7f6654132c90 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 1 -- 172.21.15.26:0/3080597641 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6648002240 con 0x7f6654132c90 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f66527fc700 10 monclient: dump: 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 4 set_mon_vals no callback set 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: shutdown 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- 172.21.15.26:0/3080597641 >> v1:172.21.15.26:6789/0 conn(0x7f6654132c90 legacy=0x7f665409ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- 172.21.15.26:0/3080597641 shutdown_connections 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- 172.21.15.26:0/3080597641 >> 172.21.15.26:0/3080597641 conn(0x7f665406e510 msgr2=0x7f66540a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- 172.21.15.26:0/3080597641 shutdown_connections 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- 172.21.15.26:0/3080597641 wait complete. 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: build_initial_monmap 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: monmap: 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:41.913 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 Processor -- start 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 1 -- start start 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.898+0000 7f6659c6f700 10 monclient: init 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient: _add_conns ranks=[5,2,7,1,0,4,3,6,8] 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient(hunting): picked mon.noname-h con 0x7f6654132c90 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient(hunting): picked mon.noname-b con 0x7f6654133060 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient(hunting): picked mon.noname-i con 0x7f6654133470 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540a1e90 con 0x7f6654133060 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665409d120 con 0x7f6654132c90 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665411fb70 con 0x7f6654133470 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient(hunting): authenticate will time out at 2026.332184s 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6652ffd700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6654133060 0x7f6654147010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:57224/0 (socket says 172.21.15.26:57224) 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6652ffd700 1 -- 172.21.15.26:0/3896068647 learned_addr learned my addr 172.21.15.26:0/3896068647 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555340621 0 0) 0x7f66540a1e90 con 0x7f6654133060 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): my global_id is 254333 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6640003570 con 0x7f6654133060 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467031281 0 0) 0x7f665411fb70 con 0x7f6654133470 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): my global_id is 254446 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66540a1e90 con 0x7f6654133470 2024-09-15T04:04:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771051032 0 0) 0x7f665409d120 con 0x7f6654132c90 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): my global_id is 274154 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665411fb70 con 0x7f6654132c90 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2772833024 0 0) 0x7f6640003570 con 0x7f6654133060 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f665409d120 con 0x7f6654133060 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6644001b20 con 0x7f6654133060 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): dump: 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.922 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3293714720 0 0) 0x7f66540a1e90 con 0x7f6654133470 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6640003570 con 0x7f6654133470 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f664c001bd0 con 0x7f6654133470 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): dump: 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2081357920 0 0) 0x7f665409d120 con 0x7f6654133060 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6792/0 conn(0x7f6654133470 legacy=0x7f665414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6791/0 conn(0x7f6654132c90 legacy=0x7f665412fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 monclient: found mon.b 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665411fe50 con 0x7f6654133060 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 5 monclient: authenticate success, global_id 254333 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient: _renew_subs 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665411e0b0 con 0x7f6654133060 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient: _renew_subs 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f665411ec80 con 0x7f6654133060 2024-09-15T04:04:41.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f66440021b0 con 0x7f6654133060 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f66440026e0 con 0x7f6654133060 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6651ffb700 4 set_mon_vals no callback set 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: dump: 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.926 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.927 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6644002840 con 0x7f6654133060 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6634006590 con 0x7f6634002840 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f6644043d80 con 0x7f6654133060 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054781809 0 0) 0x7f6634006590 con 0x7f6634002840 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient(hunting): my global_id is 274163 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665409d120 con 0x7f6634002840 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f6653fff700 10 monclient: get_auth_request con 0x7f6640037c90 auth_method 0 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 486789610 0 0) 0x7f665409d120 con 0x7f6634002840 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6634006590 con 0x7f6634002840 2024-09-15T04:04:41.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6648001e40 con 0x7f6634002840 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 10 monclient: dump: 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.929 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 3655945193 0 0) 0x7f6634006590 con 0x7f6634002840 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.902+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f663400a130 con 0x7f6634002840 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.906+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f6648002060 con 0x7f6634002840 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.906+0000 7f663b7fe700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.906+0000 7f663b7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.906+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6791/0 conn(0x7f6634002840 legacy=0x7f6634004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.906+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6791/0 conn(0x7f6634002840 legacy=0x7f6634004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.910+0000 7f6659c6f700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:41.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.910+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f6634007520 con 0x7f66340068d0 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777621089 0 0) 0x7f6634007520 con 0x7f66340068d0 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient(hunting): my global_id is 274172 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634006590 con 0x7f66340068d0 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2136528255 0 0) 0x7f6634006590 con 0x7f66340068d0 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6634007520 con 0x7f66340068d0 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6648002a40 con 0x7f66340068d0 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient: dump: 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:41.932 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:41.933 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1469946586 0 0) 0x7f6634007520 con 0x7f66340068d0 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 --> v1:172.21.15.122:6791/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6634008760 con 0x7f66340068d0 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 <== mon.5 v1:172.21.15.122:6791/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7399 (unknown 0 0 3595669189) 0x7f6648004cf0 con 0x7f66340068d0 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6791/0 conn(0x7f66340068d0 legacy=0x7f6634006d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f663b7fe700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6791/0 conn(0x7f66340068d0 legacy=0x7f6634006d50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.934 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:41.935 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 2, 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.936 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:41.938 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:41.939 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.941 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.942 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.943 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.945 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:41.946 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.946 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.946 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.948 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.949 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.950 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:41.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:41.952 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:41.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.953 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.953 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:41.953 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:41.954 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.954 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.954 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.954 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:41.955 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:41.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.122:6812/20895 conn(0x7f6640037c90 legacy=0x7f664003a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 10 monclient: shutdown 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 >> v1:172.21.15.26:6790/0 conn(0x7f6654133060 legacy=0x7f6654147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6653fff700 1 -- 172.21.15.26:0/3896068647 reap_dead start 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 shutdown_connections 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 >> 172.21.15.26:0/3896068647 conn(0x7f665406e510 msgr2=0x7f66540a4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 shutdown_connections 2024-09-15T04:04:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:41.914+0000 7f6659c6f700 1 -- 172.21.15.26:0/3896068647 wait complete. 2024-09-15T04:04:41.958 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:41.958 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-09-15T04:04:41.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.062+0000 7f1d0359e700 10 monclient: get_monmap_and_config 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.062+0000 7f1d0359e700 10 monclient: build_initial_monmap 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.062+0000 7f1d0359e700 10 monclient: monmap: 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:42.067591+0000 2024-09-15T04:04:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:42.067591+0000 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:42.072 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 Processor -- start 2024-09-15T04:04:42.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- start start 2024-09-15T04:04:42.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient: init 2024-09-15T04:04:42.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:42.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient: _add_conns ranks=[1,0,4,2,6,3,8,7,5] 2024-09-15T04:04:42.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient(hunting): picked mon.noname-f con 0x7f1d040a2a70 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient(hunting): picked mon.noname-a con 0x7f1d0409e380 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient(hunting): picked mon.noname-c con 0x7f1d0409bfc0 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0404ca10 con 0x7f1d0409e380 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0409d3e0 con 0x7f1d040a2a70 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04098880 con 0x7f1d0409bfc0 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient(hunting): authenticate will time out at 2026.496946s 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d01d9b700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1d0409e380 0x7f1d04098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49274/0 (socket says 172.21.15.26:49274) 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d01d9b700 1 -- 172.21.15.26:0/1056537908 learned_addr learned my addr 172.21.15.26:0/1056537908 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672377689 0 0) 0x7f1d04098880 con 0x7f1d0409bfc0 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): my global_id is 205141 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cec003570 con 0x7f1d0409bfc0 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420724780 0 0) 0x7f1d0404ca10 con 0x7f1d0409e380 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): my global_id is 195255 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04098880 con 0x7f1d0409e380 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048728704 0 0) 0x7f1d0409d3e0 con 0x7f1d040a2a70 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient(hunting): my global_id is 274168 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0404ca10 con 0x7f1d040a2a70 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826446853 0 0) 0x7f1cec003570 con 0x7f1d0409bfc0 2024-09-15T04:04:42.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 >> v1:172.21.15.122:6789/0 conn(0x7f1d040a2a70 legacy=0x7f1d04097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 >> v1:172.21.15.26:6789/0 conn(0x7f1d0409e380 legacy=0x7f1d04098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: _finish_hunting 0 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 monclient: found mon.noname-c 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0409d100 con 0x7f1d0409bfc0 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: _finish_auth 0 2024-09-15T04:04:42.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 5 monclient: authenticate success, global_id 205141 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cfc0017b0 con 0x7f1d0409bfc0 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: dump: 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.079 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1cfc001a90 con 0x7f1d0409bfc0 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:42.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 4 set_mon_vals no callback set 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 1 -- 172.21.15.26:0/1056537908 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cfc002240 con 0x7f1d0409bfc0 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0159a700 10 monclient: dump: 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.081 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 10 monclient: shutdown 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- 172.21.15.26:0/1056537908 >> v1:172.21.15.26:6791/0 conn(0x7f1d0409bfc0 legacy=0x7f1d04138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- 172.21.15.26:0/1056537908 shutdown_connections 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- 172.21.15.26:0/1056537908 >> 172.21.15.26:0/1056537908 conn(0x7f1d04063cd0 msgr2=0x7f1d040a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- 172.21.15.26:0/1056537908 shutdown_connections 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.066+0000 7f1d0359e700 1 -- 172.21.15.26:0/1056537908 wait complete. 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: build_initial_monmap 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: monmap: 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:42.084 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 Processor -- start 2024-09-15T04:04:42.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 -- start start 2024-09-15T04:04:42.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: init 2024-09-15T04:04:42.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:42.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: _add_conns ranks=[7,4,2,5,0,8,3,6,1] 2024-09-15T04:04:42.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient(hunting): picked mon.noname-i con 0x7f1d0409bfc0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient(hunting): picked mon.noname-c con 0x7f1d0409e380 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient(hunting): picked mon.noname-b con 0x7f1d040a2a70 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0406df30 con 0x7f1d040a2a70 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0409cad0 con 0x7f1d0409e380 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d04123b40 con 0x7f1d0409bfc0 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient(hunting): authenticate will time out at 2026.501469s 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d01d9b700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f1d0409e380 0x7f1d041470f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40558/0 (socket says 172.21.15.26:40558) 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d01d9b700 1 -- 172.21.15.26:0/1877750540 learned_addr learned my addr 172.21.15.26:0/1877750540 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128111831 0 0) 0x7f1d0409cad0 con 0x7f1d0409e380 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): my global_id is 205150 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce4003570 con 0x7f1d0409e380 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879177322 0 0) 0x7f1d0406df30 con 0x7f1d040a2a70 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): my global_id is 254342 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0409cad0 con 0x7f1d040a2a70 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861757286 0 0) 0x7f1d04123b40 con 0x7f1d0409bfc0 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): my global_id is 254455 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0406df30 con 0x7f1d0409bfc0 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2486149470 0 0) 0x7f1d0409cad0 con 0x7f1d040a2a70 2024-09-15T04:04:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d04123b40 con 0x7f1d040a2a70 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cfc001f40 con 0x7f1d040a2a70 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): dump: 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2767812618 0 0) 0x7f1ce4003570 con 0x7f1d0409e380 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d0409cad0 con 0x7f1d0409e380 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cf8001aa0 con 0x7f1d0409e380 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient(hunting): dump: 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 456673772 0 0) 0x7f1d0406df30 con 0x7f1d0409bfc0 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ce4003570 con 0x7f1d0409bfc0 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3567224254 0 0) 0x7f1d04123b40 con 0x7f1d040a2a70 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.122:6792/0 conn(0x7f1d0409bfc0 legacy=0x7f1d04141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.26:6791/0 conn(0x7f1d0409e380 legacy=0x7f1d041470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 monclient: found mon.b 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04122e70 con 0x7f1d040a2a70 2024-09-15T04:04:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 5 monclient: authenticate success, global_id 254342 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: _renew_subs 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d041219c0 con 0x7f1d040a2a70 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: _renew_subs 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0409aef0 con 0x7f1d040a2a70 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1cfc0021b0 con 0x7f1d040a2a70 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cfc002b00 con 0x7f1d040a2a70 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1d00d99700 4 set_mon_vals no callback set 2024-09-15T04:04:42.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.070+0000 7f1cf27fc700 10 monclient: dump: 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1d0359e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1ce8006590 con 0x7f1ce8002840 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f1cfc00d4c0 con 0x7f1d040a2a70 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f1cfc041640 con 0x7f1d040a2a70 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700855947 0 0) 0x7f1ce8006590 con 0x7f1ce8002840 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient(hunting): my global_id is 254464 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04123b40 con 0x7f1ce8002840 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1d01d9b700 10 monclient: get_auth_request con 0x7f1ce4037f70 auth_method 0 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4284096660 0 0) 0x7f1d04123b40 con 0x7f1ce8002840 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ce8006590 con 0x7f1ce8002840 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cf40021e0 con 0x7f1ce8002840 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient: dump: 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.101 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 2648626226 0 0) 0x7f1ce8006590 con 0x7f1ce8002840 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ce800a130 con 0x7f1ce8002840 2024-09-15T04:04:42.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+8107 (unknown 0 0 936848079) 0x7f1cf4002680 con 0x7f1ce8002840 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.122:6792/0 conn(0x7f1ce8002840 legacy=0x7f1ce8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.074+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.122:6792/0 conn(0x7f1ce8002840 legacy=0x7f1ce8004cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1d0359e700 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 1) v1 -- 0x7f1ce8009640 con 0x7f1ce80090b0 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017115418 0 0) 0x7f1ce8009640 con 0x7f1ce80090b0 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 10 monclient(hunting): my global_id is 254473 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce8006590 con 0x7f1ce80090b0 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 104307435 0 0) 0x7f1ce8006590 con 0x7f1ce80090b0 2024-09-15T04:04:42.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1ce8009640 con 0x7f1ce80090b0 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1cf40032b0 con 0x7f1ce80090b0 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 10 monclient: dump: 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.104 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (unknown 1391948521 0 0) 0x7f1ce8009640 con 0x7f1ce80090b0 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.082+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 --> v1:172.21.15.122:6792/0 -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1ce80098f0 con 0x7f1ce80090b0 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 <== mon.7 v1:172.21.15.122:6792/0 5 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+7569 (unknown 0 0 489225935) 0x7f1cf40062a0 con 0x7f1ce80090b0 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1cf27fc700 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1cf27fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.122:6792/0 conn(0x7f1ce80090b0 legacy=0x7f1ce8008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1cf27fc700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.122:6792/0 conn(0x7f1ce80090b0 legacy=0x7f1ce8008730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T04:04:42.106 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:42.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:42.106 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:42.106 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "peon", 2024-09-15T04:04:42.106 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 128, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 2, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 3, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 4, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 5, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 6, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 7, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: 8 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 2, 2024-09-15T04:04:42.107 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "required_con": "2449958755906961412", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "required_mon": [ 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:42.108 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_con": "4540138320759226367", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_mon": [ 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.109 INFO:teuthology.orchestra.run.smithi026.stdout: "outside_quorum": [], 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "extra_probe_peers": [], 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "sync_provider": [], 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "4d3becf5-10bb-4b2a-818b-afafac96b25a", 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "created": "2024-09-15T03:44:18.866916Z", 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release": 17, 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "quincy", 2024-09-15T04:04:42.110 INFO:teuthology.orchestra.run.smithi026.stdout: "election_strategy": 3, 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "disallowed_leaders: ": "", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false, 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tiebreaker_mon": "", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "removed_ranks: ": "", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "features": { 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "persistent": [ 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "kraken", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "luminous", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "mimic", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap-prune", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "nautilus", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus", 2024-09-15T04:04:42.111 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific", 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "elector-pinging", 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy" 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "optional": [] 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "mons": [ 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "a", 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.112 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789", 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6789/0", 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6789/0", 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 1, 2024-09-15T04:04:42.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "f", 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789", 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6789/0", 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6789/0", 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 2, 2024-09-15T04:04:42.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790", 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6790/0", 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6790/0", 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.116 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 3, 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "g", 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.117 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790", 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6790/0", 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6790/0", 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 4, 2024-09-15T04:04:42.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791", 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6791/0", 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6791/0", 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.119 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 5, 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "h", 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791", 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6791/0", 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6791/0", 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 6, 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.121 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792", 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6792/0", 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6792/0", 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 7, 2024-09-15T04:04:42.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "i", 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792", 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.122:6792/0", 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.122:6792/0", 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.123 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 8, 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "e", 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addrs": { 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793", 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 0 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.124 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6793/0", 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "public_addr": "172.21.15.26:6793/0", 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "priority": 0, 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "weight": 0, 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_location": "{}" 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "feature_map": { 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": [ 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:42.125 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": [ 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "client": [ 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "features": "0x3f01cfbf7ffdffff", 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "release": "luminous", 2024-09-15T04:04:42.126 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 1 2024-09-15T04:04:42.127 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-09-15T04:04:42.127 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-09-15T04:04:42.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-09-15T04:04:42.127 INFO:teuthology.orchestra.run.smithi026.stdout: "stretch_mode": false 2024-09-15T04:04:42.127 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.122:6812/20895 conn(0x7f1ce4037f70 legacy=0x7f1ce403a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 10 monclient: shutdown 2024-09-15T04:04:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 >> v1:172.21.15.26:6790/0 conn(0x7f1d040a2a70 legacy=0x7f1d0414a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d02d9d700 1 -- 172.21.15.26:0/1877750540 reap_dead start 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d02d9d700 1 -- 172.21.15.26:0/1877750540 reap_dead start 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 shutdown_connections 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 >> 172.21.15.26:0/1877750540 conn(0x7f1d04063cd0 msgr2=0x7f1d0406c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 shutdown_connections 2024-09-15T04:04:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.086+0000 7f1d0359e700 1 -- 172.21.15.26:0/1877750540 wait complete. 2024-09-15T04:04:42.129 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-09-15T04:04:42.129 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-15T04:04:42.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-15T04:04:42.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.234+0000 7f7129230700 10 monclient: get_monmap_and_config 2024-09-15T04:04:42.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.234+0000 7f7129230700 10 monclient: build_initial_monmap 2024-09-15T04:04:42.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient: monmap: 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:42.239994+0000 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:42.239994+0000 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:42.244 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:42.245 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 1 Processor -- start 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 1 -- start start 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient: init 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient: _add_conns ranks=[3,8,0,7,2,4,6,5,1] 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient(hunting): picked mon.noname-g con 0x7f7124099310 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient(hunting): picked mon.noname-e con 0x7f7124132fa0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient(hunting): picked mon.noname-a con 0x7f71241333f0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:42.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712404ca10 con 0x7f71241333f0 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7124097e50 con 0x7f7124099310 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71240a1410 con 0x7f7124132fa0 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient(hunting): authenticate will time out at 2026.669475s 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f711a59c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f7124132fa0 0x7f71241380a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39334/0 (socket says 172.21.15.26:39334) 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f711a59c700 1 -- 172.21.15.26:0/1258774533 learned_addr learned my addr 172.21.15.26:0/1258774533 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267267631 0 0) 0x7f71240a1410 con 0x7f7124132fa0 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): my global_id is 274211 2024-09-15T04:04:42.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7110003570 con 0x7f7124132fa0 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489693456 0 0) 0x7f712404ca10 con 0x7f71241333f0 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): my global_id is 195264 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71240a1410 con 0x7f71241333f0 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341248056 0 0) 0x7f7124097e50 con 0x7f7124099310 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient(hunting): my global_id is 254397 2024-09-15T04:04:42.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f712404ca10 con 0x7f7124099310 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186658583 0 0) 0x7f7110003570 con 0x7f7124132fa0 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 >> v1:172.21.15.122:6790/0 conn(0x7f7124099310 legacy=0x7f71240996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 >> v1:172.21.15.26:6789/0 conn(0x7f71241333f0 legacy=0x7f712413b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: _finish_hunting 0 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 monclient: found mon.noname-e 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71240a2710 con 0x7f7124132fa0 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: _finish_auth 0 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 5 monclient: authenticate success, global_id 274211 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f710c001770 con 0x7f7124132fa0 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: dump: 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.259 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f710c001990 con 0x7f7124132fa0 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 1 -- 172.21.15.26:0/1258774533 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f710c0048f0 con 0x7f7124132fa0 2024-09-15T04:04:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f7129230700 4 set_mon_vals no callback set 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.238+0000 7f712259c700 10 monclient: dump: 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.261 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient: shutdown 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- 172.21.15.26:0/1258774533 >> v1:172.21.15.26:6793/0 conn(0x7f7124132fa0 legacy=0x7f71241380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- 172.21.15.26:0/1258774533 shutdown_connections 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- 172.21.15.26:0/1258774533 >> 172.21.15.26:0/1258774533 conn(0x7f71240a6c30 msgr2=0x7f712406dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- 172.21.15.26:0/1258774533 shutdown_connections 2024-09-15T04:04:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- 172.21.15.26:0/1258774533 wait complete. 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient: build_initial_monmap 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient: monmap: 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 Processor -- start 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- start start 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient: init 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient: _add_conns ranks=[8,4,5,3,0,2,7,6,1] 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient(hunting): picked mon.noname-e con 0x7f7124099310 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient(hunting): picked mon.noname-c con 0x7f7124132fa0 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient(hunting): picked mon.noname-h con 0x7f71241333f0 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:42.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71241342e0 con 0x7f7124132fa0 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7124121920 con 0x7f71241333f0 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71241202d0 con 0x7f7124099310 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7129230700 10 monclient(hunting): authenticate will time out at 2026.675033s 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7122d9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f7124099310 0x7f71241314f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39336/0 (socket says 172.21.15.26:39336) 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f7122d9d700 1 -- 172.21.15.26:0/669051806 learned_addr learned my addr 172.21.15.26:0/669051806 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931351624 0 0) 0x7f71241202d0 con 0x7f7124099310 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): my global_id is 274220 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7104003570 con 0x7f7124099310 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794761705 0 0) 0x7f71241342e0 con 0x7f7124132fa0 2024-09-15T04:04:42.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): my global_id is 205159 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71241202d0 con 0x7f7124132fa0 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725415165 0 0) 0x7f7124121920 con 0x7f71241333f0 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 10 monclient(hunting): my global_id is 274181 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.242+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71241342e0 con 0x7f71241333f0 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3993508792 0 0) 0x7f71241202d0 con 0x7f7124132fa0 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7124121920 con 0x7f7124132fa0 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2965855525 0 0) 0x7f7104003570 con 0x7f7124099310 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f71241202d0 con 0x7f7124099310 2024-09-15T04:04:42.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f710c001820 con 0x7f7124132fa0 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): dump: 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.267 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f7114002b90 con 0x7f7124099310 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): dump: 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.268 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 892859575 0 0) 0x7f71241342e0 con 0x7f71241333f0 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7104003570 con 0x7f71241333f0 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f711c001c80 con 0x7f71241333f0 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient(hunting): dump: 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.270 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 255871325 0 0) 0x7f7124121920 con 0x7f7124132fa0 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 >> v1:172.21.15.26:6793/0 conn(0x7f7124099310 legacy=0x7f71241314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 >> v1:172.21.15.122:6791/0 conn(0x7f71241333f0 legacy=0x7f712414a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:42.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 monclient: found mon.c 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124121640 con 0x7f7124132fa0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 5 monclient: authenticate success, global_id 205159 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 monclient: _renew_subs 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7124142bc0 con 0x7f7124132fa0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 monclient: _renew_subs 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7124120880 con 0x7f7124132fa0 2024-09-15T04:04:42.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f710c004b30 con 0x7f7124132fa0 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f710c005060 con 0x7f7124132fa0 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7121d9b700 4 set_mon_vals no callback set 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 10 monclient: dump: 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:42.273 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:42.274 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71080050b0 con 0x7f7124132fa0 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f710c005240 con 0x7f7124132fa0 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f710c043d30 con 0x7f7124132fa0 2024-09-15T04:04:42.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.246+0000 7f7122d9d700 10 monclient: get_auth_request con 0x7f7104037fd0 auth_method 0 2024-09-15T04:04:42.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.254+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f710c014d80 con 0x7f7124132fa0 2024-09-15T04:04:42.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.254+0000 7f711b7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:42.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.254+0000 7f711b7fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.486+0000 7f7129230700 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-09-15T04:04:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.486+0000 7f7129230700 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-09-15T04:04:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.486+0000 7f7129230700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.486+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f7108005b70 con 0x7f7124132fa0 2024-09-15T04:04:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.490+0000 7f711b7fe700 1 -- 172.21.15.26:0/669051806 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (unknown 246732539 0 0) 0x7f710c014960 con 0x7f7124132fa0 2024-09-15T04:04:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.490+0000 7f711b7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-09-15T04:04:42.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.490+0000 7f711b7fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:42.498 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.498 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.498 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=27,config=2,health=10,kv=8,logm=53} crc {auth=3539280234,config=3140514009,health=1724455319,kv=2519098299,logm=571305617}) 2024-09-15T04:04:42.498 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=26,config=2,health=11,kv=8,logm=53} crc {auth=1503122117,config=3140514009,health=2172458451,kv=2519098299,logm=112039483}) 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=27,config=1,health=12,kv=7,logm=53} crc {auth=3539280234,config=1228700967,health=3102523775,kv=777372614,logm=2128624060}) 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=27,config=2,health=12,kv=6,logm=53} crc {auth=3539280234,config=3140514009,health=3102523775,kv=1266854495,logm=1416473533}) 2024-09-15T04:04:42.499 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=27,config=2,health=11,kv=8,logm=52} crc {auth=3539280234,config=3140514009,health=3068549807,kv=2519098299,logm=384968037}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=26,config=2,health=11,kv=7,logm=54} crc {auth=3972015599,config=3140514009,health=3068549807,kv=3915661961,logm=3723185407}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=27,config=1,health=12,kv=8,logm=52} crc {auth=3539280234,config=1228700967,health=3102523775,kv=2519098299,logm=2503299643}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=26,config=2,health=12,kv=8,logm=52} crc {auth=1869006511,config=3140514009,health=3102523775,kv=2519098299,logm=4137329769}) 2024-09-15T04:04:42.500 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.490+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=26,config=2,health=10,kv=8,logm=54} crc {auth=75728382,config=3140514009,health=491555799,kv=2519098299,logm=1522673013}) 2024-09-15T04:04:42.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 >> v1:172.21.15.122:6812/20895 conn(0x7f7104037fd0 legacy=0x7f710403a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 10 monclient: shutdown 2024-09-15T04:04:42.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:42.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 >> v1:172.21.15.26:6791/0 conn(0x7f7124132fa0 legacy=0x7f7124147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 shutdown_connections 2024-09-15T04:04:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 >> 172.21.15.26:0/669051806 conn(0x7f71240a6c30 msgr2=0x7f7124067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 shutdown_connections 2024-09-15T04:04:42.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:42.494+0000 7f7129230700 1 -- 172.21.15.26:0/669051806 wait complete. 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2283030049}) 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=931384133}) 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2072921232}) 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.504 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3747073749}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2361793702}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=957687696}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=154004804}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1321438008}) 2024-09-15T04:04:42.505 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.494+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=422850908}) 2024-09-15T04:04:42.506 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.506 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=11512801}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3285120908}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=619505137}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3522488641}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.507 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1650741266}) 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1704623386}) 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1216137135}) 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3527600088}) 2024-09-15T04:04:42.508 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.498+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3161416414}) 2024-09-15T04:04:42.509 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.509 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.509 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3786375517}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3765197082}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4207738639}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3522098185}) 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.510 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=400496894}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3730896599}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3633361154}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=760089463}) 2024-09-15T04:04:42.511 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.502+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2648610112}) 2024-09-15T04:04:42.512 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.512 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.512 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1475307347}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4128391198}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1457325029}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3588584085}) 2024-09-15T04:04:42.513 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4065052482}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1021184440}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2957277302}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1703171149}) 2024-09-15T04:04:42.514 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3350838645}) 2024-09-15T04:04:42.515 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.515 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3150119505}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2728590957}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2383394470}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1995638977}) 2024-09-15T04:04:42.516 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1144291908}) 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2406038546}) 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2010038104}) 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.517 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2943788037}) 2024-09-15T04:04:42.518 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.506+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3282500479}) 2024-09-15T04:04:42.519 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4173524899}) 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2672293036}) 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1617358789}) 2024-09-15T04:04:42.538 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2424004309}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3663848287}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2079189449}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2379375812}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2921337748}) 2024-09-15T04:04:42.539 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1065045320}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1327265265}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3135192652}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1767310854}) 2024-09-15T04:04:42.540 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4101414514}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1822752189}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3225122739}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2491999409}) 2024-09-15T04:04:42.541 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2696006117}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.510+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=107089290}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2510390015}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1255777950}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4246384781}) 2024-09-15T04:04:42.542 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2488105677}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4224635584}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3534022022}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2370004491}) 2024-09-15T04:04:42.543 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3352816273}) 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1713370090}) 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1637922649}) 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=827511792}) 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.544 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2614559827}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2908480036}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=375777878}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2386627630}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2540243188}) 2024-09-15T04:04:42.545 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4165489126}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.514+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4143814318}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2733903843}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1377507032}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4238127315}) 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.546 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2851043401}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2161495387}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1940744854}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3376045065}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4192853984}) 2024-09-15T04:04:42.547 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=658931062}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2635857447}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=699469940}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1605312509}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=834256898}) 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.548 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3305978855}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2670972275}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4095226434}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4015880364}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.518+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4194447451}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2150433622}) 2024-09-15T04:04:42.549 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1480047518}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2926632643}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=355374901}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3868854236}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.550 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3581755429}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1141872515}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1766392672}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1352773244}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1717118121}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2858705326}) 2024-09-15T04:04:42.551 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=20330927}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3960780308}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=602005468}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3028738874}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.552 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2228560494}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3915963539}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2885559301}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2100782549}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1649590937}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1086334671}) 2024-09-15T04:04:42.553 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3246369716}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=391920650}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1166522208}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3177632766}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3106667749}) 2024-09-15T04:04:42.554 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.522+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1596665620}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3374675477}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2014718412}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=770624065}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4151485234}) 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.555 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=937507490}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2976350102}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1819618549}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2856882940}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.526+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=187047174}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.556 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2900494293}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2064547972}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1181890261}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1134136555}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1500263878}) 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.557 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3432474772}) 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1291808193}) 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1877911334}) 2024-09-15T04:04:42.558 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=776477590}) 2024-09-15T04:04:42.614 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.614 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=688118283}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1697203269}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=878222077}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2339499183}) 2024-09-15T04:04:42.615 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2927844819}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1141419414}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1285009386}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4094781904}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.530+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3678508173}) 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.616 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1461554563}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=798546705}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1713828499}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1541147235}) 2024-09-15T04:04:42.617 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=683982491}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=353917228}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3744768128}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3145867143}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=930575590}) 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.618 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1467712787}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=655276977}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3951451027}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=717694342}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.619 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2780807289}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3342271999}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2060154911}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2916118855}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.534+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3529353866}) 2024-09-15T04:04:42.620 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=588389194}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1230790310}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3315934045}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=153584894}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.621 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2280587722}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=330878499}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3784119380}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3823592706}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3342884632}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1791167757}) 2024-09-15T04:04:42.622 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2473368372}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2649177400}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1096824001}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=217918953}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.623 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1240971594}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1776549919}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4085158909}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.538+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3978394136}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=326811897}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=568616601}) 2024-09-15T04:04:42.624 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=970039000}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=894171683}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2234279443}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3043574567}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.625 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2827327797}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3605526289}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.542+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3443122216}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3474592591}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1957218970}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1659670402}) 2024-09-15T04:04:42.626 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1146606844}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2906416161}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=567275520}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1780476555}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3608550591}) 2024-09-15T04:04:42.627 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3066551889}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2042294469}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1303887783}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=78256620}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=199266452}) 2024-09-15T04:04:42.628 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3439750227}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2578057944}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1892012003}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2173118096}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.546+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3284623877}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.629 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3321817800}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4170972194}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3694532100}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=79761629}) 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.630 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1337915570}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1559074070}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3784225229}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1493309653}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.550+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3695618530}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2696123142}) 2024-09-15T04:04:42.631 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3147976828}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3402801922}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=800881759}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1280493109}) 2024-09-15T04:04:42.632 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.633 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.633 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1791434885}) 2024-09-15T04:04:42.633 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.633 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.633 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2888350801}) 2024-09-15T04:04:42.633 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2191224234}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2500932189}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422512521}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=182725672}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2111365807}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.634 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1451458682}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1662598340}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1828961508}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2210678310}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1592128875}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.554+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=70679957}) 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.635 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1150430122}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=367372680}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=459200185}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3970141688}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.636 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4156920031}) 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1462387032}) 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2857991407}) 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1771294891}) 2024-09-15T04:04:42.637 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.558+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3477200524}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072264366}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1332796549}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=626556684}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.638 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3072961749}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3737485077}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2978874048}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2804584906}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=261491634}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3610772008}) 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.639 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3513289224}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3273545227}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4255035033}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3737416558}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.640 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3969900092}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=934469164}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=458654265}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2333807995}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.562+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1682735899}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=221194322}) 2024-09-15T04:04:42.641 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2559383614}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1267958080}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=318158164}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=175833805}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.642 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1520364306}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=660422765}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2733176853}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.566+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3192037044}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=640179011}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3264571462}) 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.643 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=114344732}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=336562223}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3404942445}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3302703449}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=275405251}) 2024-09-15T04:04:42.644 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=525778312}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=105948226}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2713113725}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3578689200}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3172427270}) 2024-09-15T04:04:42.645 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=911709411}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3720416302}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3874451315}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=853073753}) 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.646 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=470336078}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.570+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1315692776}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=871356653}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3118076154}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=737121619}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3373457997}) 2024-09-15T04:04:42.647 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1178864581}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1615117611}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=246813783}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1294582264}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=851500840}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.648 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=381463449}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2293026916}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1262218599}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=90507486}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3980558759}) 2024-09-15T04:04:42.649 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=514227795}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3276337474}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=389184462}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.574+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=891019761}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=694877380}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.650 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3507516105}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1908063406}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4143743791}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2685127081}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.651 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1436262764}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1975526269}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=212067766}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3846917937}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2382197597}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=919845352}) 2024-09-15T04:04:42.652 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=329511532}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2313487858}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.578+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1067149353}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2922310892}) 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.653 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2819130901}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=835785808}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1146034510}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1353976247}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=973865428}) 2024-09-15T04:04:42.654 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3661609363}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3461638645}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2806753634}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.656 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=248885880}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1204204327}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1919685023}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3085157006}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=192647483}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.582+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=53297697}) 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.657 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1827374141}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3428762613}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1849259961}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3069470300}) 2024-09-15T04:04:42.658 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2835631615}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4273921835}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1270813646}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1872652250}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=736415606}) 2024-09-15T04:04:42.659 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1684217679}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=461476993}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=231802063}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3336017597}) 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.660 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3612852028}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=23674912}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1664016170}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=394432774}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3384707132}) 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.661 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1807806387}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.586+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2727495228}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2023708965}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3536787711}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=928842737}) 2024-09-15T04:04:42.662 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1797471671}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1215011174}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=338096617}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2967924612}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2391246976}) 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.663 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=229310620}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=685714473}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1759929958}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3497606801}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=955683489}) 2024-09-15T04:04:42.664 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.590+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1068322244}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4190221300}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=888437385}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=146488094}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3139337736}) 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.665 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3545023458}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2781188405}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3856037086}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981819015}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1869545196}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.666 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3970821810}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2102098264}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1887560645}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3552709903}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2642997314}) 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.667 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3779175826}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3314990287}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.594+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=265832394}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=949049790}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2072479462}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.668 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2629338792}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=547707757}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=787946169}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3974704906}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1197485203}) 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.669 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=421868824}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=750786194}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.598+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1403103354}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1322185916}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3612008520}) 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.670 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2660634549}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=875396085}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2762477122}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1320222451}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.671 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2706620338}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3584550863}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=928336192}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1277780882}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2895814010}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=286836000}) 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.672 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1214303868}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3485546172}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1655097245}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=182158711}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=92530557}) 2024-09-15T04:04:42.673 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.602+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3049270052}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2184066797}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2467964352}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1627910628}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1022064595}) 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.674 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=832709753}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=74116730}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3400467741}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3057345192}) 2024-09-15T04:04:42.675 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.606+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3045594866}) 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265232285}) 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860488089}) 2024-09-15T04:04:42.807 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2481207691}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1321974995}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=169742544}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1981298874}) 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.808 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4121665261}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=6500889}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=326874403}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2739807926}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2499271255}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.809 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1801414965}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1690290886}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3838826746}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3431438443}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.810 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2615951924}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3818990647}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.610+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1526536916}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1882051256}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1184187350}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2962957889}) 2024-09-15T04:04:42.811 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1481954778}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4046468656}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2472760061}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3382095282}) 2024-09-15T04:04:42.812 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4053061939}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=821865331}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3608010253}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.614+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2827919355}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1256544034}) 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.813 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=668701956}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1738834843}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3925378600}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3116253405}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=438385005}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=807753662}) 2024-09-15T04:04:42.814 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=940139365}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2338364659}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3238616322}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3319415988}) 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.815 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1892412407}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1946203717}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1747168340}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2840599569}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.618+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2517413381}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.816 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1981134571}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=819611948}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=379736866}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2926896618}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.817 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=88995177}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1503438841}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3119433572}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3137127398}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3860090936}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1608867807}) 2024-09-15T04:04:42.818 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1184277911}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1326166863}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1988595479}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2229963050}) 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.819 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.622+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2292092133}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2537911029}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1018354058}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4288641089}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1744850176}) 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.820 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2411364898}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=632551100}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4063552519}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=930825539}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.821 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3760908854}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3838120171}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2424692070}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.626+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1619386625}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2651503115}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2571598396}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.822 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3220311954}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3440955269}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=839649633}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=640708735}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1619152513}) 2024-09-15T04:04:42.823 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1926139805}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3621397808}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1227451121}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=674255660}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2611464518}) 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.824 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2102353440}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1703477935}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1540841453}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=660235548}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=560497949}) 2024-09-15T04:04:42.825 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.630+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2620192549}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4167173877}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3705213382}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=838580953}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3805433211}) 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.826 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3897264570}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2280300944}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3713910327}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1505152059}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.634+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3843441480}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.827 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1796560975}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3295568548}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2367070505}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3521782547}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4231310381}) 2024-09-15T04:04:42.828 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4112129255}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1055843350}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2428123582}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2045960812}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=717344842}) 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.829 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=287573607}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2790221135}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1253427838}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1023013964}) 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.830 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3786106758}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4283915625}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=489979467}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.638+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1622235631}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3159328939}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.831 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3265023344}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3744178457}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2464143065}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1930722501}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4054446565}) 2024-09-15T04:04:42.832 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=815742601}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135569168}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=6061094}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2321516040}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.642+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3728665726}) 2024-09-15T04:04:42.833 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4064874126}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1740554988}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1814879942}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3558313923}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.834 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4153424561}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3103741826}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1044283809}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4284875353}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1937310175}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=582684664}) 2024-09-15T04:04:42.835 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1029497787}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=531640822}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1278297408}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=607805604}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106363797}) 2024-09-15T04:04:42.836 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.646+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3402780022}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=379867285}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1260748637}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1922621424}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=983925946}) 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.837 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=683994283}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1089770948}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3364070024}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445269272}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4272806990}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.838 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=236161016}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4223998949}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1709731819}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3721018007}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2239687522}) 2024-09-15T04:04:42.839 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4017891418}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3498501526}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4022048274}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.650+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=707230173}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2762258224}) 2024-09-15T04:04:42.840 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=214705798}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=790760601}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=377420680}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4013054483}) 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.841 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2919713986}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=640454427}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1122302913}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.654+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=50449501}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3901718230}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=237191787}) 2024-09-15T04:04:42.842 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3371976552}) 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4143732953}) 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2071262198}) 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.843 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2259348599}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1544525341}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1522569304}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=490001833}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2371098196}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.844 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1363605770}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2287609504}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2160680915}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2863783384}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=979439125}) 2024-09-15T04:04:42.845 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.658+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3338436437}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1754003590}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2452962361}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1579323313}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.846 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=376956101}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3299826228}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3985406193}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2165144497}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.847 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2708514887}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4090915776}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2852451394}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=197222406}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3629623230}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.848 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.662+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3463091142}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=556741405}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=935292191}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2176567083}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.849 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4209332021}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1764306965}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=283626602}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3947460924}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2141215454}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1129033773}) 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.850 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3163064849}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=607635404}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1511845022}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3988949339}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4081186851}) 2024-09-15T04:04:42.851 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2706839161}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.666+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2992419515}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3514176798}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4104395155}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=653799136}) 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.852 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=951479786}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1591383053}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=86965607}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=515295992}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1194459239}) 2024-09-15T04:04:42.853 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=494894800}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=101632507}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4032805874}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=359765495}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3361595921}) 2024-09-15T04:04:42.854 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.670+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1419853738}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1442876428}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1459708933}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3432070386}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=520560406}) 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.855 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2061504232}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2071070916}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1333708965}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3727394674}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.856 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3468601296}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=441266600}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1160177630}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1466936011}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.674+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=255136933}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=145155795}) 2024-09-15T04:04:42.857 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=380828874}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3362107708}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2237424758}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1654940516}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.858 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3540704469}) 2024-09-15T04:04:42.859 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.859 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.859 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1999677807}) 2024-09-15T04:04:42.860 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3448125544}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3325796383}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2900036679}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3426123181}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.861 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2973183629}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1046670265}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2696915185}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1153514367}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1747004652}) 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.862 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1402479607}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.678+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3924045831}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3083648943}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=113845229}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2067011763}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.863 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1358178750}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3883866792}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1323366620}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2899399429}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3702368579}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=694723605}) 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.864 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3500170601}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3151141513}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=485466880}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=348052820}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1016271186}) 2024-09-15T04:04:42.865 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=69176144}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=696935134}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846058648}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.682+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2107105496}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1268447272}) 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.866 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2183528160}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3948050991}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=614182737}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=467239632}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=848138108}) 2024-09-15T04:04:42.867 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3356131086}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2292682954}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1554154695}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.686+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3679317602}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2945911912}) 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.868 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=808162408}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3382882407}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=544020302}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2018233136}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=19386842}) 2024-09-15T04:04:42.869 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2262309795}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3059198014}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2278196058}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3461340245}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=662671985}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.870 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=228206967}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3007996342}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=945018042}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1515482504}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=981262151}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.690+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3824354655}) 2024-09-15T04:04:42.871 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2433454469}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4246568481}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=725178427}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1544922939}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.872 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=516809673}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1663435868}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1833046910}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3977759294}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2225020666}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3471651843}) 2024-09-15T04:04:42.873 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1432461249}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2854260046}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=649513226}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1984129295}) 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.874 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=118895777}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3363834714}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=916629546}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.694+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4254548005}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1012292511}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1225511148}) 2024-09-15T04:04:42.875 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1610267200}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1041081023}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2282758895}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3409474889}) 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.876 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1246530635}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2744215312}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1295929587}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3865042316}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39148414}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=18319277}) 2024-09-15T04:04:42.877 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2232153890}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2059783040}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=232135553}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3061918392}) 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.878 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=570112486}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.698+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1095808363}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=719564657}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1541300516}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3783986624}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3099589290}) 2024-09-15T04:04:42.879 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3251598190}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2968672788}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=369678577}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=83683503}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.702+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=440791349}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.880 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=17494830}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1444125709}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2602664769}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3498928819}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3767007940}) 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.881 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2851936621}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3069858186}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2213416851}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3977815990}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=518690987}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.882 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4130522550}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=228235274}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1299685944}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=21631941}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=139550737}) 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.883 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2135053648}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1543522738}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.706+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2649433989}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2931784122}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=182370632}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.884 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1821975014}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2541515075}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3028944325}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1954148822}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1400935353}) 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.885 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1929137178}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1318354059}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3692919006}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1771985457}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3848597791}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2063756271}) 2024-09-15T04:04:42.886 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1702828714}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3458748903}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1125860868}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3135332474}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.710+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3732293583}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.887 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2050001766}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1369300814}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1957948210}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3868743400}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4198659518}) 2024-09-15T04:04:42.888 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3797235075}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=595524570}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=929147029}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3052430508}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3499921637}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.889 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=44521882}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3880674585}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1307819277}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=220122080}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=506370870}) 2024-09-15T04:04:42.890 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3311668114}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3035995833}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.714+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2623012705}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=519608786}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3693877086}) 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.891 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1004509732}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3837108183}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2629184593}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1947160450}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.892 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2018929702}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1803344150}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.718+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2209104950}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1400915777}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2626504059}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.893 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4076438110}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=287590700}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3746743411}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3933363733}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.894 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=955649693}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3258644021}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2175442243}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=628885124}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279675166}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2724310023}) 2024-09-15T04:04:42.895 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=799180294}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2102606971}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=710026866}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.896 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3181974934}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2133300996}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.722+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2191445851}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1236374124}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=976298704}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3438793817}) 2024-09-15T04:04:42.897 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2654158149}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3770161054}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1364476751}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3206601099}) 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.898 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2516854404}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1825852620}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.726+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1701896460}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=540495009}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1987619019}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.899 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2199167534}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1413057946}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1151268446}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3041292340}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1008294820}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3833279774}) 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.900 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1472797640}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1401840858}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1430230485}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=906878641}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.901 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3472111719}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1532175190}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3618503456}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1501653339}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.730+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1873236293}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3138741166}) 2024-09-15T04:04:42.902 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3499438323}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1511274820}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=85315042}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3670195497}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.903 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3906757935}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1149700666}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2408719259}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2617438115}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=372875835}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2091082902}) 2024-09-15T04:04:42.904 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3551878942}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.734+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=602241221}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3562168023}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1306044578}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.905 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=853693081}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2723595121}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=276011987}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3788350902}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1551721069}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2332477654}) 2024-09-15T04:04:42.906 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=567075041}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1784855709}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3452874398}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1397134555}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=106747970}) 2024-09-15T04:04:42.907 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.738+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=28548018}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=70425398}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1967229056}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1389015063}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3426947765}) 2024-09-15T04:04:42.908 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1707989147}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=290531066}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=219742328}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3129474457}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=323224594}) 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.909 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=366481383}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2850526150}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=186097358}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3863781812}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3352440043}) 2024-09-15T04:04:42.910 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2572552915}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=805151362}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2117457992}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.742+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1279339377}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=830746704}) 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.911 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3974101749}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=608569492}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2305904174}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=244563990}) 2024-09-15T04:04:42.912 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=94885402}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3894478415}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3503209092}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=414075471}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3511186250}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.913 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3214739794}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3042884887}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.746+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4267640656}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3395123837}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3166527263}) 2024-09-15T04:04:42.914 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1192144570}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2375826427}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=330664288}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3258200215}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=401863759}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.915 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1503099997}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4171817665}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1819423004}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2038174847}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.916 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1797095105}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2359679907}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.750+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3798273530}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=6,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=5,osd_snap=55} crc {logm=1901497236,mdsmap=1531151339,mgr=3669162908,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=912357693,mon_config_key=733955277,monmap=4273209975,osd_metadata=1967393317,osd_snap=3494991552}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=2651494269,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=4009274175}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=8,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=1710816437,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=952913954,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=728973535}) 2024-09-15T04:04:42.917 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=53} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=524308290,mon_config_key=733955277,monmap=2173287178,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=1293658252}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=8,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=53} crc {logm=2078235436,mdsmap=1531151339,mgr=2132010929,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1477473427,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=3372259990}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=8,mdsmap=1,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=14,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=53} crc {logm=2078235436,mdsmap=2826658659,mgr=3669162909,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=488842161,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=2675870512}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=6,osd_pg_creating=1,osd_snap=50} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=3392241904}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=8,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=15,mon_config_key=5,monmap=2,osd_metadata=5,osd_pg_creating=1,osd_snap=52} crc {logm=2078235436,mdsmap=1531151339,mgr=3364684289,mgr_command_descs=47663135,mgrstat=1977270099,mon_config_key=733955277,monmap=2173287178,osd_metadata=1967393317,osd_pg_creating=4259603596,osd_snap=2495188233}) 2024-09-15T04:04:42.918 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.754+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=8,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=5,monmap=3,osd_metadata=6,osd_pg_creating=1,osd_snap=54} crc {logm=2078235436,mdsmap=3140514009,mgr=991478979,mgr_command_descs=47663135,mgr_metadata=4176136816,mgrstat=329915498,mon_config_key=733955277,monmap=4273209975,osd_metadata=2994937890,osd_pg_creating=4259603596,osd_snap=2367081349}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=100} crc {osd_snap=131331839}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=100} crc {osd_snap=4256909279}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=100} crc {osd_snap=1250801399}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=100} crc {osd_snap=2049069638}) 2024-09-15T04:04:42.919 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=100} crc {osd_snap=3156942464}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=100} crc {osd_snap=1807702799}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=100} crc {osd_snap=2351474534}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=100} crc {osd_snap=2395299066}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=100} crc {osd_snap=2358466685}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.920 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osd_snap=27,osdmap=73} crc {osd_snap=3771342342,osdmap=4060541515}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osd_snap=24,osdmap=76} crc {osd_snap=1695265373,osdmap=1757734103}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osd_snap=27,osdmap=73} crc {osd_snap=3771342342,osdmap=2285539257}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.758+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=228542835,osdmap=4120439861}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=4082620395,osdmap=3348081955}) 2024-09-15T04:04:42.921 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osd_snap=25,osdmap=75} crc {osd_snap=3930253976,osdmap=3499061890}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osd_snap=25,osdmap=75} crc {osd_snap=1054264692,osdmap=4232506896}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=407577971,osdmap=3208305589}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osd_snap=26,osdmap=74} crc {osd_snap=2758139106,osdmap=1299215696}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=829409019}) 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.922 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.762+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2219318118}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3267210907}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1112072613}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3360073656}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1364913455}) 2024-09-15T04:04:42.923 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1666032579}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=499680844}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3830602688}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2472947482}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2968794702}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.924 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=131743490}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2312974622}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1656383405}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3506094662}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=4030907638}) 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.925 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3277890342}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.766+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3102798787}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1828166622}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1181720190}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3190986745}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.926 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1906209365}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2779321123}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3837780832}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1782562790}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3214748560}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.770+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2779823014}) 2024-09-15T04:04:42.927 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1358370579}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4052989392}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1918453964}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=694195695}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.928 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4270322258}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1401898973}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1623620533}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2235267710}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.774+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=984548784}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=827851011}) 2024-09-15T04:04:42.929 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2886942784}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1605023511}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2418427232}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3702595443}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.930 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=2597824246}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2184773255}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1047610435}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.778+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1533477818}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1072796019}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3974179340}) 2024-09-15T04:04:42.931 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2717359456}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1401256201}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1069794471}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=461662034}) 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.932 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1470725044}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1761923235}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.782+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3344700996}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=23} crc {osdmap=3689792472}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=20} crc {osdmap=1588683209}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=23} crc {osdmap=3689792472}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=22} crc {osdmap=355612848}) 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.933 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=23} crc {osdmap=3689792472}) 2024-09-15T04:04:42.934 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=21} crc {osdmap=3307856632}) 2024-09-15T04:04:42.934 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-09-15T04:04:42.934 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=23} crc {osdmap=3689792472}) 2024-09-15T04:04:42.934 INFO:tasks.ceph.mon.d.smithi026.stderr:2024-09-15T04:04:42.786+0000 7fe0cc6dc700 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=22} crc {osdmap=216900388}) 2024-09-15T04:04:43.520 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-09-15T04:04:43.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T04:04:43.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient: get_monmap_and_config 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient: build_initial_monmap 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient: monmap: 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:43.634089+0000 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:43.634089+0000 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:43.638 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:43.639 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 1 Processor -- start 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 1 -- start start 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient: init 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient: _add_conns ranks=[8,5,2,3,7,0,6,4,1] 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient(hunting): picked mon.noname-e con 0x7fe1e809b6e0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient(hunting): picked mon.noname-h con 0x7fe1e806e160 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient(hunting): picked mon.noname-b con 0x7fe1e8136d30 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e804ca10 con 0x7fe1e8136d30 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8098880 con 0x7fe1e806e160 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e806e550 con 0x7fe1e809b6e0 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1eca0c700 10 monclient(hunting): authenticate will time out at 2028.063457s 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1e77fe700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fe1e809b6e0 0x7fe1e8097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39346/0 (socket says 172.21.15.26:39346) 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.630+0000 7fe1e77fe700 1 -- 172.21.15.26:0/160781879 learned_addr learned my addr 172.21.15.26:0/160781879 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804342125 0 0) 0x7fe1e806e550 con 0x7fe1e809b6e0 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): my global_id is 274229 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d4003570 con 0x7fe1e809b6e0 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860524995 0 0) 0x7fe1e8098880 con 0x7fe1e806e160 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): my global_id is 274190 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e806e550 con 0x7fe1e806e160 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898211352 0 0) 0x7fe1e804ca10 con 0x7fe1e8136d30 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient(hunting): my global_id is 254351 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8098880 con 0x7fe1e8136d30 2024-09-15T04:04:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295968757 0 0) 0x7fe1d4003570 con 0x7fe1e809b6e0 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 >> v1:172.21.15.122:6791/0 conn(0x7fe1e806e160 legacy=0x7fe1e8098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 >> v1:172.21.15.26:6790/0 conn(0x7fe1e8136d30 legacy=0x7fe1e8139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 monclient: found mon.noname-e 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e809d2e0 con 0x7fe1e809b6e0 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe1d80017b0 con 0x7fe1e809b6e0 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: dump: 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:43.648 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 5 monclient: authenticate success, global_id 274229 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe1d8001a90 con 0x7fe1e809b6e0 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 4 set_mon_vals no callback set 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 1 -- 172.21.15.26:0/160781879 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe1d8004c30 con 0x7fe1e809b6e0 2024-09-15T04:04:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1e6ffd700 10 monclient: dump: 2024-09-15T04:04:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 10 monclient: shutdown 2024-09-15T04:04:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 1 -- 172.21.15.26:0/160781879 >> v1:172.21.15.26:6793/0 conn(0x7fe1e809b6e0 legacy=0x7fe1e8097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 1 -- 172.21.15.26:0/160781879 shutdown_connections 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 1 -- 172.21.15.26:0/160781879 >> 172.21.15.26:0/160781879 conn(0x7fe1e8063d90 msgr2=0x7fe1e80a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 1 -- 172.21.15.26:0/160781879 shutdown_connections 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 1 -- 172.21.15.26:0/160781879 wait complete. 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 10 monclient: build_initial_monmap 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.634+0000 7fe1eca0c700 10 monclient: monmap: 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:43.654 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 Processor -- start 2024-09-15T04:04:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 -- start start 2024-09-15T04:04:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: init 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: _add_conns ranks=[5,8,3,6,4,2,1,0,7] 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient(hunting): picked mon.noname-h con 0x7fe1e806e160 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient(hunting): picked mon.noname-e con 0x7fe1e8136d30 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient(hunting): picked mon.noname-g con 0x7fe1e806d8d0 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e809cbe0 con 0x7fe1e806d8d0 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8135bd0 con 0x7fe1e806e160 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8134060 con 0x7fe1e8136d30 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient(hunting): authenticate will time out at 2028.070124s 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1deffd700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fe1e8136d30 0x7fe1e81326b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39352/0 (socket says 172.21.15.26:39352) 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1deffd700 1 -- 172.21.15.26:0/3713670374 learned_addr learned my addr 172.21.15.26:0/3713670374 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:43.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752024828 0 0) 0x7fe1e8134060 con 0x7fe1e8136d30 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): my global_id is 274238 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c8003570 con 0x7fe1e8136d30 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63071703 0 0) 0x7fe1e8135bd0 con 0x7fe1e806e160 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): my global_id is 274199 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8134060 con 0x7fe1e806e160 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332651795 0 0) 0x7fe1e809cbe0 con 0x7fe1e806d8d0 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:43.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): my global_id is 254406 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8135bd0 con 0x7fe1e806d8d0 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3073431787 0 0) 0x7fe1c8003570 con 0x7fe1e8136d30 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe1e809cbe0 con 0x7fe1e8136d30 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe1d0001aa0 con 0x7fe1e8136d30 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): dump: 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:43.661 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1707108825 0 0) 0x7fe1e809cbe0 con 0x7fe1e8136d30 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 >> v1:172.21.15.122:6791/0 conn(0x7fe1e806e160 legacy=0x7fe1e8131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 >> v1:172.21.15.122:6790/0 conn(0x7fe1e806d8d0 legacy=0x7fe1e81481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 monclient: found mon.e 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8132dc0 con 0x7fe1e8136d30 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: _finish_auth 0 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 5 monclient: authenticate success, global_id 274238 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: _renew_subs 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e8136860 con 0x7fe1e8136d30 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: _renew_subs 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e8134440 con 0x7fe1e8136d30 2024-09-15T04:04:43.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe1d0002d50 con 0x7fe1e8136d30 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1e5ffb700 4 set_mon_vals no callback set 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fe1d0002370 con 0x7fe1e8136d30 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.638+0000 7fe1dffff700 10 monclient: dump: 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:43.664 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fe1d00024d0 con 0x7fe1e8136d30 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fe1d0043d60 con 0x7fe1e8136d30 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1eca0c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1eca0c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1eca0c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc0050b0 con 0x7fe1e8136d30 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.642+0000 7fe1e77fe700 10 monclient: get_auth_request con 0x7fe1c8037790 auth_method 0 2024-09-15T04:04:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.646+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fe1d0014870 con 0x7fe1e8136d30 2024-09-15T04:04:43.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.646+0000 7fe1dffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:43.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.646+0000 7fe1dffff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1eca0c700 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1eca0c700 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1eca0c700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe1cc0058e0 con 0x7fe1e8136d30 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1dffff700 1 -- 172.21.15.26:0/3713670374 <== mon.8 v1:172.21.15.26:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+2701 (unknown 254746657 0 1905762553) 0x7fe1d0014690 con 0x7fe1e8136d30 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1dffff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-09-15T04:04:43.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.878+0000 7fe1dffff700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:43.886 INFO:teuthology.orchestra.run.smithi026.stdout:{"election_epoch":128,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"d","quorum_age":4,"features":{"quorum_con":"4540138320759226367","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"]},"monmap":{"epoch":1,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","modified":"2024-09-15T03:44:18.866916Z","created":"2024-09-15T03:44:18.866916Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6790","nonce":0}]},"addr":"172.21.15.26:6790/0","public_addr":"172.21.15.26:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6790","nonce":0}]},"addr":"172.21.15.122:6790/0","public_addr":"172.21.15.122:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6791","nonce":0}]},"addr":"172.21.15.26:6791/0","public_addr":"172.21.15.26:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6791","nonce":0}]},"addr":"172.21.15.122:6791/0","public_addr":"172.21.15.122:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6792","nonce":0}]},"addr":"172.21.15.26:6792/0","public_addr":"172.21.15.26:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6792","nonce":0}]},"addr":"172.21.15.122:6792/0","public_addr":"172.21.15.122:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6793","nonce":0}]},"addr":"172.21.15.26:6793/0","public_addr":"172.21.15.26:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T04:04:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 >> v1:172.21.15.122:6812/20895 conn(0x7fe1c8037790 legacy=0x7fe1c8039c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 10 monclient: shutdown 2024-09-15T04:04:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 >> v1:172.21.15.26:6793/0 conn(0x7fe1e8136d30 legacy=0x7fe1e81326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 shutdown_connections 2024-09-15T04:04:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 >> 172.21.15.26:0/3713670374 conn(0x7fe1e8063d90 msgr2=0x7fe1e812f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 shutdown_connections 2024-09-15T04:04:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:43.882+0000 7fe1eca0c700 1 -- 172.21.15.26:0/3713670374 wait complete. 2024-09-15T04:04:43.901 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-09-15T04:04:43.901 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T04:04:43.911 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-15T04:04:43.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T04:04:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient: get_monmap_and_config 2024-09-15T04:04:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient: monmap: 2024-09-15T04:04:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.029 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:44.025689+0000 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:44.025689+0000 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.030 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 1 Processor -- start 2024-09-15T04:04:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 1 -- start start 2024-09-15T04:04:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient: init 2024-09-15T04:04:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient: _add_conns ranks=[5,7,8,3,4,6,2,1,0] 2024-09-15T04:04:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient(hunting): picked mon.noname-h con 0x7ff704099910 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient(hunting): picked mon.noname-i con 0x7ff70412ff80 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient(hunting): picked mon.noname-e con 0x7ff7041303d0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70404ca10 con 0x7ff704099910 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff704098be0 con 0x7ff70412ff80 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff704099460 con 0x7ff7041303d0 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff70a4c8700 10 monclient(hunting): authenticate will time out at 2028.455615s 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff709cc7700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7ff7041303d0 0x7ff7041382d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39362/0 (socket says 172.21.15.26:39362) 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.022+0000 7ff709cc7700 1 -- 172.21.15.26:0/491879462 learned_addr learned my addr 172.21.15.26:0/491879462 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337793971 0 0) 0x7ff704099460 con 0x7ff7041303d0 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): my global_id is 274247 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f0003570 con 0x7ff7041303d0 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490503418 0 0) 0x7ff704098be0 con 0x7ff70412ff80 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): my global_id is 254482 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704099460 con 0x7ff70412ff80 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195472160 0 0) 0x7ff70404ca10 con 0x7ff704099910 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient(hunting): my global_id is 274208 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704098be0 con 0x7ff704099910 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1562039736 0 0) 0x7ff6f0003570 con 0x7ff7041303d0 2024-09-15T04:04:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 >> v1:172.21.15.122:6792/0 conn(0x7ff70412ff80 legacy=0x7ff704134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 >> v1:172.21.15.122:6791/0 conn(0x7ff704099910 legacy=0x7ff70412f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 monclient: found mon.noname-e 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff704099110 con 0x7ff7041303d0 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff6fc001b30 con 0x7ff7041303d0 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 5 monclient: authenticate success, global_id 274247 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: dump: 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff6fc001d50 con 0x7ff7041303d0 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 4 set_mon_vals no callback set 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 1 -- 172.21.15.26:0/491879462 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff6fc004c70 con 0x7ff7041303d0 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6fbfff700 10 monclient: dump: 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: shutdown 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- 172.21.15.26:0/491879462 >> v1:172.21.15.26:6793/0 conn(0x7ff7041303d0 legacy=0x7ff7041382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- 172.21.15.26:0/491879462 shutdown_connections 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- 172.21.15.26:0/491879462 >> 172.21.15.26:0/491879462 conn(0x7ff704071860 msgr2=0x7ff7040a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- 172.21.15.26:0/491879462 shutdown_connections 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- 172.21.15.26:0/491879462 wait complete. 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: monmap: 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 Processor -- start 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- start start 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: init 2024-09-15T04:04:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient: _add_conns ranks=[8,4,7,1,2,3,6,5,0] 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient(hunting): picked mon.noname-e con 0x7ff704099910 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient(hunting): picked mon.noname-c con 0x7ff70412ff80 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient(hunting): picked mon.noname-i con 0x7ff7041303d0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff704097ea0 con 0x7ff70412ff80 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70406aed0 con 0x7ff7041303d0 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70406abf0 con 0x7ff704099910 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff70a4c8700 10 monclient(hunting): authenticate will time out at 2028.459555s 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff7094c6700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7ff704099910 0x7ff70409ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39372/0 (socket says 172.21.15.26:39372) 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff7094c6700 1 -- 172.21.15.26:0/1310017177 learned_addr learned my addr 172.21.15.26:0/1310017177 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff708cc5700 1 --1- 172.21.15.26:0/1310017177 >> v1:172.21.15.26:6791/0 conn(0x7ff70412ff80 0x7ff70409b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40574/0 (socket says 172.21.15.26:40574) 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794337311 0 0) 0x7ff70406abf0 con 0x7ff704099910 2024-09-15T04:04:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6f97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6f97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6f97fa700 10 monclient(hunting): my global_id is 274256 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.026+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e8003570 con 0x7ff704099910 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312812886 0 0) 0x7ff704097ea0 con 0x7ff70412ff80 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): my global_id is 205168 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff70406abf0 con 0x7ff70412ff80 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655758088 0 0) 0x7ff70406aed0 con 0x7ff7041303d0 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): my global_id is 254491 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704097ea0 con 0x7ff7041303d0 2024-09-15T04:04:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2040428392 0 0) 0x7ff6e8003570 con 0x7ff704099910 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff70406aed0 con 0x7ff704099910 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1324221000 0 0) 0x7ff70406abf0 con 0x7ff70412ff80 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6e8003570 con 0x7ff70412ff80 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff700001cd0 con 0x7ff704099910 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): dump: 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.049 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff6f4001d10 con 0x7ff70412ff80 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:44.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): dump: 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4014358752 0 0) 0x7ff704097ea0 con 0x7ff7041303d0 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff70406abf0 con 0x7ff7041303d0 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff6fc001a30 con 0x7ff7041303d0 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient(hunting): dump: 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3730631262 0 0) 0x7ff6e8003570 con 0x7ff70412ff80 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 >> v1:172.21.15.26:6793/0 conn(0x7ff704099910 legacy=0x7ff70409ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 >> v1:172.21.15.122:6792/0 conn(0x7ff7041303d0 legacy=0x7ff70409b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 monclient: found mon.c 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70409c6a0 con 0x7ff70412ff80 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 5 monclient: authenticate success, global_id 205168 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 monclient: _renew_subs 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70406dbb0 con 0x7ff70412ff80 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 monclient: _renew_subs 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff704146a00 con 0x7ff70412ff80 2024-09-15T04:04:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ff6f4001640 con 0x7ff70412ff80 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ff6f4002530 con 0x7ff70412ff80 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6faffd700 4 set_mon_vals no callback set 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 10 monclient: dump: 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ff6f4002710 con 0x7ff70412ff80 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e40050b0 con 0x7ff70412ff80 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7ff6f4043ca0 con 0x7ff70412ff80 2024-09-15T04:04:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.030+0000 7ff7094c6700 10 monclient: get_auth_request con 0x7ff6e80380a0 auth_method 0 2024-09-15T04:04:44.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.038+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ff6f4010d90 con 0x7ff70412ff80 2024-09-15T04:04:44.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.038+0000 7ff6f97fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.038+0000 7ff6f97fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:44.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.198+0000 7ff70a4c8700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.198+0000 7ff70a4c8700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.198+0000 7ff70a4c8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.198+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6e4005360 con 0x7ff70412ff80 2024-09-15T04:04:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff6f97fa700 1 -- 172.21.15.26:0/1310017177 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v904) v1 ==== 74+0+10071 (unknown 161975460 0 69917236) 0x7ff6f4014690 con 0x7ff70412ff80 2024-09-15T04:04:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff6f97fa700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff6f97fa700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:44.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:44.208 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":904,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T04:04:08.628379+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T03:44:56.976968+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-15T03:44:57.527097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":803,"length":1},{"begin":847,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T04:04:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 >> v1:172.21.15.122:6812/20895 conn(0x7ff6e80380a0 legacy=0x7ff6e803a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 10 monclient: shutdown 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 >> v1:172.21.15.26:6791/0 conn(0x7ff70412ff80 legacy=0x7ff70409b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 shutdown_connections 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 >> 172.21.15.26:0/1310017177 conn(0x7ff704071860 msgr2=0x7ff70412cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 shutdown_connections 2024-09-15T04:04:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.202+0000 7ff70a4c8700 1 -- 172.21.15.26:0/1310017177 wait complete. 2024-09-15T04:04:44.223 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T04:04:44.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient: get_monmap_and_config 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient: monmap: 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:44.321338+0000 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:44.321338+0000 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.325 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.326 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 1 Processor -- start 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 1 -- start start 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient: init 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient: _add_conns ranks=[6,1,3,7,2,0,5,4,8] 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient(hunting): picked mon.noname-d con 0x7ffae8021f20 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:44.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient(hunting): picked mon.noname-f con 0x7ffae80204a0 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient(hunting): picked mon.noname-g con 0x7ffae80be970 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae8004080 con 0x7ffae80204a0 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae8021440 con 0x7ffae80be970 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae8026b80 con 0x7ffae8021f20 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaf4896700 10 monclient(hunting): authenticate will time out at 2028.751218s 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaeed9d700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7ffae8021f20 0x7ffae8026e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58404/0 (socket says 172.21.15.26:58404) 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaeed9d700 1 -- 172.21.15.26:0/50512369 learned_addr learned my addr 172.21.15.26:0/50512369 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092408036 0 0) 0x7ffae8026b80 con 0x7ffae8021f20 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaedd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaedd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaedd9b700 10 monclient(hunting): my global_id is 195360 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.318+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffadc003570 con 0x7ffae8021f20 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447211049 0 0) 0x7ffae8004080 con 0x7ffae80204a0 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient(hunting): my global_id is 274177 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae8026b80 con 0x7ffae80204a0 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949198495 0 0) 0x7ffae8021440 con 0x7ffae80be970 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient(hunting): my global_id is 254415 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae8004080 con 0x7ffae80be970 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4184106495 0 0) 0x7ffadc003570 con 0x7ffae8021f20 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 >> v1:172.21.15.122:6790/0 conn(0x7ffae80be970 legacy=0x7ffae80c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 >> v1:172.21.15.122:6789/0 conn(0x7ffae80204a0 legacy=0x7ffae8027530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 monclient: found mon.noname-d 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae8021160 con 0x7ffae8021f20 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffad80017b0 con 0x7ffae8021f20 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: dump: 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.332 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.333 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 5 monclient: authenticate success, global_id 195360 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffad8001a90 con 0x7ffae8021f20 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 1 -- 172.21.15.26:0/50512369 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffad8001c80 con 0x7ffae8021f20 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaedd9b700 10 monclient: dump: 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 4 set_mon_vals no callback set 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 10 monclient: shutdown 2024-09-15T04:04:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 1 -- 172.21.15.26:0/50512369 >> v1:172.21.15.26:6792/0 conn(0x7ffae8021f20 legacy=0x7ffae8026e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 1 -- 172.21.15.26:0/50512369 shutdown_connections 2024-09-15T04:04:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 1 -- 172.21.15.26:0/50512369 >> 172.21.15.26:0/50512369 conn(0x7ffae802bad0 msgr2=0x7ffae802bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 1 -- 172.21.15.26:0/50512369 shutdown_connections 2024-09-15T04:04:44.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 1 -- 172.21.15.26:0/50512369 wait complete. 2024-09-15T04:04:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 10 monclient: monmap: 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.322+0000 7ffaf4896700 1 Processor -- start 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 1 -- start start 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: init 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: _add_conns ranks=[8,3,7,2,1,0,5,4,6] 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient(hunting): picked mon.noname-e con 0x7ffae80204a0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient(hunting): picked mon.noname-g con 0x7ffae8021f20 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient(hunting): picked mon.noname-i con 0x7ffae80be970 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae8020bc0 con 0x7ffae8021f20 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80208f0 con 0x7ffae80be970 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae801b2e0 con 0x7ffae80204a0 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient(hunting): authenticate will time out at 2028.757538s 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaeed9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7ffae80204a0 0x7ffae80b2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39386/0 (socket says 172.21.15.26:39386) 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaeed9d700 1 -- 172.21.15.26:0/1779260583 learned_addr learned my addr 172.21.15.26:0/1779260583 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266576984 0 0) 0x7ffae801b2e0 con 0x7ffae80204a0 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): my global_id is 274265 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad0003570 con 0x7ffae80204a0 2024-09-15T04:04:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978156596 0 0) 0x7ffae80208f0 con 0x7ffae80be970 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): my global_id is 254500 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae801b2e0 con 0x7ffae80be970 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456933739 0 0) 0x7ffae8020bc0 con 0x7ffae8021f20 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): my global_id is 254424 2024-09-15T04:04:44.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae80208f0 con 0x7ffae8021f20 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4080294944 0 0) 0x7ffad0003570 con 0x7ffae80204a0 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffae8020bc0 con 0x7ffae80204a0 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffad8001770 con 0x7ffae80204a0 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:44.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1635260165 0 0) 0x7ffae801b2e0 con 0x7ffae80be970 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffad0003570 con 0x7ffae80be970 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffaf0067200 con 0x7ffae80be970 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.348 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3939181990 0 0) 0x7ffae80208f0 con 0x7ffae8021f20 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffae801b2e0 con 0x7ffae8021f20 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.3 v1:172.21.15.122:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffae0001920 con 0x7ffae8021f20 2024-09-15T04:04:44.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.350 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 612800691 0 0) 0x7ffae8020bc0 con 0x7ffae80204a0 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 >> v1:172.21.15.122:6792/0 conn(0x7ffae80be970 legacy=0x7ffae80b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 >> v1:172.21.15.122:6790/0 conn(0x7ffae8021f20 legacy=0x7ffae80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 monclient: found mon.e 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80b43b0 con 0x7ffae80204a0 2024-09-15T04:04:44.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 5 monclient: authenticate success, global_id 274265 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: _renew_subs 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae80b5e70 con 0x7ffae80204a0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: _renew_subs 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae801bcc0 con 0x7ffae80204a0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7ffad8004f80 con 0x7ffae80204a0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7ffad8001210 con 0x7ffae80204a0 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:44.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffaed59a700 4 set_mon_vals no callback set 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.326+0000 7ffae6ffd700 10 monclient: dump: 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.353 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7ffad800fc90 con 0x7ffae80204a0 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7ffad8044070 con 0x7ffae80204a0 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffaee59c700 10 monclient: get_auth_request con 0x7ffad0038100 auth_method 0 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffaf4896700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffaf4896700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffaf4896700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.330+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad40050b0 con 0x7ffae80204a0 2024-09-15T04:04:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.338+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7ffad8015120 con 0x7ffae80204a0 2024-09-15T04:04:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.338+0000 7ffae6ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.338+0000 7ffae6ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:44.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.498+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffad4000b90 con 0x7ffad0038100 2024-09-15T04:04:44.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffae6ffd700 1 -- 172.21.15.26:0/1779260583 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53581 (unknown 2965378022 0 2981700736) 0x7ffad4000b90 con 0x7ffad0038100 2024-09-15T04:04:44.513 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 >> v1:172.21.15.122:6812/20895 conn(0x7ffad0038100 legacy=0x7ffad003a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 10 monclient: shutdown 2024-09-15T04:04:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 >> v1:172.21.15.26:6793/0 conn(0x7ffae80204a0 legacy=0x7ffae80b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 shutdown_connections 2024-09-15T04:04:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 >> 172.21.15.26:0/1779260583 conn(0x7ffae802bad0 msgr2=0x7ffae8007840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 shutdown_connections 2024-09-15T04:04:44.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.506+0000 7ffaf4896700 1 -- 172.21.15.26:0/1779260583 wait complete. 2024-09-15T04:04:44.516 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T04:04:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":1493,"stamp":"2024-09-15T04:04:43.798473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1857728,"kb_used_data":23680,"kb_used_omap":9,"kb_used_meta":1833974,"kb_avail":564373312,"statfs":{"total":579820584960,"available":577918271488,"internally_reserved":0,"allocated":24248320,"data_stored":13776778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1877989392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003477"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633767+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T04:04:06.633767+0000","last_peered":"2024-09-15T04:04:06.633767+0000","last_clean":"2024-09-15T04:04:06.633767+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T04:04:06.633767+0000","last_undegraded":"2024-09-15T04:04:06.633767+0000","last_fullsized":"2024-09-15T04:04:06.633767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638127+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T04:04:06.638127+0000","last_peered":"2024-09-15T04:04:06.638127+0000","last_clean":"2024-09-15T04:04:06.638127+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T04:04:06.638127+0000","last_undegraded":"2024-09-15T04:04:06.638127+0000","last_fullsized":"2024-09-15T04:04:06.638127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":884,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634431+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T04:04:06.634431+0000","last_peered":"2024-09-15T04:04:06.634431+0000","last_clean":"2024-09-15T04:04:06.634431+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T04:04:06.634431+0000","last_undegraded":"2024-09-15T04:04:06.634431+0000","last_fullsized":"2024-09-15T04:04:06.634431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":893,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631849+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T04:04:06.631849+0000","last_peered":"2024-09-15T04:04:06.631849+0000","last_clean":"2024-09-15T04:04:06.631849+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T04:04:06.631849+0000","last_undegraded":"2024-09-15T04:04:06.631849+0000","last_fullsized":"2024-09-15T04:04:06.631849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":975,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633590+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T04:04:06.633590+0000","last_peered":"2024-09-15T04:04:06.633590+0000","last_clean":"2024-09-15T04:04:06.633590+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T04:04:06.633590+0000","last_undegraded":"2024-09-15T04:04:06.633590+0000","last_fullsized":"2024-09-15T04:04:06.633590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":892,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631860+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T04:04:06.631860+0000","last_peered":"2024-09-15T04:04:06.631860+0000","last_clean":"2024-09-15T04:04:06.631860+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T04:04:06.631860+0000","last_undegraded":"2024-09-15T04:04:06.631860+0000","last_fullsized":"2024-09-15T04:04:06.631860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":882,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634430+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T04:04:06.634430+0000","last_peered":"2024-09-15T04:04:06.634430+0000","last_clean":"2024-09-15T04:04:06.634430+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T04:04:06.634430+0000","last_undegraded":"2024-09-15T04:04:06.634430+0000","last_fullsized":"2024-09-15T04:04:06.634430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634028+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T04:04:06.634028+0000","last_peered":"2024-09-15T04:04:06.634028+0000","last_clean":"2024-09-15T04:04:06.634028+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T04:04:06.634028+0000","last_undegraded":"2024-09-15T04:04:06.634028+0000","last_fullsized":"2024-09-15T04:04:06.634028+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638384+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T04:04:06.638384+0000","last_peered":"2024-09-15T04:04:06.638384+0000","last_clean":"2024-09-15T04:04:06.638384+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T04:04:06.638384+0000","last_undegraded":"2024-09-15T04:04:06.638384+0000","last_fullsized":"2024-09-15T04:04:06.638384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640496,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.60099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.224,"15min":5.1609999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.374,"15min":0.372},"max":{"1min":1.0329999999999999,"5min":1.2609999999999999,"15min":3.5099999999999998},"last":0.65500000000000003}]},{"osd":1,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.64400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1799999999999999,"5min":1.6619999999999999,"15min":2.2890000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.44500000000000001,"15min":0.41199999999999998},"max":{"1min":1.165,"5min":1.264,"15min":1.341},"last":0.75800000000000001}]},{"osd":2,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.51900000000000002,"15min":0.46500000000000002},"max":{"1min":1.2050000000000001,"5min":9.8149999999999995,"15min":9.8149999999999995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.53500000000000003,"15min":0.36699999999999999},"max":{"1min":1.282,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.84999999999999998}]},{"osd":3,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.27200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3640000000000001,"15min":5.2539999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.60899999999999999,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":1.1679999999999999,"5min":1.391,"15min":2.3700000000000001},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.67100000000000004,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":1.0780000000000001,"5min":1.4279999999999999,"15min":5.2830000000000004},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.3899999999999999,"15min":5.3650000000000002},"last":0.88500000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640496,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309452,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062388,"statfs":{"total":96636764160,"available":96319885312,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81399999999999995,"15min":0.80300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.29499999999999998},"max":{"1min":1.0389999999999999,"5min":1.1679999999999999,"15min":1.3420000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.40300000000000002,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.401},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.86099999999999999,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41999999999999998},"max":{"1min":1.0860000000000001,"5min":1.327,"15min":1.8100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.69499999999999995,"5min":0.54900000000000004,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.228,"15min":2.2050000000000001},"last":1.018}]},{"osd":2,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.38600000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":3.2440000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.505,"15min":0.45500000000000002},"max":{"1min":1.105,"5min":1.226,"15min":2.5059999999999998},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.96199999999999997,"5min":1.1719999999999999,"15min":1.335},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.5409999999999999},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.308},"max":{"1min":1.095,"5min":1.1870000000000001,"15min":1.474},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.315},"max":{"1min":1.2230000000000001,"5min":1.242,"15min":1.26},"last":0.99299999999999999}]}]},{"osd":3,"up_from":11,"seq":47244640497,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318288,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94053552,"statfs":{"total":96636764160,"available":96310837248,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.61299999999999999,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.2749999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.85899999999999999},"min":{"1min":0.66700000000000004,"5min":0.42199999999999999,"15min":0.39800000000000002},"max":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3680000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.81200000000000006,"5min":0.52000000000000002,"15min":0.46999999999999997},"max":{"1min":1.0760000000000001,"5min":1.954,"15min":1.954},"last":0.95599999999999996}]},{"osd":2,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.38},"max":{"1min":1.167,"5min":1.2709999999999999,"15min":1.3799999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.68100000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.2150000000000001,"5min":1.2310000000000001,"15min":1.4330000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.503,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2490000000000001,"5min":1.337,"15min":1.337},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.40600000000000003,"15min":0.32300000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.5740000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.86399999999999999},"min":{"1min":0.78800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.621},"last":0.86599999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640496,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.40400000000000003,"5min":0.27700000000000002,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":6.3019999999999996},"last":0.505},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.01,"5min":1.113,"15min":6.3550000000000004},"last":0.69399999999999995}]},{"osd":1,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.63500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.1120000000000001,"5min":1.121,"15min":1.952},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":2.0019999999999998},"last":0.77600000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.82499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.976},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.75700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1679999999999999,"5min":1.431,"15min":1.5069999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.68799999999999994,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1160000000000001,"5min":2.391,"15min":2.391},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0429999999999999,"5min":2.419,"15min":2.419},"last":0.93400000000000005}]},{"osd":5,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.68700000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.4810000000000001,"15min":2.2330000000000001},"last":0.753},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.91700000000000004},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.51100000000000001},"max":{"1min":1.341,"5min":1.5449999999999999,"15min":1.637},"last":0.876}]}]},{"osd":1,"up_from":11,"seq":47244640498,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305808,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066032,"statfs":{"total":96636764160,"available":96323616768,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.59699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.883},"last":0.623},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":1.976},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64900000000000002,"15min":0.66400000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85699999999999998,"5min":1.5249999999999999,"15min":2.536},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.012,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.57099999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0820000000000001,"5min":1.2509999999999999,"15min":1.288},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0149999999999999,"15min":0.95499999999999996},"min":{"1min":0.65200000000000002,"5min":0.51900000000000002,"15min":0.50800000000000001},"max":{"1min":1.1970000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97199999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.024,"5min":1.677,"15min":1.677},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.68600000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2070000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.91500000000000004},"min":{"1min":0.628,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.115,"5min":1.617,"15min":1.617},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.278,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.022}]}]},{"osd":0,"up_from":11,"seq":47244640497,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305292,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066548,"statfs":{"total":96636764160,"available":96324145152,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.79700000000000004,"15min":18.481000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.247,"15min":0.247},"max":{"1min":0.79600000000000004,"5min":2.222,"15min":19.239000000000001},"last":1.0469999999999999}]},{"osd":2,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.52900000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81399999999999995,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.84099999999999997},"min":{"1min":0.68300000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.079,"5min":3.556,"15min":3.556},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85299999999999998,"15min":0.86399999999999999},"min":{"1min":0.70499999999999996,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":2.395,"15min":2.395},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.91900000000000004,"15min":0.90000000000000002},"min":{"1min":0.71699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98299999999999998,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.84799999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.55600000000000005,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.95399999999999996},"min":{"1min":0.77500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0840000000000001,"5min":2.923,"15min":2.923},"last":0.80000000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.873,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.56399999999999995,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.21,"15min":1.2989999999999999},"last":0.752},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94599999999999995,"15min":0.98299999999999998},"min":{"1min":0.81299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.282,"15min":1.6200000000000001},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T04:04:44.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.646+0000 7fc28da8f700 10 monclient: get_monmap_and_config 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.646+0000 7fc28da8f700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.646+0000 7fc28da8f700 10 monclient: monmap: 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:44.650744+0000 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:44.650744+0000 2024-09-15T04:04:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.655 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.646+0000 7fc28da8f700 1 Processor -- start 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.646+0000 7fc28da8f700 1 -- start start 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.646+0000 7fc28da8f700 10 monclient: init 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: _add_conns ranks=[0,6,5,4,1,3,8,7,2] 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient(hunting): picked mon.noname-a con 0x7fc288099910 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient(hunting): picked mon.noname-d con 0x7fc28812ff80 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient(hunting): picked mon.noname-h con 0x7fc2881303d0 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28804ca10 con 0x7fc288099910 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc288098be0 con 0x7fc2881303d0 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc288099460 con 0x7fc28812ff80 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient(hunting): authenticate will time out at 2029.080459s 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28ca8d700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc288099910 0x7fc28812f870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49296/0 (socket says 172.21.15.26:49296) 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28ca8d700 1 -- 172.21.15.26:0/2786267838 learned_addr learned my addr 172.21.15.26:0/2786267838 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462967293 0 0) 0x7fc28804ca10 con 0x7fc288099910 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient(hunting): my global_id is 195273 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc274003570 con 0x7fc288099910 2024-09-15T04:04:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922711348 0 0) 0x7fc288099460 con 0x7fc28812ff80 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient(hunting): my global_id is 195369 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28804ca10 con 0x7fc28812ff80 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909976943 0 0) 0x7fc274003570 con 0x7fc288099910 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 >> v1:172.21.15.26:6792/0 conn(0x7fc28812ff80 legacy=0x7fc288134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 >> v1:172.21.15.122:6791/0 conn(0x7fc2881303d0 legacy=0x7fc2881382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 monclient: found mon.noname-a 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc288099110 con 0x7fc288099910 2024-09-15T04:04:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 5 monclient: authenticate success, global_id 195273 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc278001b30 con 0x7fc288099910 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: dump: 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc278001da0 con 0x7fc288099910 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 1 -- 172.21.15.26:0/2786267838 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc278004fb0 con 0x7fc288099910 2024-09-15T04:04:44.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc2877fe700 10 monclient: dump: 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 4 set_mon_vals no callback set 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: shutdown 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- 172.21.15.26:0/2786267838 >> v1:172.21.15.26:6789/0 conn(0x7fc288099910 legacy=0x7fc28812f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- 172.21.15.26:0/2786267838 shutdown_connections 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- 172.21.15.26:0/2786267838 >> 172.21.15.26:0/2786267838 conn(0x7fc288071860 msgr2=0x7fc2880a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- 172.21.15.26:0/2786267838 shutdown_connections 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- 172.21.15.26:0/2786267838 wait complete. 2024-09-15T04:04:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: monmap: 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.667 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 Processor -- start 2024-09-15T04:04:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 1 -- start start 2024-09-15T04:04:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.650+0000 7fc28da8f700 10 monclient: init 2024-09-15T04:04:44.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _add_conns ranks=[5,0,8,1,4,3,6,7,2] 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient(hunting): picked mon.noname-h con 0x7fc288099910 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient(hunting): picked mon.noname-a con 0x7fc28812ff80 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient(hunting): picked mon.noname-e con 0x7fc2881303d0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc288119c30 con 0x7fc28812ff80 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc288064220 con 0x7fc288099910 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc288063ba0 con 0x7fc2881303d0 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc287fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc28812ff80 0x7fc288144fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49304/0 (socket says 172.21.15.26:49304) 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc287fff700 1 -- 172.21.15.26:0/4026905790 learned_addr learned my addr 172.21.15.26:0/4026905790 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient(hunting): authenticate will time out at 2029.084829s 2024-09-15T04:04:44.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092265542 0 0) 0x7fc288119c30 con 0x7fc28812ff80 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): my global_id is 195282 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc268003570 con 0x7fc28812ff80 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41223396 0 0) 0x7fc288063ba0 con 0x7fc2881303d0 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): my global_id is 274274 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc288119c30 con 0x7fc2881303d0 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3642346681 0 0) 0x7fc268003570 con 0x7fc28812ff80 2024-09-15T04:04:44.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc288063ba0 con 0x7fc28812ff80 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc270001ae0 con 0x7fc28812ff80 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3540113993 0 0) 0x7fc288119c30 con 0x7fc2881303d0 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc268003570 con 0x7fc2881303d0 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc280001850 con 0x7fc2881303d0 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 225961582 0 0) 0x7fc288063ba0 con 0x7fc28812ff80 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 >> v1:172.21.15.26:6793/0 conn(0x7fc2881303d0 legacy=0x7fc2881486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 >> v1:172.21.15.122:6791/0 conn(0x7fc288099910 legacy=0x7fc288140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 monclient: found mon.a 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc288119950 con 0x7fc28812ff80 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 5 monclient: authenticate success, global_id 195282 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _renew_subs 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28806d0a0 con 0x7fc28812ff80 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _renew_subs 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880635a0 con 0x7fc28812ff80 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fc270002070 con 0x7fc28812ff80 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fc270002520 con 0x7fc28812ff80 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc286ffd700 4 set_mon_vals no callback set 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 10 monclient: dump: 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fc270002700 con 0x7fc28812ff80 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c0050b0 con 0x7fc28812ff80 2024-09-15T04:04:44.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fc270043e20 con 0x7fc28812ff80 2024-09-15T04:04:44.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.654+0000 7fc28ca8d700 10 monclient: get_auth_request con 0x7fc268037b30 auth_method 0 2024-09-15T04:04:44.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.658+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fc270014d50 con 0x7fc28812ff80 2024-09-15T04:04:44.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.662+0000 7fc284ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.662+0000 7fc284ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:44.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.818+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc26c000b90 con 0x7fc268037b30 2024-09-15T04:04:44.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc284ff9700 1 -- 172.21.15.26:0/4026905790 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53581 (unknown 2965378022 0 2981700736) 0x7fc26c000b90 con 0x7fc268037b30 2024-09-15T04:04:44.832 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:44.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 >> v1:172.21.15.122:6812/20895 conn(0x7fc268037b30 legacy=0x7fc268039fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 10 monclient: shutdown 2024-09-15T04:04:44.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 >> v1:172.21.15.26:6789/0 conn(0x7fc28812ff80 legacy=0x7fc288144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 shutdown_connections 2024-09-15T04:04:44.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 >> 172.21.15.26:0/4026905790 conn(0x7fc288071860 msgr2=0x7fc2880a0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 shutdown_connections 2024-09-15T04:04:44.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.826+0000 7fc28da8f700 1 -- 172.21.15.26:0/4026905790 wait complete. 2024-09-15T04:04:44.836 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T04:04:44.846 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":1493,"stamp":"2024-09-15T04:04:43.798473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1857728,"kb_used_data":23680,"kb_used_omap":9,"kb_used_meta":1833974,"kb_avail":564373312,"statfs":{"total":579820584960,"available":577918271488,"internally_reserved":0,"allocated":24248320,"data_stored":13776778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1877989392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003477"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633767+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T04:04:06.633767+0000","last_peered":"2024-09-15T04:04:06.633767+0000","last_clean":"2024-09-15T04:04:06.633767+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T04:04:06.633767+0000","last_undegraded":"2024-09-15T04:04:06.633767+0000","last_fullsized":"2024-09-15T04:04:06.633767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638127+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T04:04:06.638127+0000","last_peered":"2024-09-15T04:04:06.638127+0000","last_clean":"2024-09-15T04:04:06.638127+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T04:04:06.638127+0000","last_undegraded":"2024-09-15T04:04:06.638127+0000","last_fullsized":"2024-09-15T04:04:06.638127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":884,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634431+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T04:04:06.634431+0000","last_peered":"2024-09-15T04:04:06.634431+0000","last_clean":"2024-09-15T04:04:06.634431+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T04:04:06.634431+0000","last_undegraded":"2024-09-15T04:04:06.634431+0000","last_fullsized":"2024-09-15T04:04:06.634431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":893,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631849+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T04:04:06.631849+0000","last_peered":"2024-09-15T04:04:06.631849+0000","last_clean":"2024-09-15T04:04:06.631849+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T04:04:06.631849+0000","last_undegraded":"2024-09-15T04:04:06.631849+0000","last_fullsized":"2024-09-15T04:04:06.631849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":975,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633590+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T04:04:06.633590+0000","last_peered":"2024-09-15T04:04:06.633590+0000","last_clean":"2024-09-15T04:04:06.633590+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T04:04:06.633590+0000","last_undegraded":"2024-09-15T04:04:06.633590+0000","last_fullsized":"2024-09-15T04:04:06.633590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":892,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631860+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T04:04:06.631860+0000","last_peered":"2024-09-15T04:04:06.631860+0000","last_clean":"2024-09-15T04:04:06.631860+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T04:04:06.631860+0000","last_undegraded":"2024-09-15T04:04:06.631860+0000","last_fullsized":"2024-09-15T04:04:06.631860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":882,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634430+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T04:04:06.634430+0000","last_peered":"2024-09-15T04:04:06.634430+0000","last_clean":"2024-09-15T04:04:06.634430+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T04:04:06.634430+0000","last_undegraded":"2024-09-15T04:04:06.634430+0000","last_fullsized":"2024-09-15T04:04:06.634430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634028+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T04:04:06.634028+0000","last_peered":"2024-09-15T04:04:06.634028+0000","last_clean":"2024-09-15T04:04:06.634028+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T04:04:06.634028+0000","last_undegraded":"2024-09-15T04:04:06.634028+0000","last_fullsized":"2024-09-15T04:04:06.634028+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638384+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T04:04:06.638384+0000","last_peered":"2024-09-15T04:04:06.638384+0000","last_clean":"2024-09-15T04:04:06.638384+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T04:04:06.638384+0000","last_undegraded":"2024-09-15T04:04:06.638384+0000","last_fullsized":"2024-09-15T04:04:06.638384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640496,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.60099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.224,"15min":5.1609999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.374,"15min":0.372},"max":{"1min":1.0329999999999999,"5min":1.2609999999999999,"15min":3.5099999999999998},"last":0.65500000000000003}]},{"osd":1,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.64400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1799999999999999,"5min":1.6619999999999999,"15min":2.2890000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.44500000000000001,"15min":0.41199999999999998},"max":{"1min":1.165,"5min":1.264,"15min":1.341},"last":0.75800000000000001}]},{"osd":2,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.51900000000000002,"15min":0.46500000000000002},"max":{"1min":1.2050000000000001,"5min":9.8149999999999995,"15min":9.8149999999999995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.53500000000000003,"15min":0.36699999999999999},"max":{"1min":1.282,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.84999999999999998}]},{"osd":3,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.27200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3640000000000001,"15min":5.2539999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.60899999999999999,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":1.1679999999999999,"5min":1.391,"15min":2.3700000000000001},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.67100000000000004,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":1.0780000000000001,"5min":1.4279999999999999,"15min":5.2830000000000004},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.3899999999999999,"15min":5.3650000000000002},"last":0.88500000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640496,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309452,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062388,"statfs":{"total":96636764160,"available":96319885312,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81399999999999995,"15min":0.80300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.29499999999999998},"max":{"1min":1.0389999999999999,"5min":1.1679999999999999,"15min":1.3420000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.40300000000000002,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.401},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.86099999999999999,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41999999999999998},"max":{"1min":1.0860000000000001,"5min":1.327,"15min":1.8100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.69499999999999995,"5min":0.54900000000000004,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.228,"15min":2.2050000000000001},"last":1.018}]},{"osd":2,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.38600000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":3.2440000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.505,"15min":0.45500000000000002},"max":{"1min":1.105,"5min":1.226,"15min":2.5059999999999998},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.96199999999999997,"5min":1.1719999999999999,"15min":1.335},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.5409999999999999},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.308},"max":{"1min":1.095,"5min":1.1870000000000001,"15min":1.474},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.315},"max":{"1min":1.2230000000000001,"5min":1.242,"15min":1.26},"last":0.99299999999999999}]}]},{"osd":3,"up_from":11,"seq":47244640497,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318288,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94053552,"statfs":{"total":96636764160,"available":96310837248,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.61299999999999999,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.2749999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.85899999999999999},"min":{"1min":0.66700000000000004,"5min":0.42199999999999999,"15min":0.39800000000000002},"max":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3680000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.81200000000000006,"5min":0.52000000000000002,"15min":0.46999999999999997},"max":{"1min":1.0760000000000001,"5min":1.954,"15min":1.954},"last":0.95599999999999996}]},{"osd":2,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.38},"max":{"1min":1.167,"5min":1.2709999999999999,"15min":1.3799999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.68100000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.2150000000000001,"5min":1.2310000000000001,"15min":1.4330000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.503,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2490000000000001,"5min":1.337,"15min":1.337},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.40600000000000003,"15min":0.32300000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.5740000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.86399999999999999},"min":{"1min":0.78800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.621},"last":0.86599999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640496,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.40400000000000003,"5min":0.27700000000000002,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":6.3019999999999996},"last":0.505},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.01,"5min":1.113,"15min":6.3550000000000004},"last":0.69399999999999995}]},{"osd":1,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.63500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.1120000000000001,"5min":1.121,"15min":1.952},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":2.0019999999999998},"last":0.77600000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.82499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.976},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.75700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1679999999999999,"5min":1.431,"15min":1.5069999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.68799999999999994,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1160000000000001,"5min":2.391,"15min":2.391},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0429999999999999,"5min":2.419,"15min":2.419},"last":0.93400000000000005}]},{"osd":5,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.68700000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.4810000000000001,"15min":2.2330000000000001},"last":0.753},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.91700000000000004},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.51100000000000001},"max":{"1min":1.341,"5min":1.5449999999999999,"15min":1.637},"last":0.876}]}]},{"osd":1,"up_from":11,"seq":47244640498,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305808,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066032,"statfs":{"total":96636764160,"available":96323616768,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.59699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.883},"last":0.623},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":1.976},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64900000000000002,"15min":0.66400000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85699999999999998,"5min":1.5249999999999999,"15min":2.536},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.012,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.57099999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0820000000000001,"5min":1.2509999999999999,"15min":1.288},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0149999999999999,"15min":0.95499999999999996},"min":{"1min":0.65200000000000002,"5min":0.51900000000000002,"15min":0.50800000000000001},"max":{"1min":1.1970000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97199999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.024,"5min":1.677,"15min":1.677},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.68600000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2070000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.91500000000000004},"min":{"1min":0.628,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.115,"5min":1.617,"15min":1.617},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.278,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.022}]}]},{"osd":0,"up_from":11,"seq":47244640497,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305292,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066548,"statfs":{"total":96636764160,"available":96324145152,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.79700000000000004,"15min":18.481000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.247,"15min":0.247},"max":{"1min":0.79600000000000004,"5min":2.222,"15min":19.239000000000001},"last":1.0469999999999999}]},{"osd":2,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.52900000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81399999999999995,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.84099999999999997},"min":{"1min":0.68300000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.079,"5min":3.556,"15min":3.556},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85299999999999998,"15min":0.86399999999999999},"min":{"1min":0.70499999999999996,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":2.395,"15min":2.395},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.91900000000000004,"15min":0.90000000000000002},"min":{"1min":0.71699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98299999999999998,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.84799999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.55600000000000005,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.95399999999999996},"min":{"1min":0.77500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0840000000000001,"5min":2.923,"15min":2.923},"last":0.80000000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.873,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.56399999999999995,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.21,"15min":1.2989999999999999},"last":0.752},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94599999999999995,"15min":0.98299999999999998},"min":{"1min":0.81299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.282,"15min":1.6200000000000001},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T04:04:44.848 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T04:04:44.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T04:04:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: get_monmap_and_config 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: monmap: 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:44.953254+0000 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:44.953254+0000 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.958 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 1 Processor -- start 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 1 -- start start 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: init 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: _add_conns ranks=[0,8,4,5,3,7,1,2,6] 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient(hunting): picked mon.noname-a con 0x7fbce009b6e0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient(hunting): picked mon.noname-e con 0x7fbce006e160 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient(hunting): picked mon.noname-c con 0x7fbce0136d30 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce004ca10 con 0x7fbce009b6e0 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce0098880 con 0x7fbce0136d30 2024-09-15T04:04:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce006e550 con 0x7fbce006e160 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient(hunting): authenticate will time out at 2029.382593s 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce49b8700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fbce006e160 0x7fbce0098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39400/0 (socket says 172.21.15.26:39400) 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce51b9700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbce009b6e0 0x7fbce0097a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49318/0 (socket says 172.21.15.26:49318) 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce49b8700 1 -- 172.21.15.26:0/412753418 learned_addr learned my addr 172.21.15.26:0/412753418 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035544122 0 0) 0x7fbce004ca10 con 0x7fbce009b6e0 2024-09-15T04:04:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): my global_id is 195291 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbccc003570 con 0x7fbce009b6e0 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008631359 0 0) 0x7fbce0098880 con 0x7fbce0136d30 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): my global_id is 205177 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce004ca10 con 0x7fbce0136d30 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781583611 0 0) 0x7fbce006e550 con 0x7fbce006e160 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient(hunting): my global_id is 274283 2024-09-15T04:04:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce0098880 con 0x7fbce006e160 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 784676601 0 0) 0x7fbccc003570 con 0x7fbce009b6e0 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 >> v1:172.21.15.26:6793/0 conn(0x7fbce006e160 legacy=0x7fbce0098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 >> v1:172.21.15.26:6791/0 conn(0x7fbce0136d30 legacy=0x7fbce0139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 monclient: found mon.noname-a 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce009d2e0 con 0x7fbce009b6e0 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbcd00017b0 con 0x7fbce009b6e0 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 5 monclient: authenticate success, global_id 195291 2024-09-15T04:04:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: dump: 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.969 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbcd00019d0 con 0x7fbce009b6e0 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 1 -- 172.21.15.26:0/412753418 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbcd0004830 con 0x7fbce009b6e0 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbce61bb700 4 set_mon_vals no callback set 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.950+0000 7fbcd7fff700 10 monclient: dump: 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: shutdown 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/412753418 >> v1:172.21.15.26:6789/0 conn(0x7fbce009b6e0 legacy=0x7fbce0097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/412753418 shutdown_connections 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/412753418 >> 172.21.15.26:0/412753418 conn(0x7fbce0063d90 msgr2=0x7fbce00a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/412753418 shutdown_connections 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/412753418 wait complete. 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: build_initial_monmap 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: monmap: 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 Processor -- start 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- start start 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: init 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: _add_conns ranks=[3,0,6,5,1,7,4,2,8] 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient(hunting): picked mon.noname-g con 0x7fbce006e160 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient(hunting): picked mon.noname-a con 0x7fbce0136d30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient(hunting): picked mon.noname-d con 0x7fbce006d6c0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce009cbe0 con 0x7fbce0136d30 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce009c810 con 0x7fbce006e160 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce011a670 con 0x7fbce006d6c0 2024-09-15T04:04:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient(hunting): authenticate will time out at 2029.386346s 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce59ba700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fbce006d6c0 0x7fbce014a200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58428/0 (socket says 172.21.15.26:58428) 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce59ba700 1 -- 172.21.15.26:0/1373070777 learned_addr learned my addr 172.21.15.26:0/1373070777 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce49b8700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbce0136d30 0x7fbce006cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49322/0 (socket says 172.21.15.26:49322) 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471248429 0 0) 0x7fbce009cbe0 con 0x7fbce0136d30 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): my global_id is 195300 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc4003570 con 0x7fbce0136d30 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345033660 0 0) 0x7fbce011a670 con 0x7fbce006d6c0 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): my global_id is 195378 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce009cbe0 con 0x7fbce006d6c0 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217886596 0 0) 0x7fbce009c810 con 0x7fbce006e160 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): my global_id is 254433 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce011a670 con 0x7fbce006e160 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3589244765 0 0) 0x7fbcc4003570 con 0x7fbce0136d30 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbce009c810 con 0x7fbce0136d30 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbcdc001a60 con 0x7fbce0136d30 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): dump: 2024-09-15T04:04:44.979 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.980 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2701250111 0 0) 0x7fbce009cbe0 con 0x7fbce006d6c0 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbcc4003570 con 0x7fbce006d6c0 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbcd8001d00 con 0x7fbce006d6c0 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient(hunting): dump: 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.981 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.982 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 916464550 0 0) 0x7fbce009c810 con 0x7fbce0136d30 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 >> v1:172.21.15.26:6792/0 conn(0x7fbce006d6c0 legacy=0x7fbce014a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 >> v1:172.21.15.122:6790/0 conn(0x7fbce006e160 legacy=0x7fbce006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient: _finish_hunting 0 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 monclient: found mon.a 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce011a390 con 0x7fbce0136d30 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient: _finish_auth 0 2024-09-15T04:04:44.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 5 monclient: authenticate success, global_id 195300 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: _renew_subs 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce01188f0 con 0x7fbce0136d30 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: _renew_subs 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce012e460 con 0x7fbce0136d30 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fbcdc001cd0 con 0x7fbce0136d30 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fbcdc004ee0 con 0x7fbce0136d30 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd77fe700 4 set_mon_vals no callback set 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.954+0000 7fbcd57fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:44.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbcd57fa700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbcd57fa700 10 monclient: dump: 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbce61bb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbce61bb700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbce61bb700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc00050b0 con 0x7fbce0136d30 2024-09-15T04:04:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fbcdc00f920 con 0x7fbce0136d30 2024-09-15T04:04:44.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fbcdc043dd0 con 0x7fbce0136d30 2024-09-15T04:04:44.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.958+0000 7fbce51b9700 10 monclient: get_auth_request con 0x7fbcc4037bd0 auth_method 0 2024-09-15T04:04:44.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.962+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fbcdc014940 con 0x7fbce0136d30 2024-09-15T04:04:44.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.962+0000 7fbcd57fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:44.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:44.962+0000 7fbcd57fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:45.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.126+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcc0000b90 con 0x7fbcc4037bd0 2024-09-15T04:04:45.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.130+0000 7fbcd57fa700 1 -- 172.21.15.26:0/1373070777 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53581 (unknown 2965378022 0 2981700736) 0x7fbcc0000b90 con 0x7fbcc4037bd0 2024-09-15T04:04:45.138 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 >> v1:172.21.15.122:6812/20895 conn(0x7fbcc4037bd0 legacy=0x7fbcc403a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 10 monclient: shutdown 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 >> v1:172.21.15.26:6789/0 conn(0x7fbce0136d30 legacy=0x7fbce006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 shutdown_connections 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 >> 172.21.15.26:0/1373070777 conn(0x7fbce0063d90 msgr2=0x7fbce009ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:45.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 shutdown_connections 2024-09-15T04:04:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.134+0000 7fbce61bb700 1 -- 172.21.15.26:0/1373070777 wait complete. 2024-09-15T04:04:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T04:04:45.152 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":1493,"stamp":"2024-09-15T04:04:43.798473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1857728,"kb_used_data":23680,"kb_used_omap":9,"kb_used_meta":1833974,"kb_avail":564373312,"statfs":{"total":579820584960,"available":577918271488,"internally_reserved":0,"allocated":24248320,"data_stored":13776778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1877989392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003477"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633767+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T04:04:06.633767+0000","last_peered":"2024-09-15T04:04:06.633767+0000","last_clean":"2024-09-15T04:04:06.633767+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T04:04:06.633767+0000","last_undegraded":"2024-09-15T04:04:06.633767+0000","last_fullsized":"2024-09-15T04:04:06.633767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638127+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T04:04:06.638127+0000","last_peered":"2024-09-15T04:04:06.638127+0000","last_clean":"2024-09-15T04:04:06.638127+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T04:04:06.638127+0000","last_undegraded":"2024-09-15T04:04:06.638127+0000","last_fullsized":"2024-09-15T04:04:06.638127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":884,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634431+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T04:04:06.634431+0000","last_peered":"2024-09-15T04:04:06.634431+0000","last_clean":"2024-09-15T04:04:06.634431+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T04:04:06.634431+0000","last_undegraded":"2024-09-15T04:04:06.634431+0000","last_fullsized":"2024-09-15T04:04:06.634431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":893,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631849+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T04:04:06.631849+0000","last_peered":"2024-09-15T04:04:06.631849+0000","last_clean":"2024-09-15T04:04:06.631849+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T04:04:06.631849+0000","last_undegraded":"2024-09-15T04:04:06.631849+0000","last_fullsized":"2024-09-15T04:04:06.631849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":975,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633590+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T04:04:06.633590+0000","last_peered":"2024-09-15T04:04:06.633590+0000","last_clean":"2024-09-15T04:04:06.633590+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T04:04:06.633590+0000","last_undegraded":"2024-09-15T04:04:06.633590+0000","last_fullsized":"2024-09-15T04:04:06.633590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":892,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631860+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T04:04:06.631860+0000","last_peered":"2024-09-15T04:04:06.631860+0000","last_clean":"2024-09-15T04:04:06.631860+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T04:04:06.631860+0000","last_undegraded":"2024-09-15T04:04:06.631860+0000","last_fullsized":"2024-09-15T04:04:06.631860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":882,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634430+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T04:04:06.634430+0000","last_peered":"2024-09-15T04:04:06.634430+0000","last_clean":"2024-09-15T04:04:06.634430+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T04:04:06.634430+0000","last_undegraded":"2024-09-15T04:04:06.634430+0000","last_fullsized":"2024-09-15T04:04:06.634430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634028+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T04:04:06.634028+0000","last_peered":"2024-09-15T04:04:06.634028+0000","last_clean":"2024-09-15T04:04:06.634028+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T04:04:06.634028+0000","last_undegraded":"2024-09-15T04:04:06.634028+0000","last_fullsized":"2024-09-15T04:04:06.634028+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638384+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T04:04:06.638384+0000","last_peered":"2024-09-15T04:04:06.638384+0000","last_clean":"2024-09-15T04:04:06.638384+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T04:04:06.638384+0000","last_undegraded":"2024-09-15T04:04:06.638384+0000","last_fullsized":"2024-09-15T04:04:06.638384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640496,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.60099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.224,"15min":5.1609999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.374,"15min":0.372},"max":{"1min":1.0329999999999999,"5min":1.2609999999999999,"15min":3.5099999999999998},"last":0.65500000000000003}]},{"osd":1,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.64400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1799999999999999,"5min":1.6619999999999999,"15min":2.2890000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.44500000000000001,"15min":0.41199999999999998},"max":{"1min":1.165,"5min":1.264,"15min":1.341},"last":0.75800000000000001}]},{"osd":2,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.51900000000000002,"15min":0.46500000000000002},"max":{"1min":1.2050000000000001,"5min":9.8149999999999995,"15min":9.8149999999999995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.53500000000000003,"15min":0.36699999999999999},"max":{"1min":1.282,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.84999999999999998}]},{"osd":3,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.27200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3640000000000001,"15min":5.2539999999999996},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.60899999999999999,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":1.1679999999999999,"5min":1.391,"15min":2.3700000000000001},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.67100000000000004,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":1.0780000000000001,"5min":1.4279999999999999,"15min":5.2830000000000004},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.3899999999999999,"15min":5.3650000000000002},"last":0.88500000000000001}]}]},{"osd":4,"up_from":11,"seq":47244640496,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309452,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062388,"statfs":{"total":96636764160,"available":96319885312,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81399999999999995,"15min":0.80300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.29499999999999998},"max":{"1min":1.0389999999999999,"5min":1.1679999999999999,"15min":1.3420000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.40300000000000002,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.401},"last":0.67700000000000005}]},{"osd":1,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.86099999999999999,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41999999999999998},"max":{"1min":1.0860000000000001,"5min":1.327,"15min":1.8100000000000001},"last":0.752},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.69499999999999995,"5min":0.54900000000000004,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.228,"15min":2.2050000000000001},"last":1.018}]},{"osd":2,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.38600000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":3.2440000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.505,"15min":0.45500000000000002},"max":{"1min":1.105,"5min":1.226,"15min":2.5059999999999998},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.96199999999999997,"5min":1.1719999999999999,"15min":1.335},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.5409999999999999},"last":0.92000000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.308},"max":{"1min":1.095,"5min":1.1870000000000001,"15min":1.474},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.315},"max":{"1min":1.2230000000000001,"5min":1.242,"15min":1.26},"last":0.99299999999999999}]}]},{"osd":3,"up_from":11,"seq":47244640497,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318288,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94053552,"statfs":{"total":96636764160,"available":96310837248,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.61299999999999999,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.2749999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.85899999999999999},"min":{"1min":0.66700000000000004,"5min":0.42199999999999999,"15min":0.39800000000000002},"max":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3680000000000001},"last":0.89900000000000002}]},{"osd":1,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.81200000000000006,"5min":0.52000000000000002,"15min":0.46999999999999997},"max":{"1min":1.0760000000000001,"5min":1.954,"15min":1.954},"last":0.95599999999999996}]},{"osd":2,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.38},"max":{"1min":1.167,"5min":1.2709999999999999,"15min":1.3799999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.68100000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.2150000000000001,"5min":1.2310000000000001,"15min":1.4330000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.503,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2490000000000001,"5min":1.337,"15min":1.337},"last":0.69999999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.40600000000000003,"15min":0.32300000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.5740000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.86399999999999999},"min":{"1min":0.78800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.621},"last":0.86599999999999999}]}]},{"osd":2,"up_from":11,"seq":47244640496,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.40400000000000003,"5min":0.27700000000000002,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":6.3019999999999996},"last":0.505},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.01,"5min":1.113,"15min":6.3550000000000004},"last":0.69399999999999995}]},{"osd":1,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.63500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.1120000000000001,"5min":1.121,"15min":1.952},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":2.0019999999999998},"last":0.77600000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.82499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.976},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.75700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1679999999999999,"5min":1.431,"15min":1.5069999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.68799999999999994,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1160000000000001,"5min":2.391,"15min":2.391},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0429999999999999,"5min":2.419,"15min":2.419},"last":0.93400000000000005}]},{"osd":5,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.68700000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.4810000000000001,"15min":2.2330000000000001},"last":0.753},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.91700000000000004},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.51100000000000001},"max":{"1min":1.341,"5min":1.5449999999999999,"15min":1.637},"last":0.876}]}]},{"osd":1,"up_from":11,"seq":47244640498,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305808,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066032,"statfs":{"total":96636764160,"available":96323616768,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57699999999999996,"15min":0.59699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.88400000000000001,"5min":0.90200000000000002,"15min":1.883},"last":0.623},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.67400000000000004},"min":{"1min":0.35599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83499999999999996,"5min":0.93500000000000005,"15min":1.976},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64900000000000002,"15min":0.66400000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85699999999999998,"5min":1.5249999999999999,"15min":2.536},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.012,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92700000000000005,"15min":0.93100000000000005},"min":{"1min":0.57099999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0820000000000001,"5min":1.2509999999999999,"15min":1.288},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0149999999999999,"15min":0.95499999999999996},"min":{"1min":0.65200000000000002,"5min":0.51900000000000002,"15min":0.50800000000000001},"max":{"1min":1.1970000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.97199999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84499999999999997,"15min":0.83799999999999997},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.024,"5min":1.677,"15min":1.677},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95399999999999996,"15min":0.94099999999999995},"min":{"1min":0.68600000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2070000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Sun Sep 15 04:03:40 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.89500000000000002,"15min":0.91500000000000004},"min":{"1min":0.628,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.115,"5min":1.617,"15min":1.617},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.995,"15min":0.98799999999999999},"min":{"1min":0.61299999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.278,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.022}]}]},{"osd":0,"up_from":11,"seq":47244640497,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305292,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066548,"statfs":{"total":96636764160,"available":96324145152,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.79700000000000004,"15min":18.481000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.247,"15min":0.247},"max":{"1min":0.79600000000000004,"5min":2.222,"15min":19.239000000000001},"last":1.0469999999999999}]},{"osd":2,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.52900000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81399999999999995,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.84099999999999997},"min":{"1min":0.68300000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.079,"5min":3.556,"15min":3.556},"last":0.98199999999999998}]},{"osd":3,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85299999999999998,"15min":0.86399999999999999},"min":{"1min":0.70499999999999996,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":2.395,"15min":2.395},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.91900000000000004,"15min":0.90000000000000002},"min":{"1min":0.71699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98299999999999998,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.84799999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.55600000000000005,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.95399999999999996},"min":{"1min":0.77500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0840000000000001,"5min":2.923,"15min":2.923},"last":0.80000000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.873,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.56399999999999995,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.21,"15min":1.2989999999999999},"last":0.752},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94599999999999995,"15min":0.98299999999999998},"min":{"1min":0.81299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.282,"15min":1.6200000000000001},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T04:04:45.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T04:04:45.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: get_monmap_and_config 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: build_initial_monmap 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: monmap: 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:45.260185+0000 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:45.260185+0000 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:45.265 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 Processor -- start 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- start start 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: init 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: _add_conns ranks=[7,8,6,3,1,4,0,2,5] 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient(hunting): picked mon.noname-i con 0x7f6e2c068a30 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient(hunting): picked mon.noname-e con 0x7f6e2c132b80 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient(hunting): picked mon.noname-d con 0x7f6e2c138ec0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c04ca10 con 0x7f6e2c138ec0 2024-09-15T04:04:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0a2920 con 0x7f6e2c068a30 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0a2640 con 0x7f6e2c132b80 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient(hunting): authenticate will time out at 2029.689530s 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2bfff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f6e2c138ec0 0x7f6e2c13b2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:58444/0 (socket says 172.21.15.26:58444) 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2bfff700 1 -- 172.21.15.26:0/2757972980 learned_addr learned my addr 172.21.15.26:0/2757972980 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2affd700 1 --1- 172.21.15.26:0/2757972980 >> v1:172.21.15.26:6793/0 conn(0x7f6e2c132b80 0x7f6e2c132f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:39408/0 (socket says 172.21.15.26:39408) 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925779439 0 0) 0x7f6e2c04ca10 con 0x7f6e2c138ec0 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): my global_id is 195387 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003570 con 0x7f6e2c138ec0 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63255941 0 0) 0x7f6e2c0a2640 con 0x7f6e2c132b80 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:45.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): my global_id is 274292 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c04ca10 con 0x7f6e2c132b80 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403179885 0 0) 0x7f6e2c0a2920 con 0x7f6e2c068a30 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient(hunting): my global_id is 254509 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c0a2640 con 0x7f6e2c068a30 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185592517 0 0) 0x7f6e18003570 con 0x7f6e2c138ec0 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 >> v1:172.21.15.26:6793/0 conn(0x7f6e2c132b80 legacy=0x7f6e2c132f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 >> v1:172.21.15.122:6792/0 conn(0x7f6e2c068a30 legacy=0x7f6e2c09fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 monclient: found mon.noname-d 2024-09-15T04:04:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0a0320 con 0x7f6e2c138ec0 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 5 monclient: authenticate success, global_id 195387 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6e240017b0 con 0x7f6e2c138ec0 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: dump: 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6e24001a90 con 0x7f6e2c138ec0 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 4 set_mon_vals no callback set 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 1 -- 172.21.15.26:0/2757972980 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6e24004c30 con 0x7f6e2c138ec0 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e2a7fc700 10 monclient: dump: 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 10 monclient: shutdown 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- 172.21.15.26:0/2757972980 >> v1:172.21.15.26:6792/0 conn(0x7f6e2c138ec0 legacy=0x7f6e2c13b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- 172.21.15.26:0/2757972980 shutdown_connections 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.258+0000 7f6e30df8700 1 -- 172.21.15.26:0/2757972980 >> 172.21.15.26:0/2757972980 conn(0x7f6e2c06e430 msgr2=0x7f6e2c0a6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- 172.21.15.26:0/2757972980 shutdown_connections 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- 172.21.15.26:0/2757972980 wait complete. 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: build_initial_monmap 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: monmap: 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:45.276 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 Processor -- start 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- start start 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: init 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: _add_conns ranks=[8,7,4,1,0,2,6,3,5] 2024-09-15T04:04:45.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient(hunting): picked mon.noname-e con 0x7f6e2c068a30 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient(hunting): picked mon.noname-i con 0x7f6e2c132b80 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient(hunting): picked mon.noname-c con 0x7f6e2c138ec0 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c098880 con 0x7f6e2c138ec0 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c1328f0 con 0x7f6e2c132b80 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c099a10 con 0x7f6e2c068a30 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient(hunting): authenticate will time out at 2029.693542s 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e2bfff700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f6e2c138ec0 0x7f6e2c14a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:40580/0 (socket says 172.21.15.26:40580) 2024-09-15T04:04:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e2bfff700 1 -- 172.21.15.26:0/4061492810 learned_addr learned my addr 172.21.15.26:0/4061492810 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279123332 0 0) 0x7f6e2c098880 con 0x7f6e2c138ec0 2024-09-15T04:04:45.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): my global_id is 205186 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e0c003570 con 0x7f6e2c138ec0 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403740162 0 0) 0x7f6e2c099a10 con 0x7f6e2c068a30 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): my global_id is 274301 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c098880 con 0x7f6e2c068a30 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694152301 0 0) 0x7f6e2c1328f0 con 0x7f6e2c132b80 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): my global_id is 254518 2024-09-15T04:04:45.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c099a10 con 0x7f6e2c132b80 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2487444662 0 0) 0x7f6e2c098880 con 0x7f6e2c068a30 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e2c1328f0 con 0x7f6e2c068a30 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6e1c001de0 con 0x7f6e2c068a30 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): dump: 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:45.284 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2489646155 0 0) 0x7f6e0c003570 con 0x7f6e2c138ec0 2024-09-15T04:04:45.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e2c098880 con 0x7f6e2c138ec0 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6e24004b70 con 0x7f6e2c138ec0 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): dump: 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:45.286 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1381851683 0 0) 0x7f6e2c099a10 con 0x7f6e2c132b80 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e0c003570 con 0x7f6e2c132b80 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6e14001a60 con 0x7f6e2c132b80 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient(hunting): dump: 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:45.287 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:45.288 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4291016076 0 0) 0x7f6e2c098880 con 0x7f6e2c138ec0 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 >> v1:172.21.15.26:6793/0 conn(0x7f6e2c068a30 legacy=0x7f6e2c097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 >> v1:172.21.15.122:6792/0 conn(0x7f6e2c132b80 legacy=0x7f6e2c098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 monclient: found mon.c 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c099cf0 con 0x7f6e2c138ec0 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 5 monclient: authenticate success, global_id 205186 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: _renew_subs 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:45.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c099fd0 con 0x7f6e2c138ec0 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: _renew_subs 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c0994d0 con 0x7f6e2c138ec0 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6e24001860 con 0x7f6e2c138ec0 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6e24001160 con 0x7f6e2c138ec0 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e29ffb700 4 set_mon_vals no callback set 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.262+0000 7f6e237fe700 10 monclient: dump: 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:45.290 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:45.291 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6e2400fa00 con 0x7f6e2c138ec0 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e30df8700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e30df8700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e30df8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e100050b0 con 0x7f6e2c138ec0 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f6e24043e20 con 0x7f6e2c138ec0 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.266+0000 7f6e2b7fe700 10 monclient: get_auth_request con 0x7f6e0c0380b0 auth_method 0 2024-09-15T04:04:45.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.270+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f6e24014a50 con 0x7f6e2c138ec0 2024-09-15T04:04:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.270+0000 7f6e237fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:45.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.270+0000 7f6e237fe700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.434+0000 7f6e30df8700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.438+0000 7f6e30df8700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.438+0000 7f6e30df8700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.438+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e10005360 con 0x7f6e2c138ec0 2024-09-15T04:04:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.438+0000 7f6e237fe700 1 -- 172.21.15.26:0/4061492810 <== mon.4 v1:172.21.15.26:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v904) v1 ==== 74+0+10071 (unknown 161975460 0 69917236) 0x7f6e24014870 con 0x7f6e2c138ec0 2024-09-15T04:04:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.438+0000 7f6e237fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-09-15T04:04:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.438+0000 7f6e237fe700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:04:45.445 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:45.445 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":904,"fsid":"4d3becf5-10bb-4b2a-818b-afafac96b25a","created":"2024-09-15T03:44:47.145432+0000","modified":"2024-09-15T04:04:08.628379+0000","last_up_change":"2024-09-15T03:44:55.536946+0000","last_in_change":"2024-09-15T03:44:48.670274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T03:44:56.976968+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-15T03:44:57.527097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b65c9d59-0082-4f20-9069-40cf97297ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6804","nonce":21131}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6805","nonce":21131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6807","nonce":21131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6806","nonce":21131}]},"public_addr":"172.21.15.26:6804/21131","cluster_addr":"172.21.15.26:6805/21131","heartbeat_back_addr":"172.21.15.26:6807/21131","heartbeat_front_addr":"172.21.15.26:6806/21131","state":["exists","up"]},{"osd":1,"uuid":"57399273-1f9a-4dfb-bef9-0ced7ef1a7c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6800","nonce":21130}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6801","nonce":21130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6803","nonce":21130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6802","nonce":21130}]},"public_addr":"172.21.15.26:6800/21130","cluster_addr":"172.21.15.26:6801/21130","heartbeat_back_addr":"172.21.15.26:6803/21130","heartbeat_front_addr":"172.21.15.26:6802/21130","state":["exists","up"]},{"osd":2,"uuid":"e72df2db-c0f8-48fe-84a2-7de252df9821","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6808","nonce":21148}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6809","nonce":21148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6811","nonce":21148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.26:6810","nonce":21148}]},"public_addr":"172.21.15.26:6808/21148","cluster_addr":"172.21.15.26:6809/21148","heartbeat_back_addr":"172.21.15.26:6811/21148","heartbeat_front_addr":"172.21.15.26:6810/21148","state":["exists","up"]},{"osd":3,"uuid":"f50cfade-e9a7-4aaa-8816-2172cb4eace1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":21255}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":21255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":21255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":21255}]},"public_addr":"172.21.15.122:6800/21255","cluster_addr":"172.21.15.122:6801/21255","heartbeat_back_addr":"172.21.15.122:6803/21255","heartbeat_front_addr":"172.21.15.122:6802/21255","state":["exists","up"]},{"osd":4,"uuid":"c473a258-e2aa-45ab-8209-98faa4e1a34d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":21254}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":21254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":21254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":21254}]},"public_addr":"172.21.15.122:6808/21254","cluster_addr":"172.21.15.122:6809/21254","heartbeat_back_addr":"172.21.15.122:6811/21254","heartbeat_front_addr":"172.21.15.122:6810/21254","state":["exists","up"]},{"osd":5,"uuid":"c49841a6-c038-49c5-9b02-b124380dad0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":21256}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":21256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":21256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":21256}]},"public_addr":"172.21.15.122:6804/21256","cluster_addr":"172.21.15.122:6805/21256","heartbeat_back_addr":"172.21.15.122:6807/21256","heartbeat_front_addr":"172.21.15.122:6806/21256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.131487+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048447+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.161242+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:54.048910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.975374+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T03:44:53.875679+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":803,"length":1},{"begin":847,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 >> v1:172.21.15.122:6812/20895 conn(0x7f6e0c0380b0 legacy=0x7f6e0c03a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 10 monclient: shutdown 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 >> v1:172.21.15.26:6791/0 conn(0x7f6e2c138ec0 legacy=0x7f6e2c14a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 shutdown_connections 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 >> 172.21.15.26:0/4061492810 conn(0x7f6e2c06e430 msgr2=0x7f6e2c0677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 shutdown_connections 2024-09-15T04:04:45.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:45.442+0000 7f6e30df8700 1 -- 172.21.15.26:0/4061492810 wait complete. 2024-09-15T04:04:46.461 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T04:04:46.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T04:04:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient: get_monmap_and_config 2024-09-15T04:04:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient: build_initial_monmap 2024-09-15T04:04:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient: monmap: 2024-09-15T04:04:46.584 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:46.580614+0000 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:46.580614+0000 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:46.585 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:46.586 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 1 Processor -- start 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 1 -- start start 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient: init 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient: _add_conns ranks=[4,0,3,1,6,8,5,7,2] 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient(hunting): picked mon.noname-c con 0x7f665409fa20 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:46.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient(hunting): picked mon.noname-a con 0x7f6654132b20 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient(hunting): picked mon.noname-g con 0x7f6654138c30 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6654070420 con 0x7f6654132b20 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540a0de0 con 0x7f6654138c30 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540a2430 con 0x7f665409fa20 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6659de0700 10 monclient(hunting): authenticate will time out at 2031.010880s 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6658dde700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f665409fa20 0x7f665409fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:54726/0 (socket says 172.21.15.26:54726) 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f6658dde700 1 -- 172.21.15.26:0/4188193378 learned_addr learned my addr 172.21.15.26:0/4188193378 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571903291 0 0) 0x7f66540a2430 con 0x7f665409fa20 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): my global_id is 205195 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6640003570 con 0x7f665409fa20 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929753029 0 0) 0x7f6654070420 con 0x7f6654132b20 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): my global_id is 195309 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66540a2430 con 0x7f6654132b20 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528356806 0 0) 0x7f66540a0de0 con 0x7f6654138c30 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient(hunting): my global_id is 254442 2024-09-15T04:04:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6654070420 con 0x7f6654138c30 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3865370343 0 0) 0x7f66540a2430 con 0x7f6654132b20 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 >> v1:172.21.15.26:6791/0 conn(0x7f665409fa20 legacy=0x7f665409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 >> v1:172.21.15.122:6790/0 conn(0x7f6654138c30 legacy=0x7f665413b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 1 monclient: found mon.noname-a 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.578+0000 7f66537fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540a2150 con 0x7f6654132b20 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 5 monclient: authenticate success, global_id 195309 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f663c001790 con 0x7f6654132b20 2024-09-15T04:04:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: dump: 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.592 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f663c001a30 con 0x7f6654132b20 2024-09-15T04:04:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 1 -- 172.21.15.26:0/4188193378 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f663c001f60 con 0x7f6654132b20 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 4 set_mon_vals no callback set 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f66537fe700 10 monclient: dump: 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.594 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.595 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 10 monclient: shutdown 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 1 -- 172.21.15.26:0/4188193378 >> v1:172.21.15.26:6789/0 conn(0x7f6654132b20 legacy=0x7f6654132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 1 -- 172.21.15.26:0/4188193378 shutdown_connections 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 1 -- 172.21.15.26:0/4188193378 >> 172.21.15.26:0/4188193378 conn(0x7f665406e6e0 msgr2=0x7f66540a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 1 -- 172.21.15.26:0/4188193378 shutdown_connections 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 1 -- 172.21.15.26:0/4188193378 wait complete. 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 10 monclient: build_initial_monmap 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 10 monclient: monmap: 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:46.602 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.582+0000 7f6659de0700 1 Processor -- start 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 1 -- start start 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: init 2024-09-15T04:04:46.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: _add_conns ranks=[3,5,2,1,8,7,6,0,4] 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient(hunting): picked mon.noname-g con 0x7f665409fa20 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient(hunting): picked mon.noname-h con 0x7f6654132b20 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient(hunting): picked mon.noname-b con 0x7f6654138c30 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665409cce0 con 0x7f6654138c30 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665411ca40 con 0x7f665409fa20 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665411cd20 con 0x7f6654132b20 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient(hunting): authenticate will time out at 2031.017210s 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f66595df700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6654138c30 0x7f665414a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42246/0 (socket says 172.21.15.26:42246) 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f66595df700 1 -- 172.21.15.26:0/2181049515 learned_addr learned my addr 172.21.15.26:0/2181049515 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285301607 0 0) 0x7f665409cce0 con 0x7f6654138c30 2024-09-15T04:04:46.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): my global_id is 254360 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003570 con 0x7f6654138c30 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445057087 0 0) 0x7f665411cd20 con 0x7f6654132b20 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): my global_id is 274217 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665409cce0 con 0x7f6654132b20 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007013286 0 0) 0x7f665411ca40 con 0x7f665409fa20 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): my global_id is 254451 2024-09-15T04:04:46.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665411cd20 con 0x7f665409fa20 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1179621284 0 0) 0x7f6634003570 con 0x7f6654138c30 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f665411ca40 con 0x7f6654138c30 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f664c002df0 con 0x7f6654138c30 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.606 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3467835923 0 0) 0x7f665409cce0 con 0x7f6654132b20 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6634003570 con 0x7f6654132b20 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f663c001ee0 con 0x7f6654132b20 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.608 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.3 v1:172.21.15.122:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4104524135 0 0) 0x7f665411cd20 con 0x7f665409fa20 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.122:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f665409cce0 con 0x7f665409fa20 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 100137255 0 0) 0x7f665411ca40 con 0x7f6654138c30 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 >> v1:172.21.15.122:6791/0 conn(0x7f6654132b20 legacy=0x7f6654099db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 >> v1:172.21.15.122:6790/0 conn(0x7f665409fa20 legacy=0x7f66540996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 monclient: found mon.b 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665409af80 con 0x7f6654138c30 2024-09-15T04:04:46.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 5 monclient: authenticate success, global_id 254360 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: _renew_subs 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665409b490 con 0x7f6654138c30 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: _renew_subs 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6654132570 con 0x7f6654138c30 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f664c001900 con 0x7f6654138c30 2024-09-15T04:04:46.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f664c001e30 con 0x7f6654138c30 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6652ffd700 4 set_mon_vals no callback set 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.586+0000 7f6650ff9700 10 monclient: dump: 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.612 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.613 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f664c002010 con 0x7f6654138c30 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6659de0700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6638000f20 con 0x7f6654138c30 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f664c044120 con 0x7f6654138c30 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6804/21131 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66340427c0 con 0x7f663403eb60 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_get_version_reply(handle=1 version=904) v2 ==== 24+0+0 (unknown 607715002 0 0) 0x7f664c044540 con 0x7f6654138c30 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6650ff9700 10 monclient: handle_get_version_reply finishing 1 version 904 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6658dde700 10 monclient: get_auth_request con 0x7f6634037f50 auth_method 0 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6653fff700 10 monclient: get_auth_request con 0x7f663403eb60 auth_method 0 2024-09-15T04:04:46.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.590+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== osd.0 v1:172.21.15.26:6804/21131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f66340427c0 con 0x7f663403eb60 2024-09-15T04:04:46.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 --> v1:172.21.15.26:6804/21131 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6638002bd0 con 0x7f663403eb60 2024-09-15T04:04:46.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6650ff9700 1 -- 172.21.15.26:0/2181049515 <== osd.0 v1:172.21.15.26:6804/21131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f6638002bd0 con 0x7f663403eb60 2024-09-15T04:04:46.616 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:46.616 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T04:04:46.616 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 >> v1:172.21.15.26:6804/21131 conn(0x7f663403eb60 legacy=0x7f6634040f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 >> v1:172.21.15.122:6812/20895 conn(0x7f6634037f50 legacy=0x7f663403a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 10 monclient: shutdown 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 >> v1:172.21.15.26:6790/0 conn(0x7f6654138c30 legacy=0x7f665414a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f66595df700 1 -- 172.21.15.26:0/2181049515 reap_dead start 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 shutdown_connections 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 >> 172.21.15.26:0/2181049515 conn(0x7f665406e6e0 msgr2=0x7f66540a4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 shutdown_connections 2024-09-15T04:04:46.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.610+0000 7f6659de0700 1 -- 172.21.15.26:0/2181049515 wait complete. 2024-09-15T04:04:46.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T04:04:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.734+0000 7fde5bfff700 10 monclient: get_monmap_and_config 2024-09-15T04:04:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.734+0000 7fde5bfff700 10 monclient: build_initial_monmap 2024-09-15T04:04:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.734+0000 7fde5bfff700 10 monclient: monmap: 2024-09-15T04:04:46.742 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:46.738853+0000 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:46.738853+0000 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:46.743 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:46.744 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:46.744 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 Processor -- start 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- start start 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient: init 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient: _add_conns ranks=[1,0,6,8,7,5,4,3,2] 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient(hunting): picked mon.noname-f con 0x7fde5c132b40 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient(hunting): picked mon.noname-a con 0x7fde5c09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient(hunting): picked mon.noname-d con 0x7fde5c138d40 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c04ca10 con 0x7fde5c09faa0 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c09d5d0 con 0x7fde5c132b40 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c133620 con 0x7fde5c138d40 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient(hunting): authenticate will time out at 2031.169114s 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5b7fe700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7fde5c138d40 0x7fde5c13b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48098/0 (socket says 172.21.15.26:48098) 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde53fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde5c09faa0 0x7fde5c09fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42534/0 (socket says 172.21.15.26:42534) 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5b7fe700 1 -- 172.21.15.26:0/605156906 learned_addr learned my addr 172.21.15.26:0/605156906 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983925744 0 0) 0x7fde5c04ca10 con 0x7fde5c09faa0 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): my global_id is 195318 2024-09-15T04:04:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde48003570 con 0x7fde5c09faa0 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618532954 0 0) 0x7fde5c133620 con 0x7fde5c138d40 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): my global_id is 195396 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde5c04ca10 con 0x7fde5c138d40 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728455619 0 0) 0x7fde5c09d5d0 con 0x7fde5c132b40 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient(hunting): my global_id is 274186 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde5c133620 con 0x7fde5c132b40 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1046860 0 0) 0x7fde48003570 con 0x7fde5c09faa0 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 >> v1:172.21.15.26:6792/0 conn(0x7fde5c138d40 legacy=0x7fde5c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 >> v1:172.21.15.122:6789/0 conn(0x7fde5c132b40 legacy=0x7fde5c132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 monclient: found mon.noname-a 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0a0cc0 con 0x7fde5c09faa0 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde440017b0 con 0x7fde5c09faa0 2024-09-15T04:04:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: dump: 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 5 monclient: authenticate success, global_id 195318 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fde44001a20 con 0x7fde5c09faa0 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 4 set_mon_vals no callback set 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 1 -- 172.21.15.26:0/605156906 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde44004b70 con 0x7fde5c09faa0 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5a7fc700 10 monclient: dump: 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 10 monclient: shutdown 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- 172.21.15.26:0/605156906 >> v1:172.21.15.26:6789/0 conn(0x7fde5c09faa0 legacy=0x7fde5c09fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- 172.21.15.26:0/605156906 shutdown_connections 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- 172.21.15.26:0/605156906 >> 172.21.15.26:0/605156906 conn(0x7fde5c06e370 msgr2=0x7fde5c0a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- 172.21.15.26:0/605156906 shutdown_connections 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.738+0000 7fde5bfff700 1 -- 172.21.15.26:0/605156906 wait complete. 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: build_initial_monmap 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: monmap: 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 1 Processor -- start 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 1 -- start start 2024-09-15T04:04:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: init 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: _add_conns ranks=[0,8,6,2,1,5,7,4,3] 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient(hunting): picked mon.noname-a con 0x7fde5c09faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient(hunting): picked mon.noname-e con 0x7fde5c132b40 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient(hunting): picked mon.noname-d con 0x7fde5c138d40 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c09b920 con 0x7fde5c09faa0 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c09cfa0 con 0x7fde5c138d40 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c09ccc0 con 0x7fde5c132b40 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient(hunting): authenticate will time out at 2031.174040s 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5affd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde5c09faa0 0x7fde5c0995f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42546/0 (socket says 172.21.15.26:42546) 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5affd700 1 -- 172.21.15.26:0/388853702 learned_addr learned my addr 172.21.15.26:0/388853702 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561753789 0 0) 0x7fde5c09b920 con 0x7fde5c09faa0 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): my global_id is 195327 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3c003570 con 0x7fde5c09faa0 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244469859 0 0) 0x7fde5c09cfa0 con 0x7fde5c138d40 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): my global_id is 195405 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde5c09b920 con 0x7fde5c138d40 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391696 0 0) 0x7fde5c09ccc0 con 0x7fde5c132b40 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): my global_id is 274310 2024-09-15T04:04:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde5c09cfa0 con 0x7fde5c132b40 2024-09-15T04:04:46.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2782433660 0 0) 0x7fde5c09b920 con 0x7fde5c138d40 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde5c09ccc0 con 0x7fde5c138d40 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde54001c80 con 0x7fde5c138d40 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.769 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.770 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 397817913 0 0) 0x7fde5c09cfa0 con 0x7fde5c132b40 2024-09-15T04:04:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde5c09b920 con 0x7fde5c132b40 2024-09-15T04:04:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde44004b70 con 0x7fde5c132b40 2024-09-15T04:04:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.773 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.774 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.774 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.774 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.774 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.775 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2446876705 0 0) 0x7fde3c003570 con 0x7fde5c09faa0 2024-09-15T04:04:46.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde5c09cfa0 con 0x7fde5c09faa0 2024-09-15T04:04:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde4c001aa0 con 0x7fde5c09faa0 2024-09-15T04:04:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.776 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.777 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.778 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 532655794 0 0) 0x7fde5c09cfa0 con 0x7fde5c09faa0 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 >> v1:172.21.15.26:6793/0 conn(0x7fde5c132b40 legacy=0x7fde5c099d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 >> v1:172.21.15.26:6792/0 conn(0x7fde5c138d40 legacy=0x7fde5c14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 monclient: found mon.a 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c09b430 con 0x7fde5c09faa0 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:46.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde52ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 5 monclient: authenticate success, global_id 195327 2024-09-15T04:04:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: _renew_subs 2024-09-15T04:04:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:46.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.742+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c132660 con 0x7fde5c09faa0 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 10 monclient: _renew_subs 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c09baf0 con 0x7fde5c09faa0 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fde4c002cf0 con 0x7fde5c09faa0 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fde4c002370 con 0x7fde5c09faa0 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:46.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 10 monclient: dump: 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:46.782 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde59ffb700 4 set_mon_vals no callback set 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fde4c0024d0 con 0x7fde5c09faa0 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fde4c043c50 con 0x7fde5c09faa0 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:46.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde400050b0 con 0x7fde5c09faa0 2024-09-15T04:04:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.746+0000 7fde53fff700 10 monclient: get_auth_request con 0x7fde3c038100 auth_method 0 2024-09-15T04:04:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.754+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fde4c014760 con 0x7fde5c09faa0 2024-09-15T04:04:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.754+0000 7fde52ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:46.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.754+0000 7fde52ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:46.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.918+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fde40000b90 con 0x7fde3c038100 2024-09-15T04:04:46.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.918+0000 7fde52ffd700 1 -- 172.21.15.26:0/388853702 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (unknown 2057319476 0 0) 0x7fde40000b90 con 0x7fde3c038100 2024-09-15T04:04:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 >> v1:172.21.15.122:6812/20895 conn(0x7fde3c038100 legacy=0x7fde3c03a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 10 monclient: shutdown 2024-09-15T04:04:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 >> v1:172.21.15.26:6789/0 conn(0x7fde5c09faa0 legacy=0x7fde5c0995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 shutdown_connections 2024-09-15T04:04:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 >> 172.21.15.26:0/388853702 conn(0x7fde5c06e370 msgr2=0x7fde5c09da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 shutdown_connections 2024-09-15T04:04:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:46.922+0000 7fde5bfff700 1 -- 172.21.15.26:0/388853702 wait complete. 2024-09-15T04:04:46.930 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T04:04:46.945 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T04:04:46.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient: get_monmap_and_config 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient: monmap: 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:47.053234+0000 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:47.053234+0000 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.058 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 1 Processor -- start 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 1 -- start start 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient: init 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient: _add_conns ranks=[8,6,2,1,4,5,3,7,0] 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient(hunting): picked mon.noname-e con 0x7f5b2c09fa20 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient(hunting): picked mon.noname-d con 0x7f5b2c132b20 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient(hunting): picked mon.noname-b con 0x7f5b2c138c30 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c070420 con 0x7f5b2c138c30 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0a0de0 con 0x7f5b2c132b20 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0a2430 con 0x7f5b2c09fa20 2024-09-15T04:04:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b322fd700 10 monclient(hunting): authenticate will time out at 2031.483500s 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b31afc700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f5b2c138c30 0x7f5b2c13b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42254/0 (socket says 172.21.15.26:42254) 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b312fb700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f5b2c09fa20 0x7f5b2c09fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33124/0 (socket says 172.21.15.26:33124) 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.050+0000 7f5b31afc700 1 -- 172.21.15.26:0/2936292498 learned_addr learned my addr 172.21.15.26:0/2936292498 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118202059 0 0) 0x7f5b2c0a2430 con 0x7f5b2c09fa20 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): my global_id is 274319 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b18003570 con 0x7f5b2c09fa20 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620113391 0 0) 0x7f5b2c0a0de0 con 0x7f5b2c132b20 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): my global_id is 195414 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b2c0a2430 con 0x7f5b2c132b20 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799586145 0 0) 0x7f5b2c070420 con 0x7f5b2c138c30 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient(hunting): my global_id is 254369 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b2c0a0de0 con 0x7f5b2c138c30 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918184208 0 0) 0x7f5b18003570 con 0x7f5b2c09fa20 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 >> v1:172.21.15.26:6792/0 conn(0x7f5b2c132b20 legacy=0x7f5b2c132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 >> v1:172.21.15.26:6790/0 conn(0x7f5b2c138c30 legacy=0x7f5b2c13b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 monclient: found mon.noname-e 2024-09-15T04:04:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0a2150 con 0x7f5b2c09fa20 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 5 monclient: authenticate success, global_id 274319 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b280017f0 con 0x7f5b2c09fa20 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: dump: 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.067 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5b28001a80 con 0x7f5b2c09fa20 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 1 -- 172.21.15.26:0/2936292498 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b28001c70 con 0x7f5b2c09fa20 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 4 set_mon_vals no callback set 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b23fff700 10 monclient: dump: 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 10 monclient: shutdown 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 1 -- 172.21.15.26:0/2936292498 >> v1:172.21.15.26:6793/0 conn(0x7f5b2c09fa20 legacy=0x7f5b2c09fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 1 -- 172.21.15.26:0/2936292498 shutdown_connections 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 1 -- 172.21.15.26:0/2936292498 >> 172.21.15.26:0/2936292498 conn(0x7f5b2c06e6e0 msgr2=0x7f5b2c0a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 1 -- 172.21.15.26:0/2936292498 shutdown_connections 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 1 -- 172.21.15.26:0/2936292498 wait complete. 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.054+0000 7f5b322fd700 10 monclient: monmap: 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.071 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 Processor -- start 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 -- start start 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: init 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: _add_conns ranks=[3,1,6,7,5,8,4,2,0] 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient(hunting): picked mon.noname-g con 0x7f5b2c09fa20 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient(hunting): picked mon.noname-f con 0x7f5b2c132b20 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient(hunting): picked mon.noname-d con 0x7f5b2c138c30 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:47.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0a17d0 con 0x7f5b2c132b20 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c09cce0 con 0x7f5b2c09fa20 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c132450 con 0x7f5b2c138c30 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient(hunting): authenticate will time out at 2031.489495s 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b31afc700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f5b2c138c30 0x7f5b2c135dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48126/0 (socket says 172.21.15.26:48126) 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b31afc700 1 -- 172.21.15.26:0/1426287259 learned_addr learned my addr 172.21.15.26:0/1426287259 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154355680 0 0) 0x7f5b2c132450 con 0x7f5b2c138c30 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): my global_id is 195423 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b10003570 con 0x7f5b2c138c30 2024-09-15T04:04:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546500420 0 0) 0x7f5b2c0a17d0 con 0x7f5b2c132b20 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): my global_id is 274195 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b2c132450 con 0x7f5b2c132b20 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392710894 0 0) 0x7f5b2c09cce0 con 0x7f5b2c09fa20 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): my global_id is 254460 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b2c0a17d0 con 0x7f5b2c09fa20 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2436249919 0 0) 0x7f5b10003570 con 0x7f5b2c138c30 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b2c09cce0 con 0x7f5b2c138c30 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b24001cf0 con 0x7f5b2c138c30 2024-09-15T04:04:47.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): dump: 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.079 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1309881792 0 0) 0x7f5b2c09cce0 con 0x7f5b2c138c30 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 >> v1:172.21.15.122:6790/0 conn(0x7f5b2c09fa20 legacy=0x7f5b2c134fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 >> v1:172.21.15.122:6789/0 conn(0x7f5b2c132b20 legacy=0x7f5b2c1356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 monclient: found mon.d 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c136f60 con 0x7f5b2c138c30 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 5 monclient: authenticate success, global_id 195423 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: _renew_subs 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c1373a0 con 0x7f5b2c138c30 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f5b24002080 con 0x7f5b2c138c30 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b237fe700 4 set_mon_vals no callback set 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: _renew_subs 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f5b240051a0 con 0x7f5b2c138c30 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c137620 con 0x7f5b2c138c30 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.058+0000 7f5b217fa700 10 monclient: dump: 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b322fd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b2c11c7f0 con 0x7f5b2c138c30 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f5b2400fb60 con 0x7f5b2c138c30 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f5b24043f00 con 0x7f5b2c138c30 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6800/21130 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b10042000 con 0x7f5b1003e3a0 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_get_version_reply(handle=1 version=904) v2 ==== 24+0+0 (unknown 607715002 0 0) 0x7f5b24044330 con 0x7f5b2c138c30 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b217fa700 10 monclient: handle_get_version_reply finishing 1 version 904 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b312fb700 10 monclient: get_auth_request con 0x7f5b10037730 auth_method 0 2024-09-15T04:04:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b30afa700 10 monclient: get_auth_request con 0x7f5b1003e3a0 auth_method 0 2024-09-15T04:04:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.062+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== osd.1 v1:172.21.15.26:6800/21130 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f5b10042000 con 0x7f5b1003e3a0 2024-09-15T04:04:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 --> v1:172.21.15.26:6800/21130 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5b2c0a5500 con 0x7f5b1003e3a0 2024-09-15T04:04:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b217fa700 1 -- 172.21.15.26:0/1426287259 <== osd.1 v1:172.21.15.26:6800/21130 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f5b2c0a5500 con 0x7f5b1003e3a0 2024-09-15T04:04:47.089 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:47.089 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T04:04:47.089 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 >> v1:172.21.15.26:6800/21130 conn(0x7f5b1003e3a0 legacy=0x7f5b100407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 >> v1:172.21.15.122:6812/20895 conn(0x7f5b10037730 legacy=0x7f5b10039be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 10 monclient: shutdown 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 >> v1:172.21.15.26:6792/0 conn(0x7f5b2c138c30 legacy=0x7f5b2c135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b31afc700 1 -- 172.21.15.26:0/1426287259 reap_dead start 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 shutdown_connections 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 >> 172.21.15.26:0/1426287259 conn(0x7f5b2c06e6e0 msgr2=0x7f5b2c09ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 shutdown_connections 2024-09-15T04:04:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.082+0000 7f5b322fd700 1 -- 172.21.15.26:0/1426287259 wait complete. 2024-09-15T04:04:47.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T04:04:47.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.202+0000 7fb2e133f700 10 monclient: get_monmap_and_config 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.202+0000 7fb2e133f700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.202+0000 7fb2e133f700 10 monclient: monmap: 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:47.205994+0000 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:47.205994+0000 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.210 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.211 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.211 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.202+0000 7fb2e133f700 1 Processor -- start 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.202+0000 7fb2e133f700 1 -- start start 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.202+0000 7fb2e133f700 10 monclient: init 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient: _add_conns ranks=[0,7,3,4,5,8,1,2,6] 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient(hunting): picked mon.noname-a con 0x7fb2dc070de0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient(hunting): picked mon.noname-i con 0x7fb2dc09c580 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient(hunting): picked mon.noname-g con 0x7fb2dc09c9b0 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc04ca10 con 0x7fb2dc070de0 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc0711b0 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc09ea10 con 0x7fb2dc09c580 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient(hunting): authenticate will time out at 2031.637155s 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2daffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb2dc070de0 0x7fb2dc09be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42548/0 (socket says 172.21.15.26:42548) 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2daffd700 1 -- 172.21.15.26:0/2924774683 learned_addr learned my addr 172.21.15.26:0/2924774683 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585069972 0 0) 0x7fb2dc04ca10 con 0x7fb2dc070de0 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient(hunting): my global_id is 195336 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c4003570 con 0x7fb2dc070de0 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422805386 0 0) 0x7fb2dc0711b0 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient(hunting): my global_id is 254469 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2dc04ca10 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572118785 0 0) 0x7fb2c4003570 con 0x7fb2dc070de0 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 >> v1:172.21.15.122:6792/0 conn(0x7fb2dc09c580 legacy=0x7fb2dc137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 >> v1:172.21.15.122:6790/0 conn(0x7fb2dc09c9b0 legacy=0x7fb2dc13b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 monclient: found mon.noname-a 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc09ecf0 con 0x7fb2dc070de0 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 5 monclient: authenticate success, global_id 195336 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2cc0017b0 con 0x7fb2dc070de0 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: dump: 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.219 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb2cc0019a0 con 0x7fb2dc070de0 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 1 -- 172.21.15.26:0/2924774683 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2cc0048f0 con 0x7fb2dc070de0 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 4 set_mon_vals no callback set 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:47.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2d9ffb700 10 monclient: dump: 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.221 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 10 monclient: shutdown 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- 172.21.15.26:0/2924774683 >> v1:172.21.15.26:6789/0 conn(0x7fb2dc070de0 legacy=0x7fb2dc09be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- 172.21.15.26:0/2924774683 shutdown_connections 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- 172.21.15.26:0/2924774683 >> 172.21.15.26:0/2924774683 conn(0x7fb2dc063e10 msgr2=0x7fb2dc0a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- 172.21.15.26:0/2924774683 shutdown_connections 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.206+0000 7fb2e133f700 1 -- 172.21.15.26:0/2924774683 wait complete. 2024-09-15T04:04:47.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: monmap: 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.223 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.224 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.224 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.224 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 Processor -- start 2024-09-15T04:04:47.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 -- start start 2024-09-15T04:04:47.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: init 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: _add_conns ranks=[0,5,6,2,7,4,8,1,3] 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient(hunting): picked mon.noname-a con 0x7fb2dc070de0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient(hunting): picked mon.noname-h con 0x7fb2dc09c580 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient(hunting): picked mon.noname-d con 0x7fb2dc09c9b0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc09e280 con 0x7fb2dc070de0 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc1356f0 con 0x7fb2dc09c580 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc142a30 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient(hunting): authenticate will time out at 2031.641558s 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2daffd700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb2dc070de0 0x7fb2dc142320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42554/0 (socket says 172.21.15.26:42554) 2024-09-15T04:04:47.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2daffd700 1 -- 172.21.15.26:0/825562790 learned_addr learned my addr 172.21.15.26:0/825562790 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274604036 0 0) 0x7fb2dc09e280 con 0x7fb2dc070de0 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): my global_id is 195345 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c8003570 con 0x7fb2dc070de0 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303767505 0 0) 0x7fb2dc142a30 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): my global_id is 195432 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2dc09e280 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997601788 0 0) 0x7fb2dc1356f0 con 0x7fb2dc09c580 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): my global_id is 274226 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2dc142a30 con 0x7fb2dc09c580 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1790900532 0 0) 0x7fb2c8003570 con 0x7fb2dc070de0 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2dc1356f0 con 0x7fb2dc070de0 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2cc0016b0 con 0x7fb2dc070de0 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.230 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 634217532 0 0) 0x7fb2dc09e280 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb2c8003570 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2d4001d00 con 0x7fb2dc09c9b0 2024-09-15T04:04:47.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.232 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2205377696 0 0) 0x7fb2dc1356f0 con 0x7fb2dc070de0 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 >> v1:172.21.15.26:6792/0 conn(0x7fb2dc09c9b0 legacy=0x7fb2dc14a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 >> v1:172.21.15.122:6791/0 conn(0x7fb2dc09c580 legacy=0x7fb2dc146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 monclient: found mon.a 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:47.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc119240 con 0x7fb2dc070de0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 5 monclient: authenticate success, global_id 195345 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: _renew_subs 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc12c770 con 0x7fb2dc070de0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: _renew_subs 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc118ea0 con 0x7fb2dc070de0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fb2cc004e70 con 0x7fb2dc070de0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fb2cc005320 con 0x7fb2dc070de0 2024-09-15T04:04:47.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2d97fa700 4 set_mon_vals no callback set 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.210+0000 7fb2c2ffd700 10 monclient: dump: 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.235 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2e133f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2e133f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2e133f700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.26:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc0050b0 con 0x7fb2dc070de0 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fb2cc00fce0 con 0x7fb2dc070de0 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fb2cc044030 con 0x7fb2dc070de0 2024-09-15T04:04:47.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.214+0000 7fb2da7fc700 10 monclient: get_auth_request con 0x7fb2c8037d10 auth_method 0 2024-09-15T04:04:47.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.222+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fb2cc015080 con 0x7fb2dc070de0 2024-09-15T04:04:47.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.222+0000 7fb2c2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:47.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.222+0000 7fb2c2ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.386+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2bc000b90 con 0x7fb2c8037d10 2024-09-15T04:04:47.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.386+0000 7fb2c2ffd700 1 -- 172.21.15.26:0/825562790 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (unknown 489605429 0 0) 0x7fb2bc000b90 con 0x7fb2c8037d10 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 >> v1:172.21.15.122:6812/20895 conn(0x7fb2c8037d10 legacy=0x7fb2c803a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 10 monclient: shutdown 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 >> v1:172.21.15.26:6789/0 conn(0x7fb2dc070de0 legacy=0x7fb2dc142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 shutdown_connections 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 >> 172.21.15.26:0/825562790 conn(0x7fb2dc063e10 msgr2=0x7fb2dc12ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 shutdown_connections 2024-09-15T04:04:47.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.390+0000 7fb2e133f700 1 -- 172.21.15.26:0/825562790 wait complete. 2024-09-15T04:04:47.397 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T04:04:47.410 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T04:04:47.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient: get_monmap_and_config 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient: monmap: 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:47.521606+0000 2024-09-15T04:04:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:47.521606+0000 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.526 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 1 Processor -- start 2024-09-15T04:04:47.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 1 -- start start 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient: init 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient: _add_conns ranks=[3,2,4,8,6,1,7,0,5] 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient(hunting): picked mon.noname-g con 0x7f217409fa20 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient(hunting): picked mon.noname-b con 0x7f2174132b20 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 10 monclient(hunting): picked mon.noname-c con 0x7f2174138c30 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.518+0000 7f217359e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2174070420 con 0x7f2174132b20 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740a0de0 con 0x7f217409fa20 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f2171d9b700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f2174132b20 0x7f2174132ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42266/0 (socket says 172.21.15.26:42266) 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f2171d9b700 1 -- 172.21.15.26:0/847986657 learned_addr learned my addr 172.21.15.26:0/847986657 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- 172.21.15.26:0/847986657 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21740a2430 con 0x7f2174138c30 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 10 monclient(hunting): authenticate will time out at 2031.952225s 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832216518 0 0) 0x7f2174070420 con 0x7f2174132b20 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient(hunting): my global_id is 254378 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f215c003570 con 0x7f2174132b20 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917841175 0 0) 0x7f21740a2430 con 0x7f2174138c30 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient(hunting): my global_id is 205204 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2174070420 con 0x7f2174138c30 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408474674 0 0) 0x7f215c003570 con 0x7f2174132b20 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 >> v1:172.21.15.26:6791/0 conn(0x7f2174138c30 legacy=0x7f217413b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 >> v1:172.21.15.122:6790/0 conn(0x7f217409fa20 legacy=0x7f217409fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 monclient: found mon.noname-b 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740a2150 con 0x7f2174132b20 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f21640017b0 con 0x7f2174132b20 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: dump: 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.533 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.534 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 5 monclient: authenticate success, global_id 254378 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2164001a90 con 0x7f2174132b20 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 1 -- 172.21.15.26:0/847986657 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2164004c30 con 0x7f2174132b20 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 4 set_mon_vals no callback set 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217159a700 10 monclient: dump: 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.536 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 10 monclient: shutdown 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- 172.21.15.26:0/847986657 >> v1:172.21.15.26:6790/0 conn(0x7f2174132b20 legacy=0x7f2174132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- 172.21.15.26:0/847986657 shutdown_connections 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- 172.21.15.26:0/847986657 >> 172.21.15.26:0/847986657 conn(0x7f217406e6e0 msgr2=0x7f21740a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- 172.21.15.26:0/847986657 shutdown_connections 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 1 -- 172.21.15.26:0/847986657 wait complete. 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.522+0000 7f217359e700 10 monclient: monmap: 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.538 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 Processor -- start 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- start start 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: init 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _add_conns ranks=[8,5,4,1,3,6,2,7,0] 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient(hunting): picked mon.noname-e con 0x7f217409fa20 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient(hunting): picked mon.noname-h con 0x7f2174132b20 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient(hunting): picked mon.noname-c con 0x7f2174138c30 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2174132490 con 0x7f2174138c30 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21741374b0 con 0x7f2174132b20 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21741389f0 con 0x7f217409fa20 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient(hunting): authenticate will time out at 2031.957269s 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217259c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f217409fa20 0x7f2174135430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33126/0 (socket says 172.21.15.26:33126) 2024-09-15T04:04:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217259c700 1 -- 172.21.15.26:0/3762028640 learned_addr learned my addr 172.21.15.26:0/3762028640 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448420354 0 0) 0x7f21741389f0 con 0x7f217409fa20 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): my global_id is 274328 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154003570 con 0x7f217409fa20 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415069485 0 0) 0x7f2174132490 con 0x7f2174138c30 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): my global_id is 205213 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21741389f0 con 0x7f2174138c30 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211166929 0 0) 0x7f21741374b0 con 0x7f2174132b20 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): my global_id is 274235 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2174132490 con 0x7f2174132b20 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 132198904 0 0) 0x7f2154003570 con 0x7f217409fa20 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21741374b0 con 0x7f217409fa20 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2168001aa0 con 0x7f217409fa20 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): dump: 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.545 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3873204866 0 0) 0x7f2174132490 con 0x7f2174132b20 2024-09-15T04:04:47.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2154003570 con 0x7f2174132b20 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2164004b70 con 0x7f2174132b20 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient(hunting): dump: 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2635978859 0 0) 0x7f21741389f0 con 0x7f2174138c30 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2174132490 con 0x7f2174138c30 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4220467041 0 0) 0x7f21741374b0 con 0x7f217409fa20 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 >> v1:172.21.15.122:6791/0 conn(0x7f2174132b20 legacy=0x7f2174135b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 >> v1:172.21.15.26:6791/0 conn(0x7f2174138c30 legacy=0x7f2174136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 monclient: found mon.e 2024-09-15T04:04:47.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740a17d0 con 0x7f217409fa20 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 5 monclient: authenticate success, global_id 274328 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _renew_subs 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217409cce0 con 0x7f217409fa20 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _renew_subs 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2174137b00 con 0x7f217409fa20 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:47.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6793/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f217411c650 con 0x7f217409fa20 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f2168002230 con 0x7f217409fa20 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f2168002760 con 0x7f217409fa20 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f21627fc700 10 monclient: dump: 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.526+0000 7f2170d99700 4 set_mon_vals no callback set 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f2168002940 con 0x7f217409fa20 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f2168043e70 con 0x7f217409fa20 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6808/21148 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2154042700 con 0x7f215403eaa0 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_get_version_reply(handle=1 version=904) v2 ==== 24+0+0 (unknown 607715002 0 0) 0x7f2168044290 con 0x7f217409fa20 2024-09-15T04:04:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f21627fc700 10 monclient: handle_get_version_reply finishing 1 version 904 2024-09-15T04:04:47.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f2171d9b700 10 monclient: get_auth_request con 0x7f2154037e90 auth_method 0 2024-09-15T04:04:47.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f2172d9d700 10 monclient: get_auth_request con 0x7f215403eaa0 auth_method 0 2024-09-15T04:04:47.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.530+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== osd.2 v1:172.21.15.26:6808/21148 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f2154042700 con 0x7f215403eaa0 2024-09-15T04:04:47.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.546+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 --> v1:172.21.15.26:6808/21148 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2174068140 con 0x7f215403eaa0 2024-09-15T04:04:47.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.546+0000 7f21627fc700 1 -- 172.21.15.26:0/3762028640 <== osd.2 v1:172.21.15.26:6808/21148 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f2174068140 con 0x7f215403eaa0 2024-09-15T04:04:47.555 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:47.555 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T04:04:47.555 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 >> v1:172.21.15.26:6808/21148 conn(0x7f215403eaa0 legacy=0x7f2154040eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 >> v1:172.21.15.122:6812/20895 conn(0x7f2154037e90 legacy=0x7f215403a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 10 monclient: shutdown 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 >> v1:172.21.15.26:6793/0 conn(0x7f217409fa20 legacy=0x7f2174135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f2172d9d700 1 -- 172.21.15.26:0/3762028640 reap_dead start 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 shutdown_connections 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 >> 172.21.15.26:0/3762028640 conn(0x7f217406e6e0 msgr2=0x7f21740677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 shutdown_connections 2024-09-15T04:04:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.550+0000 7f217359e700 1 -- 172.21.15.26:0/3762028640 wait complete. 2024-09-15T04:04:47.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T04:04:47.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.666+0000 7f53d9cee700 10 monclient: get_monmap_and_config 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.666+0000 7f53d9cee700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.666+0000 7f53d9cee700 10 monclient: monmap: 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:47.671222+0000 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:47.671222+0000 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.676 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.666+0000 7f53d9cee700 1 Processor -- start 2024-09-15T04:04:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.666+0000 7f53d9cee700 1 -- start start 2024-09-15T04:04:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.666+0000 7f53d9cee700 10 monclient: init 2024-09-15T04:04:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient: _add_conns ranks=[0,3,1,6,2,4,5,7,8] 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient(hunting): picked mon.noname-a con 0x7f53d406e530 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient(hunting): picked mon.noname-g con 0x7f53d4099240 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient(hunting): picked mon.noname-f con 0x7f53d4138d70 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d404ca10 con 0x7f53d406e530 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d409e310 con 0x7f53d4138d70 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d4133730 con 0x7f53d4099240 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient(hunting): authenticate will time out at 2032.100581s 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d8cec700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53d406e530 0x7f53d4133020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42562/0 (socket says 172.21.15.26:42562) 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d8cec700 1 -- 172.21.15.26:0/1820745388 learned_addr learned my addr 172.21.15.26:0/1820745388 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655380938 0 0) 0x7f53d404ca10 con 0x7f53d406e530 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): my global_id is 195354 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53bc003570 con 0x7f53d406e530 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526469340 0 0) 0x7f53d409e310 con 0x7f53d4138d70 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): my global_id is 274204 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d404ca10 con 0x7f53d4138d70 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483754035 0 0) 0x7f53d4133730 con 0x7f53d4099240 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient(hunting): my global_id is 254487 2024-09-15T04:04:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d409e310 con 0x7f53d4099240 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4291296346 0 0) 0x7f53bc003570 con 0x7f53d406e530 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 >> v1:172.21.15.122:6790/0 conn(0x7f53d4099240 legacy=0x7f53d4099610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 >> v1:172.21.15.122:6789/0 conn(0x7f53d4138d70 legacy=0x7f53d413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 monclient: found mon.noname-a 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4097b00 con 0x7f53d406e530 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 5 monclient: authenticate success, global_id 195354 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c80017b0 con 0x7f53d406e530 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: dump: 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.687 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53c8001a90 con 0x7f53d406e530 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 4 set_mon_vals no callback set 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 1 -- 172.21.15.26:0/1820745388 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c8004c30 con 0x7f53d406e530 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d37fe700 10 monclient: dump: 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.688 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.689 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient: shutdown 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- 172.21.15.26:0/1820745388 >> v1:172.21.15.26:6789/0 conn(0x7f53d406e530 legacy=0x7f53d4133020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- 172.21.15.26:0/1820745388 shutdown_connections 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- 172.21.15.26:0/1820745388 >> 172.21.15.26:0/1820745388 conn(0x7f53d4070de0 msgr2=0x7f53d40711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- 172.21.15.26:0/1820745388 shutdown_connections 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 1 -- 172.21.15.26:0/1820745388 wait complete. 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.670+0000 7f53d9cee700 10 monclient: monmap: 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:47.690 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 Processor -- start 2024-09-15T04:04:47.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 -- start start 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: init 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: _add_conns ranks=[2,4,5,3,6,7,1,8,0] 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient(hunting): picked mon.noname-b con 0x7f53d406e530 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient(hunting): picked mon.noname-c con 0x7f53d4099240 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient(hunting): picked mon.noname-h con 0x7f53d4138d70 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d409ea60 con 0x7f53d406e530 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d409e790 con 0x7f53d4099240 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d4137940 con 0x7f53d4138d70 2024-09-15T04:04:47.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient(hunting): authenticate will time out at 2032.104942s 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d8cec700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f53d406e530 0x7f53d412ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42282/0 (socket says 172.21.15.26:42282) 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d8cec700 1 -- 172.21.15.26:0/2150122601 learned_addr learned my addr 172.21.15.26:0/2150122601 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d3fff700 1 --1- 172.21.15.26:0/2150122601 >> v1:172.21.15.26:6791/0 conn(0x7f53d4099240 0x7f53d412f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:54760/0 (socket says 172.21.15.26:54760) 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719369050 0 0) 0x7f53d409ea60 con 0x7f53d406e530 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): my global_id is 254387 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c0003570 con 0x7f53d406e530 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454447178 0 0) 0x7f53d409e790 con 0x7f53d4099240 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): my global_id is 205222 2024-09-15T04:04:47.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d409ea60 con 0x7f53d4099240 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446005008 0 0) 0x7f53d4137940 con 0x7f53d4138d70 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): my global_id is 274244 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d409e790 con 0x7f53d4138d70 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 251824054 0 0) 0x7f53c0003570 con 0x7f53d406e530 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53d4137940 con 0x7f53d406e530 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c8004b70 con 0x7f53d406e530 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2314749278 0 0) 0x7f53d409ea60 con 0x7f53d4099240 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53c0003570 con 0x7f53d4099240 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c4002ab0 con 0x7f53d4099240 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.696 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 539481872 0 0) 0x7f53d409e790 con 0x7f53d4138d70 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53d409ea60 con 0x7f53d4138d70 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53cc001d00 con 0x7f53d4138d70 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient(hunting): dump: 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.698 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3305096816 0 0) 0x7f53d4137940 con 0x7f53d406e530 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 >> v1:172.21.15.122:6791/0 conn(0x7f53d4138d70 legacy=0x7f53d414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 >> v1:172.21.15.26:6791/0 conn(0x7f53d4099240 legacy=0x7f53d412f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 monclient: found mon.b 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4134110 con 0x7f53d406e530 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 5 monclient: authenticate success, global_id 254387 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: _renew_subs 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d412fc60 con 0x7f53d406e530 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: _renew_subs 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d41309f0 con 0x7f53d406e530 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f53c80011d0 con 0x7f53d406e530 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f53c8005230 con 0x7f53d406e530 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d2ffd700 4 set_mon_vals no callback set 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 10 monclient: dump: 2024-09-15T04:04:47.700 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.701 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.674+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f53c800fbf0 con 0x7f53d406e530 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.678+0000 7f53d9cee700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.678+0000 7f53d9cee700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.678+0000 7f53d9cee700 10 monclient: _send_mon_message to mon.b at v1:172.21.15.26:6790/0 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.678+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.26:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b40050b0 con 0x7f53d406e530 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.678+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f53c8043e60 con 0x7f53d406e530 2024-09-15T04:04:47.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.678+0000 7f53d3fff700 10 monclient: get_auth_request con 0x7f53c00380d0 auth_method 0 2024-09-15T04:04:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.686+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mon.2 v1:172.21.15.26:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f53c8014af0 con 0x7f53d406e530 2024-09-15T04:04:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.686+0000 7f53d0ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:47.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.686+0000 7f53d0ff9700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.850+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f53b4000b90 con 0x7f53c00380d0 2024-09-15T04:04:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d0ff9700 1 -- 172.21.15.26:0/2150122601 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (unknown 3049115190 0 0) 0x7f53b4000b90 con 0x7f53c00380d0 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 >> v1:172.21.15.122:6812/20895 conn(0x7f53c00380d0 legacy=0x7f53c003a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 10 monclient: shutdown 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 >> v1:172.21.15.26:6790/0 conn(0x7f53d406e530 legacy=0x7f53d412ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 shutdown_connections 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 >> 172.21.15.26:0/2150122601 conn(0x7f53d4070de0 msgr2=0x7f53d4098a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 shutdown_connections 2024-09-15T04:04:47.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.854+0000 7f53d9cee700 1 -- 172.21.15.26:0/2150122601 wait complete. 2024-09-15T04:04:47.863 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T04:04:47.873 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T04:04:47.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient: get_monmap_and_config 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient: build_initial_monmap 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient: monmap: 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:47.982402+0000 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:47.982402+0000 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.986 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:47.987 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 1 Processor -- start 2024-09-15T04:04:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 1 -- start start 2024-09-15T04:04:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient: init 2024-09-15T04:04:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient: _add_conns ranks=[7,6,0,3,4,5,8,2,1] 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient(hunting): picked mon.noname-i con 0x7f4f9009a1d0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.978+0000 7f4f949f7700 10 monclient(hunting): picked mon.noname-d con 0x7f4f9009aca0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient(hunting): picked mon.noname-a con 0x7f4f9009a5a0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f90070420 con 0x7f4f9009a5a0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f90069680 con 0x7f4f9009aca0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f90068e00 con 0x7f4f9009a1d0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient(hunting): authenticate will time out at 2032.412133s 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8e59c700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f4f9009aca0 0x7f4f9009f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48148/0 (socket says 172.21.15.26:48148) 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8e59c700 1 -- 172.21.15.26:0/1297563853 learned_addr learned my addr 172.21.15.26:0/1297563853 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8f59e700 1 --1- 172.21.15.26:0/1297563853 >> v1:172.21.15.26:6789/0 conn(0x7f4f9009a5a0 0x7f4f9009f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42572/0 (socket says 172.21.15.26:42572) 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635172046 0 0) 0x7f4f90069680 con 0x7f4f9009aca0 2024-09-15T04:04:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): my global_id is 195441 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f78003570 con 0x7f4f9009aca0 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910852707 0 0) 0x7f4f90070420 con 0x7f4f9009a5a0 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): my global_id is 195363 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f90069680 con 0x7f4f9009a5a0 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504474986 0 0) 0x7f4f90068e00 con 0x7f4f9009a1d0 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient(hunting): my global_id is 254536 2024-09-15T04:04:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f90070420 con 0x7f4f9009a1d0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224403121 0 0) 0x7f4f78003570 con 0x7f4f9009aca0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 >> v1:172.21.15.122:6792/0 conn(0x7f4f9009a1d0 legacy=0x7f4f9009e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 >> v1:172.21.15.26:6789/0 conn(0x7f4f9009a5a0 legacy=0x7f4f9009f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: _finish_hunting 0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 monclient: found mon.noname-d 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90067640 con 0x7f4f9009aca0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: _finish_auth 0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 5 monclient: authenticate success, global_id 195441 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f840017b0 con 0x7f4f9009aca0 2024-09-15T04:04:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: dump: 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f84001a20 con 0x7f4f9009aca0 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 1 -- 172.21.15.26:0/1297563853 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f84004c30 con 0x7f4f9009aca0 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 4 set_mon_vals no callback set 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:47.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f8dd9b700 10 monclient: dump: 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:47.997 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient: shutdown 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- 172.21.15.26:0/1297563853 >> v1:172.21.15.26:6792/0 conn(0x7f4f9009aca0 legacy=0x7f4f9009f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- 172.21.15.26:0/1297563853 shutdown_connections 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- 172.21.15.26:0/1297563853 >> 172.21.15.26:0/1297563853 conn(0x7f4f900a4a80 msgr2=0x7f4f900a0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- 172.21.15.26:0/1297563853 shutdown_connections 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 -- 172.21.15.26:0/1297563853 wait complete. 2024-09-15T04:04:47.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 10 monclient: monmap: 2024-09-15T04:04:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:48.000 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.982+0000 7f4f949f7700 1 Processor -- start 2024-09-15T04:04:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 1 -- start start 2024-09-15T04:04:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: init 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: _add_conns ranks=[4,6,8,7,5,3,2,1,0] 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient(hunting): picked mon.noname-c con 0x7f4f9009a1d0 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient(hunting): picked mon.noname-d con 0x7f4f9009a5a0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient(hunting): picked mon.noname-e con 0x7f4f9009aca0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f90122d50 con 0x7f4f9009a1d0 2024-09-15T04:04:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9013e0e0 con 0x7f4f9009a5a0 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9013ede0 con 0x7f4f9009aca0 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient(hunting): authenticate will time out at 2032.416866s 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f8ed9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f4f9009a1d0 0x7f4f9013d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:54762/0 (socket says 172.21.15.26:54762) 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f8ed9d700 1 -- 172.21.15.26:0/3658202088 learned_addr learned my addr 172.21.15.26:0/3658202088 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951908498 0 0) 0x7f4f90122d50 con 0x7f4f9009a1d0 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): my global_id is 205231 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f7c003570 con 0x7f4f9009a1d0 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230666817 0 0) 0x7f4f9013e0e0 con 0x7f4f9009a5a0 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): my global_id is 195450 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f90122d50 con 0x7f4f9009a5a0 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148633803 0 0) 0x7f4f9013ede0 con 0x7f4f9009aca0 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): my global_id is 274337 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f9013e0e0 con 0x7f4f9009aca0 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 581798346 0 0) 0x7f4f7c003570 con 0x7f4f9009a1d0 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f9013ede0 con 0x7f4f9009a1d0 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f80002b90 con 0x7f4f9009a1d0 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2698327281 0 0) 0x7f4f90122d50 con 0x7f4f9009a5a0 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f7c003570 con 0x7f4f9009a5a0 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4024879319 0 0) 0x7f4f9013e0e0 con 0x7f4f9009aca0 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4f90122d50 con 0x7f4f9009aca0 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f840017b0 con 0x7f4f9009a5a0 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f88001890 con 0x7f4f9009aca0 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1088307596 0 0) 0x7f4f7c003570 con 0x7f4f9009a5a0 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 >> v1:172.21.15.26:6793/0 conn(0x7f4f9009aca0 legacy=0x7f4f9014a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 >> v1:172.21.15.26:6791/0 conn(0x7f4f9009a1d0 legacy=0x7f4f9013d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 monclient: found mon.d 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90122a70 con 0x7f4f9009a5a0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 5 monclient: authenticate success, global_id 195450 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: _renew_subs 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f90123030 con 0x7f4f9009a5a0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: _renew_subs 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f90120680 con 0x7f4f9009a5a0 2024-09-15T04:04:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f4f84004c60 con 0x7f4f9009a5a0 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f4f840051a0 con 0x7f4f9009a5a0 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f8d59a700 4 set_mon_vals no callback set 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 10 monclient: dump: 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.986+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f4f8400fb60 con 0x7f4f9009a5a0 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f949f7700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.26:6792/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f70000f20 con 0x7f4f9009a5a0 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f4f84043db0 con 0x7f4f9009a5a0 2024-09-15T04:04:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.122:6800/21255 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f7c0429b0 con 0x7f4f7c03ed50 2024-09-15T04:04:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_get_version_reply(handle=1 version=904) v2 ==== 24+0+0 (unknown 607715002 0 0) 0x7f4f84014b70 con 0x7f4f9009a5a0 2024-09-15T04:04:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f76ffd700 10 monclient: handle_get_version_reply finishing 1 version 904 2024-09-15T04:04:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f8ed9d700 10 monclient: get_auth_request con 0x7f4f7c038140 auth_method 0 2024-09-15T04:04:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f8f59e700 10 monclient: get_auth_request con 0x7f4f7c03ed50 auth_method 0 2024-09-15T04:04:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:47.990+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== osd.3 v1:172.21.15.122:6800/21255 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f4f7c0429b0 con 0x7f4f7c03ed50 2024-09-15T04:04:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 --> v1:172.21.15.122:6800/21255 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4f70002bd0 con 0x7f4f7c03ed50 2024-09-15T04:04:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f76ffd700 1 -- 172.21.15.26:0/3658202088 <== osd.3 v1:172.21.15.122:6800/21255 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f4f70002bd0 con 0x7f4f7c03ed50 2024-09-15T04:04:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:48.020 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T04:04:48.020 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 >> v1:172.21.15.122:6800/21255 conn(0x7f4f7c03ed50 legacy=0x7f4f7c041160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 >> v1:172.21.15.122:6812/20895 conn(0x7f4f7c038140 legacy=0x7f4f7c03a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 10 monclient: shutdown 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 >> v1:172.21.15.26:6792/0 conn(0x7f4f9009a5a0 legacy=0x7f4f90146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f8f59e700 1 -- 172.21.15.26:0/3658202088 reap_dead start 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 shutdown_connections 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 >> 172.21.15.26:0/3658202088 conn(0x7f4f900a4a80 msgr2=0x7f4f9006d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 shutdown_connections 2024-09-15T04:04:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.014+0000 7f4f949f7700 1 -- 172.21.15.26:0/3658202088 wait complete. 2024-09-15T04:04:48.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.142+0000 7f116ffff700 10 monclient: get_monmap_and_config 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.142+0000 7f116ffff700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.142+0000 7f116ffff700 10 monclient: monmap: 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:48.146615+0000 2024-09-15T04:04:48.150 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:48.146615+0000 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.151 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.142+0000 7f116ffff700 1 Processor -- start 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.142+0000 7f116ffff700 1 -- start start 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.142+0000 7f116ffff700 10 monclient: init 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: _add_conns ranks=[4,7,5,0,6,1,2,8,3] 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): picked mon.noname-c con 0x7f11700a0100 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): picked mon.noname-i con 0x7f11700992f0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): picked mon.noname-h con 0x7f1170138d70 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117004ca10 con 0x7f11700a0100 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117009e070 con 0x7f1170138d70 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1170133750 con 0x7f11700992f0 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): authenticate will time out at 2032.576163s 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116effd700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f11700a0100 0x7f1170133040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:54772/0 (socket says 172.21.15.26:54772) 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116effd700 1 -- 172.21.15.26:0/2532614918 learned_addr learned my addr 172.21.15.26:0/2532614918 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221459277 0 0) 0x7f117004ca10 con 0x7f11700a0100 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): my global_id is 205240 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f115c003570 con 0x7f11700a0100 2024-09-15T04:04:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700164122 0 0) 0x7f117009e070 con 0x7f1170138d70 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): my global_id is 274253 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117004ca10 con 0x7f1170138d70 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692198181 0 0) 0x7f1170133750 con 0x7f11700992f0 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient(hunting): my global_id is 254545 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117009e070 con 0x7f11700992f0 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2626209379 0 0) 0x7f115c003570 con 0x7f11700a0100 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 >> v1:172.21.15.122:6792/0 conn(0x7f11700992f0 legacy=0x7f11700996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 >> v1:172.21.15.122:6791/0 conn(0x7f1170138d70 legacy=0x7f117013b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 monclient: found mon.noname-c 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117009db80 con 0x7f11700a0100 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f11600017b0 con 0x7f11700a0100 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: dump: 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.160 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 5 monclient: authenticate success, global_id 205240 2024-09-15T04:04:48.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1160001a20 con 0x7f11700a0100 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 4 set_mon_vals no callback set 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 1 -- 172.21.15.26:0/2532614918 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1160004c30 con 0x7f11700a0100 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116dffb700 10 monclient: dump: 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.162 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: shutdown 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- 172.21.15.26:0/2532614918 >> v1:172.21.15.26:6791/0 conn(0x7f11700a0100 legacy=0x7f1170133040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- 172.21.15.26:0/2532614918 shutdown_connections 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- 172.21.15.26:0/2532614918 >> 172.21.15.26:0/2532614918 conn(0x7f11700a3430 msgr2=0x7f11700a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- 172.21.15.26:0/2532614918 shutdown_connections 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- 172.21.15.26:0/2532614918 wait complete. 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: monmap: 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:48.164 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 Processor -- start 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- start start 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: init 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient: _add_conns ranks=[6,1,5,0,3,4,2,7,8] 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): picked mon.noname-d con 0x7f11700992f0 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): picked mon.noname-f con 0x7f11700a0100 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): picked mon.noname-h con 0x7f1170138d70 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117009e4f0 con 0x7f11700a0100 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1170137f20 con 0x7f1170138d70 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1170136580 con 0x7f11700992f0 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116ffff700 10 monclient(hunting): authenticate will time out at 2032.579687s 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116effd700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f11700992f0 0x7f1170135760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48166/0 (socket says 172.21.15.26:48166) 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.146+0000 7f116effd700 1 -- 172.21.15.26:0/3339947900 learned_addr learned my addr 172.21.15.26:0/3339947900 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815007770 0 0) 0x7f1170136580 con 0x7f11700992f0 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): my global_id is 195459 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1150003570 con 0x7f11700992f0 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454005213 0 0) 0x7f1170137f20 con 0x7f1170138d70 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): my global_id is 274262 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1170136580 con 0x7f1170138d70 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754259992 0 0) 0x7f117009e4f0 con 0x7f11700a0100 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): my global_id is 274213 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1170137f20 con 0x7f11700a0100 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3771559794 0 0) 0x7f1150003570 con 0x7f11700992f0 2024-09-15T04:04:48.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f117009e4f0 con 0x7f11700992f0 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1160001830 con 0x7f11700992f0 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.168 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 616953186 0 0) 0x7f1170136580 con 0x7f1170138d70 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1150003570 con 0x7f1170138d70 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1168001d00 con 0x7f1170138d70 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:04:48.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient(hunting): dump: 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.170 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 573931619 0 0) 0x7f117009e4f0 con 0x7f11700992f0 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 >> v1:172.21.15.122:6791/0 conn(0x7f1170138d70 legacy=0x7f117014a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 >> v1:172.21.15.122:6789/0 conn(0x7f11700a0100 legacy=0x7f1170135e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 monclient: found mon.d 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117011a280 con 0x7f11700992f0 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 5 monclient: authenticate success, global_id 195459 2024-09-15T04:04:48.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 monclient: _renew_subs 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.26:6792/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117011a560 con 0x7f11700992f0 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 monclient: _renew_subs 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.26:6792/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f11701372c0 con 0x7f11700992f0 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f11600011d0 con 0x7f11700992f0 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f1160005240 con 0x7f11700992f0 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116cff9700 4 set_mon_vals no callback set 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 10 monclient: dump: 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.172 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.173 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f116000fc00 con 0x7f11700992f0 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 10 monclient: _send_mon_message to mon.d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.26:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11540050b0 con 0x7f11700992f0 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f1160043fd0 con 0x7f11700992f0 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.150+0000 7f116e7fc700 10 monclient: get_auth_request con 0x7f1150037c40 auth_method 0 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.154+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mon.6 v1:172.21.15.26:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f1160011080 con 0x7f11700992f0 2024-09-15T04:04:48.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.154+0000 7f1166ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:48.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.154+0000 7f1166ffd700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:48.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.322+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1154000b90 con 0x7f1150037c40 2024-09-15T04:04:48.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f1166ffd700 1 -- 172.21.15.26:0/3339947900 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (unknown 3526570807 0 0) 0x7f1154000b90 con 0x7f1150037c40 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 >> v1:172.21.15.122:6812/20895 conn(0x7f1150037c40 legacy=0x7f115003a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 10 monclient: shutdown 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 >> v1:172.21.15.26:6792/0 conn(0x7f11700992f0 legacy=0x7f1170135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 shutdown_connections 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 >> 172.21.15.26:0/3339947900 conn(0x7f11700a3430 msgr2=0x7f1170138340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 shutdown_connections 2024-09-15T04:04:48.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.326+0000 7f116ffff700 1 -- 172.21.15.26:0/3339947900 wait complete. 2024-09-15T04:04:48.335 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T04:04:48.346 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T04:04:48.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient: get_monmap_and_config 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient: monmap: 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:48.457454+0000 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:48.457454+0000 2024-09-15T04:04:48.461 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.462 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 1 Processor -- start 2024-09-15T04:04:48.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 1 -- start start 2024-09-15T04:04:48.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient: init 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient: _add_conns ranks=[5,3,0,4,7,8,1,2,6] 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient(hunting): picked mon.noname-h con 0x7f6f4809fa20 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.454+0000 7f6f4d096700 10 monclient(hunting): picked mon.noname-g con 0x7f6f48132b20 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 10 monclient(hunting): picked mon.noname-a con 0x7f6f48138c30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f48070420 con 0x7f6f48138c30 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480a0de0 con 0x7f6f48132b20 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480a2430 con 0x7f6f4809fa20 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 10 monclient(hunting): authenticate will time out at 2032.888325s 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4c895700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f48138c30 0x7f6f4813b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42580/0 (socket says 172.21.15.26:42580) 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4c895700 1 -- 172.21.15.26:0/167510568 learned_addr learned my addr 172.21.15.26:0/167510568 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f47fff700 1 --1- 172.21.15.26:0/167510568 >> v1:172.21.15.122:6791/0 conn(0x7f6f4809fa20 0x7f6f4809fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6791/0 says I am v1:172.21.15.26:54538/0 (socket says 172.21.15.26:54538) 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104158748 0 0) 0x7f6f48070420 con 0x7f6f48138c30 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): my global_id is 195372 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f34003570 con 0x7f6f48138c30 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.3 v1:172.21.15.122:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268598459 0 0) 0x7f6f480a0de0 con 0x7f6f48132b20 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): my global_id is 254496 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 --> v1:172.21.15.122:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f48070420 con 0x7f6f48132b20 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376623454 0 0) 0x7f6f480a2430 con 0x7f6f4809fa20 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient(hunting): my global_id is 274271 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f480a0de0 con 0x7f6f4809fa20 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248296538 0 0) 0x7f6f34003570 con 0x7f6f48138c30 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 >> v1:172.21.15.122:6791/0 conn(0x7f6f4809fa20 legacy=0x7f6f4809fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 >> v1:172.21.15.122:6790/0 conn(0x7f6f48132b20 legacy=0x7f6f48132ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 monclient: found mon.noname-a 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.26:6789/0 2024-09-15T04:04:48.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480a2150 con 0x7f6f48138c30 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 5 monclient: authenticate success, global_id 195372 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6f40001b30 con 0x7f6f48138c30 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: dump: 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.471 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.0 v1:172.21.15.26:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6f40001e10 con 0x7f6f48138c30 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 1 -- 172.21.15.26:0/167510568 <== mon.0 v1:172.21.15.26:6789/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6f40004fb0 con 0x7f6f48138c30 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 4 set_mon_vals no callback set 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f46ffd700 10 monclient: dump: 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.473 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.474 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 10 monclient: shutdown 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- 172.21.15.26:0/167510568 >> v1:172.21.15.26:6789/0 conn(0x7f6f48138c30 legacy=0x7f6f4813b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- 172.21.15.26:0/167510568 shutdown_connections 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- 172.21.15.26:0/167510568 >> 172.21.15.26:0/167510568 conn(0x7f6f4806e6e0 msgr2=0x7f6f480a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- 172.21.15.26:0/167510568 shutdown_connections 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.458+0000 7f6f4d096700 1 -- 172.21.15.26:0/167510568 wait complete. 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: monmap: 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.476 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.477 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 1 Processor -- start 2024-09-15T04:04:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 1 -- start start 2024-09-15T04:04:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: init 2024-09-15T04:04:48.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: _add_conns ranks=[7,2,0,6,1,5,4,8,3] 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient(hunting): picked mon.noname-i con 0x7f6f4809fa20 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient(hunting): picked mon.noname-b con 0x7f6f48132b20 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient(hunting): picked mon.noname-a con 0x7f6f48138c30 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480a17d0 con 0x7f6f48138c30 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4809cce0 con 0x7f6f48132b20 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f48137e60 con 0x7f6f4809fa20 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient(hunting): authenticate will time out at 2032.894240s 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f477fe700 1 --1- >> v1:172.21.15.26:6790/0 conn(0x7f6f48132b20 0x7f6f48146830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6790/0 says I am v1:172.21.15.26:42298/0 (socket says 172.21.15.26:42298) 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f477fe700 1 -- 172.21.15.26:0/1990043921 learned_addr learned my addr 172.21.15.26:0/1990043921 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694351965 0 0) 0x7f6f4809cce0 con 0x7f6f48132b20 2024-09-15T04:04:48.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): my global_id is 254396 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f28003570 con 0x7f6f48132b20 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351616571 0 0) 0x7f6f480a17d0 con 0x7f6f48138c30 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): my global_id is 195381 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f4809cce0 con 0x7f6f48138c30 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985244774 0 0) 0x7f6f48137e60 con 0x7f6f4809fa20 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): my global_id is 254554 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f480a17d0 con 0x7f6f4809fa20 2024-09-15T04:04:48.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3000506965 0 0) 0x7f6f4809cce0 con 0x7f6f48138c30 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f48137e60 con 0x7f6f48138c30 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6f40004ef0 con 0x7f6f48138c30 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): dump: 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.484 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1773172449 0 0) 0x7f6f28003570 con 0x7f6f48132b20 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f4809cce0 con 0x7f6f48132b20 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6f30002b10 con 0x7f6f48132b20 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): dump: 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.486 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 804013917 0 0) 0x7f6f480a17d0 con 0x7f6f4809fa20 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6f28003570 con 0x7f6f4809fa20 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6f38001a20 con 0x7f6f4809fa20 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient(hunting): dump: 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.487 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.488 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 645984494 0 0) 0x7f6f48137e60 con 0x7f6f48138c30 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 >> v1:172.21.15.122:6792/0 conn(0x7f6f4809fa20 legacy=0x7f6f4809c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 >> v1:172.21.15.26:6790/0 conn(0x7f6f48132b20 legacy=0x7f6f48146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 monclient: found mon.a 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f481379e0 con 0x7f6f48138c30 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f3ffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 5 monclient: authenticate success, global_id 195381 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: _renew_subs 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.462+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f48137290 con 0x7f6f48138c30 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f4d096700 10 monclient: _renew_subs 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f4d096700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f481323f0 con 0x7f6f48138c30 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6f40001bc0 con 0x7f6f48138c30 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f6f400051e0 con 0x7f6f48138c30 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f467fc700 4 set_mon_vals no callback set 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 10 monclient: dump: 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f4d096700 10 monclient: _send_mon_message to mon.a at v1:172.21.15.26:6789/0 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.26:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f2c000f20 con 0x7f6f48138c30 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f6f4000fc20 con 0x7f6f48138c30 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f6f40043fe0 con 0x7f6f48138c30 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.122:6808/21254 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f280429b0 con 0x7f6f2803ed50 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== mon.0 v1:172.21.15.26:6789/0 9 ==== mon_get_version_reply(handle=1 version=904) v2 ==== 24+0+0 (unknown 607715002 0 0) 0x7f6f400443d0 con 0x7f6f48138c30 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f3ffff700 10 monclient: handle_get_version_reply finishing 1 version 904 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f47fff700 10 monclient: get_auth_request con 0x7f6f28038140 auth_method 0 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.466+0000 7f6f477fe700 10 monclient: get_auth_request con 0x7f6f2803ed50 auth_method 0 2024-09-15T04:04:48.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.470+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== osd.4 v1:172.21.15.122:6808/21254 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f6f280429b0 con 0x7f6f2803ed50 2024-09-15T04:04:48.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 --> v1:172.21.15.122:6808/21254 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6f2c002bd0 con 0x7f6f2803ed50 2024-09-15T04:04:48.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f3ffff700 1 -- 172.21.15.26:0/1990043921 <== osd.4 v1:172.21.15.122:6808/21254 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7f6f2c002bd0 con 0x7f6f2803ed50 2024-09-15T04:04:48.494 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:48.494 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T04:04:48.494 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 >> v1:172.21.15.122:6808/21254 conn(0x7f6f2803ed50 legacy=0x7f6f28041160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 >> v1:172.21.15.122:6812/20895 conn(0x7f6f28038140 legacy=0x7f6f2803a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 10 monclient: shutdown 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 >> v1:172.21.15.26:6789/0 conn(0x7f6f48138c30 legacy=0x7f6f48149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4c895700 1 -- 172.21.15.26:0/1990043921 reap_dead start 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 shutdown_connections 2024-09-15T04:04:48.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 >> 172.21.15.26:0/1990043921 conn(0x7f6f4806e6e0 msgr2=0x7f6f4813b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 shutdown_connections 2024-09-15T04:04:48.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.486+0000 7f6f4d096700 1 -- 172.21.15.26:0/1990043921 wait complete. 2024-09-15T04:04:48.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: get_monmap_and_config 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: monmap: 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:48.616224+0000 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:48.616224+0000 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.620 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.621 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 1 Processor -- start 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 1 -- start start 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: init 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: _add_conns ranks=[6,1,4,8,7,3,5,0,2] 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient(hunting): picked mon.noname-d con 0x7f9d18132b40 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient(hunting): picked mon.noname-f con 0x7f9d1809faa0 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient(hunting): picked mon.noname-c con 0x7f9d18138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1804ca10 con 0x7f9d1809faa0 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1809d5d0 con 0x7f9d18138d40 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d18133620 con 0x7f9d18132b40 2024-09-15T04:04:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient(hunting): authenticate will time out at 2033.046496s 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d17fff700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f9d18132b40 0x7f9d18132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48176/0 (socket says 172.21.15.26:48176) 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d17fff700 1 -- 172.21.15.26:0/2358668565 learned_addr learned my addr 172.21.15.26:0/2358668565 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039310687 0 0) 0x7f9d18133620 con 0x7f9d18132b40 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): my global_id is 195468 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d00003570 con 0x7f9d18132b40 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424111641 0 0) 0x7f9d1809d5d0 con 0x7f9d18138d40 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): my global_id is 205249 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18133620 con 0x7f9d18138d40 2024-09-15T04:04:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117474250 0 0) 0x7f9d1804ca10 con 0x7f9d1809faa0 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient(hunting): my global_id is 274222 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d1809d5d0 con 0x7f9d1809faa0 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110996269 0 0) 0x7f9d00003570 con 0x7f9d18132b40 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 >> v1:172.21.15.26:6791/0 conn(0x7f9d18138d40 legacy=0x7f9d1813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 >> v1:172.21.15.122:6789/0 conn(0x7f9d1809faa0 legacy=0x7f9d1809fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 monclient: found mon.noname-d 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.26:6792/0 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 --> v1:172.21.15.26:6792/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180a0cc0 con 0x7f9d18132b40 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 5 monclient: authenticate success, global_id 195468 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d080017b0 con 0x7f9d18132b40 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: dump: 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.628 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.629 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.6 v1:172.21.15.26:6792/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d080019d0 con 0x7f9d18132b40 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 1 -- 172.21.15.26:0/2358668565 <== mon.6 v1:172.21.15.26:6792/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d080048f0 con 0x7f9d18132b40 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d1d148700 4 set_mon_vals no callback set 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.614+0000 7f9d16ffd700 10 monclient: dump: 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.630 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 10 monclient: shutdown 2024-09-15T04:04:48.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 -- 172.21.15.26:0/2358668565 >> v1:172.21.15.26:6792/0 conn(0x7f9d18132b40 legacy=0x7f9d18132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 -- 172.21.15.26:0/2358668565 shutdown_connections 2024-09-15T04:04:48.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 -- 172.21.15.26:0/2358668565 >> 172.21.15.26:0/2358668565 conn(0x7f9d1806e370 msgr2=0x7f9d180a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 -- 172.21.15.26:0/2358668565 shutdown_connections 2024-09-15T04:04:48.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 -- 172.21.15.26:0/2358668565 wait complete. 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 10 monclient: monmap: 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.637 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 Processor -- start 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 1 -- start start 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.618+0000 7f9d1d148700 10 monclient: init 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _add_conns ranks=[0,4,1,7,5,6,2,3,8] 2024-09-15T04:04:48.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient(hunting): picked mon.noname-a con 0x7f9d1809faa0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient(hunting): picked mon.noname-c con 0x7f9d18132b40 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient(hunting): picked mon.noname-f con 0x7f9d18138d40 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d18139620 con 0x7f9d1809faa0 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1809cfa0 con 0x7f9d18138d40 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d17fff700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d1809faa0 0x7f9d18131790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42610/0 (socket says 172.21.15.26:42610) 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1809cce0 con 0x7f9d18132b40 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d17fff700 1 -- 172.21.15.26:0/3594977489 learned_addr learned my addr 172.21.15.26:0/3594977489 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient(hunting): authenticate will time out at 2033.052835s 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304976436 0 0) 0x7f9d1809cce0 con 0x7f9d18132b40 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): my global_id is 205258 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d04003570 con 0x7f9d18132b40 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522251996 0 0) 0x7f9d18139620 con 0x7f9d1809faa0 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): my global_id is 195390 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d1809cce0 con 0x7f9d1809faa0 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515674648 0 0) 0x7f9d1809cfa0 con 0x7f9d18138d40 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): my global_id is 274231 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18139620 con 0x7f9d18138d40 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3080398704 0 0) 0x7f9d04003570 con 0x7f9d18132b40 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d1809cfa0 con 0x7f9d18132b40 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d0c001a60 con 0x7f9d18132b40 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): dump: 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.641 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 214874679 0 0) 0x7f9d1809cce0 con 0x7f9d1809faa0 2024-09-15T04:04:48.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d04003570 con 0x7f9d1809faa0 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d080016b0 con 0x7f9d1809faa0 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient(hunting): dump: 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.643 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3676445508 0 0) 0x7f9d1809cfa0 con 0x7f9d18132b40 2024-09-15T04:04:48.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 >> v1:172.21.15.122:6789/0 conn(0x7f9d18138d40 legacy=0x7f9d1814ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 >> v1:172.21.15.26:6789/0 conn(0x7f9d1809faa0 legacy=0x7f9d18131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 monclient: found mon.c 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1811ce70 con 0x7f9d18132b40 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 5 monclient: authenticate success, global_id 205258 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _renew_subs 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d18137940 con 0x7f9d18132b40 2024-09-15T04:04:48.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _renew_subs 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d0c001ff0 con 0x7f9d18132b40 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1811c890 con 0x7f9d18132b40 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d0c002520 con 0x7f9d18132b40 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d167fc700 4 set_mon_vals no callback set 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 10 monclient: dump: 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.646 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9d0c0026a0 con 0x7f9d18132b40 2024-09-15T04:04:48.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.622+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1804bbf0 con 0x7f9d18132b40 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.626+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f9d0c0411e0 con 0x7f9d18132b40 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.626+0000 7f9d17fff700 10 monclient: get_auth_request con 0x7f9d04037d50 auth_method 0 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.630+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f9d0c0121d0 con 0x7f9d18132b40 2024-09-15T04:04:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.630+0000 7f9cfffff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.630+0000 7f9cfffff700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:48.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d18137940 con 0x7f9d04037d50 2024-09-15T04:04:48.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9cfffff700 1 -- 172.21.15.26:0/3594977489 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (unknown 3782719169 0 0) 0x7f9d18137940 con 0x7f9d04037d50 2024-09-15T04:04:48.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 >> v1:172.21.15.122:6812/20895 conn(0x7f9d04037d50 legacy=0x7f9d0403a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 10 monclient: shutdown 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 >> v1:172.21.15.26:6791/0 conn(0x7f9d18132b40 legacy=0x7f9d181473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 shutdown_connections 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 >> 172.21.15.26:0/3594977489 conn(0x7f9d1806e370 msgr2=0x7f9d1809da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 shutdown_connections 2024-09-15T04:04:48.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.794+0000 7f9d1d148700 1 -- 172.21.15.26:0/3594977489 wait complete. 2024-09-15T04:04:48.804 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T04:04:48.814 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T04:04:48.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient: get_monmap_and_config 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient: monmap: 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:48.925947+0000 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:48.925947+0000 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.930 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 1 Processor -- start 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 1 -- start start 2024-09-15T04:04:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient: init 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient: _add_conns ranks=[3,2,0,5,7,6,1,4,8] 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient(hunting): picked mon.noname-g con 0x7fd5d0097a60 addr v1:172.21.15.122:6790/0 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient(hunting): picked mon.noname-b con 0x7fd5d0097e30 addr v1:172.21.15.26:6790/0 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient(hunting): picked mon.noname-a con 0x7fd5d0135bd0 addr v1:172.21.15.26:6789/0 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d0070420 con 0x7fd5d0135bd0 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d0099e60 con 0x7fd5d0097e30 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 1 -- --> v1:172.21.15.122:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d009afa0 con 0x7fd5d0097a60 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d6f1d700 10 monclient(hunting): authenticate will time out at 2033.355260s 2024-09-15T04:04:48.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d671c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5d0135bd0 0x7fd5d0137fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42620/0 (socket says 172.21.15.26:42620) 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d671c700 1 -- 172.21.15.26:0/2239452493 learned_addr learned my addr 172.21.15.26:0/2239452493 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166172804 0 0) 0x7fd5d0099e60 con 0x7fd5d0097e30 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 10 monclient(hunting): my global_id is 254405 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5bc003570 con 0x7fd5d0097e30 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628994703 0 0) 0x7fd5d0070420 con 0x7fd5d0135bd0 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 10 monclient(hunting): my global_id is 195399 2024-09-15T04:04:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.922+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d0099e60 con 0x7fd5d0135bd0 2024-09-15T04:04:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 424054246 0 0) 0x7fd5bc003570 con 0x7fd5d0097e30 2024-09-15T04:04:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 >> v1:172.21.15.122:6790/0 conn(0x7fd5d0097a60 legacy=0x7fd5d009a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 >> v1:172.21.15.26:6789/0 conn(0x7fd5d0135bd0 legacy=0x7fd5d0137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 monclient: found mon.noname-b 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.26:6790/0 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 --> v1:172.21.15.26:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d009da60 con 0x7fd5d0097e30 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 5 monclient: authenticate success, global_id 254405 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5cc0017b0 con 0x7fd5d0097e30 2024-09-15T04:04:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: dump: 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 <== mon.2 v1:172.21.15.26:6790/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd5cc0019e0 con 0x7fd5d0097e30 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 4 set_mon_vals no callback set 2024-09-15T04:04:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 1 -- 172.21.15.26:0/2239452493 <== mon.2 v1:172.21.15.26:6790/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5cc002240 con 0x7fd5d0097e30 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d4f19700 10 monclient: dump: 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: shutdown 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/2239452493 >> v1:172.21.15.26:6790/0 conn(0x7fd5d0097e30 legacy=0x7fd5d0098220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/2239452493 shutdown_connections 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/2239452493 >> 172.21.15.26:0/2239452493 conn(0x7fd5d0063af0 msgr2=0x7fd5d00a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/2239452493 shutdown_connections 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/2239452493 wait complete. 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: build_initial_monmap 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: monmap: 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:48.945 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 Processor -- start 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- start start 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: init 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient: _add_conns ranks=[8,4,1,0,2,5,7,6,3] 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient(hunting): picked mon.noname-e con 0x7fd5d0097a60 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient(hunting): picked mon.noname-c con 0x7fd5d0097e30 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient(hunting): picked mon.noname-f con 0x7fd5d0135bd0 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d0069e00 con 0x7fd5d0135bd0 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d0130990 con 0x7fd5d0097e30 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d0099400 con 0x7fd5d0097a60 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d6f1d700 10 monclient(hunting): authenticate will time out at 2033.359419s 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d5f1b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fd5d0097a60 0x7fd5d013f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33142/0 (socket says 172.21.15.26:33142) 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5d5f1b700 1 -- 172.21.15.26:0/482106094 learned_addr learned my addr 172.21.15.26:0/482106094 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528010395 0 0) 0x7fd5d0099400 con 0x7fd5d0097a60 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5c5ffb700 10 monclient(hunting): my global_id is 274346 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.926+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b4003570 con 0x7fd5d0097a60 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968056046 0 0) 0x7fd5d0130990 con 0x7fd5d0097e30 2024-09-15T04:04:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): my global_id is 205267 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d0099400 con 0x7fd5d0097e30 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122842594 0 0) 0x7fd5d0069e00 con 0x7fd5d0135bd0 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): my global_id is 274240 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d0130990 con 0x7fd5d0135bd0 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 432682693 0 0) 0x7fd5b4003570 con 0x7fd5d0097a60 2024-09-15T04:04:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5d0069e00 con 0x7fd5d0097a60 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 895561074 0 0) 0x7fd5d0099400 con 0x7fd5d0097e30 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd5b4003570 con 0x7fd5d0097e30 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5cc001e90 con 0x7fd5d0097e30 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): dump: 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.951 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5c0001a60 con 0x7fd5d0097a60 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient(hunting): dump: 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3990071066 0 0) 0x7fd5d0069e00 con 0x7fd5d0097a60 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 >> v1:172.21.15.26:6791/0 conn(0x7fd5d0097e30 legacy=0x7fd5d013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 >> v1:172.21.15.122:6789/0 conn(0x7fd5d0135bd0 legacy=0x7fd5d0147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: _finish_hunting 0 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 monclient: found mon.e 2024-09-15T04:04:48.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d013fe50 con 0x7fd5d0097a60 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: _finish_auth 0 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 5 monclient: authenticate success, global_id 274346 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 10 monclient: _renew_subs 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d006a5f0 con 0x7fd5d0097a60 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 10 monclient: _renew_subs 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d0134ed0 con 0x7fd5d0097a60 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd5c0001c50 con 0x7fd5d0097a60 2024-09-15T04:04:48.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fd5c0002570 con 0x7fd5d0097a60 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c77fe700 4 set_mon_vals no callback set 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: dump: 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:48.956 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fd5c00026d0 con 0x7fd5d0097a60 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.26:6793/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5b0000f20 con 0x7fd5d0097a60 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fd5c0043bd0 con 0x7fd5d0097a60 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.122:6804/21256 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5b40447e0 con 0x7fd5b4040b80 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_get_version_reply(handle=1 version=904) v2 ==== 24+0+0 (unknown 607715002 0 0) 0x7fd5c0014810 con 0x7fd5d0097a60 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5c5ffb700 10 monclient: handle_get_version_reply finishing 1 version 904 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d571a700 10 monclient: get_auth_request con 0x7fd5b4039f70 auth_method 0 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.930+0000 7fd5d671c700 10 monclient: get_auth_request con 0x7fd5b4040b80 auth_method 0 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.934+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== osd.5 v1:172.21.15.122:6804/21256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fd5b40447e0 con 0x7fd5b4040b80 2024-09-15T04:04:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.950+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 --> v1:172.21.15.122:6804/21256 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd5b0002ba0 con 0x7fd5b4040b80 2024-09-15T04:04:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.950+0000 7fd5c5ffb700 1 -- 172.21.15.26:0/482106094 <== osd.5 v1:172.21.15.122:6804/21256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (unknown 0 0 119112422) 0x7fd5b0002ba0 con 0x7fd5b4040b80 2024-09-15T04:04:48.959 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-09-15T04:04:48.959 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T04:04:48.959 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-09-15T04:04:48.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 >> v1:172.21.15.122:6804/21256 conn(0x7fd5b4040b80 legacy=0x7fd5b4042f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 >> v1:172.21.15.122:6812/20895 conn(0x7fd5b4039f70 legacy=0x7fd5b403c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 10 monclient: shutdown 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 >> v1:172.21.15.26:6793/0 conn(0x7fd5d0097a60 legacy=0x7fd5d013f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d671c700 1 -- 172.21.15.26:0/482106094 reap_dead start 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 shutdown_connections 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 >> 172.21.15.26:0/482106094 conn(0x7fd5d0063af0 msgr2=0x7fd5d009b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 shutdown_connections 2024-09-15T04:04:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:48.954+0000 7fd5d6f1d700 1 -- 172.21.15.26:0/482106094 wait complete. 2024-09-15T04:04:48.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T04:04:49.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient: get_monmap_and_config 2024-09-15T04:04:49.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient: build_initial_monmap 2024-09-15T04:04:49.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient: monmap: 2024-09-15T04:04:49.086 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:49.086 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.086 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:49.081705+0000 2024-09-15T04:04:49.086 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:49.081705+0000 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:49.087 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 1 Processor -- start 2024-09-15T04:04:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 1 -- start start 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient: init 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient: _add_conns ranks=[8,4,1,2,3,0,5,6,7] 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient(hunting): picked mon.noname-e con 0x7f9da8132c90 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient(hunting): picked mon.noname-c con 0x7f9da8133060 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient(hunting): picked mon.noname-f con 0x7f9da8133470 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da804ca10 con 0x7f9da8133470 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da8068f90 con 0x7f9da8133060 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80a1430 con 0x7f9da8132c90 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9dacb5e700 10 monclient(hunting): authenticate will time out at 2033.511330s 2024-09-15T04:04:49.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9da659c700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f9da8132c90 0x7f9da809ff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33156/0 (socket says 172.21.15.26:33156) 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.078+0000 7f9da659c700 1 -- 172.21.15.26:0/3107240692 learned_addr learned my addr 172.21.15.26:0/3107240692 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022071683 0 0) 0x7f9da8068f90 con 0x7f9da8133060 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): my global_id is 205276 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d94003570 con 0x7f9da8133060 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926095025 0 0) 0x7f9da80a1430 con 0x7f9da8132c90 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): my global_id is 274355 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da8068f90 con 0x7f9da8132c90 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163247617 0 0) 0x7f9da804ca10 con 0x7f9da8133470 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient(hunting): my global_id is 274249 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da80a1430 con 0x7f9da8133470 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 782572148 0 0) 0x7f9d94003570 con 0x7f9da8133060 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 >> v1:172.21.15.26:6793/0 conn(0x7f9da8132c90 legacy=0x7f9da809ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 >> v1:172.21.15.122:6789/0 conn(0x7f9da8133470 legacy=0x7f9da813b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: _finish_hunting 0 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 monclient: found mon.noname-c 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da809da90 con 0x7f9da8133060 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: _finish_auth 0 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 5 monclient: authenticate success, global_id 205276 2024-09-15T04:04:49.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d900017b0 con 0x7f9da8133060 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: dump: 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.099 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d90001a90 con 0x7f9da8133060 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 4 set_mon_vals no callback set 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 1 -- 172.21.15.26:0/3107240692 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d90001fc0 con 0x7f9da8133060 2024-09-15T04:04:49.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9da559a700 10 monclient: dump: 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.101 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.102 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.102 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.102 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.102 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.102 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 10 monclient: shutdown 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 -- 172.21.15.26:0/3107240692 >> v1:172.21.15.26:6791/0 conn(0x7f9da8133060 legacy=0x7f9da8137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 -- 172.21.15.26:0/3107240692 shutdown_connections 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 -- 172.21.15.26:0/3107240692 >> 172.21.15.26:0/3107240692 conn(0x7f9da806e510 msgr2=0x7f9da80a6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 -- 172.21.15.26:0/3107240692 shutdown_connections 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 -- 172.21.15.26:0/3107240692 wait complete. 2024-09-15T04:04:49.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 10 monclient: build_initial_monmap 2024-09-15T04:04:49.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 10 monclient: monmap: 2024-09-15T04:04:49.104 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:49.104 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:49.104 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:49.104 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:49.104 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:49.105 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:49.105 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:49.105 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:49.105 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:49.105 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:49.105 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 Processor -- start 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 1 -- start start 2024-09-15T04:04:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.082+0000 7f9dacb5e700 10 monclient: init 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _add_conns ranks=[5,6,4,1,7,2,0,8,3] 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient(hunting): picked mon.noname-h con 0x7f9da8132c90 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient(hunting): picked mon.noname-d con 0x7f9da8133060 addr v1:172.21.15.26:6792/0 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient(hunting): picked mon.noname-c con 0x7f9da8133470 addr v1:172.21.15.26:6791/0 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da811fe20 con 0x7f9da8133470 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da811e080 con 0x7f9da8132c90 2024-09-15T04:04:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 1 -- --> v1:172.21.15.26:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da811e7d0 con 0x7f9da8133060 2024-09-15T04:04:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient(hunting): authenticate will time out at 2033.516380s 2024-09-15T04:04:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9da6d9d700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f9da8133470 0x7f9da814a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:54814/0 (socket says 172.21.15.26:54814) 2024-09-15T04:04:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9da6d9d700 1 -- 172.21.15.26:0/1417607179 learned_addr learned my addr 172.21.15.26:0/1417607179 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9da5d9b700 1 --1- >> v1:172.21.15.26:6792/0 conn(0x7f9da8133060 0x7f9da8147100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6792/0 says I am v1:172.21.15.26:48190/0 (socket says 172.21.15.26:48190) 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247527813 0 0) 0x7f9da811fe20 con 0x7f9da8133470 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): my global_id is 205285 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d88003570 con 0x7f9da8133470 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.6 v1:172.21.15.26:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878327906 0 0) 0x7f9da811e7d0 con 0x7f9da8133060 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): my global_id is 195477 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da811fe20 con 0x7f9da8133060 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622908183 0 0) 0x7f9da811e080 con 0x7f9da8132c90 2024-09-15T04:04:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): my global_id is 274280 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da811e7d0 con 0x7f9da8132c90 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1988484089 0 0) 0x7f9d88003570 con 0x7f9da8133470 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9da811e080 con 0x7f9da8133470 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d980029f0 con 0x7f9da8133470 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): dump: 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.110 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.6 v1:172.21.15.26:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 881158668 0 0) 0x7f9da811fe20 con 0x7f9da8133060 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d88003570 con 0x7f9da8133060 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.6 v1:172.21.15.26:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d90001770 con 0x7f9da8133060 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient(hunting): dump: 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.113 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 682816186 0 0) 0x7f9da811e080 con 0x7f9da8133470 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 >> v1:172.21.15.26:6792/0 conn(0x7f9da8133060 legacy=0x7f9da8147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 >> v1:172.21.15.122:6791/0 conn(0x7f9da8132c90 legacy=0x7f9da81420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 monclient: found mon.c 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da811fb40 con 0x7f9da8133470 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 5 monclient: authenticate success, global_id 205285 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _renew_subs 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6791/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da809d120 con 0x7f9da8133470 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _renew_subs 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6791/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da8120f90 con 0x7f9da8133470 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f9d98001540 con 0x7f9da8133470 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f9d98001a70 con 0x7f9da8133470 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9da4d99700 4 set_mon_vals no callback set 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:04:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 10 monclient: dump: 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 10 monclient: _send_mon_message to mon.c at v1:172.21.15.26:6791/0 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.26:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c0050b0 con 0x7f9da8133470 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.086+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f9d98001c50 con 0x7f9da8133470 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.090+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f9d98043d20 con 0x7f9da8133470 2024-09-15T04:04:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.090+0000 7f9da659c700 10 monclient: get_auth_request con 0x7f9d88037cb0 auth_method 0 2024-09-15T04:04:49.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.094+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mon.4 v1:172.21.15.26:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f9d98011340 con 0x7f9da8133470 2024-09-15T04:04:49.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.094+0000 7f9d9e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:49.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.094+0000 7f9d9e7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:49.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.262+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d8c000b90 con 0x7f9d88037cb0 2024-09-15T04:04:49.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.262+0000 7f9d9e7fc700 1 -- 172.21.15.26:0/1417607179 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (unknown 2264484800 0 0) 0x7f9d8c000b90 con 0x7f9d88037cb0 2024-09-15T04:04:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.262+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 >> v1:172.21.15.122:6812/20895 conn(0x7f9d88037cb0 legacy=0x7f9d8803a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.262+0000 7f9dacb5e700 10 monclient: shutdown 2024-09-15T04:04:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.262+0000 7f9dacb5e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.262+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 >> v1:172.21.15.26:6791/0 conn(0x7f9da8133470 legacy=0x7f9da814a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.266+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 shutdown_connections 2024-09-15T04:04:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.266+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 >> 172.21.15.26:0/1417607179 conn(0x7f9da806e510 msgr2=0x7f9da80a4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:49.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.266+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 shutdown_connections 2024-09-15T04:04:49.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.266+0000 7f9dacb5e700 1 -- 172.21.15.26:0/1417607179 wait complete. 2024-09-15T04:04:49.273 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T04:04:49.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.378+0000 7fcae097f700 10 monclient: get_monmap_and_config 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.378+0000 7fcae097f700 10 monclient: build_initial_monmap 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.378+0000 7fcae097f700 10 monclient: monmap: 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:04:49.383647+0000 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:04:49.383647+0000 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:49.387 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:49.388 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 1 Processor -- start 2024-09-15T04:04:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 1 -- start start 2024-09-15T04:04:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient: init 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient: _add_conns ranks=[5,7,8,2,3,0,6,4,1] 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient(hunting): picked mon.noname-h con 0x7fcadc0a21d0 addr v1:172.21.15.122:6791/0 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient(hunting): picked mon.noname-i con 0x7fcadc06bce0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient(hunting): picked mon.noname-e con 0x7fcadc135fe0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc04ca10 con 0x7fcadc0a21d0 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc0a25a0 con 0x7fcadc06bce0 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc06a620 con 0x7fcadc135fe0 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcae097f700 10 monclient(hunting): authenticate will time out at 2033.814793s 2024-09-15T04:04:49.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcadad9d700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fcadc135fe0 0x7fcadc1383c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33170/0 (socket says 172.21.15.26:33170) 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcadad9d700 1 -- 172.21.15.26:0/4287685506 learned_addr learned my addr 172.21.15.26:0/4287685506 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447551677 0 0) 0x7fcadc06a620 con 0x7fcadc135fe0 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): my global_id is 274364 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac4003570 con 0x7fcadc135fe0 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509668104 0 0) 0x7fcadc04ca10 con 0x7fcadc0a21d0 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): my global_id is 274289 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc06a620 con 0x7fcadc0a21d0 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414775421 0 0) 0x7fcadc0a25a0 con 0x7fcadc06bce0 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 10 monclient(hunting): my global_id is 254563 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.382+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc04ca10 con 0x7fcadc06bce0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2337060073 0 0) 0x7fcac4003570 con 0x7fcadc135fe0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 >> v1:172.21.15.122:6792/0 conn(0x7fcadc06bce0 legacy=0x7fcadc069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 >> v1:172.21.15.122:6791/0 conn(0x7fcadc0a21d0 legacy=0x7fcadc069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: _finish_hunting 0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 monclient: found mon.noname-e 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0a1d40 con 0x7fcadc135fe0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: _finish_auth 0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcad4001b30 con 0x7fcadc135fe0 2024-09-15T04:04:49.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: dump: 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 5 monclient: authenticate success, global_id 274364 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcad4001e10 con 0x7fcadc135fe0 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: get_monmap_and_config success 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 1 -- 172.21.15.26:0/4287685506 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcad4004fb0 con 0x7fcadc135fe0 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 4 set_mon_vals no callback set 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcad959a700 10 monclient: dump: 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.397 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.398 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: shutdown 2024-09-15T04:04:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:49.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- 172.21.15.26:0/4287685506 >> v1:172.21.15.26:6793/0 conn(0x7fcadc135fe0 legacy=0x7fcadc1383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- 172.21.15.26:0/4287685506 shutdown_connections 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- 172.21.15.26:0/4287685506 >> 172.21.15.26:0/4287685506 conn(0x7fcadc0a2d10 msgr2=0x7fcadc0a3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- 172.21.15.26:0/4287685506 shutdown_connections 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- 172.21.15.26:0/4287685506 wait complete. 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: build_initial_monmap 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: monmap: 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:04:49.403 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 Processor -- start 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- start start 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: init 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: _reopen_session rank -1 2024-09-15T04:04:49.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient: _add_conns ranks=[1,8,7,4,3,5,0,6,2] 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient(hunting): picked mon.noname-f con 0x7fcadc06bce0 addr v1:172.21.15.122:6789/0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient(hunting): picked mon.noname-e con 0x7fcadc0a21d0 addr v1:172.21.15.26:6793/0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient(hunting): picked mon.noname-i con 0x7fcadc135fe0 addr v1:172.21.15.122:6792/0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc06aa40 con 0x7fcadc06bce0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc09cdc0 con 0x7fcadc135fe0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc11a1a0 con 0x7fcadc0a21d0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient(hunting): _renew_subs 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.386+0000 7fcae097f700 10 monclient(hunting): authenticate will time out at 2033.819906s 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcad9d9b700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7fcadc0a21d0 0x7fcadc143cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:33184/0 (socket says 172.21.15.26:33184) 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcad9d9b700 1 -- 172.21.15.26:0/2072940439 learned_addr learned my addr 172.21.15.26:0/2072940439 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522437974 0 0) 0x7fcadc11a1a0 con 0x7fcadc0a21d0 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): my global_id is 274373 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcabc003570 con 0x7fcadc0a21d0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360930349 0 0) 0x7fcadc09cdc0 con 0x7fcadc135fe0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): my global_id is 254572 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc11a1a0 con 0x7fcadc135fe0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010520362 0 0) 0x7fcadc06aa40 con 0x7fcadc06bce0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): my global_id is 274258 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc09cdc0 con 0x7fcadc06bce0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 518375166 0 0) 0x7fcabc003570 con 0x7fcadc0a21d0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcadc06aa40 con 0x7fcadc0a21d0 2024-09-15T04:04:49.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcad0002bb0 con 0x7fcadc0a21d0 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): dump: 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.407 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 538991986 0 0) 0x7fcadc06aa40 con 0x7fcadc0a21d0 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 >> v1:172.21.15.122:6792/0 conn(0x7fcadc135fe0 legacy=0x7fcadc1473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 >> v1:172.21.15.122:6789/0 conn(0x7fcadc06bce0 legacy=0x7fcadc119300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: _finish_hunting 0 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 monclient: found mon.e 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:49.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc131bf0 con 0x7fcadc0a21d0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: _finish_auth 0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 5 monclient: authenticate success, global_id 274373 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 monclient: _renew_subs 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc11a860 con 0x7fcadc0a21d0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 monclient: _renew_subs 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc119bb0 con 0x7fcadc0a21d0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcad0001660 con 0x7fcadc0a21d0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7fcad0001b90 con 0x7fcadc0a21d0 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:04:49.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcad8d99700 4 set_mon_vals no callback set 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 10 monclient: dump: 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:04:49.410 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7fcad0001d70 con 0x7fcadc0a21d0 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac00050b0 con 0x7fcadc0a21d0 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7fcad0043e30 con 0x7fcadc0a21d0 2024-09-15T04:04:49.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.390+0000 7fcada59c700 10 monclient: get_auth_request con 0x7fcabc0376f0 auth_method 0 2024-09-15T04:04:49.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.398+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7fcad0014e20 con 0x7fcadc0a21d0 2024-09-15T04:04:49.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.398+0000 7fcaca7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:04:49.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.398+0000 7fcaca7fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:04:49.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.558+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcac0000b90 con 0x7fcabc0376f0 2024-09-15T04:04:49.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcaca7fc700 1 -- 172.21.15.26:0/2072940439 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53658 (unknown 2965378022 0 3232412550) 0x7fcac0000b90 con 0x7fcabc0376f0 2024-09-15T04:04:49.572 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 >> v1:172.21.15.122:6812/20895 conn(0x7fcabc0376f0 legacy=0x7fcabc039ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 10 monclient: shutdown 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 >> v1:172.21.15.26:6793/0 conn(0x7fcadc0a21d0 legacy=0x7fcadc143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 shutdown_connections 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 >> 172.21.15.26:0/2072940439 conn(0x7fcadc0a2d10 msgr2=0x7fcadc135390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 shutdown_connections 2024-09-15T04:04:49.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:04:49.566+0000 7fcae097f700 1 -- 172.21.15.26:0/2072940439 wait complete. 2024-09-15T04:04:49.575 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T04:04:49.594 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":1495,"stamp":"2024-09-15T04:04:47.799513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1857728,"kb_used_data":23680,"kb_used_omap":9,"kb_used_meta":1833974,"kb_avail":564373312,"statfs":{"total":579820584960,"available":577918271488,"internally_reserved":0,"allocated":24248320,"data_stored":13776778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1877989392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003434"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633767+0000","last_change":"2024-09-15T03:45:00.748035+0000","last_active":"2024-09-15T04:04:06.633767+0000","last_peered":"2024-09-15T04:04:06.633767+0000","last_clean":"2024-09-15T04:04:06.633767+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T04:04:06.633767+0000","last_undegraded":"2024-09-15T04:04:06.633767+0000","last_fullsized":"2024-09-15T04:04:06.633767+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:23.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638127+0000","last_change":"2024-09-15T03:45:00.901393+0000","last_active":"2024-09-15T04:04:06.638127+0000","last_peered":"2024-09-15T04:04:06.638127+0000","last_clean":"2024-09-15T04:04:06.638127+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T04:04:06.638127+0000","last_undegraded":"2024-09-15T04:04:06.638127+0000","last_fullsized":"2024-09-15T04:04:06.638127+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:48:06.013517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"16'2","reported_seq":884,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634431+0000","last_change":"2024-09-15T03:45:00.748385+0000","last_active":"2024-09-15T04:04:06.634431+0000","last_peered":"2024-09-15T04:04:06.634431+0000","last_clean":"2024-09-15T04:04:06.634431+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T04:04:06.634431+0000","last_undegraded":"2024-09-15T04:04:06.634431+0000","last_fullsized":"2024-09-15T04:04:06.634431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:20.245862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"14'1","reported_seq":893,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631849+0000","last_change":"2024-09-15T03:45:00.747287+0000","last_active":"2024-09-15T04:04:06.631849+0000","last_peered":"2024-09-15T04:04:06.631849+0000","last_clean":"2024-09-15T04:04:06.631849+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T04:04:06.631849+0000","last_undegraded":"2024-09-15T04:04:06.631849+0000","last_fullsized":"2024-09-15T04:04:06.631849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:50:09.470513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"15'72","reported_seq":975,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.633590+0000","last_change":"2024-09-15T03:44:58.601750+0000","last_active":"2024-09-15T04:04:06.633590+0000","last_peered":"2024-09-15T04:04:06.633590+0000","last_clean":"2024-09-15T04:04:06.633590+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T04:04:06.633590+0000","last_undegraded":"2024-09-15T04:04:06.633590+0000","last_fullsized":"2024-09-15T04:04:06.633590+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T13:13:57.842614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":892,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.631860+0000","last_change":"2024-09-15T03:45:00.747232+0000","last_active":"2024-09-15T04:04:06.631860+0000","last_peered":"2024-09-15T04:04:06.631860+0000","last_clean":"2024-09-15T04:04:06.631860+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T04:04:06.631860+0000","last_undegraded":"2024-09-15T04:04:06.631860+0000","last_fullsized":"2024-09-15T04:04:06.631860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:01:41.948413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":882,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634430+0000","last_change":"2024-09-15T03:45:00.747898+0000","last_active":"2024-09-15T04:04:06.634430+0000","last_peered":"2024-09-15T04:04:06.634430+0000","last_clean":"2024-09-15T04:04:06.634430+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T04:04:06.634430+0000","last_undegraded":"2024-09-15T04:04:06.634430+0000","last_fullsized":"2024-09-15T04:04:06.634430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T12:19:24.167652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":890,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.634028+0000","last_change":"2024-09-15T03:45:00.748023+0000","last_active":"2024-09-15T04:04:06.634028+0000","last_peered":"2024-09-15T04:04:06.634028+0000","last_clean":"2024-09-15T04:04:06.634028+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T04:04:06.634028+0000","last_undegraded":"2024-09-15T04:04:06.634028+0000","last_fullsized":"2024-09-15T04:04:06.634028+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:52.853509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":894,"reported_epoch":903,"state":"active+clean","last_fresh":"2024-09-15T04:04:06.638384+0000","last_change":"2024-09-15T03:45:00.901386+0000","last_active":"2024-09-15T04:04:06.638384+0000","last_peered":"2024-09-15T04:04:06.638384+0000","last_clean":"2024-09-15T04:04:06.638384+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T04:04:06.638384+0000","last_undegraded":"2024-09-15T04:04:06.638384+0000","last_fullsized":"2024-09-15T04:04:06.638384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T03:44:57.572235+0000","last_clean_scrub_stamp":"2024-09-15T03:44:57.572235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:49:50.128663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640497,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.60099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.224,"15min":5.1609999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.374,"15min":0.372},"max":{"1min":1.0329999999999999,"5min":1.2609999999999999,"15min":3.5099999999999998},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.64400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1799999999999999,"5min":1.6619999999999999,"15min":2.2890000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.44500000000000001,"15min":0.41199999999999998},"max":{"1min":1.165,"5min":1.264,"15min":1.341},"last":1.0629999999999999}]},{"osd":2,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.51900000000000002,"15min":0.46500000000000002},"max":{"1min":1.2050000000000001,"5min":9.8149999999999995,"15min":9.8149999999999995},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.53500000000000003,"15min":0.36699999999999999},"max":{"1min":1.282,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.89500000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.27200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3640000000000001,"15min":5.2539999999999996},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.60899999999999999,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":1.1679999999999999,"5min":1.391,"15min":2.3700000000000001},"last":0.81799999999999995}]},{"osd":4,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.67100000000000004,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":1.0780000000000001,"5min":1.4279999999999999,"15min":5.2830000000000004},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.3899999999999999,"15min":5.3650000000000002},"last":0.93899999999999995}]}]},{"osd":4,"up_from":11,"seq":47244640497,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309452,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062388,"statfs":{"total":96636764160,"available":96319885312,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81399999999999995,"15min":0.80300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.29499999999999998},"max":{"1min":1.0389999999999999,"5min":1.1679999999999999,"15min":1.3420000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.40300000000000002,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.401},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.86099999999999999,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41999999999999998},"max":{"1min":1.0860000000000001,"5min":1.327,"15min":1.8100000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.69499999999999995,"5min":0.54900000000000004,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.228,"15min":2.2050000000000001},"last":0.63}]},{"osd":2,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.38600000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":3.2440000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.505,"15min":0.45500000000000002},"max":{"1min":1.105,"5min":1.226,"15min":2.5059999999999998},"last":0.67900000000000005}]},{"osd":3,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.96199999999999997,"5min":1.1719999999999999,"15min":1.335},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.5409999999999999},"last":0.876}]},{"osd":5,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.308},"max":{"1min":1.095,"5min":1.1870000000000001,"15min":1.474},"last":0.752},{"interface":"front","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.315},"max":{"1min":1.2230000000000001,"5min":1.242,"15min":1.26},"last":0.92800000000000005}]}]},{"osd":3,"up_from":11,"seq":47244640498,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318288,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94053552,"statfs":{"total":96636764160,"available":96310837248,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.61299999999999999,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.2749999999999999},"last":0.751},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.85899999999999999},"min":{"1min":0.66700000000000004,"5min":0.42199999999999999,"15min":0.39800000000000002},"max":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3680000000000001},"last":0.88900000000000001}]},{"osd":1,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.81200000000000006,"5min":0.52000000000000002,"15min":0.46999999999999997},"max":{"1min":1.0760000000000001,"5min":1.954,"15min":1.954},"last":1.0429999999999999}]},{"osd":2,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.38},"max":{"1min":1.167,"5min":1.2709999999999999,"15min":1.3799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.68100000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.2150000000000001,"5min":1.2310000000000001,"15min":1.4330000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.503,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2490000000000001,"5min":1.337,"15min":1.337},"last":0.97399999999999998}]},{"osd":5,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.40600000000000003,"15min":0.32300000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.5740000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.86399999999999999},"min":{"1min":0.78800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.621},"last":0.92900000000000005}]}]},{"osd":2,"up_from":11,"seq":47244640497,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.40400000000000003,"5min":0.27700000000000002,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":6.3019999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.01,"5min":1.113,"15min":6.3550000000000004},"last":0.69899999999999995}]},{"osd":1,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.63500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.1120000000000001,"5min":1.121,"15min":1.952},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":2.0019999999999998},"last":0.77900000000000003}]},{"osd":3,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.82499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.976},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.75700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1679999999999999,"5min":1.431,"15min":1.5069999999999999},"last":1.095}]},{"osd":4,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.68799999999999994,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1160000000000001,"5min":2.391,"15min":2.391},"last":1.02},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0429999999999999,"5min":2.419,"15min":2.419},"last":0.98699999999999999}]},{"osd":5,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.68700000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.4810000000000001,"15min":2.2330000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.91700000000000004},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.51100000000000001},"max":{"1min":1.341,"5min":1.5449999999999999,"15min":1.637},"last":0.95899999999999996}]}]},{"osd":1,"up_from":11,"seq":47244640499,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305808,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066032,"statfs":{"total":96636764160,"available":96323616768,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.51100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.68300000000000005,"5min":0.90200000000000002,"15min":1.883},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68000000000000005,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.29799999999999999},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":1.976},"last":0.65600000000000003}]},{"osd":2,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64200000000000002,"15min":0.66000000000000003},"min":{"1min":0.5,"5min":0.25,"15min":0.25},"max":{"1min":0.81599999999999995,"5min":1.5249999999999999,"15min":2.536},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91200000000000003,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.44900000000000001,"15min":0.437},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.878},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0029999999999999,"15min":0.95099999999999996},"min":{"1min":0.69899999999999995,"5min":0.60499999999999998,"15min":0.50800000000000001},"max":{"1min":1.2250000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.86199999999999999}]},{"osd":4,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84699999999999998,"15min":0.83799999999999997},"min":{"1min":0.74299999999999999,"5min":0.47399999999999998,"15min":0.42099999999999999},"max":{"1min":1.0620000000000001,"5min":1.677,"15min":1.677},"last":0.748},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95499999999999996,"15min":0.93799999999999994},"min":{"1min":0.67600000000000005,"5min":0.56100000000000005,"15min":0.48299999999999998},"max":{"1min":1.2210000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.91200000000000003},"min":{"1min":0.75600000000000001,"5min":0.53600000000000003,"15min":0.46500000000000002},"max":{"1min":1.2789999999999999,"5min":1.617,"15min":1.617},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98099999999999998,"15min":0.98399999999999999},"min":{"1min":0.76700000000000002,"5min":0.55900000000000005,"15min":0.55000000000000004},"max":{"1min":1.119,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.82699999999999996}]}]},{"osd":0,"up_from":11,"seq":47244640498,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305292,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066548,"statfs":{"total":96636764160,"available":96324145152,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.79700000000000004,"15min":18.481000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.247,"15min":0.247},"max":{"1min":0.79600000000000004,"5min":2.222,"15min":19.239000000000001},"last":0.69499999999999995}]},{"osd":2,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.52900000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81399999999999995,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.84099999999999997},"min":{"1min":0.68300000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.079,"5min":3.556,"15min":3.556},"last":0.93200000000000005}]},{"osd":3,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85299999999999998,"15min":0.86399999999999999},"min":{"1min":0.70499999999999996,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":2.395,"15min":2.395},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.91900000000000004,"15min":0.90000000000000002},"min":{"1min":0.71699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98299999999999998,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.79700000000000004}]},{"osd":4,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.55600000000000005,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.95399999999999996},"min":{"1min":0.77500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0840000000000001,"5min":2.923,"15min":2.923},"last":0.751}]},{"osd":5,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.873,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.56399999999999995,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.21,"15min":1.2989999999999999},"last":0.624},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94599999999999995,"15min":0.98299999999999998},"min":{"1min":0.81299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.282,"15min":1.6200000000000001},"last":0.82699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T04:04:49.598 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.598 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.598 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-15T03:44:57.572235+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=3, tm_min=44, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=4, tm_min=4, tm_sec=45, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T04:04:49.599 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T04:05:09.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T04:05:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient: get_monmap_and_config 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient: build_initial_monmap 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient: monmap: 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:05:09.714153+0000 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:05:09.714153+0000 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:05:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:05:09.719 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 1 Processor -- start 2024-09-15T04:05:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 1 -- start start 2024-09-15T04:05:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient: init 2024-09-15T04:05:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient: _reopen_session rank -1 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient: _add_conns ranks=[1,8,2,5,6,0,7,3,4] 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient(hunting): picked mon.noname-f con 0x7f82c809daf0 addr v1:172.21.15.122:6789/0 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient(hunting): picked mon.noname-e con 0x7f82c806de90 addr v1:172.21.15.26:6793/0 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient(hunting): picked mon.noname-b con 0x7f82c8136640 addr v1:172.21.15.26:6790/0 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c804ca10 con 0x7f82c809daf0 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c809dfa0 con 0x7f82c8136640 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 1 -- --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c8098880 con 0x7f82c806de90 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient(hunting): _renew_subs 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82ce086700 10 monclient(hunting): authenticate will time out at 2054.143641s 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82cc883700 1 --1- >> v1:172.21.15.26:6793/0 conn(0x7f82c806de90 0x7f82c8098170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6793/0 says I am v1:172.21.15.26:47332/0 (socket says 172.21.15.26:47332) 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.709+0000 7f82cc883700 1 -- 172.21.15.26:0/3649349334 learned_addr learned my addr 172.21.15.26:0/3649349334 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:05:09.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932145068 0 0) 0x7f82c8098880 con 0x7f82c806de90 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): my global_id is 274382 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b4003570 con 0x7f82c806de90 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381773955 0 0) 0x7f82c804ca10 con 0x7f82c809daf0 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): my global_id is 274267 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c8098880 con 0x7f82c809daf0 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925854947 0 0) 0x7f82c809dfa0 con 0x7f82c8136640 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient(hunting): my global_id is 254414 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c804ca10 con 0x7f82c8136640 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1555561477 0 0) 0x7f82b4003570 con 0x7f82c806de90 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 >> v1:172.21.15.26:6790/0 conn(0x7f82c8136640 legacy=0x7f82c8138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 >> v1:172.21.15.122:6789/0 conn(0x7f82c809daf0 legacy=0x7f82c8097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: _finish_hunting 0 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 monclient: found mon.noname-e 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.26:6793/0 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c809e180 con 0x7f82c806de90 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: _finish_auth 0 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 5 monclient: authenticate success, global_id 274382 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:05:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82c40017b0 con 0x7f82c806de90 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: dump: 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.8 v1:172.21.15.26:6793/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f82c4001a90 con 0x7f82c806de90 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 10 monclient: get_monmap_and_config success 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 1 -- 172.21.15.26:0/3649349334 <== mon.8 v1:172.21.15.26:6793/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82c4004c30 con 0x7f82c806de90 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 4 set_mon_vals no callback set 2024-09-15T04:05:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82bffff700 10 monclient: dump: 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 10 monclient: shutdown 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 1 -- 172.21.15.26:0/3649349334 >> v1:172.21.15.26:6793/0 conn(0x7f82c806de90 legacy=0x7f82c8098170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 1 -- 172.21.15.26:0/3649349334 shutdown_connections 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 1 -- 172.21.15.26:0/3649349334 >> 172.21.15.26:0/3649349334 conn(0x7f82c8063d90 msgr2=0x7f82c80a1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 1 -- 172.21.15.26:0/3649349334 shutdown_connections 2024-09-15T04:05:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 1 -- 172.21.15.26:0/3649349334 wait complete. 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 10 monclient: build_initial_monmap 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.713+0000 7f82ce086700 10 monclient: monmap: 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:05:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 1 Processor -- start 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 1 -- start start 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient: init 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient: _reopen_session rank -1 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient: _add_conns ranks=[1,7,5,0,2,4,8,3,6] 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient(hunting): picked mon.noname-f con 0x7f82c806de90 addr v1:172.21.15.122:6789/0 2024-09-15T04:05:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient(hunting): picked mon.noname-i con 0x7f82c8136640 addr v1:172.21.15.122:6792/0 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient(hunting): picked mon.noname-h con 0x7f82c811d6b0 addr v1:172.21.15.122:6791/0 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c8099650 con 0x7f82c806de90 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c806e500 con 0x7f82c811d6b0 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 1 -- --> v1:172.21.15.122:6792/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c811ebd0 con 0x7f82c8136640 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient(hunting): _renew_subs 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82ce086700 10 monclient(hunting): authenticate will time out at 2054.150118s 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82cd084700 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f82c806de90 0x7f82c813e090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.26:53466/0 (socket says 172.21.15.26:53466) 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82cd084700 1 -- 172.21.15.26:0/589392676 learned_addr learned my addr 172.21.15.26:0/589392676 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047761300 0 0) 0x7f82c8099650 con 0x7f82c806de90 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): my global_id is 274276 2024-09-15T04:05:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82ac003570 con 0x7f82c806de90 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523410847 0 0) 0x7f82c806e500 con 0x7f82c811d6b0 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): my global_id is 274298 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c8099650 con 0x7f82c811d6b0 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.7 v1:172.21.15.122:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666369917 0 0) 0x7f82c811ebd0 con 0x7f82c8136640 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): my global_id is 254581 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6792/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c806e500 con 0x7f82c8136640 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1331169927 0 0) 0x7f82ac003570 con 0x7f82c806de90 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82c811ebd0 con 0x7f82c806de90 2024-09-15T04:05:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82b8001b20 con 0x7f82c806de90 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): dump: 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.7 v1:172.21.15.122:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2774677620 0 0) 0x7f82c806e500 con 0x7f82c8136640 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6792/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82ac003570 con 0x7f82c8136640 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.5 v1:172.21.15.122:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3030236419 0 0) 0x7f82c8099650 con 0x7f82c811d6b0 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82c806e500 con 0x7f82c811d6b0 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.7 v1:172.21.15.122:6792/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82c4004b70 con 0x7f82c8136640 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 10 monclient(hunting): dump: 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.717+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.5 v1:172.21.15.122:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82c0001cc0 con 0x7f82c811d6b0 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient(hunting): dump: 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:09.742 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4178224590 0 0) 0x7f82c811ebd0 con 0x7f82c806de90 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 >> v1:172.21.15.122:6792/0 conn(0x7f82c8136640 legacy=0x7f82c8142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 >> v1:172.21.15.122:6791/0 conn(0x7f82c811d6b0 legacy=0x7f82c81463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: _finish_hunting 0 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 monclient: found mon.f 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c811cff0 con 0x7f82c806de90 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: _finish_auth 0 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 5 monclient: authenticate success, global_id 274276 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 monclient: _renew_subs 2024-09-15T04:05:09.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c811cb20 con 0x7f82c806de90 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 monclient: _renew_subs 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c8135b50 con 0x7f82c806de90 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f82b8002100 con 0x7f82c806de90 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f82b80025b0 con 0x7f82c806de90 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bf7fe700 4 set_mon_vals no callback set 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 10 monclient: dump: 2024-09-15T04:05:09.746 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:09.747 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f82b8002710 con 0x7f82c806de90 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 10 monclient: _send_mon_message to mon.f at v1:172.21.15.122:6789/0 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a80050b0 con 0x7f82c806de90 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f82b8043df0 con 0x7f82c806de90 2024-09-15T04:05:09.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.721+0000 7f82cc883700 10 monclient: get_auth_request con 0x7f82ac038100 auth_method 0 2024-09-15T04:05:09.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.733+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mon.1 v1:172.21.15.122:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f82b8014d20 con 0x7f82c806de90 2024-09-15T04:05:09.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.733+0000 7f82bd7fa700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:05:09.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.733+0000 7f82bd7fa700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:05:09.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.897+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 --> v1:172.21.15.122:6812/20895 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82a8000b90 con 0x7f82ac038100 2024-09-15T04:05:09.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.905+0000 7f82bd7fa700 1 -- 172.21.15.26:0/589392676 <== mgr.4124 v1:172.21.15.122:6812/20895 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+53823 (unknown 2965378022 0 1750292771) 0x7f82a8000b90 con 0x7f82ac038100 2024-09-15T04:05:09.912 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-09-15T04:05:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 >> v1:172.21.15.122:6812/20895 conn(0x7f82ac038100 legacy=0x7f82ac03a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 10 monclient: shutdown 2024-09-15T04:05:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:05:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 >> v1:172.21.15.122:6789/0 conn(0x7f82c806de90 legacy=0x7f82c813e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 shutdown_connections 2024-09-15T04:05:09.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 >> 172.21.15.26:0/589392676 conn(0x7f82c8063d90 msgr2=0x7f82c809af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:05:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 shutdown_connections 2024-09-15T04:05:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:09.909+0000 7f82ce086700 1 -- 172.21.15.26:0/589392676 wait complete. 2024-09-15T04:05:09.917 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-09-15T04:05:09.932 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":1506,"stamp":"2024-09-15T04:05:09.805667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1857712,"kb_used_data":23680,"kb_used_omap":9,"kb_used_meta":1833974,"kb_avail":564373328,"statfs":{"total":579820584960,"available":577918287872,"internally_reserved":0,"allocated":24248320,"data_stored":13776778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1877989392},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003547"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":901,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:49.229390+0000","last_change":"2024-09-15T04:04:49.229308+0000","last_active":"2024-09-15T04:04:49.229390+0000","last_peered":"2024-09-15T04:04:49.229390+0000","last_clean":"2024-09-15T04:04:49.229390+0000","last_became_active":"2024-09-15T03:44:58.600357+0000","last_became_peered":"2024-09-15T03:44:58.600357+0000","last_unstale":"2024-09-15T04:04:49.229390+0000","last_undegraded":"2024-09-15T04:04:49.229390+0000","last_fullsized":"2024-09-15T04:04:49.229390+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T04:04:49.228971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T04:04:49.228971+0000","last_clean_scrub_stamp":"2024-09-15T04:04:49.228971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:17:49.044427+0000","scrub_duration":0.0033488070000000001,"objects_trimmed":0,"snaptrim_duration":0.00053311999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":905,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:53.919806+0000","last_change":"2024-09-15T04:04:53.919745+0000","last_active":"2024-09-15T04:04:53.919806+0000","last_peered":"2024-09-15T04:04:53.919806+0000","last_clean":"2024-09-15T04:04:53.919806+0000","last_became_active":"2024-09-15T03:44:58.607366+0000","last_became_peered":"2024-09-15T03:44:58.607366+0000","last_unstale":"2024-09-15T04:04:53.919806+0000","last_undegraded":"2024-09-15T04:04:53.919806+0000","last_fullsized":"2024-09-15T04:04:53.919806+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T04:04:53.919529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T04:04:53.919529+0000","last_clean_scrub_stamp":"2024-09-15T04:04:53.919529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:26:45.748463+0000","scrub_duration":0.0035157970000000002,"objects_trimmed":0,"snaptrim_duration":0.00077087300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"904'3","reported_seq":896,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:50.214771+0000","last_change":"2024-09-15T04:04:50.214640+0000","last_active":"2024-09-15T04:04:50.214771+0000","last_peered":"2024-09-15T04:04:50.214771+0000","last_clean":"2024-09-15T04:04:50.214771+0000","last_became_active":"2024-09-15T03:44:58.597728+0000","last_became_peered":"2024-09-15T03:44:58.597728+0000","last_unstale":"2024-09-15T04:04:50.214771+0000","last_undegraded":"2024-09-15T04:04:50.214771+0000","last_fullsized":"2024-09-15T04:04:50.214771+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"904'3","last_scrub_stamp":"2024-09-15T04:04:50.214403+0000","last_deep_scrub":"904'3","last_deep_scrub_stamp":"2024-09-15T04:04:50.214403+0000","last_clean_scrub_stamp":"2024-09-15T04:04:50.214403+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:59:04.511829+0000","scrub_duration":0.0069003149999999997,"objects_trimmed":0,"snaptrim_duration":0.0010665869999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"904'2","reported_seq":905,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:51.895534+0000","last_change":"2024-09-15T04:04:51.895412+0000","last_active":"2024-09-15T04:04:51.895534+0000","last_peered":"2024-09-15T04:04:51.895534+0000","last_clean":"2024-09-15T04:04:51.895534+0000","last_became_active":"2024-09-15T03:44:58.606814+0000","last_became_peered":"2024-09-15T03:44:58.606814+0000","last_unstale":"2024-09-15T04:04:51.895534+0000","last_undegraded":"2024-09-15T04:04:51.895534+0000","last_fullsized":"2024-09-15T04:04:51.895534+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"904'2","last_scrub_stamp":"2024-09-15T04:04:51.895176+0000","last_deep_scrub":"904'2","last_deep_scrub_stamp":"2024-09-15T04:04:51.895176+0000","last_clean_scrub_stamp":"2024-09-15T04:04:51.895176+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:51:26.677133+0000","scrub_duration":0.0069272159999999999,"objects_trimmed":0,"snaptrim_duration":0.00049012800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"904'74","reported_seq":988,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:53.244030+0000","last_change":"2024-09-15T04:04:53.243912+0000","last_active":"2024-09-15T04:04:53.244030+0000","last_peered":"2024-09-15T04:04:53.244030+0000","last_clean":"2024-09-15T04:04:53.244030+0000","last_became_active":"2024-09-15T03:44:58.601454+0000","last_became_peered":"2024-09-15T03:44:58.601454+0000","last_unstale":"2024-09-15T04:04:53.244030+0000","last_undegraded":"2024-09-15T04:04:53.244030+0000","last_fullsized":"2024-09-15T04:04:53.244030+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"904'74","last_scrub_stamp":"2024-09-15T04:04:53.243707+0000","last_deep_scrub":"904'74","last_deep_scrub_stamp":"2024-09-15T04:04:53.243707+0000","last_clean_scrub_stamp":"2024-09-15T04:04:53.243707+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:44:44.850286+0000","scrub_duration":0.0084995460000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":903,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:49.918067+0000","last_change":"2024-09-15T04:04:49.917929+0000","last_active":"2024-09-15T04:04:49.918067+0000","last_peered":"2024-09-15T04:04:49.918067+0000","last_clean":"2024-09-15T04:04:49.918067+0000","last_became_active":"2024-09-15T03:44:58.600847+0000","last_became_peered":"2024-09-15T03:44:58.600847+0000","last_unstale":"2024-09-15T04:04:49.918067+0000","last_undegraded":"2024-09-15T04:04:49.918067+0000","last_fullsized":"2024-09-15T04:04:49.918067+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T04:04:49.917656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T04:04:49.917656+0000","last_clean_scrub_stamp":"2024-09-15T04:04:49.917656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:51:07.883801+0000","scrub_duration":0.0036390039999999999,"objects_trimmed":0,"snaptrim_duration":0.00044461000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":893,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:55.189337+0000","last_change":"2024-09-15T04:04:55.189268+0000","last_active":"2024-09-15T04:04:55.189337+0000","last_peered":"2024-09-15T04:04:55.189337+0000","last_clean":"2024-09-15T04:04:55.189337+0000","last_became_active":"2024-09-15T03:44:58.597505+0000","last_became_peered":"2024-09-15T03:44:58.597505+0000","last_unstale":"2024-09-15T04:04:55.189337+0000","last_undegraded":"2024-09-15T04:04:55.189337+0000","last_fullsized":"2024-09-15T04:04:55.189337+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T04:04:55.189086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T04:04:55.189086+0000","last_clean_scrub_stamp":"2024-09-15T04:04:55.189086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:30.604595+0000","scrub_duration":0.0036847389999999998,"objects_trimmed":0,"snaptrim_duration":0.00058925399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":901,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:54.197333+0000","last_change":"2024-09-15T04:04:54.197248+0000","last_active":"2024-09-15T04:04:54.197333+0000","last_peered":"2024-09-15T04:04:54.197333+0000","last_clean":"2024-09-15T04:04:54.197333+0000","last_became_active":"2024-09-15T03:44:58.600473+0000","last_became_peered":"2024-09-15T03:44:58.600473+0000","last_unstale":"2024-09-15T04:04:54.197333+0000","last_undegraded":"2024-09-15T04:04:54.197333+0000","last_fullsized":"2024-09-15T04:04:54.197333+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T04:04:54.197053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T04:04:54.197053+0000","last_clean_scrub_stamp":"2024-09-15T04:04:54.197053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:07:08.337159+0000","scrub_duration":0.003648784,"objects_trimmed":0,"snaptrim_duration":0.000671782,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":905,"reported_epoch":904,"state":"active+clean","last_fresh":"2024-09-15T04:04:50.895223+0000","last_change":"2024-09-15T04:04:50.895138+0000","last_active":"2024-09-15T04:04:50.895223+0000","last_peered":"2024-09-15T04:04:50.895223+0000","last_clean":"2024-09-15T04:04:50.895223+0000","last_became_active":"2024-09-15T03:44:58.604184+0000","last_became_peered":"2024-09-15T03:44:58.604184+0000","last_unstale":"2024-09-15T04:04:50.895223+0000","last_undegraded":"2024-09-15T04:04:50.895223+0000","last_fullsized":"2024-09-15T04:04:50.895223+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T04:04:50.894921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T04:04:50.894921+0000","last_clean_scrub_stamp":"2024-09-15T04:04:50.894921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T15:44:46.047060+0000","scrub_duration":0.003505326,"objects_trimmed":0,"snaptrim_duration":0.00065078800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6}],"osd_stats":[{"osd":5,"up_from":11,"seq":47244640501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.60099999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.224,"15min":5.1609999999999996},"last":1.028},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.374,"15min":0.372},"max":{"1min":1.0329999999999999,"5min":1.2609999999999999,"15min":3.5099999999999998},"last":0.88200000000000001}]},{"osd":1,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.64400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1799999999999999,"5min":1.6619999999999999,"15min":2.2890000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.83999999999999997,"15min":0.83899999999999997},"min":{"1min":0.64800000000000002,"5min":0.44500000000000001,"15min":0.41199999999999998},"max":{"1min":1.165,"5min":1.264,"15min":1.341},"last":0.81699999999999995}]},{"osd":2,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93100000000000005,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.51900000000000002,"15min":0.46500000000000002},"max":{"1min":1.2050000000000001,"5min":9.8149999999999995,"15min":9.8149999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.88100000000000001},"min":{"1min":0.64600000000000002,"5min":0.53500000000000003,"15min":0.36699999999999999},"max":{"1min":1.282,"5min":6.6660000000000004,"15min":6.6660000000000004},"last":0.92200000000000004}]},{"osd":3,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.74299999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.40000000000000002,"15min":0.27200000000000002},"max":{"1min":1.1160000000000001,"5min":1.3640000000000001,"15min":5.2539999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.79600000000000004,"15min":0.78800000000000003},"min":{"1min":0.60899999999999999,"5min":0.40600000000000003,"15min":0.36099999999999999},"max":{"1min":1.1679999999999999,"5min":1.391,"15min":2.3700000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:04:24 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.67100000000000004,"5min":0.34899999999999998,"15min":0.34300000000000003},"max":{"1min":1.0780000000000001,"5min":1.4279999999999999,"15min":5.2830000000000004},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.93400000000000005,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.51500000000000001,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.3899999999999999,"15min":5.3650000000000002},"last":0.72399999999999998}]}]},{"osd":4,"up_from":11,"seq":47244640501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309448,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062392,"statfs":{"total":96636764160,"available":96319889408,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81399999999999995,"15min":0.80300000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.29499999999999998},"max":{"1min":1.0389999999999999,"5min":1.1679999999999999,"15min":1.3420000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.84999999999999998},"min":{"1min":0.57999999999999996,"5min":0.40300000000000002,"15min":0.35099999999999998},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.401},"last":0.95599999999999996}]},{"osd":1,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.86099999999999999,"15min":0.84399999999999997},"min":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.41999999999999998},"max":{"1min":1.0860000000000001,"5min":1.327,"15min":1.8100000000000001},"last":1.075},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.69499999999999995,"5min":0.54900000000000004,"15min":0.42599999999999999},"max":{"1min":1.1890000000000001,"5min":1.228,"15min":2.2050000000000001},"last":1.018}]},{"osd":2,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.55100000000000005,"5min":0.51500000000000001,"15min":0.38600000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":3.2440000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.875,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.505,"15min":0.45500000000000002},"max":{"1min":1.105,"5min":1.226,"15min":2.5059999999999998},"last":0.81799999999999995}]},{"osd":3,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.36499999999999999,"15min":0.35499999999999998},"max":{"1min":0.96199999999999997,"5min":1.1719999999999999,"15min":1.335},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.5409999999999999},"last":0.91600000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.77800000000000002},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.308},"max":{"1min":1.095,"5min":1.1870000000000001,"15min":1.474},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.83099999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.315},"max":{"1min":1.2230000000000001,"5min":1.242,"15min":1.26},"last":1.038}]}]},{"osd":3,"up_from":11,"seq":47244640502,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":318284,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":313982,"kb_avail":94053556,"statfs":{"total":96636764160,"available":96310841344,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":321517912},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78500000000000003},"min":{"1min":0.61299999999999999,"5min":0.35099999999999998,"15min":0.32000000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.2749999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.872,"15min":0.85899999999999999},"min":{"1min":0.66700000000000004,"5min":0.42199999999999999,"15min":0.39800000000000002},"max":{"1min":1.194,"5min":1.3129999999999999,"15min":1.3680000000000001},"last":0.95699999999999996}]},{"osd":1,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84299999999999997,"15min":0.83299999999999996},"min":{"1min":0.57599999999999996,"5min":0.46899999999999997,"15min":0.40699999999999997},"max":{"1min":1.05,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.81200000000000006,"5min":0.52000000000000002,"15min":0.46999999999999997},"max":{"1min":1.0760000000000001,"5min":1.954,"15min":1.954},"last":1.0089999999999999}]},{"osd":2,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.85699999999999998},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.38},"max":{"1min":1.167,"5min":1.2709999999999999,"15min":1.3799999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81999999999999995,"15min":0.84799999999999998},"min":{"1min":0.68100000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.2150000000000001,"5min":1.2310000000000001,"15min":1.4330000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.503,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.625},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.77500000000000002,"15min":0.745},"min":{"1min":0.629,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2490000000000001,"5min":1.337,"15min":1.337},"last":0.89600000000000002}]},{"osd":5,"last update":"Sun Sep 15 04:04:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.40600000000000003,"15min":0.32300000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.5740000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.86399999999999999},"min":{"1min":0.78800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.621},"last":0.76100000000000001}]}]},{"osd":2,"up_from":11,"seq":47244640501,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309444,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94062396,"statfs":{"total":96636764160,"available":96319893504,"internally_reserved":0,"allocated":3862528,"data_stored":2119814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.40400000000000003,"5min":0.27700000000000002,"15min":0.253},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":6.3019999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.61899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.01,"5min":1.113,"15min":6.3550000000000004},"last":0.81899999999999995}]},{"osd":1,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.63500000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.1120000000000001,"5min":1.121,"15min":1.952},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.751,"15min":0.78800000000000003},"min":{"1min":0.48699999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":2.0019999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96199999999999997,"15min":0.95999999999999996},"min":{"1min":0.82499999999999996,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.976},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.93100000000000005,"15min":0.92700000000000005},"min":{"1min":0.75700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.1679999999999999,"5min":1.431,"15min":1.5069999999999999},"last":1.01}]},{"osd":4,"last update":"Sun Sep 15 04:04:35 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.68799999999999994,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1160000000000001,"5min":2.391,"15min":2.391},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.95099999999999996,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.0429999999999999,"5min":2.419,"15min":2.419},"last":0.73899999999999999}]},{"osd":5,"last update":"Sun Sep 15 04:04:29 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.68700000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0009999999999999,"5min":1.4810000000000001,"15min":2.2330000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.95199999999999996,"15min":0.91700000000000004},"min":{"1min":0.81499999999999995,"5min":0.52600000000000002,"15min":0.51100000000000001},"max":{"1min":1.341,"5min":1.5449999999999999,"15min":1.637},"last":0.96699999999999997}]}]},{"osd":1,"up_from":11,"seq":47244640503,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305804,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066036,"statfs":{"total":96636764160,"available":96323620864,"internally_reserved":0,"allocated":4395008,"data_stored":2648742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.59499999999999997},"min":{"1min":0.51100000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.68300000000000005,"5min":0.90200000000000002,"15min":1.883},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68000000000000005,"15min":0.67100000000000004},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.29799999999999999},"max":{"1min":0.84499999999999997,"5min":0.90800000000000003,"15min":1.976},"last":0.65100000000000002}]},{"osd":2,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64200000000000002,"15min":0.66000000000000003},"min":{"1min":0.5,"5min":0.25,"15min":0.25},"max":{"1min":0.81599999999999995,"5min":1.5249999999999999,"15min":2.536},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.53600000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.91200000000000003,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77500000000000002}]},{"osd":3,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.93600000000000005,"15min":0.92800000000000005},"min":{"1min":0.78600000000000003,"5min":0.44900000000000001,"15min":0.437},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0029999999999999,"15min":0.95099999999999996},"min":{"1min":0.69899999999999995,"5min":0.60499999999999998,"15min":0.50800000000000001},"max":{"1min":1.2250000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84699999999999998,"15min":0.83799999999999997},"min":{"1min":0.74299999999999999,"5min":0.47399999999999998,"15min":0.42099999999999999},"max":{"1min":1.0620000000000001,"5min":1.677,"15min":1.677},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95499999999999996,"15min":0.93799999999999994},"min":{"1min":0.67600000000000005,"5min":0.56100000000000005,"15min":0.48299999999999998},"max":{"1min":1.2210000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.85099999999999998}]},{"osd":5,"last update":"Sun Sep 15 04:04:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.88700000000000001,"15min":0.91200000000000003},"min":{"1min":0.75600000000000001,"5min":0.53600000000000003,"15min":0.46500000000000002},"max":{"1min":1.2789999999999999,"5min":1.617,"15min":1.617},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98099999999999998,"15min":0.98399999999999999},"min":{"1min":0.76700000000000002,"5min":0.55900000000000005,"15min":0.55000000000000004},"max":{"1min":1.119,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.92700000000000005}]}]},{"osd":0,"up_from":11,"seq":47244640502,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305288,"kb_used_data":3776,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94066552,"statfs":{"total":96636764160,"available":96324149248,"internally_reserved":0,"allocated":3866624,"data_stored":2119833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56299999999999994,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.32500000000000001,"15min":0.28100000000000003},"max":{"1min":0.69899999999999995,"5min":0.79700000000000004,"15min":18.481000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68500000000000005,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.247,"15min":0.247},"max":{"1min":0.79600000000000004,"5min":2.222,"15min":19.239000000000001},"last":0.503}]},{"osd":2,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67400000000000004,"15min":0.68700000000000006},"min":{"1min":0.52900000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.81399999999999995,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.89900000000000002,"15min":0.84099999999999997},"min":{"1min":0.68300000000000005,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.079,"5min":3.556,"15min":3.556},"last":0.99399999999999999}]},{"osd":3,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.85299999999999998,"15min":0.86399999999999999},"min":{"1min":0.70499999999999996,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.044,"5min":2.395,"15min":2.395},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.91900000000000004,"15min":0.90000000000000002},"min":{"1min":0.71699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98299999999999998,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.872}]},{"osd":4,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.89800000000000002,"15min":0.88500000000000001},"min":{"1min":0.628,"5min":0.55600000000000005,"15min":0.46800000000000003},"max":{"1min":0.99299999999999999,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.032},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.96599999999999997,"15min":0.95399999999999996},"min":{"1min":0.77500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0840000000000001,"5min":2.923,"15min":2.923},"last":0.79800000000000004}]},{"osd":5,"last update":"Sun Sep 15 04:04:32 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.873,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.56399999999999995,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.21,"15min":1.2989999999999999},"last":1.212},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.94599999999999995,"15min":0.98299999999999998},"min":{"1min":0.81299999999999994,"5min":0.495,"15min":0.495},"max":{"1min":1.123,"5min":1.282,"15min":1.6200000000000001},"last":1.286}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T04:05:09.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T04:05:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient: get_monmap_and_config 2024-09-15T04:05:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient: build_initial_monmap 2024-09-15T04:05:10.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient: monmap: 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T04:05:10.036853+0000 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T04:05:10.036853+0000 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:05:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:05:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:05:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:05:10.042 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 1 Processor -- start 2024-09-15T04:05:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 1 -- start start 2024-09-15T04:05:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient: init 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient: _reopen_session rank -1 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient: _add_conns ranks=[0,5,4,1,2,7,8,6,3] 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient(hunting): picked mon.noname-a con 0x7f27c0132b40 addr v1:172.21.15.26:6789/0 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient(hunting): picked mon.noname-h con 0x7f27c009faa0 addr v1:172.21.15.122:6791/0 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient(hunting): picked mon.noname-c con 0x7f27c0138d40 addr v1:172.21.15.26:6791/0 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c004ca10 con 0x7f27c0132b40 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 1 -- --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c009d5d0 con 0x7f27c0138d40 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 1 -- --> v1:172.21.15.122:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c0133620 con 0x7f27c009faa0 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient(hunting): _renew_subs 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bf59e700 10 monclient(hunting): authenticate will time out at 2054.467288s 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27be59c700 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f27c0132b40 0x7f27c0132f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54860/0 (socket says 172.21.15.26:54860) 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27be59c700 1 -- 172.21.15.26:0/726629832 learned_addr learned my addr 172.21.15.26:0/726629832 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:05:10.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189082887 0 0) 0x7f27c004ca10 con 0x7f27c0132b40 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bd59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bd59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bd59a700 10 monclient(hunting): my global_id is 195408 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.033+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ac003570 con 0x7f27c0132b40 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348696725 0 0) 0x7f27c009d5d0 con 0x7f27c0138d40 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient(hunting): my global_id is 205294 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c004ca10 con 0x7f27c0138d40 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.5 v1:172.21.15.122:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142820115 0 0) 0x7f27c0133620 con 0x7f27c009faa0 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:10.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient(hunting): my global_id is 274307 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 --> v1:172.21.15.122:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c009d5d0 con 0x7f27c009faa0 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2869517252 0 0) 0x7f27c004ca10 con 0x7f27c0138d40 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 >> v1:172.21.15.122:6791/0 conn(0x7f27c009faa0 legacy=0x7f27c009fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 >> v1:172.21.15.26:6789/0 conn(0x7f27c0132b40 legacy=0x7f27c0132f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: _finish_hunting 0 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 monclient: found mon.noname-c 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.26:6791/0 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 --> v1:172.21.15.26:6791/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00a0cc0 con 0x7f27c0138d40 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: _finish_auth 0 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27b8001b10 con 0x7f27c0138d40 2024-09-15T04:05:10.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 5 monclient: authenticate success, global_id 205294 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: dump: 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 20 monclient: get_monmap_and_config waiting for monmap|config 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.4 v1:172.21.15.26:6791/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27b8001d40 con 0x7f27c0138d40 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 1 -- 172.21.15.26:0/726629832 <== mon.4 v1:172.21.15.26:6791/0 5 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27b8002270 con 0x7f27c0138d40 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 10 monclient: get_monmap_and_config success 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 4 set_mon_vals no callback set 2024-09-15T04:05:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bd59a700 10 monclient: dump: 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 10 monclient: shutdown 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 -- 172.21.15.26:0/726629832 >> v1:172.21.15.26:6791/0 conn(0x7f27c0138d40 legacy=0x7f27c013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 -- 172.21.15.26:0/726629832 shutdown_connections 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 -- 172.21.15.26:0/726629832 >> 172.21.15.26:0/726629832 conn(0x7f27c006e370 msgr2=0x7f27c00a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 -- 172.21.15.26:0/726629832 shutdown_connections 2024-09-15T04:05:10.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 -- 172.21.15.26:0/726629832 wait complete. 2024-09-15T04:05:10.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 10 monclient: build_initial_monmap 2024-09-15T04:05:10.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 10 monclient: monmap: 2024-09-15T04:05:10.057 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 0 2024-09-15T04:05:10.057 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-09-15T04:05:10.057 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 0.000000 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:created 0.000000 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 0 (unknown) 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 1 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.noname-a 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.noname-f 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.noname-b 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.noname-g 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.noname-c 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.noname-h 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.noname-d 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.noname-i 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.noname-e 2024-09-15T04:05:10.058 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 Processor -- start 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 1 -- start start 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.037+0000 7f27bf59e700 10 monclient: init 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient: _reopen_session rank -1 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient: _add_conns ranks=[4,2,8,6,7,5,3,1,0] 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient(hunting): picked mon.noname-c con 0x7f27c009faa0 addr v1:172.21.15.26:6791/0 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient(hunting): picked mon.noname-b con 0x7f27c0132b40 addr v1:172.21.15.26:6790/0 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient(hunting): picked mon.noname-e con 0x7f27c0138d40 addr v1:172.21.15.26:6793/0 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 1 -- --> v1:172.21.15.26:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c009cfa0 con 0x7f27c0132b40 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27be59c700 1 --1- >> v1:172.21.15.26:6791/0 conn(0x7f27c009faa0 0x7f27c0098300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.26:6791/0 says I am v1:172.21.15.26:55096/0 (socket says 172.21.15.26:55096) 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27be59c700 1 -- 172.21.15.26:0/3739008792 learned_addr learned my addr 172.21.15.26:0/3739008792 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6791/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c009ccc0 con 0x7f27c009faa0 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c009b8b0 con 0x7f27c0138d40 2024-09-15T04:05:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient(hunting): _renew_subs 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient(hunting): authenticate will time out at 2054.473506s 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560847714 0 0) 0x7f27c009b8b0 con 0x7f27c0138d40 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): my global_id is 274391 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a0003570 con 0x7f27c0138d40 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.2 v1:172.21.15.26:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859456738 0 0) 0x7f27c009cfa0 con 0x7f27c0132b40 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): my global_id is 254423 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c009b8b0 con 0x7f27c0132b40 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.4 v1:172.21.15.26:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367088693 0 0) 0x7f27c009ccc0 con 0x7f27c009faa0 2024-09-15T04:05:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _init_auth method 2 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _init_auth creating new auth 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): my global_id is 205303 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6791/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c009cfa0 con 0x7f27c009faa0 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2381621305 0 0) 0x7f27a0003570 con 0x7f27c0138d40 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27c009ccc0 con 0x7f27c0138d40 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27b80021f0 con 0x7f27c0138d40 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): dump: 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.061 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): _finish_auth 0 2024-09-15T04:05:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.4 v1:172.21.15.26:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2826202607 0 0) 0x7f27c009cfa0 con 0x7f27c009faa0 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6791/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27a0003570 con 0x7f27c009faa0 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.2 v1:172.21.15.26:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3224129973 0 0) 0x7f27c009b8b0 con 0x7f27c0132b40 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27c009cfa0 con 0x7f27c0132b40 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.4 v1:172.21.15.26:6791/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27b0001900 con 0x7f27c009faa0 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): dump: 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.2 v1:172.21.15.26:6790/0 3 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27a80019e0 con 0x7f27c0132b40 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-09-15T04:05:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient(hunting): dump: 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 824866716 0 0) 0x7f27c009ccc0 con 0x7f27c0138d40 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 >> v1:172.21.15.26:6791/0 conn(0x7f27c009faa0 legacy=0x7f27c0098300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 >> v1:172.21.15.26:6790/0 conn(0x7f27c0132b40 legacy=0x7f27c0098a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: _finish_hunting 0 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 monclient: found mon.e 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0099d80 con 0x7f27c0138d40 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: _finish_auth 0 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 20 monclient: _check_auth_rotating not needed by client.admin 2024-09-15T04:05:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 5 monclient: authenticate success, global_id 274391 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient: _renew_subs 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c009c570 con 0x7f27c0138d40 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient: _renew_subs 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c009aaf0 con 0x7f27c0138d40 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f27b8002750 con 0x7f27c0138d40 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: handle_config config(0 keys) v1 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 6 ==== mon_map magic: 0 v1 ==== 722+0+0 (unknown 1782804122 0 0) 0x7f27b8002c00 con 0x7f27c0138d40 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27b67fc700 10 monclient: dump: 2024-09-15T04:05:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:epoch 1 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:fsid 4d3becf5-10bb-4b2a-818b-afafac96b25a 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:last_changed 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:created 2024-09-15T03:44:18.866916+0000 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:min_mon_release 17 (quincy) 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:election_strategy: 3 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:0: v1:172.21.15.26:6789/0 mon.a 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:1: v1:172.21.15.122:6789/0 mon.f 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2: v1:172.21.15.26:6790/0 mon.b 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:3: v1:172.21.15.122:6790/0 mon.g 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:4: v1:172.21.15.26:6791/0 mon.c 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:5: v1:172.21.15.122:6791/0 mon.h 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:6: v1:172.21.15.26:6792/0 mon.d 2024-09-15T04:05:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:7: v1:172.21.15.122:6792/0 mon.i 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:8: v1:172.21.15.26:6793/0 mon.e 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.041+0000 7f27bcd99700 4 set_mon_vals no callback set 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 7 ==== mgrmap(e 5) v1 ==== 43092+0+0 (unknown 994166761 0 0) 0x7f27b800d640 con 0x7f27c0138d40 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27bf59e700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27bf59e700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27bf59e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 8 ==== osd_map(904..904 src has 435..904) v4 ==== 3546+0+0 (unknown 1049492764 0 0) 0x7f27b8041960 con 0x7f27c0138d40 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c009bd50 con 0x7f27c0138d40 2024-09-15T04:05:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.045+0000 7f27be59c700 10 monclient: get_auth_request con 0x7f27a00381e0 auth_method 0 2024-09-15T04:05:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.049+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (unknown 1092875540 0 118188397) 0x7f27b8012950 con 0x7f27c0138d40 2024-09-15T04:05:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.049+0000 7f27b67fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-09-15T04:05:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.049+0000 7f27b67fc700 10 monclient: _finish_command 1 = system:0 2024-09-15T04:05:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.217+0000 7f27bf59e700 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T04:05:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.217+0000 7f27bf59e700 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T04:05:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.217+0000 7f27bf59e700 10 monclient: _send_mon_message to mon.e at v1:172.21.15.26:6793/0 2024-09-15T04:05:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.217+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 --> v1:172.21.15.26:6793/0 -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f27c004bbf0 con 0x7f27c0138d40 2024-09-15T04:05:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 10 ==== config(1 keys) v1 ==== 35+0+0 (unknown 2555425225 0 0) 0x7f27b8012950 con 0x7f27c0138d40 2024-09-15T04:05:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27b67fc700 10 monclient: handle_config config(1 keys) v1 2024-09-15T04:05:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27bcd99700 4 set_mon_vals no callback set 2024-09-15T04:05:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27bcd99700 10 set_mon_vals mon_health_to_clog = false 2024-09-15T04:05:10.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27b67fc700 1 -- 172.21.15.26:0/3739008792 <== mon.8 v1:172.21.15.26:6793/0 11 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v2)=0 v2) v1 ==== 123+0+0 (unknown 691139944 0 0) 0x7f27b8012530 con 0x7f27c0138d40 2024-09-15T04:05:10.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27b67fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-09-15T04:05:10.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.229+0000 7f27b67fc700 10 monclient: _finish_command 2 = system:0 2024-09-15T04:05:10.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.233+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 >> v1:172.21.15.122:6812/20895 conn(0x7f27a00381e0 legacy=0x7f27a003a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.233+0000 7f27bf59e700 10 monclient: shutdown 2024-09-15T04:05:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.233+0000 7f27bf59e700 20 monclient: shutdown discarding 0 pending message(s) 2024-09-15T04:05:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.233+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 >> v1:172.21.15.26:6793/0 conn(0x7f27c0138d40 legacy=0x7f27c0099120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T04:05:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.233+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 shutdown_connections 2024-09-15T04:05:10.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.237+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 >> 172.21.15.26:0/3739008792 conn(0x7f27c006e370 msgr2=0x7f27c012eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T04:05:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.241+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 shutdown_connections 2024-09-15T04:05:10.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-09-15T04:05:10.241+0000 7f27bf59e700 1 -- 172.21.15.26:0/3739008792 wait complete. 2024-09-15T04:05:10.269 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T04:05:10.270 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T04:05:10.270 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T04:05:10.270 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.351 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T04:05:10.351 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T04:05:10.351 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.422 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T04:05:10.422 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T04:05:10.422 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.511 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T04:05:10.512 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T04:05:10.512 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.593 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T04:05:10.593 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T04:05:10.594 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.668 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T04:05:10.668 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T04:05:10.668 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.750 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T04:05:10.750 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T04:05:10.751 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T04:05:10.751 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.786 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T04:05:10.786 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T04:05:10.786 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T04:05:10.786 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.809 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T04:05:10.810 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T04:05:10.810 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.823 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T04:05:10.823 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T04:05:10.824 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.844 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T04:05:10.844 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-09-15T04:05:10.844 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.908 INFO:tasks.ceph.mon.d:Stopped 2024-09-15T04:05:10.908 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-09-15T04:05:10.908 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.925 INFO:tasks.ceph.mon.e:Stopped 2024-09-15T04:05:10.925 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-09-15T04:05:10.925 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.946 INFO:tasks.ceph.mon.f:Stopped 2024-09-15T04:05:10.946 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-09-15T04:05:10.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:10.965 INFO:tasks.ceph.mon.g:Stopped 2024-09-15T04:05:10.965 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-09-15T04:05:10.965 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:11.026 INFO:tasks.ceph.mon.h:Stopped 2024-09-15T04:05:11.027 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-09-15T04:05:11.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T04:05:11.046 INFO:tasks.ceph.mon.i:Stopped 2024-09-15T04:05:11.046 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T04:05:11.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-15T04:05:11.090 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T04:05:11.090 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T04:05:11.464 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T04:05:11.464 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T04:05:11.897 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T04:05:11.897 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T04:05:12.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T04:05:12.101 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T04:05:12.376 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T04:05:12.376 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T04:05:12.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T04:05:12.625 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T04:05:12.845 INFO:tasks.ceph:Archiving mon data... 2024-09-15T04:05:12.846 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.a.tgz 2024-09-15T04:05:12.847 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-15T04:05:12.853 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.oshWoWvhGY 2024-09-15T04:05:12.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.oshWoWvhGY 2024-09-15T04:05:12.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.oshWoWvhGY 2024-09-15T04:05:13.018 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.oshWoWvhGY is 489KB 2024-09-15T04:05:13.086 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.oshWoWvhGY 2024-09-15T04:05:13.093 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.b.tgz 2024-09-15T04:05:13.093 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-15T04:05:13.100 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.rAwalI2hl5 2024-09-15T04:05:13.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rAwalI2hl5 2024-09-15T04:05:13.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.rAwalI2hl5 2024-09-15T04:05:13.263 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.rAwalI2hl5 is 478KB 2024-09-15T04:05:13.325 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.rAwalI2hl5 2024-09-15T04:05:13.332 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.c.tgz 2024-09-15T04:05:13.332 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-15T04:05:13.338 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.9ohqBaFoCH 2024-09-15T04:05:13.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9ohqBaFoCH 2024-09-15T04:05:13.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.9ohqBaFoCH 2024-09-15T04:05:13.502 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.9ohqBaFoCH is 484KB 2024-09-15T04:05:13.525 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.9ohqBaFoCH 2024-09-15T04:05:13.530 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.d.tgz 2024-09-15T04:05:13.530 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-15T04:05:13.578 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.TAS7zPxc4Y 2024-09-15T04:05:13.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.TAS7zPxc4Y 2024-09-15T04:05:13.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.TAS7zPxc4Y 2024-09-15T04:05:13.698 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.TAS7zPxc4Y is 489KB 2024-09-15T04:05:13.766 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.TAS7zPxc4Y 2024-09-15T04:05:13.773 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.e.tgz 2024-09-15T04:05:13.773 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-09-15T04:05:13.779 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.3IvMwFLQXo 2024-09-15T04:05:13.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.3IvMwFLQXo 2024-09-15T04:05:13.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.3IvMwFLQXo 2024-09-15T04:05:13.896 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.3IvMwFLQXo is 478KB 2024-09-15T04:05:13.961 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.3IvMwFLQXo 2024-09-15T04:05:13.969 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.f.tgz 2024-09-15T04:05:13.969 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T04:05:13.976 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.mfhY4g85ns 2024-09-15T04:05:13.977 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.mfhY4g85ns 2024-09-15T04:05:14.071 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.mfhY4g85ns 2024-09-15T04:05:14.090 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.mfhY4g85ns is 478KB 2024-09-15T04:05:14.113 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.mfhY4g85ns 2024-09-15T04:05:14.119 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.g.tgz 2024-09-15T04:05:14.119 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T04:05:14.168 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.EQUtCT4p2a 2024-09-15T04:05:14.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.EQUtCT4p2a 2024-09-15T04:05:14.264 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.EQUtCT4p2a 2024-09-15T04:05:14.290 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.EQUtCT4p2a is 477KB 2024-09-15T04:05:14.353 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.EQUtCT4p2a 2024-09-15T04:05:14.359 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.h.tgz 2024-09-15T04:05:14.359 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T04:05:14.366 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.wAgF7eVPCD 2024-09-15T04:05:14.367 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.wAgF7eVPCD 2024-09-15T04:05:14.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.wAgF7eVPCD 2024-09-15T04:05:14.525 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.wAgF7eVPCD is 475KB 2024-09-15T04:05:14.589 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.wAgF7eVPCD 2024-09-15T04:05:14.595 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/data/mon.i.tgz 2024-09-15T04:05:14.597 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T04:05:14.603 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.KXhsWYAyUL 2024-09-15T04:05:14.603 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.KXhsWYAyUL 2024-09-15T04:05:14.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.KXhsWYAyUL 2024-09-15T04:05:14.756 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.KXhsWYAyUL is 478KB 2024-09-15T04:05:14.821 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.KXhsWYAyUL 2024-09-15T04:05:14.830 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T04:05:14.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T04:05:14.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T04:05:14.847 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-15T04:05:14.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T04:05:14.889 DEBUG:teuthology.orchestra.run.smithi122:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-15T04:05:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T04:05:14.921 INFO:teuthology.orchestra.run.smithi122.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-15T04:05:14.927 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T04:05:14.929 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/remote/smithi026/crash 2024-09-15T04:05:14.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T04:05:14.946 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/remote/smithi122/crash 2024-09-15T04:05:14.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T04:05:14.962 INFO:tasks.ceph:Compressing logs... 2024-09-15T04:05:14.962 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T04:05:14.965 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T04:05:14.983 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31969.log 2024-09-15T04:05:14.984 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28442.log 2024-09-15T04:05:14.984 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29002.log 2024-09-15T04:05:14.985 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31969.log: /var/log/ceph/ceph-client.admin.28442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27153.log 2024-09-15T04:05:14.985 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28442.log.gz 2024-09-15T04:05:14.985 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21213.log 2024-09-15T04:05:14.986 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29002.log: gzip -5 88.8%--verbose -- /var/log/ceph/ceph-client.admin.30081.log 2024-09-15T04:05:14.986 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31969.log.gz 2024-09-15T04:05:14.986 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27153.log: /var/log/ceph/ceph-client.admin.21213.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29002.log.gz 2024-09-15T04:05:14.986 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27153.log.gz -- replaced with /var/log/ceph/ceph-client.admin.21213.log.gz 2024-09-15T04:05:14.987 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:14.987 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27018.log 2024-09-15T04:05:14.987 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27531.log 2024-09-15T04:05:14.987 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29946.log 2024-09-15T04:05:14.988 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27018.log: /var/log/ceph/ceph-client.admin.30081.log: /var/log/ceph/ceph-client.admin.27531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24062.log 2024-09-15T04:05:14.988 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29946.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27018.log.gz 2024-09-15T04:05:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25059.log 2024-09-15T04:05:14.989 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27531.log.gz 2024-09-15T04:05:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24062.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30081.log.gz 2024-09-15T04:05:14.989 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29946.log.gz 2024-09-15T04:05:14.989 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.24062.log.gz 2024-09-15T04:05:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27261.log 2024-09-15T04:05:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32401.log 2024-09-15T04:05:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27261.log: /var/log/ceph/ceph-client.admin.25059.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27261.log.gz 2024-09-15T04:05:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25559.log 2024-09-15T04:05:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32401.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25059.log.gz 2024-09-15T04:05:14.991 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32401.log.gz 2024-09-15T04:05:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29191.log 2024-09-15T04:05:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24208.log 2024-09-15T04:05:14.993 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25559.log: /var/log/ceph/ceph-client.admin.29191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26266.log 2024-09-15T04:05:14.993 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27315.log 2024-09-15T04:05:14.993 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29191.log.gz 2024-09-15T04:05:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24208.log: /var/log/ceph/ceph-client.admin.26266.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T04:05:14.994 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.25559.log.gz 2024-09-15T04:05:14.994 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.26266.log.gz 2024-09-15T04:05:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27315.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.24208.log.gz 2024-09-15T04:05:14.995 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27315.log.gz 2024-09-15T04:05:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26401.log 2024-09-15T04:05:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27477.log 2024-09-15T04:05:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27587.log 2024-09-15T04:05:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31004.log 2024-09-15T04:05:14.996 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26401.log.gz 2024-09-15T04:05:14.997 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27477.log.gz 2024-09-15T04:05:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23897.log 2024-09-15T04:05:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27587.log: /var/log/ceph/ceph-client.admin.31004.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27587.log.gz 2024-09-15T04:05:14.998 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.31004.log.gz 2024-09-15T04:05:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27668.log 2024-09-15T04:05:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23897.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.23897.log.gz 2024-09-15T04:05:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T04:05:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23895.log 2024-09-15T04:05:15.000 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27668.log.gz 2024-09-15T04:05:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.24216.log 2024-09-15T04:05:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25965.log 2024-09-15T04:05:15.001 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23895.log.gz 2024-09-15T04:05:15.001 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24216.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24216.log.gz 2024-09-15T04:05:15.003 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27504.log 2024-09-15T04:05:15.004 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32563.log 2024-09-15T04:05:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27504.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25965.log.gz 2024-09-15T04:05:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30464.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29110.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27504.log.gz 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32563.log: gzip -5 /var/log/ceph/ceph-client.admin.30464.log: --verbose -- /var/log/ceph/ceph-client.admin.28361.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29110.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30464.log.gz 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32563.log.gz 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29245.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29110.log.gz 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30896.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28361.log.gz 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23818.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28731.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31672.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24252.log 2024-09-15T04:05:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28731.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29245.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26131.log 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31672.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30896.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28731.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28947.log 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24252.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.23818.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26131.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31672.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17183.log 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24252.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28947.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.26131.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24854.log 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28947.log.gz 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.tmp-client.admin.17183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25586.log 2024-09-15T04:05:15.012 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17183.log.gz 2024-09-15T04:05:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27559.log 2024-09-15T04:05:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24854.log: gzip -5 --verbose /var/log/ceph/ceph-client.admin.25586.log: -- /var/log/ceph/ceph-client.admin.25424.log 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33025.log 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24854.log.gz 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.25586.log.gz 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28280.log 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27559.log.gz 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23669.log 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.33025.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25424.log.gz 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28920.log 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30189.log 2024-09-15T04:05:15.022 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28280.log.gz 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23669.log: gzip /var/log/ceph/ceph-client.admin.28920.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32914.log 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33025.log.gz 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31834.log 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30189.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28920.log.gz/var/log/ceph/ceph-client.admin.32914.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.23669.log.gz 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30950.log 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30189.log.gz 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31807.log 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32914.log.gz 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29703.log 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30950.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31834.log.gz 2024-09-15T04:05:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32034.log 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4%/var/log/ceph/ceph-client.admin.29703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29380.log 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31807.log.gz 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30950.log.gz 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27874.log 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29703.log.gz 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29380.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32034.log.gz 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29380.log.gz 2024-09-15T04:05:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28388.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26563.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28388.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27874.log.gz 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28388.log.gz 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26239.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26509.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26563.log: /var/log/ceph/ceph-client.admin.26239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30518.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26563.log.gz 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26455.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26239.log.gz 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26428.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30518.log: /var/log/ceph/ceph-client.admin.26455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24718.log 2024-09-15T04:05:15.025 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26509.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32455.log 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30518.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26428.log: /var/log/ceph/ceph-client.admin.24718.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.26455.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24951.log 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26428.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30923.log 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32455.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.24718.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32671.log 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24951.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32455.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30923.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24951.log.gz 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30545.log 2024-09-15T04:05:15.026 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24555.log 2024-09-15T04:05:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32671.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30923.log.gz 2024-09-15T04:05:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25884.log 2024-09-15T04:05:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30545.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32671.log.gz 2024-09-15T04:05:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29784.log 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31347.log 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30545.log.gz 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24555.log.gz 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29542.log /var/log/ceph/ceph-client.admin.29784.log: 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25884.log.gz 2024-09-15T04:05:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25775.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31564.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29542.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29784.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23845.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31347.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29434.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29542.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25775.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.23845.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31753.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31564.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31618.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29434.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23845.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25262.log 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29434.log.gz 2024-09-15T04:05:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.31618.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32320.log 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31753.log.gz 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30707.log 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25262.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31618.log.gz 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31455.log 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32320.log: gzip /var/log/ceph/ceph-client.admin.30707.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25505.log 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25262.log.gz 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip 89.4%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.32320.log.gz-- 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26536.log 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31455.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30707.log.gz 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.25505.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27234.log 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31455.log.gz 2024-09-15T04:05:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30027.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26536.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.25505.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28036.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27234.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27342.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26536.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27288.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27234.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28470.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27342.log: 89.1%/var/log/ceph/ceph-client.admin.27288.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.30027.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.28117.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28036.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27342.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28470.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27288.log.gz 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26747.log 2024-09-15T04:05:15.085 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28470.log.gz 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31536.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26991.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26747.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28117.log.gz 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31915.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26747.log.gz 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24586.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26991.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31536.log.gz 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 /var/log/ceph/ceph-client.admin.31915.log: --verbose -- /var/log/ceph/ceph-client.admin.26964.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26991.log.gz 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25370.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24586.log: gzip 88.9%-5 -- replaced with /var/log/ceph/ceph-client.admin.31915.log.gz--verbose 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:-- /var/log/ceph/ceph-client.admin.24222.log 2024-09-15T04:05:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26964.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.24586.log.gz 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30491.log 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26964.log.gz 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.24222.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32061.log 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.25370.log.gz 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28702.log 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24222.log.gz 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29299.log 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32061.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30491.log.gz-5 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:--verbose -- /var/log/ceph/ceph-client.admin.31726.log 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28702.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.32061.log.gz 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23898.log 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28702.log.gz 2024-09-15T04:05:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29488.log 2024-09-15T04:05:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31726.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29299.log.gz 2024-09-15T04:05:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29757.log 2024-09-15T04:05:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23898.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31726.log.gz 2024-09-15T04:05:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24054.log 2024-09-15T04:05:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29488.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23898.log.gz 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25748.log 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29757.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27450.log 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29488.log.gz 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24054.log: /var/log/ceph/ceph-client.admin.25748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28524.log 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29757.log.gz 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7%/var/log/ceph/ceph-client.admin.27450.log: -- replaced with /var/log/ceph/ceph-client.admin.24054.log.gz 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20887.log 2024-09-15T04:05:15.089 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25748.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28974.log 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28524.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.27450.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30324.log 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.20887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27207.log 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28524.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28974.log: /var/log/ceph/ceph-client.admin.30324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27045.log 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.20887.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28974.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30351.log /var/log/ceph/ceph-client.admin.27207.log: 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30324.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32428.log 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27045.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27207.log.gz 2024-09-15T04:05:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29892.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30351.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27045.log.gz 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28497.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32293.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9%/var/log/ceph/ceph-client.admin.29892.log: -- replaced with /var/log/ceph/ceph-client.admin.30351.log.gz 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32995.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28497.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32428.log.gz 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29892.log.gz 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32293.log: gzip -5 --verbose -- 88.6%/var/log/ceph/ceph-client.admin.30734.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28497.log.gz 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32293.log.gz 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31112.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31199.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30405.log 2024-09-15T04:05:15.091 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.32995.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31112.log: /var/log/ceph/ceph-client.admin.31199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28063.log 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30734.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31942.log 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30405.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31112.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.31199.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26698.log /var/log/ceph/ceph-client.admin.28063.log: 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31942.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30405.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24204.log 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28063.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32860.log 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26698.log: /var/log/ceph/ceph-client.admin.24204.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31942.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.26698.log.gz 2024-09-15T04:05:15.092 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24204.log.gz 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31085.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30869.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31085.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.32860.log.gz 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:-5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31888.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30869.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31085.log.gz 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24826.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25857.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31888.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30869.log.gz 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26774.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25857.log: -- replaced with /var/log/ceph/ceph-client.admin.31888.log.gz 2024-09-15T04:05:15.093 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24826.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26774.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.25857.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26774.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30761.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29029.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30761.log: /var/log/ceph/ceph-client.admin.29029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31996.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27180.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30761.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.audit.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29029.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30842.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31996.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31996.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27180.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27180.log.gz 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32698.log 2024-09-15T04:05:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26671.log 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32698.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30842.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr: 96.4% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32698.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24637.log 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26671.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26671.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24060.log 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31428.log /var/log/ceph/ceph-client.admin.24637.log: 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26937.log 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24060.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.24637.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31428.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.24060.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31428.log.gz 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31139.log 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31482.log 2024-09-15T04:05:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32347.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26937.log.gz 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27614.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29461.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31139.log.gz 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32347.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31482.log.gz 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27614.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32347.log.gz 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27614.log.gz 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26617.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30297.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29461.log.gz 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25803.log 2024-09-15T04:05:15.096 INFO:teuthology.orchestra.run.smithi026.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30297.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26617.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30297.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27928.log 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25911.log 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.27928.log: -- /var/log/ceph/ceph-client.admin.25667.log 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.25803.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25911.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27928.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25911.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32617.log 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24585.log /var/log/ceph/ceph-client.admin.25667.log: 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25721.log 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24585.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25667.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32617.log.gz 2024-09-15T04:05:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28866.log 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24585.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26293.log 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25721.log: /var/log/ceph/ceph-client.admin.28866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29164.log 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29272.log 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28866.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25721.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29164.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26293.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27695.log 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29865.log 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29272.log: /var/log/ceph/ceph-client.admin.27695.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29164.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29272.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27695.log.gz 2024-09-15T04:05:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24664.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26482.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29865.log: /var/log/ceph/ceph-client.admin.24664.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29865.log.gz 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27737.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.24664.log.gz 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25316.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.27737.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26828.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26482.log.gz 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25316.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32088.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27737.log.gz 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28334.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26828.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.25316.log.gz 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29919.log 2024-09-15T04:05:15.099 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32088.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26828.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30243.log 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.32088.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28307.log 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29919.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28334.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30243.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29919.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30243.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24420.log 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28839.log 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27955.log 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24420.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28307.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31401.log 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24420.log.gz 2024-09-15T04:05:15.100 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27955.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28839.log.gz 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27955.log.gz 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25343.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29649.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28758.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25343.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.31401.log.gz 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.29649.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32158.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25343.log.gz 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28009.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28758.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29649.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29218.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32158.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29594.log 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28009.log: -- replaced with /var/log/ceph/ceph-client.admin.28758.log.gz 2024-09-15T04:05:15.101 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32158.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26320.log 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28009.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23695.log 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29218.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32509.log 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26320.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29594.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32887.log 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31780.log 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32509.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26320.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23695.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32887.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32509.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32887.log.gz 2024-09-15T04:05:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30977.log 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31861.log 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.30977.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31780.log.gz 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24978.log 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30977.log.gz 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31861.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31861.log.gz 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32941.log 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30054.log 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27099.log 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32941.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.24978.log.gz 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30054.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32941.log.gz 2024-09-15T04:05:15.103 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30054.log.gz 2024-09-15T04:05:15.108 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.h.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.i.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23906.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21159.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21191.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21134.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21083.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-09-15T04:05:15.109 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T04:05:15.110 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T04:05:15.110 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.f.log: /var/log/ceph/ceph-mon.g.log: /var/log/ceph/ceph.log: /var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.audit.log: 95.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T04:05:15.110 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.23906.log: /var/log/ceph/ceph-client.admin.21159.log: /var/log/ceph/ceph-client.admin.21191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21218.log 2024-09-15T04:05:15.110 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.21191.log.gz 2024-09-15T04:05:15.110 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.21159.log.gz 2024-09-15T04:05:15.125 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.21134.log: /var/log/ceph/ceph-client.admin.21083.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T04:05:15.125 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.21083.log.gz 2024-09-15T04:05:15.125 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.21134.log.gz 2024-09-15T04:05:15.125 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.21218.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.21218.log.gz 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23879.log 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27099.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27099.log.gz 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24501.log 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30653.log 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24908.log 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24501.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.23879.log.gz 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31266.log 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.24501.log.gz 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24908.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30653.log.gz 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28253.log 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.24908.log.gz 2024-09-15T04:05:15.129 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30788.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26077.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28253.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31266.log.gz 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30788.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28253.log.gz 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29083.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30788.log.gz 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27641.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29083.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26077.log.gz 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29083.log.gz 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28226.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31591.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28226.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27641.log.gz 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30270.log 2024-09-15T04:05:15.130 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28226.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31591.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31591.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26104.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27072.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30270.log: /var/log/ceph/ceph-client.admin.26104.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30270.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26104.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23889.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27072.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27072.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25992.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29730.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip /var/log/ceph/ceph-client.admin.25992.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.24366.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.23889.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29730.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25992.log.gz 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25005.log 2024-09-15T04:05:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23723.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9%/var/log/ceph/ceph-client.admin.24366.log: -- replaced with /var/log/ceph/ceph-client.admin.29730.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24366.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32482.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.25005.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32725.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.23723.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24266.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32482.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32725.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32725.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29811.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24266.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24266.log.gz 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29676.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-09-15T04:05:15.132 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27423.log 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29811.log.gz/var/log/ceph/ceph-client.admin.28893.log: 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29676.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31293.log 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27423.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27423.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26185.log 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28578.log 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31031.log 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26185.log: 89.2%/var/log/ceph/ceph-client.admin.28578.log: -- replaced with /var/log/ceph/ceph-client.admin.31293.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26185.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32968.log 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28578.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31031.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31031.log.gz 2024-09-15T04:05:15.133 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29137.log 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30572.log 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30599.log 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29137.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32968.log.gz 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30572.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29137.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26801.log 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30572.log.gz 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30599.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30599.log.gz 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28172.log 2024-09-15T04:05:15.134 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30626.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26801.log: /var/log/ceph/ceph-client.admin.28172.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26801.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28172.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30378.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24447.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30626.log: /var/log/ceph/ceph-client.admin.30378.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30626.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30378.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24061.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30815.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24447.log: /var/log/ceph/ceph-client.admin.24061.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24447.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.24061.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29353.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32185.log 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30815.log: /var/log/ceph/ceph-client.admin.29353.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30815.log.gz 2024-09-15T04:05:15.135 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29353.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25613.log 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27126.log 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32185.log: /var/log/ceph/ceph-client.admin.25613.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32185.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25613.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31509.log 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30680.log 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27126.log: /var/log/ceph/ceph-client.admin.31509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25938.log 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30680.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27126.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24745.log 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31509.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30680.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25938.log.gz 2024-09-15T04:05:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31320.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29622.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30162.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31320.log: /var/log/ceph/ceph-client.admin.29622.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.24745.log.gz 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31320.log.gz 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29622.log.gz 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27982.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30162.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30162.log.gz 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25532.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25032.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27982.log: /var/log/ceph/ceph-client.admin.25532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28812.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27982.log.gz 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25032.log: gzip -5 --verbose 89.4%-- /var/log/ceph/ceph.log 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25532.log.gz 2024-09-15T04:05:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24528.log 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28812.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.25032.log.gz 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28812.log.gz 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28551.log 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24528.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.24528.log.gz 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30437.log 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28551.log.gz 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25478.log 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32752.log 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25478.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30437.log.gz 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24881.log 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.25478.log.gz 2024-09-15T04:05:15.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29326.log 2024-09-15T04:05:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21109.log 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32752.log: /var/log/ceph/ceph-client.admin.24881.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32752.log.gz 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.24881.log.gz 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32212.log 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29326.log.gz 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30135.log 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30216.log 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32212.log: /var/log/ceph/ceph-client.admin.30135.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32212.log.gz 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30135.log.gz 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26910.log 2024-09-15T04:05:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30216.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30216.log.gz 2024-09-15T04:05:15.144 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26644.log 2024-09-15T04:05:15.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26910.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26910.log.gz 2024-09-15T04:05:15.144 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32644.log 2024-09-15T04:05:15.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26644.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26644.log.gz 2024-09-15T04:05:15.144 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29838.log 2024-09-15T04:05:15.144 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32644.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32644.log.gz 2024-09-15T04:05:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29056.log 2024-09-15T04:05:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24474.log 2024-09-15T04:05:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29056.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29056.log.gz 2024-09-15T04:05:15.145 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29838.log.gz 2024-09-15T04:05:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29515.log 2024-09-15T04:05:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24474.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24474.log.gz 2024-09-15T04:05:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32779.log 2024-09-15T04:05:15.148 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29515.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29515.log.gz 2024-09-15T04:05:15.148 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29973.log 2024-09-15T04:05:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32779.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32779.log.gz 2024-09-15T04:05:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31058.log 2024-09-15T04:05:15.150 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29973.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29973.log.gz 2024-09-15T04:05:15.151 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.21109.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.21109.log.gz 2024-09-15T04:05:15.155 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24393.log 2024-09-15T04:05:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31058.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31058.log.gz 2024-09-15T04:05:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24778.log 2024-09-15T04:05:15.158 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24393.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24393.log.gz 2024-09-15T04:05:15.159 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27396.log 2024-09-15T04:05:15.159 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24778.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.24778.log.gz 2024-09-15T04:05:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31699.log 2024-09-15T04:05:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27396.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27396.log.gz 2024-09-15T04:05:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24056.log 2024-09-15T04:05:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31699.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31699.log.gz 2024-09-15T04:05:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32806.log 2024-09-15T04:05:15.163 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24056.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24056.log.gz 2024-09-15T04:05:15.163 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23791.log 2024-09-15T04:05:15.165 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32806.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32806.log.gz 2024-09-15T04:05:15.165 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24059.log 2024-09-15T04:05:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.23791.log.gz 2024-09-15T04:05:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25830.log 2024-09-15T04:05:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24059.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.24059.log.gz 2024-09-15T04:05:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29407.log 2024-09-15T04:05:15.168 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25830.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.25830.log.gz 2024-09-15T04:05:15.169 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25206.log 2024-09-15T04:05:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.29407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29407.log.gz 2024-09-15T04:05:15.170 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 2024-09-15T04:05:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25206.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.25206.log.gz 2024-09-15T04:05:15.171 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32266.log 2024-09-15T04:05:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32833.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-09-15T04:05:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32590.log 2024-09-15T04:05:15.173 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32266.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32266.log.gz 2024-09-15T04:05:15.174 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30108.log 2024-09-15T04:05:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32590.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32590.log.gz 2024-09-15T04:05:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26374.log 2024-09-15T04:05:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30108.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30108.log.gz 2024-09-15T04:05:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32536.log 2024-09-15T04:05:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26374.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26374.log.gz 2024-09-15T04:05:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31645.log 2024-09-15T04:05:15.178 INFO:teuthology.orchestra.run.smithi122.stderr: 90.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T04:05:15.179 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32536.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32536.log.gz 2024-09-15T04:05:15.179 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27369.log 2024-09-15T04:05:15.180 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31645.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31645.log.gz 2024-09-15T04:05:15.180 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32239.log 2024-09-15T04:05:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27369.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27369.log.gz 2024-09-15T04:05:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25451.log 2024-09-15T04:05:15.183 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32239.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32239.log.gz 2024-09-15T04:05:15.183 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21129.log 2024-09-15T04:05:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25451.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.25451.log.gz 2024-09-15T04:05:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28415.log 2024-09-15T04:05:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.21129.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.21129.log.gz 2024-09-15T04:05:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26590.log 2024-09-15T04:05:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28415.log.gz 2024-09-15T04:05:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28090.log 2024-09-15T04:05:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26590.log.gz 2024-09-15T04:05:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24772.log 2024-09-15T04:05:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28090.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28090.log.gz 2024-09-15T04:05:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27901.log 2024-09-15T04:05:15.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24772.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24772.log.gz 2024-09-15T04:05:15.191 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33050.log 2024-09-15T04:05:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.27901.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27901.log.gz 2024-09-15T04:05:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28785.log 2024-09-15T04:05:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.33050.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33050.log.gz 2024-09-15T04:05:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.d.log 2024-09-15T04:05:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28785.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28785.log.gz 2024-09-15T04:05:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30000.log 2024-09-15T04:05:15.211 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26212.log 2024-09-15T04:05:15.296 INFO:teuthology.orchestra.run.smithi122.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.23906.log.gz 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.30000.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30000.log.gz 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31374.log 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26212.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26212.log.gz 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr: 90.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32374.log 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.31374.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31374.log.gz 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25235.log 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.32374.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32374.log.gz 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26158.log 2024-09-15T04:05:15.303 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25235.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25235.log.gz 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23896.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26158.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26158.log.gz 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25694.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.23896.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.23896.log.gz 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28144.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25694.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.25694.log.gz 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25397.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28144.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28144.log.gz 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25397.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.25397.log.gz 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24691.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26347.log 2024-09-15T04:05:15.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.24691.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.24691.log.gz 2024-09-15T04:05:15.305 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28199.log 2024-09-15T04:05:15.305 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.26347.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26347.log.gz 2024-09-15T04:05:15.305 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25640.log 2024-09-15T04:05:15.305 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.28199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28199.log.gz 2024-09-15T04:05:15.306 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25289.log 2024-09-15T04:05:15.306 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25640.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25640.log.gz 2024-09-15T04:05:15.320 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.25289.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25289.log.gz 2024-09-15T04:05:15.346 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T04:05:15.729 INFO:teuthology.orchestra.run.smithi122.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-09-15T04:05:15.746 INFO:teuthology.orchestra.run.smithi026.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T04:05:15.748 INFO:teuthology.orchestra.run.smithi122.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-09-15T04:05:15.782 INFO:teuthology.orchestra.run.smithi026.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T04:05:15.890 INFO:teuthology.orchestra.run.smithi122.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-09-15T04:05:15.931 INFO:teuthology.orchestra.run.smithi026.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T04:05:16.081 INFO:teuthology.orchestra.run.smithi026.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-09-15T04:05:16.362 INFO:teuthology.orchestra.run.smithi122.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-09-15T04:05:16.620 INFO:teuthology.orchestra.run.smithi026.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-09-15T04:05:17.564 INFO:teuthology.orchestra.run.smithi122.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T04:05:18.093 INFO:teuthology.orchestra.run.smithi026.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T04:05:18.517 INFO:teuthology.orchestra.run.smithi122.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T04:05:18.741 INFO:teuthology.orchestra.run.smithi122.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T04:05:18.742 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T04:05:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m3.772s 2024-09-15T04:05:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m12.878s 2024-09-15T04:05:18.742 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.349s 2024-09-15T04:05:20.761 INFO:teuthology.orchestra.run.smithi026.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T04:05:21.137 INFO:teuthology.orchestra.run.smithi026.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T04:05:21.138 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-09-15T04:05:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m6.170s 2024-09-15T04:05:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m19.345s 2024-09-15T04:05:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.637s 2024-09-15T04:05:21.140 INFO:tasks.ceph:Archiving logs... 2024-09-15T04:05:21.140 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/remote/smithi026/log 2024-09-15T04:05:21.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T04:05:24.461 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/remote/smithi122/log 2024-09-15T04:05:24.461 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T04:05:26.576 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T04:05:26.586 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T04:05:26.586 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T04:05:26.588 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T04:05:26.669 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T04:05:26.678 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T04:05:26.678 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T04:05:26.681 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T04:05:26.701 INFO:teuthology.orchestra.run.smithi026.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T04:05:26.702 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================== 2024-09-15T04:05:26.702 INFO:teuthology.orchestra.run.smithi026.stdout:*hv01.front.sepi 67.205.162.81 3 u 3 64 377 0.060 3.628 0.287 2024-09-15T04:05:26.702 INFO:teuthology.orchestra.run.smithi026.stdout:+hv02.front.sepi 63.231.80.2 3 u 10 64 377 0.086 3.920 0.348 2024-09-15T04:05:26.702 INFO:teuthology.orchestra.run.smithi026.stdout:+hv03.front.sepi 74.6.168.72 3 u 59 64 377 0.091 4.010 0.340 2024-09-15T04:05:26.702 INFO:teuthology.orchestra.run.smithi026.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2024-09-15T04:05:26.704 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-15T04:05:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-15T04:05:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:+hv01.front.sepi 67.205.162.81 3 u 10 128 377 0.080 -0.156 0.041 2024-09-15T04:05:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:*hv02.front.sepi 63.231.80.2 3 u 13 128 377 0.117 0.077 0.046 2024-09-15T04:05:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:+hv03.front.sepi 74.6.168.72 3 u 11 128 377 0.080 0.334 0.056 2024-09-15T04:05:26.704 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 0.000 0.000 2024-09-15T04:05:26.704 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T04:05:26.713 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T04:05:26.714 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T04:05:26.721 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T04:05:26.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T04:05:26.738 INFO:teuthology.task.internal:Duration was 1734.628734 seconds 2024-09-15T04:05:26.739 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T04:05:26.747 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T04:05:26.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T04:05:26.750 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T04:05:26.801 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T04:05:26.802 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-09-15T04:05:26.802 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T04:05:26.814 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T04:05:26.814 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T04:05:26.827 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T04:05:26.828 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T04:05:26.830 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T04:05:26.868 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T04:05:26.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T04:05:26.871 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T04:05:26.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T04:05:26.987 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T04:05:26.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T04:05:27.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T04:05:27.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T04:05:27.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T04:05:27.049 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T04:05:27.062 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-09-15T04:05:27.063 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-15T04:05:27.096 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T04:05:27.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T04:05:27.102 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T04:05:27.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T04:05:27.139 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T04:05:27.149 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T04:05:27.150 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/remote/smithi026 2024-09-15T04:05:27.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T04:05:27.177 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-15_00:54:39-rados-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7905302/remote/smithi122 2024-09-15T04:05:27.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T04:05:27.205 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T04:05:27.205 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T04:05:27.217 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T04:05:27.223 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T04:05:27.235 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T04:05:27.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T04:05:27.282 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T04:05:27.282 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T04:05:27.285 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T04:05:27.290 INFO:teuthology.orchestra.run.smithi026.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 04:05 /home/ubuntu/cephtest 2024-09-15T04:05:27.293 INFO:teuthology.orchestra.run.smithi122.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 04:05 /home/ubuntu/cephtest 2024-09-15T04:05:27.295 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T04:05:27.305 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T04:05:27.327 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/sync-many workloads/snaps-few-objects} duration: 1734.6287343502045 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-15T04:05:27.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T04:05:27.448 INFO:teuthology.run:pass